When you connect a hardware wallet with a hot wallet, most of them will fetch the balances of all addresses in the device, which doesn't sound too bad. However, this gives the node provider visibility of all the addresses on your hardware wallet, doxing all the addresses on your hardware wallet.
Something I learned from using BlockWallet (a privacy non-custodial web3 wallet) that connects to only the address you choose. This is not to suggest that the hardware wallet is unsafe but rather to suggest that more of your privacy is within your control depending on the Hot Wallet you connect to your hardwallet.