Key control is definitely important, yeah, but it's not a silver bullet either. A cold wallet like Ledger is pretty safe against remote hacks because the device stays offline, but you've gotta be really careful about where you store your seed phrase and not lose it, because if you do it's game over for your access and nobody can get it back for you. The realistic take is that a cold wallet is secure if you're actually responsible yourself; if you're terrible at keeping important things safe, at least an exchange has protections if they get hacked, even though obviously you're betting they won't.