This is a rough situation, sorry for your loss. Legally speaking: the accounts are part of the estate, so basically the heirs have to handle it - and that's probably you and your family as statutory heirs (unless a will says otherwise). That means you could theoretically access the accounts, but usually you have to be able to prove it. With email and banking, providers normally require a certified copy of the inheritance certificate or will + your ID. Some companies have special procedures for deaths.
The first step would be to collect all the accounts you can find. Look through his emails, bills, browser history (if you have access to the computer), bank statements - you'll usually find something there. Google and Facebook have special "Legacy Contact" options where you can set what happens after death; but if that wasn't set up, you'll have to contact support directly. With PayPal and the bank it's similar - just call, say who you are, and ask what they need from you to manage or delete the accounts.
If he had debts or subscriptions - important to know before you delete everything. And if you don't want to get an inheritance certificate at all: some banks also accept a certified copy of the death certificate + birth certificate + possibly a statutory declaration that you're the next of kin. Just call the individual companies and ask what they need - they're usually cooperative about this stuff.