What nobody's really making clear is that you have a deadline to make this decision. It's usually a few months (depends on where you live in Portugal), and during that time you can ask the bank for complete information about the debts before deciding whether to accept or reject the inheritance. It's not something you need to figure out right away.
If you accept the inheritance, it's true that the debts come out of what he left behind - the house, savings, all of that. If the total value of the debts is bigger than the estate (like, owing 100k and only leaving 50k in assets), then you don't lose your own money, the inheritance just doesn't cover everything and the bank doesn't get paid the rest. Now, if you reject the inheritance, you don't inherit anything, neither the assets nor the debts - it all stays behind.
The catch is this: before you sign anything with the bank or do things like paying his bills or messing with his accounts, talk to your lawyer, because some actions can be interpreted as tacit acceptance of the inheritance. Better to have that straight before you do anything.