Technically, it's already his apartment and his things, so you can't just throw them away - that could be considered damage to someone else's property. But there's a perfectly legal way to solve this problem. The safest option is to send him a letter (preferably registered mail with a delivery confirmation) demanding that he pick up his belongings within a reasonable timeframe, say two weeks. In the letter, list exactly what's in the cabinets and storage closet, and make it clear that after the deadline expires, you'll start taking measures.
If he doesn't respond, you can invite him along with two witnesses, draw up a report documenting the property left in the apartment, and move everything to a storage facility (paid storage). You can then recover the storage costs from him. Or, if the things are clearly nobody's concern and are taking up space, you can invite a police officer who will file a report on the discovered property - this documents the fact that you made a legal attempt to return it to its owner.
The key thing is to document everything in writing and with witnesses. This will protect you if he later tries to make claims or complain. Usually people respond pretty quickly once they realize they'll have to pay for storage or face other hassles.