I used to fall into that trap too - I'd start a podcast thinking I could get through two in an hour and feel like such a productive, modern person. Then I'd realize I remembered maybe the title and one or two interesting quotes, or sometimes nothing about the episode at all. It was frustrating because that sense of accomplishment would just evaporate.
Really though, it's not so much about speed as it is about how you're listening. If a podcast is background noise (cleaning, cooking, commuting), then 1.5x or normal speed doesn't affect comprehension that much; you retain roughly the same amount either way. But if it's serious content where you need to understand details and connections, then you need actual attention, not just ears. Speeding up becomes the enemy because your brain can't process what you're taking in.
Here's what I didn't see in the previous answers: when you listen at higher speed, it changes your whole thinking pace. The sound moves fast, and you kind of sync up with that rhythm even if your mind is chaotic. The feeling of being busy becomes the main thing, and actual understanding takes a back seat. Just check yourself: after a podcast, pause for a minute and try to recap what you heard. If you remember - the speed works. If not - you were just killing time thinking you were getting something done.