Voltage drop at idle is totally normal, but it shouldn't cause visible flickering like you're describing. When the engine's running slower, the alternator generates less power, so the system voltage naturally dips a bit - that's just physics. What matters is whether it's dipping enough to make your lights actually flicker, which suggests something's working harder than it should be.
I had this exact problem a couple years back. My voltage was sitting around 12.8V at idle instead of the 13.5+ it should be, and everything would dim when sitting still. Turned out my battery terminals were corroded underneath the cable clamps - not visibly corroded on top, but enough to create resistance. Cleaned those up, replaced the battery as a precaution since it was old anyway, and boom, problem gone. So yeah, check your battery terminals first - loosen the cables, clean the connections really well with a wire brush, and make sure they're tight. If that doesn't fix it, then you're looking at either a weak battery or an alternator that's losing output.
The other folks mentioning the alternator aren't wrong - that's definitely possible. But before you spend money on a replacement, grab a cheap multimeter and check what your charging voltage actually is while idling. If it's under 13V consistently, your alternator's probably on its way out. If it's closer to normal but the battery's old (more than 4-5 years), the battery itself might not be holding charge well enough to handle the load when RPMs are down.