The thing is, there's a difference between "health agencies say it's safe" and "we actually know everything about the long term." Reference doses (ADI - acceptable daily intake) are calculated with safety margins, but they're mostly based on studies from years ago. Aspartame for example, we've been talking about it since the 70s-80s. Since then, consumption conditions have changed - people drink way more of it than they used to, and often combined with other additives. So technically, at the "recommended" doses, it's not like it's poison, but saying it's "risk-free" is a bit much.
What bugs me is that a lot of studies are funded or orchestrated by the industries themselves, so there's real bias. And then a few recent studies suggest that some sweeteners could affect gut microbiota or glycemic response, even if it's not necessarily obvious. Nothing dramatic enough to make you freak out, but enough to make you wonder.
Honestly, if you have the choice it's probably not stupid to not overdo it. You can mix it up a bit - diet sodas occasionally, but not every day by the liter. And try to actually check what you're drinking rather than relying on one ingredient. The dose makes the poison, as they say, but with this stuff you never really know exactly what dose you're hitting.