It's not that referees are slow - the truth is that reviewing a controversial goal involves way more than it looks like from your couch. When there's a disputed play, the VAR team doesn't just look at one angle: they have to review multiple camera views, freeze the image at the exact moment of contact or when the ball is touched, and sometimes they need to wait for the cameras to capture the precise instant. If it's offside, they have to see where all the players were in that microsecond. If there's a foul, they have to determine whether it happened inside or outside the area, if it was deliberate, how intense the contact was. All of that takes time.
Plus, when you're watching those five minutes that feel like forever, the on-field referee generally already knows it's going to be reviewed and is just waiting. Meanwhile, in the VAR room they're analyzing, communicating with each other, requesting other angles from the camera operators. The Premier League and other big leagues have teams that work fast, but even so there are plays that are incredibly complicated to sort out. I've seen goals where you literally can't tell if the ball touched someone by millimeters, and that doesn't get resolved in 30 seconds.
What is true is that sometimes it feels like they're taking longer than they should, but most leagues are already trying to improve that. The system works better when referees know exactly what they're going to review and the protocol is clear from the start. Five minutes is still a lot for someone watching the match, but usually it's because there really is doubt.