I've bought refurbished phones more than once and the price difference is totally worth it if you know where to look. The key thing is that they have real warranty, not that "30 days" stuff some sellers try to pull. Big brick-and-mortar stores like Media Markt or Carrefour have refurbished sections with official manufacturer warranty, so if something breaks you've got somewhere to make a claim without hassle. There's also Amazon Renewed, which is pretty reliable because if you get something that's not what you paid for, they'll send it back no questions asked.
If you prefer specialized sites, Back Market is the one most people use here in Europe. They sell from certified sellers and everyone offers warranty between one and three years depending on the product. Sure, it takes a bit longer to get your order, but the price drops quite a bit compared to new. Another option is to check eBay directly with sellers who have really good ratings, though that depends more on luck.
What I'd do is rule out anything offering less than a year warranty, even if it's cheaper. And if you find a phone with battery at 85% or higher, even better. A lot of specialized shops do technical reports for each device that they show before you buy, so you can see exactly what condition it's in.