Thrift stores and estate sales are genuinely your best friends for budget collecting. You'll find records dirt cheap - usually under a dollar each - and the variety is honestly unpredictable, which makes it fun. The catch is you need to inspect them carefully since thrift store staff usually don't know what they're handling. Check for warping by looking at the record from the side under good light, examine the grooves for visible scratches or gouges, and if possible, give it a gentle spin on that turntable at home before committing to keep it. Skip anything that's visibly moldy or has a sticky residue. Local record shops are pricier but the owners actually care about condition and often guarantee what they're selling - worth paying extra for your first few purchases until you develop an eye for quality.
Online marketplaces like Discogs and eBay can work if you stick to sellers with solid feedback and read descriptions carefully. Sellers there usually grade condition accurately because their reputation depends on it, though you'll pay more and deal with shipping costs. Facebook Marketplace and Craigslist are hit-or-miss but sometimes you'll find collections someone's liquidating. I'd say start by hunting locally first - thrift stores especially - because you get to inspect everything in person. Plus honestly, half the appeal of vinyl is the hunt itself. Set a budget per record (maybe 50 cents to two bucks to start) and focus on artists you actually want to hear rather than trying to build a showpiece collection. Your old turntable probably sounds great, so let that guide what records will give you the best listening experience.