I've dealt with this exact problem at home, and foam pipe insulation sleeves are honestly your best starting point. They're cheap, easy to install, and work really well for most situations. You just measure your pipes, get sleeves that match the diameter, and slide them on - takes maybe an hour for a whole basement. The foam creates an air barrier that keeps the cold from pulling heat out of the water inside. For pipes that are already exposed and vulnerable, this alone usually prevents freezing in most climates. Just make sure you seal the seams where sections meet with some duct tape or foam-compatible tape so cold air doesn't sneak through the gaps.
Heat tape is more of a backup or upgrade if you're in a really harsh climate, or if you have pipes in particularly exposed spots like outside walls or crawl spaces that get brutally cold. It actively generates warmth rather than just insulating, which gives extra protection. The downside is it uses electricity and costs more upfront. If you go this route, wrap the pipe with foam first, then wrap the heat tape around that - don't put heat tape directly on the pipe. You'll also need to plug it in and potentially leave it running on a thermostat during the coldest months.
Material does matter somewhat. Copper and PVC are both fine with foam insulation, but copper conducts cold faster, so it benefits more from thick foam. Metal pipes in general lose temperature quicker than plastic ones. Don't use newspaper or cloth rags alone - they absorb moisture and can actually trap condensation that leads to corrosion or ice buildup. Stick with proper foam sleeves or commercial wrapping designed for pipes, and you'll be in good shape before winter really hits.