The groove at the bottom of a sliding door track is basically a tunnel for air, and that's almost always where drafts win out against weatherstripping alone. What you're dealing with is that the track has open channels on both sides plus an exposed bottom, so even if you seal around the door's edges, cold air is still flowing underneath through the groove itself.
The most practical fix is to fill that bottom channel with something flexible. Silicone caulk works well - run a bead along the entire bottom groove where the door slides. It stays somewhat flexible so the door can still move smoothly, and if you ever need to remove it later, it peels out without too much hassle. Another option is foam backer rod stuffed into the groove first, then caulked over, which gives you better insulation and uses less caulk. Just make sure whatever you use isn't so thick that it prevents the door from rolling freely.
The earlier suggestion about checking whether the door is level is worth a quick look too. If the door has settled unevenly over time, one side of the track gap might be wider than the other, which would make sealing harder. But honestly, even a perfectly level door still needs that bottom channel addressed - weatherstripping just can't reach into it.