With an RTX 4060 and i5-12400F you should be able to play both games just fine, at least at 1080p with medium-high settings. The fact that your framerate drops halfway through a match makes me think it's not a hardware shortage issue, but rather something that builds up during gameplay. Probably RAM - how much do you have? Baldur's Gate 3 and Starfield are pretty hungry, especially BG3 where the difference between 16 and 24 GB is night and day. If you only have 8GB, that's already cause number one. Also check if you're using non-optimal RAM speed - sometimes the BIOS doesn't set it properly by default.
Another thing I see a lot is thermal throttling. After months of use your PC accumulates dust and ventilation gets worse, like when I was lazy about cleaning my case a few years ago - I just had to do a good internal cleaning and the problem disappeared. Check your GPU and CPU temperatures under load with a tool like HWInfo64. If they go over 80-85°C on the GPU you're already in the danger zone. Try lowering your graphics settings a bit, especially ray tracing and upscaling - sometimes badly configured DLSS or FSR causes instability. And yeah, updating your drivers is good, but make sure you uninstall the old ones first, don't just overwrite them.
One last thing: Starfield in particular had quite a few optimization issues at launch in 2023, and it's gotten better with updates but it's not perfect. If the problem is more localized to that game, it could also be a bug with your specific setup. Also take a look at how much VRAM you're using - if you go over your card's 6GB (which is the limit anyway), Windows starts swapping to main RAM and everything slows down.