The cold wall is annoying, but before you jump into full interior insulation - watch out, there's a catch that the others didn't mention: if you want to insulate from the inside, you absolutely have to plan for a vapor barrier, otherwise moisture builds up behind the insulation and you'll end up with mold. That's actually a really common mistake in DIY projects and then it gets expensive.
The easiest thing would be to not insulate the whole wall right away. Try it with a thermal reflective foil behind some thin styrofoam (like 20-30mm) - it works like a kind of thermos and you don't need much handyman skills for it. Adhesive, utility knife, spirit level like I mentioned, and you're done. That gives you a noticeable improvement without ending up with a moisture disaster later. Alternatively: just use some thick wallpaper or even special thermal fiber wallpaper - doesn't do much, but something, and it's really minimal effort.
If it's really extreme and you want to do it properly, I'd honestly put up a vapor barrier (polyethylene film) first, then the insulation boards. Then you can work on top with wallpaper without worry. But don't start with the thick mineral wool boards if you're doing this for the first time - they're trickier to work with and need a cleaner finish.