Is a heat pump worth it with poor insulation?

How bad are we talking here - old windows and walls with no insulation, or more like small gaps? With really poor insulation, a heat pump is pretty much a waste of money because you'd just be generating heat expensively that escapes right away. It makes more sense to insulate first - roof, walls, windows - then a heat pump will run efficiently afterwards. If you're currently stuck with expensive oil or gas, a heat pump might save you money in the short term, but long-term you're just making it harder on yourself.