Peonies can grow in one spot for decades with almost no intervention, but phlox and lupines start to degrade and get smaller after about 4-5 years without division - they need to be periodically rejuvenated by digging up and splitting the clump. Peonies are the easiest: they're real long-lifers and can easily live in a garden for half a century if you leave them alone; phlox and lupines use up the soil more actively and get dense in the center, so it's worth dividing them every few years so they don't lose their ornamental appeal.