My mom redid her kitchen with precast concrete panels a few years back and hasn't had any issues so far, but honestly it really depends on your region's climate and whether you seal the joints properly. The mason is right that it's definitely faster, but the durability thing is more about maintenance - if you let moisture get into the seams, that's when it can start falling apart over time.
What really matters is that the installation is done right (the joints properly sealed with silicone or specific sealant) and that you have good ventilation in the spaces. If your house already has a history of moisture problems, maybe it's not the best choice, but in a dry place with good workmanship it works just fine.