How do you design sustainable architecture without dramatically increasing costs?

I'm trying to figure out how to integrate sustainability principles into a renovation project without breaking the bank. What are the elements that actually make a difference? I'm especially interested in thermal insulation and materials.