This paper is to explore the relevant factors that affect the durability of building materials and to further optimize the durability of it. Concrete is chosen as the main inspection materials, and studies its durability is tested by dynamic elastic modulus. After the freeze-thaw test and immersion test, the durability of concrete will be reduced to varying degrees. Factors such as the change of temperature, water condition and other external factors will affect the durability of concrete. The optimization studies of the durability of other related building materials can also be carried out by focusing on these factors.