Aromatase-dependent biosynthesis of estrogen plays an important role in maintenance of the male skeleton, and Cytochrome p450 aromatase is the key enzyme to catalyze the conversion of androgen precursors to estrogens. We investigated the association of polymorphisms in the CYP19A1 gene and bone mineral density in a Chinese cohort. 2392 extreme low femoral neck BMD cases or extreme high femoral neck BMD controls were selected from a population-based cohort and genotyped for eight SNPs in the CYP19A1 gene. Significant associations for rs17703883, rs12594287 and rs16964201 SNPs with BMD were found in men only. Men with TC/CC genotypes in the rs17703883 SNP had a 1.5 times higher risk of having extreme low femoral neck BMD (P = 0.003, empirical P value = 0.05), and decreased BMDs at total body (P = 0.004, empirical P value = 0.07) and total hip (P = 0.003, empirical P value = 0.05). Men carrying AA/AG genotypes in the rs12594287 SNP had a 30% reduced risk of having extreme low femoral neck BMD (P = 0.007, empirical P value = 0.12), and increased BMDs at total body (P = 0.0009, empirical P value = 0.018) and total hip (P = 0.001, empirical P value = 0.02). Men carrying TT/TC genotypes in the rs16964201 SNP had a 40% reduced risk of having extreme low femoral neck BMD (P = 0.005, empirical P value = 0.087), and increased BMDs at total body (P = 0.0001, empirical P value = 0.002) and total hip (P = 0.0006, empirical P value = 0.012). Haplotype analysis showed that the G-C-T-A-T haplotype was significantly related to higher BMD. Our finding suggests that genetic variations in the CYP19A1 gene are significantly associated with BMD at different skeletal sites in adult men, but not in women.