The difference was said to be significant.
Researchers say just one cup of soy milk or one serving of tofu or soy burgers every other day made a difference.
Researchers also found the association between soy foods and lower sperm count was stronger in overweight men.