No, infertility is not always a woman's problem. In only about one-third of cases is infertility due to the woman (female factors). In another one third of cases, infertility is due to the man (male factors). The remaining cases are caused by a mixture of male and female factors or by unknown factors.
Source: The National Women's Health Information Center