We are not able to fully measure “equal pay for equal work” because we can’t measure all relevant job characteristics. As a simple example, in the same occupation, “physician,” in the same industry, “health care,” women are more likely to be pediatricians and gerontologists and less likely to be surgeons. Surgeons get paid more. Broadly speaking there are all sorts of job characteristics that women select moreso than men, and that imply lower wages (for example, women are more likely to select jobs with better non-wage benefits like health care, women are more likely to avoid dangerous jobs like construction). This all falls under the theory of compensating differentials. As a result, it is still in general not possible to reject the free market argument you’re putting forth. Which, by the way, is Becker’s famous theory from 1957. Of course, based on the data it’s also not possible to reject the theory that the gender wage gap exists because of pure discrimination.
Among other possible explanations, there are different social expectations about child care according to gender, which results in gender differences in experience profiles and different investment in workplace skills, and so on. There are hundreds of college classes that try to address this question. Just google the syllabi of courses that use the two most common textbooks, The Economics of Women, Men and Work and [The Economics of Gender] (https://www.google.com/search?&q=economics+of+gender+syllabus)
The unadjusted wage gap is simply the raw difference between men and women, and this comes about through a wide range of different factors, with the main ones being the ones mentioned by the other users: career choices, working hours, maternity leave, etc.
However, these factors are all accounted for when we calculate the adjusted wage gap. The adjusted wage gap is essentially everything we can possibly think of that separates the working behaviors of men and women that could explain the difference, and what’s left is (more or less) basic discrimination.
There’s a good paper that discusses the topic here: Addressing the gender pay gap: Government and social partner actions. When we account for all the factors we can think of, we’re still typically left with an adjusted wage gap of around 5-10%. So the “77 cents” figure refers to the unadjusted gap, and the “991 cents” figure refers to the adjusted gap.
Importantly, the paper I link to also discusses the myth that the unadjusted wage gap is indicative of non-discriminatory processes, as this simply isn’t true. For example, even though career choices do affect wage levels, it doesn’t change the fact there are social norms that pressure men and women into making different choices, nor does it change the fact that as the number of women in a field increases, the lower the value (and thus the wage) that is attributed to that field.
What all this means, to answer your title briefly, is the reason the wage gap exists is partly regular differences between men and women (regardless of whether it’s innate or cultural), but also partly because of discrimination.
If we can pay women less, why don’t people hire them? Well, primarily because people don’t realise that they are getting the same value at a lower price. They pay women less precisely because they think they are not as qualified or not as suitable for the job, regardless of their actual characteristics or attributes compared to other potential candidates.
This is why, for example, when faced with the exact same CV that only differs in having a male or female name attached, the “male” applicant will significantly be more successful and be paid more money when successful. The reasoning that employers give in this situation is not “well, they’re a woman so I can pay them less” but rather there is a post-hoc justification, like selecting positive attributes from a CV with a male name and selecting negative attributes from a CV with a female name, and concluding that the male applicant is better suited.
The reason why economic forces aren’t getting rid of wage gaps is because economic forces are driven by humans who are necessarily biased. They aren’t making decisions based on who is the most qualified at the best price, they are making decisions on who they think is the best qualified at what they think is a fair price.
To explain like you’re five, for the sake of argument imagine that iPhones and a similar non-branded android were actually identical in every way (they somehow got around copyright and recreated the design, had the same functioning parts, to the same build quality, same warranty, etc). You’re saying that since economic forces should be pushing for the most efficient option (the cheaper non-branded android), why don’t people just buy that? And the answer is that because people think that iPhones are better because of all the preconceptions and ideas they’ve built around the phone. These preconceptions in the wage gap are prejudices and stereotypes.
The products are equal but the buyer still believes it’s worth spending the extra amount on the iPhone for the perceived quality it brings with it.
The Sociological Mail