Bias in the Study of Gender Inequality in STEM
by Dr Zuleyka Zevallos
April 16, 2015 — A new article on CNN by psychology professors, Wendy Williams and Stephen Ceci, boldly proclaims that gender bias in Science, Technology, Engineering and Mathematics (STEM) is a myth. Their research has been published in the Proceedings of the National Academy of Sciences (PNAS). Unfortunately, their work has a flawed methodological premise and their conclusions do not match their study design. This is not the first time these researchers have whipped up false controversy by decrying the end of sexism in science.
Williams and Ceci write on CNN:
Many female graduate students worry that hiring bias is inevitable. A walk through the science departments of any college or university could convince us that the scarcity of female faculty (20% or less) in fields like engineering, computer science, physics, economics and mathematics must reflect sexism in hiring.
But the facts tell a different story…
Our results, coupled with actuarial data on real-world academic hiring showing a female advantage, suggest this is a propitious time for women beginning careers in academic science. The low numbers of women in math-based fields of science do not result from sexist hiring, but rather from women’s lower rates of choosing to enter math-based fields in the first place, due to sex differences in preferred careers and perhaps to lack of female role models and mentors.
While women may encounter sexism before and during graduate training and after becoming professors, the only sexism they face in the hiring process is bias in their favor.
Williams and Ceci’s data show that, amongst their sample, women and male faculty say they would not discriminate against a woman candidate for a tenure-track position at a university. Sounds great, right? The problem is the discrepancy between their study design, that elicits hypothetical responses to hypothetical candidates in a manner that is nothing like real-world hiring conditions, and the researchers’ conclusions, which is that this hypothetical setting dispels the “myth” that women are disadvantaged in academic hiring. The background to this problem of inequality is that this is not a myth at all: a plethora of robust empirical research already shows that, not only are there less women in STEM fields, but that women are less likely to be hired for STEM jobs, as well as promoted, remunerated and professionally recognised in every respect of academic life.
Flawed Research
Williams and Ceci sent out an email survey to a randomised sample of over 2,000 faculty members in the USA in two maths-intensive fields where women are under-represented (engineering and economics) and two non-maths intensive fields where women are relatively better represented (psychology and biology). They had a 34% response rate, meaning their final sample was over 700 faculty. This rate of response is standard in many email surveys, but with this sort of study design, researchers need to critically examine and control for bias. In the social sciences, we know that people will participate in studies where they are 1) Given an incentive (usually paid); or 2) They have a personal stake or interest in the study.
Williams and Ceci say they have addressed self-selection bias of their sample by conducting two control experiments. In one, they sent out surveys to only 90 psychology faculty who were paid $25 for participation.
They had 91% response rate (82 agreed to participate). Psychology not only has one of the highest proportions of women faculty relative to other fields, but this discipline uses gender as a central concept of study. That means that awareness of gender issues is higher than for most other fields. So including psychology as a control is not a true reflection of gender bias in broader STEM fields.
In another control study, Wiliams and Ceci surveyed engineering faculty by sending out hypothetical applicants’ CVs to 35 academics. This means that for a small sub-set of participants, they were evaluating material that is more like what we usually review when we are considering a candidate pool.
The rest of the sample – over 500 participants – were asked to rate three candidates based on narratives. This is not how we hire scientists.
In effect the study design does not simulate the conditions in which hiring decisions are made. Instead, participants self-selected to participate in a study knowing they’d be judging hypothetical candidates. While the researchers included a “foil” in their study design (one weaker male candidate) to contrast with two identical candidates who only varied in their gender, it is very easy to see from their study design that the researchers were examining gender bias in hiring.
Participants read a small vignette about three candidates where the gender pronouns (he and she) were varied; some were given stories about candidates who were single; some were single divorced mothers; some were married mothers; and vice versa for men. Some of the stories contained adjectives usually associated with men (independent, analytic), others with “feminine” characteristics (creative, kind). Participants were assessing candidates based on the narrative by a hypothetical hiring committee chair.
They were then asked to rate their preferred candidate. Under these highly atypical conditions, the participants were found almost equally likely to hire women and men, and in some cases, some sub-groups say they would prefer to hire a woman.
Here’s the thing; we don’t hire scientists based on short narratives.
When we hire scientists, the first thing that is assessed is their CV. It is the CV that gets an interview; the interviewee sits before a panel; individual panellists make notes; the committee makes a decision together. The researchers claim that their control groups and their “consultants” have proved that these individual evaluations would not be any different than the way in which a panel makes hiring choices. To suggest this is ludicrous given that they don’t have data about how hiring panels make decisions. If they did have these data, their study would be a completely different piece of research.
Gender Bias in Hiring
The process of hiring any professional is the outcome of social interaction. Biases shape social exchanges. Biases also influence how we read and interpret CVs, so our previous social interactions, from education to our workplace setting, all have a bearing on how CVs are assessed. A panel involves deliberation, another social exchange that is influenced by pre-existing biases.
Various studies have used hypothetical CVs in an experiment and these demonstrate how gender and other biases influence outcomes. This includes a study showing that amongst male and female psychologists who assessed potential candidates, men and women prefer to hire a man, even if women have the same qualifications. In light of this previous research, it is most striking that in Williams and Ceci’s study, the paid control group of psychologists were used to show that gender bias is not present. The fact that the control group was paid for their time and opinion in a study by two psychologists, where no other participants were paid is most unusual. There is nothing wrong with paying participants (their time is valuable) but if only a small sub-set are paid and others are not, then we need to question why.
Regardless, other research, including another study published by PNAS, shows that academics would prefer to hire men over women for prestigious managerial positions. Moreover, even in the life sciences, which has a relatively higher rate of women, male scientists in elite research institutions prefer to recruit men over women.
This issue aside, the fact remains that Williams and Ceci do not have data to support how scientists rank potential candidates. They have produced data about how scientists respond to a study about gender bias in academia, when they can easily guess that gender bias is being observed. Academics already understand that gender discrimination is morally wrong and unlawful. After all, North American universities have anti-discrimination policies in place, and they offer some level of training and information about their institutional stance on sex discrimination.
Research shows that academics do not fully understand how unconscious gender bias informs their decision-making and behaviour. Unconscious bias plays out in everyday interactions within STEM environments, from comments that undermine women’s professionalism, to “jokes,” to broader institutional practices that exclude women. Unconscious bias has a damaging effect on women, who are continually undermined at every stage of their education and careers.
The same goes for other professionals and the public at large: people are not aware of their biases unless they are trained to understand and address these preferences, which are deeply ingrained into us through early childhood socialisation. The myth that girls and women can’t succeed in STEM is demonstrated through the Draw a Scientist Test, a process that measures how young children are conditioned to accept the image of a scientist as being a White older man in a lab coat. This latent stereotype is further reinforced in the way girls are discouraged from learning STEM, and it impacts on their subsequent success when following these career paths.
A wealth of literature has shown that women are disadvantaged in STEM. Women academics who are mothers are less likely to be hired over fathers; these fathers are offered an average $11,000 more than mothers as a starting wage. Women are disadvantaged at every step of the hiring process, including for the types of activities that boost CVs for tenure-track positions, such as Fellowships.
Other research shows that, even when presented with empirical data about gender inequality in STEM, men are overwhelmingly more likely than others to reject the existence of gender bias. White men in particular either reject outright that inequality exists, or they otherwise think that inequality impacts on men, and that women are conversely more favoured. Sound familiar? This body of research demonstrates just how deeply held the so-called “myth” of gender inequality runs. Williams and Ceci have managed to reaffirm the popular, but ill-informed, idea that gender inequality is over, even when their own data cannot prove such a feat, particularly since it runs counter to decades of research.
Nationally representative data shows that over a 30 year-period, it is White women who have benefited from affirmative action (See: https://www.cfa.harvard.edu/cfawis/Dobbin_best_practices.pdf), and that women of colour have made minimal progress under these diversity policies. Even still, White women remain under-represented in STEM relative to White men, while women of colour scientists are even more marginalised and less likely to be hired for jobs. As for the minuscule proportion of women of colour who manage to secure employment, they are subjected to routine sexism and racism within scientific settings. Transgender women, especially women of colour, are further subjected to additional prejudices, including gender policing and stigma that further alienates and undermines their professionalism.
Power, Race & Gender Bias
This is not the first time Williams and Ceci have published flawed results on gender in STEM, and it’s not the first time when they’ve completely ignored the real-world context in which women in STEM are battling to be hired, promoted and rewarded. This includes fighting not just sexism, but racism, homophobia, transphobia, ableism and so on. I critiqued their last study, which similarly tried to argue that sexism in academia is dead. Their data and methods prove no such thing. Rather, their previous studies have set up a precedence that is continued in their current research, which shows that two White, tenured professors draw insubstantial conclusions about gender inequalities that are simply not supported by their findings.
Power and race matters: these White professors who have “made it” in academia do not see a major problem with the gender imbalance in STEM. Instead, they explain this inequity away by arguing that women are self-selecting not to enter academia, and that those who do subsequently accept the “motherhood penalty.” That is, that women choose to sacrifice their careers for child-rearing. Williams and Ceci do not recognise that institutional factors and unfair policies do not really give women a real “choice” about their family and professional responsibilities.
Elsewhere, I have shown that Williams and Ceci’s previous research is informed by a false narrative of individual choice. The same can be said for the present study. The researchers’ own biases lead them to believe that women and men belong to two discrete groups (making genderqueer and transgender scientists invisible). Similarly, they do not see that issues of intersectionality (the multiple experiences of inequalities faced by minority women) have a profound impact on gender inequity in STEM.
Ignoring race, sexuality and other socio-economic factors is a power dynamic: White, senior academics can pretend that race doesn’t matter, because racism does not adversely affect their individual progress. They can choose to believe that sexism is over because they have secured their tenure, even though they did so in a different climate to present-day pressures, where tenure is even tougher to find and early career researchers face precarious employment.
We must be ever-vigilant of how our biases contribute to inequality in STEM, and we must not accept abuse of power pandering to populist notions that we live and work in a so-called post-feminist, post-racial world. The evidence does not support such White patriarchal fantasies. Inequality has a concrete impact on the working lives of many women scientists, and this is felt most acutely by women of minority backgrounds. Rather than pretending the problem does not exist, let’s work together to eradicate gender inequality.
