For more information please call  800.727.2766

 

Stanford Study Found Bias in AI Tools

Employer Insight: A paper led by Stanford researchers evaluated over 4 million job applications from 3 million job applicants across 156 employers. The “Algorithmic Monocultures in Hiring” paper showed outcomes for 1,746 individual job positions. It concluded that 10.62% of outcomes reflected an adverse impact on Black applicants based on an analysis of the applications position by position. The paper found “clear racial disparities in applicant outcomes.” Fortune reported that more than one in four applications submitted by Black job seekers ended up routed to positions where outcomes could trigger federal discrimination scrutiny.

Diving deeper into the data, the study found that 26% of Black applicants and 15% of Asian applicants applied for positions where AI discriminated against their racial group. To provide more context, if AI recommended Black and Asian candidates at the same rate as White candidates, 40,000 more of their applications would have moved to the next stage of the hiring process. These groups needed to apply to 25 positions before obtaining at least one recommendation to move on for further evaluation.

Researchers from Stanford University, Chapman University, and Northeastern University participated in evaluating the data for the paper. The data came from employers with at least 5 billion dollars in annual revenue and used assessments provided by the talent platform vendor Pymetrics.

Pymetrics screens job candidates using “neuroscience and machine learning” to help employers. It asks applicants to play mini online games to measure cognitive traits, including risk tolerance, processing speed, and altruism. The platform does not rely on resumes. Researchers compared data position by position rather than pooling outcomes across employers and roles. By measuring the data this way, an increased share of positions showed an adverse impact on Black applicants, as opposed to an analysis of the entire pool of positions.