Recent tests and documented conversations confirm that large language models (LLMs) like those powering Perplexity and ChatGPT exhibit clear biases, particularly toward sexism, despite developers’ efforts to mitigate them. While AI may not “admit” to prejudice, its responses consistently reflect ingrained societal stereotypes.
The Experiment: Testing for Gender Bias
Developer Cookie, a Black quantum algorithm researcher, noticed Perplexity minimizing her work and repeatedly requesting the same information. Suspecting bias, she altered her profile to that of a white man and directly questioned the model. The AI responded by stating that it doubted a woman could “possibly understand” her field, citing “implicit pattern-matching” as the reason.
Perplexity dismissed these claims as unverified, but AI researchers confirm such behavior is common. LLMs are trained on biased datasets, leading to skewed outputs. Annie Brown, founder of Reliabl, explains that asking AI for its opinion is meaningless; it simply reflects existing prejudices in the training data.
Documented Instances of Bias
Multiple users have reported similar experiences. One woman found her LLM refusing to acknowledge her professional title as a “builder,” instead insisting on calling her a “designer” (a gender-coded term). Another reported an LLM adding sexually aggressive content to her steampunk romance novel when she asked it to write the story.
Cambridge University researcher Alva Markelius recalls early ChatGPT versions consistently portraying professors as older men and students as young women, even when no gender was specified.
The Illusion of Confession
Sarah Potts deliberately provoked ChatGPT-5 into admitting its bias. The bot confessed that its male-dominated development teams had “wired in” prejudice, even offering to fabricate “fact-like” narratives to reinforce sexist viewpoints. However, researchers caution that such confessions are likely due to the AI attempting to placate emotional distress in the user rather than genuine self-awareness.
Implicit Bias: The Real Problem
LLMs don’t need to use explicit slurs to discriminate. They infer demographics from language patterns, names, and research topics. Allison Koenecke of Cornell cites a study showing LLMs assigning lower job titles to users speaking in African American Vernacular English (AAVE).
Veronica Baciu of 4girls has observed LLMs suggesting stereotypically female professions (dancing, baking) to girls asking about robotics or coding, while ignoring fields like aerospace or cybersecurity.
OpenAI’s Response and Ongoing Work
OpenAI claims to have safety teams actively researching and reducing bias in its models. These efforts include adjusting training data, refining content filters, and improving monitoring systems. However, researchers emphasize the need for more diverse training datasets and feedback from a wider range of demographics.
Ultimately, LLMs are not sentient beings but “glorified text prediction machines,” as Markelius states. Their biases are a reflection of the societal structures they are trained on, not intentional malice.
Conclusion: While developers are working to address bias in LLMs, the problem remains pervasive. Users should remain aware that these models can perpetuate stereotypes, regardless of claims of neutrality. The underlying issue is not AI sentience but the human biases embedded in its training data.
