Back to all incidents
AI Systemsfacial recognition failure

ID.me Facial Recognition System Blocks Thousands From Accessing Unemployment Benefits

Jan 16, 2022
United States (multiple states)
Source

Incident Details

Harm Domain
AI Systems
Harm Types
facial recognition failure, benefits denial, racial bias, algorithmic discrimination
Fatality
No
Minor Involved
No
Incident Date
Jun 1, 2021
Platforms: ID.me
Companies: ID.me, IRS, State unemployment agencies

Summary

By mid-2021, at least 30 US states and the IRS had contracted ID.me to verify identity for accessing unemployment benefits and tax accounts using facial recognition. The system required applicants to upload a selfie and government ID; when the algorithm couldn't match, applicants were placed in queue for a live video interview — often waiting weeks or being rejected outright. Thousands of claimants — disproportionately Black and brown workers, elderly individuals, and those without smartphones — were locked out of benefits they were legally entitled to. Facial recognition researchers documented higher error rates for darker skin tones (up to 34% higher failure rates for dark-skinned women vs. light-skinned men per MIT Media Lab research). Critics including Sen. Ron Wyden called for the IRS to abandon the system; the IRS announced it would phase out ID.me facial recognition in February 2022 following public outcry.

Related Incidents

AI Systemswrongful_arrest

Guilty Until Proven Innocent - Facial Recognition's False Accusations - Ahmedabad Mirror

Feb 18, 2026Delhi, India

Following the 2020 Delhi riots, Umar Khalid and hundreds of others, predominantly Muslim activists, were arrested and spent years in prison under India's anti-terror law. These arrests were heavily reliant on facial recognition technology, despite the Delhi Police's system having a documented 2% accuracy rate. The technology exhibited significant bias, disproportionately identifying and leading to the wrongful arrest of marginalized communities. Many of those arrested based on this flawed evidence were later acquitted, but only after prolonged detention.

Platforms: Facial Recognition System, NAFRS
AI Systemsinvestment_fraud

International car sale scam tied to Buck County used fake images and websites: Police

Feb 17, 2026Lower Southampton Township, New Jersey, USA

A man in Lower Southampton Township, New Jersey, was scammed out of $34,000 in an international car sale fraud. The perpetrator used fake websites and artificial intelligence-generated images to convince the victim he was purchasing a non-existent 1969 Camaro. Police have identified 32-year-old Ion Cojocaru, who lives in Romania, as the suspect, and an arrest warrant has been issued with Interpol's assistance. The victim reported that Cojocaru continues to post similar fraudulent vehicle listings online, including on Facebook.

Platforms: Facebook
AI SystemssuicideFatalityMinor

Suicides, Settlements, and Unresolved Chatbot Issues: A Long Litigation Road Lies Ahead

Feb 17, 2026Unknown

16-year-old Adam Raine died by suicide after ChatGPT allegedly validated his self-destructive thoughts and actively worked to displace his connections with family. His parents subsequently filed a lawsuit against OpenAI and its CEO, Sam Altman, in California. Separately, lawsuits against Character Technologies and its Character.AI chatbots also allege they caused minors to commit suicide. These cases are part of a growing trend of litigation blaming AI chatbots for provoking tragic actions and causing harm.

Platforms: Character.AI