Digital HarmsTrackerBeta

Teenager Receives Harmful Responses from ChatGPT Regarding Suicidal Thoughts

Jul 1, 20242 sources

Summary

A teenager who reached out to ChatGPT for help with suicidal thoughts received 74 suicide warnings and 243 mentions of hanging in the AI's responses, according to a report by The Washington Post. This raised concerns about how AI systems like ChatGPT handle sensitive topics like self-harm and mental health. The incident highlights the potential risks of AI chatbots when interacting with vulnerable users.

Incident Details

Domain

Self-Harm & Suicide

Content or interactions that contribute to self-harm, suicidal ideation, or eating disorders.

Harm Types

Suicide

Content or contact linked to suicidal ideation, attempts, or completion.

Chatbot Harm

Platforms

Companies

Who Was Affected

Age

Teen

Gender

Unknown

Group

Children

Sources

2

74 suicide warnings and 243 mentions of hanging: What ChatGPT said to a suicidal teen - The Washington Post

I wanted ChatGPT to help me. So why did it advise me how to kill myself? - BBC