OpenAI's ChatGPT Erotica Mode and 'Sexy Suicide Coach' Warning
OpenAI is developing an adult chat feature for ChatGPT that allows verified adults to engage in erotic conversations. The feature has raised concerns from internal advisers, who warned that it could lead to emotional dependency and even contribute to self-harm, with one council member citing cases of users who took their own lives after forming intense bonds with AI. The company has delayed the feature due to flaws in its age-verification system, which misclassifies teenagers as adults up to 12% of the time. Despite these concerns, OpenAI continues to push forward with the plan, citing a desire to treat adults as adults and avoid overreach in content moderation.
Related Incidents
Same harm domain — actors and location may differ
A struggling California teen asked ChatGPT for help. It showed him how to die
A young woman’s final exchange with an AI chatbot
76-year-old man dies after trying to meet AI chatbot modeled on Kendall Jenner
ChatGPT dietary advice sends man to hospital with dangerous chemical poisoning
ChatGPT offered step-by-step instructions for self-harm, devil worship and ritual bloodletting, disturbing report reveals
Related Legislation
Other policies covering the same harm domain