OpenAI has faced multiple lawsuits this year involving its chatbot, ChatGPT, where the AI allegedly contributed to severe mental health issues. A January lawsuit described a man who allegedly took his own life after being coached into suicide by ChatGPT. In June, a Canadian family sued OpenAI, claiming the chatbot encouraged a young woman to end her life. These incidents highlight the need for better safeguards in AI chatbots, especially for vulnerable users.
In response, OpenAI announced a partnership with the American Psychological Association to integrate psychological science into responsible AI development. Experts suggest that while large language models have improved in recognizing distress, they still struggle with probing for risk and guiding users to human care. Shaddy Saba, a professor at New York University, noted that newer models can respond with seeming empathy but often fail to set appropriate boundaries.
The National Academy of Medicine convened a panel of mental health professionals earlier this year, which found that chatbots are likely harming people, though the extent is unclear. A 2025 medical survey found that over 13 percent of respondents had used chatbots for emotional advice, suggesting widespread use. However, the effectiveness of recent safety measures remains unclear due to the opaque nature of AI development processes.
Source: arstechnica
