TechAmerica
SearchSubscribe
OpenAI CSAM image classifier
Image credit OpenAI

OpenAI CSAM Image Classifier Launches With New ChatGPT Safety Warnings

OpenAI has introduced a new image classifier to detect child sexual abuse material, including AI-generated imagery. The company is also adding ChatGPT warnings, temporary account suspensions, developer guidance and investigative tools for law enforcement.

OpenAI has introduced a new image classifier to detect child sexual abuse material (CSAM), including AI-generated images. The company is also adding ChatGPT warnings, temporary account suspensions and new safety guidance for developers.

The OpenAI CSAM image classifier is designed to identify previously unseen abusive imagery. Select organizations will receive free access through OpenAI’s API.

OpenAI CSAM Image Classifier Reduces Review Work

Developed with the National Center for Missing and Exploited Children (NCMEC), the classifier examines images for potential abuse indicators. It assigns scores to help human reviewers prioritize suspicious content.

OpenAI reported that early testing identified the same number of abusive images while flagging 48% fewer images overall. Flagged images were also twice as likely to contain abusive material.

The company is working with the Tech Coalition and ROOST to make the technology available to additional organizations.

These results come from early testing and do not establish performance across every platform or image type.

ChatGPT Adds Warnings and Account Suspensions

ChatGPT will display warnings when users attempt to find material involving child sexual abuse. The intervention stops the conversation and directs users toward specialized support services.

Some violations can trigger a temporary 24-hour suspension. Repeated offenses may lead to longer restrictions or permanent account closure.

OpenAI developed the interventions with researchers and child protection organizations, including Finland-based Protect Children and the Lucy Faithfull Foundation.

The approach draws on research evaluating whether warnings can discourage harmful searches. A University of Tasmania study found that 82% of users receiving a warning did not appear to repeat the prohibited search during the trial.

OpenAI Expands Developer and Law Enforcement Safety Tools

OpenAI is also publishing guidance to help developers prevent, detect and report abusive material. The recommendations cover content screening, account enforcement, human review and reporting obligations.

The company is providing law enforcement investigators with AI-assisted research tools, dedicated plugins and training resources. Program administrators will manage access to these tools.

The new protections build onOpenAI’ss existing image moderation systems. However, automated detection cannot eliminate every risk, making human review and specialist intervention important parts of the process.

Shivangi Yadav

Shivangi Yadav is a technology writer at TechAmerica.ai, covering artificial intelligence, startups, digital platforms, consumer technology, mobility, and emerging technologies. Her reporting follows major developments across the global technology industry… All stories by Shivangi

Related articles

More on OpenAI, ChatGPT, Child Safety.

See More - All Apps stories

Newsletters

The day's tech news, in five minutes

Written by the reporters who cover each beat. No sponsored stories, and you can unsubscribe in one click.

Send me

By subscribing you agree to our Terms and Privacy policy.

Thank you

Your newsletter subscription has been received. Thank you for subscribing to TechAmerica.ai.