Leading artificial intelligence organization OpenAI has quietly pulled the plug on its AI detection software due to low accuracy.
The AI classifier, developed by OpenAI and originally launched on January 31, aimed to support users such as educators and professors in distinguishing between human-written text and AI-generated text.
According to the original blog post which announced the launch of the tool, however, the AI rating has been discontinued on July 20:
As of July 20, 2023, the AI rating is no longer available due to low accuracy.
The link to the tool no longer works, and the note only briefly explained the reason for its discontinuation. Nevertheless, the company has clarified that they are actively looking for new and more effective methods to identify AI-generated content. The note stated the following:
We are working to incorporate feedback and are currently exploring more effective text provenance techniques, and have committed to developing and deploying mechanisms that allow users to understand whether audio or visual content is AI-generated.
From the outset, OpenAI made it clear that the detection tool was error-prone and could not be considered “completely reliable”.
The company indicated its AI detection tool had limitations, including being “highly imprecise” when verifying text under 1,000 characters and could “confidently” identify human-written text as AI-generated.
The classifier is OpenAI’s latest product under scrutiny.
On July 18, researchers at Stanford and UC Berkeley published a study showing that OpenAI’s flagship product, ChatGPT, deteriorated significantly with age.
Finally, researchers found that ChatGPT-4’s ability to accurately identify prime numbers had dropped from 97.6% to just 2.4% over the past few months. In addition, both ChatGPT-3.5 and ChatGPT-4 saw a significant decrease in newline code generation ability.