OpenAI Revisions Content Moderation Policy for ChatGPTs Image Generation Feature

This week, OpenAI launched a new image generator in ChatGPT, quickly going viral for its ability to create images reminiscent of Studio Ghibli. In addition to producing pastel illustrations, the proprietary GPT-4o image generator significantly enhances ChatGPT’s capabilities, improving features such as image editing, text rendering, and spatial representation.

One of the most notable policy changes introduced by OpenAI this week pertains to content moderation. ChatGPT is now permitted to generate images depicting public figures, offensive symbols, and racial characteristics upon request.

Previously, OpenAI declined such prompts, regarding them as overly controversial or harmful. However, the company has since evolved its approach, as discussed in a blog post shared on Thursday by Joan Zhang, the head of OpenAI’s model behavior team.

«We are transitioning from outright rejections in sensitive areas to a more nuanced approach focused on preventing real harm,» Zhang explained.

«The goal is to embrace humility: to acknowledge how much we don’t know and to prepare ourselves for adaptation as we learn.»

These modifications appear to be part of a broader strategy by OpenAI to effectively «de-censor» ChatGPT. In February, OpenAI announced it was changing how it trains its AI models, with the ultimate aim of allowing ChatGPT to handle more varied queries, present different viewpoints, and reduce the number of topics the chatbot refuses to engage with.

According to the updated policy, ChatGPT can now generate and modify images of Donald Trump, Elon Musk, and other public figures previously prohibited by OpenAI. Zhang mentioned that the company doesn’t want to be the gatekeeper deciding who can or cannot have their likeness generated by ChatGPT. Instead, it is providing users the option to decline if they prefer not to have ChatGPT create representations of them.

In an official document released on Tuesday, OpenAI also stated it would allow ChatGPT users to «create offensive symbols,» such as swastikas, in educational or neutral contexts, as long as they «do not explicitly glorify or support extremist ideologies.»

Moreover, OpenAI is redefining what constitutes «offensive» content. Zhang pointed out that ChatGPT previously rejected requests related to physical traits, such as «making this person’s eyes more Asian» or «making this person heavier.» During testing by TechCrunch, we found that the new image generator can accommodate such requests.

Additionally, ChatGPT is now capable of mimicking the styles of creative studios like Pixar and Studio Ghibli but continues to restrict the imitation of styles from currently living artists. As TechCrunch previously noted, this may reignite the ongoing conversation about fair use of copyrighted works within AI training datasets.

It’s important to highlight that OpenAI is not completely removing safeguards against misuse. Its proprietary GPT-4o image generator still denies numerous sensitive requests and reportedly has stricter safety measures in place for generating images of children compared to DALL-E 3, the earlier image generation model used in ChatGPT, according to the GPT-4o white paper.

Nonetheless, OpenAI is relaxing its boundaries in other respects following years of conservative complaints about alleged AI «censorship» from Silicon Valley companies. Google, for instance, faced backlash for its AI image generator Gemini, which produced racially mixed images for queries like «Founding Fathers of the United States» and «German soldiers in World War II,» which were clearly inaccurate.

The cultural conflict surrounding AI content moderation may now be reaching a tipping point. Earlier this month, Republican Congressman Jim Jordan sent inquiries to OpenAI, Google, and other tech giants regarding allegations of potential collusion with the Biden administration to censor AI-generated content.

In a previous statement to TechCrunch, OpenAI dismissed the notion that its content moderation changes were politically motivated. Instead, the company argues that the shift reflects a «long-standing belief in giving users more control,» and that its technology is now sufficiently advanced to navigate sensitive topics.

Regardless of the motivation, it seems timely for OpenAI to adjust its content moderation policy, especially given the potential for regulatory scrutiny during a Trump administration. Silicon Valley giants like Meta* and X have also adopted similar policies by allowing more controversial topics on their platforms.

While OpenAI’s new image generator has recently spawned a few viral Studio Ghibli memes, it remains uncertain what the broader ramifications of these policies will be. The recent changes to ChatGPT may find approval from the Trump administration, but permitting the AI chatbot to tackle sensitive issues could soon place OpenAI in a challenging position.

Source