Authorities in Malaysia and Indonesia have moved to restrict nationwide access to an artificial intelligence chatbot linked to X, after raising alarms that the tool is being exploited to produce sexually explicit fake images of real people.
The action represents the first complete national ban of the AI system, underscoring growing regulatory unease over generative AI tools that officials say are evolving more quickly than safety controls and content moderation policies.
Governments Warn of Threats to Women and Minors
Regulators in Malaysia and Indonesia said the chatbot allows users to generate and alter images, but is increasingly being misused to turn ordinary photos into sexualised content without consent.
Officials stressed that the technology poses heightened dangers to women and children, noting how quickly pornographic deepfakes can be created and distributed at scale.
Malaysia Criticises X’s Reliance on User Reporting
Malaysia’s communications authority disclosed that it had earlier cautioned X about abuse linked to its AI features. However, regulators said the company’s response focused largely on post-by-post user reporting, rather than limiting the AI’s capacity to generate harmful material in the first place.
According to officials, once explicit fake images are produced, takedown mechanisms often fail to stop further circulation, allowing the damage to spread.
Indonesia Grounds Decision in Human Rights Protection
Indonesia’s digital affairs ministry said its decision was rooted in respect for human dignity and fundamental rights, arguing that AI-generated non-consensual sexual content erodes public trust and exposes vulnerable communities to harm.
The government has formally asked X to clarify how the chatbot is governed, moderated, and constrained, indicating that future access could hinge on stricter safeguards.
Ban Fits Indonesia’s Strict Online Content History
The move is consistent with Indonesia’s long-standing stance on digital content regulation. The country has previously blocked platforms including Pornhub and OnlyFans, and officials see AI-generated sexual imagery as a modern extension of the same challenge, enabled by more advanced tools.
International Scrutiny Begins to Mount
Attention around the chatbot is now spreading beyond Southeast Asia. In the United Kingdom, regulators are reviewing whether X is meeting online safety obligations, while political leaders have voiced concerns over the use of technology to fabricate explicit fake images.
The AI tool is associated with Elon Musk, whose companies are facing growing scrutiny worldwide over AI governance, content moderation, and platform responsibility.
Victims Say Reporting Can Amplify Harm
For individuals affected by AI-altered images, the fallout has been personal and distressing. Some users say that reporting explicit content did little to curb its spread. In several cases, asking others to flag the posts only increased visibility, drawing more attention to the fabricated images and compounding the harm
A Signal of Tougher AI Oversight Ahead
The growing backlash is pushing X to reconsider how its AI products operate, as governments increasingly treat AI systems as regulated digital services rather than experimental tools.
Regulators argue that the issue goes beyond one chatbot, exposing gaps in online safety and consent laws and accelerating calls for clearer global rules around AI-generated content.
With more countries watching closely, the actions taken by Malaysia and Indonesia may mark an early sign of stricter global regulation of generative AI technologies.
