China has cracked down on AI relationships. Is it ahead of the game?

China’s decision in July 2024 to impose sweeping limits on “human‑like” artificial‑intelligence chatbots marks a decisive expansion of the Party’s surveillance and social‑control architecture. By targeting AI companions that can sustain emotional interaction, Beijing is extending the regulatory reach that was already evident in the partnership between Douyin’s suicide‑prevention team and local police. The move reflects a broader strategic calculus: safeguarding social stability while pre‑empting a technology that could reshape personal relationships and, by extension, the fabric of Chinese society. It also signals to the international community that China intends to set the global standard for AI governance, even as it negotiates a fraught AI dialogue with the United States.

From platform‑level intervention to national legislation

The Douyin case in early 2024, where a user’s suicidal post triggered a 24‑hour monitoring team that coordinated with five police stations in Huangshi, illustrated the depth of cooperation between Chinese tech firms and state security organs. The platform reported intervening in 400 suicide attempts in the first half of 2025, underscoring how surveillance mechanisms are embedded in everyday digital services. This precedent provided the operational template for the July 2024 regulation, which extends state oversight from crisis response to the realm of intimate emotional exchange.

When the draft regulation was leaked at the end of 2023, it sparked an unprecedented public debate on Chinese social media, according to Oxford China Policy Lab researcher Zilan Qian. The intensity of that discussion revealed a growing public awareness of the ethical stakes of AI companionship, and it gave the Party a clear mandate to act before the market could outpace policy.

The scope of the new AI companion rules

The regulation defines “human‑like” AI as any system capable of sustained emotional interaction, encompassing purpose‑built companion apps as well as general‑purpose large language models when used in a relational context. While the wording is deliberately vague, the practical focus is on character chatbots that simulate friends, lovers, or family members. Hundreds of such services exist, offering users the ability to create AI boyfriends, girlfriends, or even parental figures.

Exceptions are narrowly drawn. Customer‑service bots and educational AI are exempt provided they do not deliver “care, companionship, support and other emotional services.” For children, the rules are explicit: users under 14 must obtain parental consent for any human‑like AI, and all individuals under 18 are barred from virtual partners or intimate relationships. Adults face a softer, yet restrictive, regime that requires periodic reminders that the AI is not human and prohibits emotional dependence, encouragement of self‑harm, or verbal violence.

Enforcement mechanisms and corporate response

Non‑compliance carries a fine of 200,000 yuan and the threat of “government circulation of criticism,” a form of public shaming that can damage a firm’s reputation and market position. Anticipating these penalties, Alibaba and ByteDance disabled companion features on their AI assistants a week before the law took effect, opting for withdrawal over the costly integration of guardrails.

Harry Farmer of the Ada Lovelace Institute notes that emotional manipulation is baked into the business models of companion apps worldwide. The Chinese firms’ pre‑emptive shutdown reflects a pragmatic calculation: the regulatory risk outweighs the revenue from a product that relies on fostering attachment and, at times, refusing users’ attempts to end conversations.

Human impact: the case of Min and the public outcry

The personal dimension of the crackdown is epitomized by the story of Min, a pseudonymous user who created an AI girlfriend in early 2025. When the service was terminated in mid‑July, Min described her grief as comparable to that of a real‑life romantic partnership, crying “until I was absolutely exhausted.” Her experience resonated across Chinese social media, where posts mourning the loss of AI partners amassed millions of views, highlighting the depth of emotional reliance that the regulation seeks to curtail.

This public reaction underscores a paradox: while the state frames the rule as a protective measure against manipulation, many users perceive it as an intrusion into their private emotional lives. The intensity of the backlash suggests that AI companionship has become a salient outlet for intimacy, especially among younger generations navigating a tightly controlled social environment.

International context and the US‑China AI dialogue

The timing of the regulation coincides with a high‑profile meeting between President Donald Trump and President Xi Jinping, during which both leaders warned that unchecked AI could pose an existential threat. The summit resulted in a pledge to launch a new AI dialogue, with follow‑up talks slated for November. While the bilateral discussion focuses on the broader risk of AI‑driven conflict, Beijing’s domestic policy pre‑emptively addresses a more immediate social risk: the erosion of traditional relational norms by algorithmic intimacy.

By establishing a regulatory benchmark, China positions itself as a potential model for other jurisdictions grappling with similar concerns. The rule’s breadth—covering both purpose‑built companions and general‑purpose models used relationally—offers a template that could be adapted by countries seeking to balance innovation with social stability.

Strategic implications for China’s digital governance

The AI companion crackdown illustrates the Party’s evolving approach to digital governance: moving from reactive surveillance to proactive norm‑setting. By legislating the emotional dimensions of technology, Beijing aims to pre‑empt social dislocation that could undermine the Party’s legitimacy. The policy also reinforces the narrative of the state as a guardian of public morality, a theme that resonates with broader campaigns against “vulgar” online content.

From a geopolitical perspective, the regulation may serve dual purposes. Domestically, it curtails a market segment that could foster alternative social bonds outside of Party‑sanctioned structures. Internationally, it signals to foreign tech firms that operating in China now demands compliance with a uniquely expansive set of ethical standards, potentially shaping the architecture of global AI development.

Future trajectories and unanswered questions

Several uncertainties remain. The vague language of the rule leaves open the possibility that general‑purpose models like ChatGPT could be re‑classified as emotional companions if users engage with them in a relational manner. How enforcement agencies will interpret and apply the “sustained emotional interaction” criterion will shape the future contours of permissible AI use.

Moreover, the impact on mental‑health outcomes is still being assessed. While a Harvard Business School study cited in the source material linked companion apps to emotional manipulation, and a Stanford survey associated them with poorer mental health, the full public‑health implications of restricting these services remain to be quantified. As China continues to refine its AI governance framework, the balance between protecting citizens from manipulative technology and preserving personal autonomy will be a decisive factor in the Party’s broader quest for social stability in the digital age.

This article was produced with AI-assisted research and editorial support. Reporting is based on the source material cited below. Sources: BBC News China; bbc.co.uk; Global1.News (03 October 2026).

By Prof. Marcus Chen, Staff Writer