Are There Any Good Ideas for AI Governance?

President Trump returned from his state visit to Beijing last month with few concrete deliverables, beyond the unveiling of a new formulation of the bilateral relationship as one of “constructive strategic stability” and high-level nods to Iran, rare earths, and agriculture purchases. But one area where there may be cause for cautious optimism is the renewed potential for U.S.-China engagement on AI risk.

In an interview on the sidelines of the summit, U.S. Treasury Secretary Scott Bessent observed that the “two AI superpowers are going to start talking” and would “set up a protocol” to determine “best practices for AI to make sure nonstate actors don’t get ahold of these models.”

When Trump entered office in 2025, his administration appeared opposed to restrictions constraining the development of frontier labs, citing the need to unleash the U.S. AI industry to compete with China. Yet after Anthropic announced in April 2026 that it had built Mythos, a model the company said would be so destabilizing for cybersecurity that it could not be publicly released, members of the administration began voicing concerns that a degree of regulation might be prudent. This culminated in an Executive Order that took steps toward imposing regulation in the form of 30-day voluntary pre-deployment oversight.

Now, in the aftermath of the Trump-Xi summit, the question remains: Can the U.S. and China meaningfully pursue dialogue that produces substantive outcomes to manage a general-purpose technology that promises advancements for human progress while posing catastrophic risks?

Critics argue that Beijing has had a long history of dissembling—in areas from military-to-military communication to arms control—and will use engagement as a performative cover to raise objections about export controls. That may be true. The sole round of AI talks in Geneva in 2024 failed to produce meaningful progress, and the Chinese delegation was led by Foreign Ministry bureaucrats rather than technical experts. Challenges of verification also cloud any potential agreement reached by the two sides.

On the other hand, since the Biden administration, frontier AI has evolved so rapidly that China has grown more attentive to risks posed by AI—and increasingly elevated AI safety on its policy agenda. Moreover, the U.S. and China have previously reached limited agreements on specific issues, such as a joint statement at the late-2024 Biden-Xi Lima Summit about the need to maintain human control over nuclear weapons. This demonstrates that some degree of positive progress is possible, even if challenges persist.

One domain of potential early engagement lies in biosecurity. Last week, CEOs of major AI firms released a letter sounding the alarm about AI’s potential to erode knowledge barriers to biological weapons. Since both countries have an interest in ensuring that frontier models do not enable a non-state actor to manufacture bioweapons, there is a small but genuine possibility that the two countries might come to an initial consensus to screen synthetic DNA or establish model safeguards. If Washington and Beijing can achieve even a limited agreement, that would mark an important first step toward managing a potent technology and preventing catastrophic outcomes neither country wants.

When I think about AI risk and governance, I picture a pyramid.

At the tip are frontier risks: agentic AI capabilities, AI-enabled biosecurity threats, and scenarios in which increasingly powerful models could make it easier for bad actors to develop dangerous biological agents or other harmful technologies. This is where much of the U.S.-China AI safety dialogue is understandably focused. These risks are urgent, high-stakes, and deserve serious attention.

The middle layer is the expanding universe of governance questions across the AI stack: data rules, privacy, user consent, copyright, liability, model transparency, and platform responsibility. As the technology moves at breakneck speed, legal and regulatory systems are still struggling to catch up.

The base of the pyramid is broader and messier: AI’s impact on human society. This is where labor displacement, social mobility, mental health, inequality, human relationships, and institutional reform all come in. Arguably, this bottom layer may prove the most consequential over the long run. Yet compared with frontier safety, it has seen far less concrete governance experimentation.

That is what makes a series of recent labor rulings in China especially interesting. The case that has received considerable media attention is a recent court ruling from Hangzhou, which involved a quality-control supervisor for a large language model. After a system upgrade, the company argued that AI could now perform much of his work, offered him a lower-paid position, and dismissed him when he refused. The court found the termination illegal and ordered compensation. In a similar Beijing case, an arbitration commission rejected a company’s argument that AI replacement constituted a “major change in objective circumstances.” AI adoption, it ruled, was a voluntary business decision, not an unforeseeable external shock.

Together, these cases offer a glimpse into China’s emerging legal infrastructure for AI and employment. The message seems to be that companies cannot simply fire employees because AI replacement is more cost-efficient. Chinese legal counsel to firms have told me they believe employers are increasingly expected to consider reassignment, retraining, and additional compensation if they want to show good-faith negotiation.

The economic stakes are substantial, particularly for China. The AI boom is widening the gap between high-tech strength and broader macroeconomic weakness. Advanced manufacturing, robotics, AI, and other capital- and tech-intensive sectors are gaining momentum and policy tailwinds, while labor-intensive traditional sectors are under strain. AI may help make the cake bigger, but it does not automatically improve the system for dividing it.

This is the deeper governance challenge of the AI era. In the language of factors of production, productivity gains accrue first to capital and technology, while adjustment costs fall disproportionately on labor—especially workers whose tasks are routine, standardized, and therefore more vulnerable to replacement by AI. Left unmanaged, AI could aggravate China’s structural problem of “strong supply and weak demand.” The youth job market is already fragile, and AI replacement is pressing hardest on white-collar, entry-level work—the very jobs that often serve as the first rung into the middle class.

The problem is neither uniquely Chinese nor entirely new. Technological disruption has always tested employment resilience. But AI’s sweeping scale and breakneck speed could make this disruption much harder to manage.

Chinese legal scholars have proposed requiring employment-impact assessments before major AI deployment. Companies whose AI systems may significantly affect jobs or working conditions could be required to submit labor-impact reports, consult with unions, design transition plans, and provide retraining and reemployment support. Economists have gone further, suggesting that tax incentives for AI investment be tied to employment-protection obligations, and that fiscal subsidies prioritize firms that raise salaries or retrain workers to use AI technology. Globally, there is also a growing debate about whether extraordinary AI-driven profits should be partly redirected through tax or other redistribution mechanisms to prevent excessive concentration of productivity gains.

The Communist Party has a clear stake in this debate, because it goes directly to social stability and, ultimately, political legitimacy. As AI moves deeper into the economy, Beijing will have to reconcile two imperatives that do not always sit comfortably together: the drive for innovation (Xi’s top economic priority), and the need for political stability (his top security priority). These court cases are worth watching because they show that this tension is already entering the legal system. They are sketching the early contours of a new policy frontier where technology, economics, society, and politics converge.

Several open questions about AI governance lack settled answers anywhere. But Chinese AI governance has a less-visible layer that is often overlooked by Western coverage: academic legal groups drafting comprehensive AI laws meant to feed into future legislation. Three of these drafts—the Scholars’ Suggestion Draft from the China University of Political Science and Law (CUPL), the Model Artificial Intelligence Law (MAIL) from the Chinese Academy of Social Sciences (CASS), and the Basic AI Law from Nanjing University School of Law (hereafter Nanjing)—address four key questions.

When AI causes harm, who pays? The drafts diverge on a question neither the EU nor the U.S. has settled. CUPL makes foundation-model providers fully liable alongside downstream actors when they knew or should have known of misuse (with open-source exemptions). MAIL reverses the burden of proof for providers and holds developers liable for specific failures, such as inadequate safety testing. And Nanjing allocates liability proportionally across developers, providers, deployers, and users by fault, control, and contribution to harm.

Can AI train on copyrighted works without permission? Litigation is still testing the extent of fair use in the U.S. and the Text and Data Mining (TDM) exception in the EU. CUPL’s position (refined in its 2025 Recommendations) is that training on copyrighted works is permissible if the works are obtained legally and use doesn’t unreasonably harm rights holders. MAIL goes further, calling for a statutory licensing regime that lets foundation-model developers use published works without authorization, albeit with remuneration and the option to opt out; open-source training gets full fair use without remuneration.

How should AI lab employees be protected for disclosing critical safety information? MAIL mandates whistleblower systems, with a penalty of minimum twelve months’ wages for retaliating against employees who report broad safety and legal concerns. This is more extensive than the EU AI Act, which extends protections to those reporting concrete AI Act violations. Meanwhile the U.S. is still debating how to extend whistleblower protections to AI workers.

How should agentic AI be regulated? Systems that act autonomously on behalf of users have barely been touched by Western statute, but Chinese academic legal scholars are actively engaging. MAIL requires agent providers to offer authorization revocation and operation termination functions and that users “exercise reasonable control” over their agents. CUPL’s 2025 Key Institutional Recommendations on AI Legislation, a follow-up to its 2024 draft law, mostly addresses data protection by requiring edge models and intelligent agents to keep personal information local, but they are broadening the discussion. Additionally, CUPL’s March 2026 Ten Research Topics on AI Rule of Law makes “agent behavioral boundaries, authorization mechanisms, and competition governance” its first priority research topic.

Whether these drafts become official Chinese regulation remains uncertain, but there is precedent. In 2019, Professor Zhang Xinbao at Renmin University led an expert suggestion draft of the Personal Information Protection Law (PIPL) and “participated in the entire legislative process” of the final law that was adopted in 2021; the enacted law bears some similarity to the draft. Western frontier-AI safety scholarship lives mostly on the open-access archive arXiv, and binding rules emerge slowly. Even short of a comprehensive AI law, the three drafts will likely shape narrower regulation in China. Thus, they deserve Western attention as potential signals of what’s to come in Chinese regulation and as a source of regulatory ideas other jurisdictions haven’t yet tried—after all, a governance race would be a good race to be in.

What I love about observing Silicon Valley and China side by side is that the two tech industries are topologically different, even when they look similar from a distance.

I should be honest: I don’t have a strong answer about whether good governance ideas or citizen tools will emerge from the noise. The two state governments have formally opened a dialogue—Guo Jiakun’s mid-May statement that China and the U.S. had agreed to set up a government-to-government dialogue on AI followed the Xi-Trump summit in Beijing—and there are regulatory developments in both countries worth tracking. But I’m not sure the answer to the question of who is giving citizens tools to shape AI is meaningful to anyone right now.

The deeper trend I see is moving in the opposite direction, which is concentration. Anthropic’s Mythos cybersecurity model remains largely obscure even to the most engaged analysts. The most recent rollout added 150 organizations to a small initial set that included open-source maintainers and the U.S. government, but the rollout is slow and the model is exceptionally expensive. The age in which the best AI models were widely accessible is ending. Expect more Mythos-like cases where powerful frontier models are released with deliberate obscurity and rationed availability.

So here we come to China. In my recent post, “Mandate of AI,” I argued that China’s AI industry still depends on Silicon Valley in nearly every dimension that matters: the taxonomic power to name reality (for example, Constitutional AI, vibe coding, jaggedness: all these terms come from the Valley); the very definition of what “frontier” means (after Anthropic defined coding capability as the key to Recursive Self-Improvement (RSI) AI, all of the Chinese labs started chasing coding capability); and, to some degree, source models from which Chinese labs distill capability. That dependency is widening, and the model capability gap is widening (when was the last time you heard someone praise DeepSeek’s V4 and actually use the model?)

Mythos is the kind of model that, by design, Chinese researchers will not be able to legally access. So frontier AI is concentrating into a smaller set of authorized hands within the United States, and Chinese AI, even as it produces dazzling open-weight releases and dominates deployment, remains intellectually and creatively passive at the actual frontier.

First thing’s first: If we’re considering who might be ahead in AI, we have to be explicit about what we mean. As I’ve recently argued, “artificial intelligence” means many different things: facial recognition, drug discovery, code generation, LLM chatbots. Various U.S. and Chinese actors have different strengths in different areas, and speaking of a single “AI race” can obscure this variegated reality. It can also skirt the question of whether a marginal lead on some metric makes any practical difference, or whether a leading company’s technological edge translates to the national interest.

The most genuine nation-vs.-nation racing dynamics are in largely unrealized military applications. Both governments are looking at LLM-enabled cybersecurity gains, various forms of autonomy in weapons systems, and other ways to leverage advanced models, and these efforts are likely to matter in both peacetime jockeying and any potential conflict. Civilian applications of already-existing LLM-based systems will be many, and researchers are only beginning to understand how their diffusion may affect national outcomes.

The conceptual elephant in the room is “artificial general intelligence” or “artificial superintelligence” (AGI or ASI), contentious and often vague ideas that broadly signal an unprecedented transformation of the relationship between technology and humanity. Some strategists postulate that such a transformation is coming and then speculate about which country might harness it first—or about who can be trusted to avoid catastrophic risks of such systems. In this futurecasting view, any concept of a national lead is fundamentally speculative. Many people honestly believe a great deal is at stake, but few are explicit about how uncertain and multifaceted this development story actually is.

What is certain is that the Chinese and U.S. governments have walked very different paths in regulating the development and use of generative AI. The Chinese government has introduced dozens of binding and nonbinding regulatory documents, while the U.S. government has generally relied on the light and uneven discipline of preexisting law and policy.

On the face of it, it’s a stark hands-on vs. hands-off contrast. Yet these approaches have something in common: Neither is a departure from how each country previously regulated internet technologies. China has gradually built a voluminous framework for cyberspace, focusing on controlling the information environment (for both political repression and genuine social ills) and regulating the market. Its generative AI regulations are thoughtfully structured but mostly stop short of anticipating medium- to long-term results of the technology’s use. The United States, on the other hand, has largely left things to the market and to lawsuits and regulatory agencies to address harms, with more targeted interventions mostly voluntary.

The world can learn from China’s efforts to regulate these technologies. Even when other governments have different goals, China may show which levers work and which don’t. The United States is home to many thinkers working hard on AI risks and governance—just not many doers in government so far. Neither country, however, has figured out how to predict the future.

Mixed reactions in China to Anthropic’s non-release of Claude Mythos Preview reveal some broader truths about the state and possible direction of frontier AI governance. Concern regarding the possible implications for China’s cyber defenses, and therefore national security, coexists with renewed urgency to develop comparative offensive capabilities domestically. In this adversarial environment defined by U.S.-China strategic competition, corporate power, and rapid technological change, both governments are being challenged to balance the development of strategic AI capabilities with the establishment of appropriate national and bilateral guardrails.

The first truth is that both frontier AI capabilities and related security know-how today are extremely concentrated. Anthropic’s choice of providing a select few with access to a limited version of Mythos Preview for “defensive security work” has sparked controversy. Even the EU cyber agency had not been invited to access Mythos until last week. Given China’s longstanding preoccupation with cyber and critical infrastructure security at home, it is not surprising that Chinese officials have also sought access—without success—into the powerful AI model. While Beijing’s official reaction to Mythos has been muted, the prospect of it or similar AI cyberweapons being used against China is likely to keep the leadership awake at night.

Beyond the technical and geopolitical fault lines lies a second, important truth. AI’s emerging abilities to independently identify, patch, and exploit software vulnerabilities usher in a new era of cyber insecurity. China’s cybersecurity contractors are already adapting to this reality, candidly acknowledging that vulnerabilities can never be fully patched, but also that Chinese actors lack tools comparable to Mythos that would allow them to respond in kind to disruptive AI-driven, agentic attacks. In fact, Chinese vendors, like 360 Security Technology, have been actively exploring the integration of advanced AI technology in vulnerability discovery and exploitation, aiming to close the gap with frontier U.S. labs. In the words of 360’s founder Zhou Hongyi, “winners and losers will be defined based on whether they can use agents and computing power to take the lead in building a new generation of offensive and defensive systems.”

This game-changing moment in cyber warfare invites us to consider a third element: The possibly catastrophic implications for citizens, companies, and countries around the world should compel Beijing and Washington to find at least some common ground. Indeed, a Chinese Foreign Ministry spokesperson confirmed in May that President Xi Jinping and President Donald Trump agreed to resume intergovernmental talks on AI oversight, confirming statements made by U.S. officials without disclosing more details. This is an important signal, but it remains to be seen whether any dialogue will succeed at galvanizing support for basic yet meaningful safeguard measures within two systems that are locked in a strategic rivalry.

As things stand, citizens worldwide have little room to shape these critical conversations on issues that affect their lives. Amid the AI “arms race,” which extends beyond the cybersecurity domain, knowledge and power increasingly rest in the hands of a few powerful technology firms. Meanwhile, the two governments with the most power to act are finding it more and more difficult to balance transparency and security, competition and cooperation, as well as technological progress and safety measures.