The push for a moratorium aims to halt the development of systems that could theoretically evolve beyond human control or oversight.
The urgency behind this ban stems from the potential for self-improving systems to rapidly discover and exploit security vulnerabilities, threatening global digital infrastructure.
Critics of the technology argue that once a model enters a cycle of autonomous enhancement, the pace of its development could outstrip the ability of engineers to implement safety guardrails.
In a competitive landscape where companies race to achieve the most powerful AI, such a ban would fundamentally shift the trajectory of capital expenditure—the money spent on physical assets like data centers and specialized chips—toward safety-oriented research instead of raw performance gains.
If enacted, the regulation would likely target the specific technical mechanisms that allow models to modify their core algorithms without direct human intervention.
This shift would primarily affect major cloud computing providers and AI developers who currently prioritize scaling model size and efficiency.
The next phase of this debate involves determining how to enforce such a ban globally, as the decentralized nature of AI research makes it difficult to monitor the internal code-writing activities of private firms.