Anthropic Automates Formal Verification for Open-Source Software

The central tension in modern software development lies in the disparity between the speed of code generation and the sluggish pace of verification. While artificial intelligence models can now produce code at unprecedented rates, proving that this code is mathematically correct under all possible conditions has remained a prohibitively expensive undertaking. Anthropic has sought to resolve this imbalance by leveraging its Claude model to automate the formal verification process, effectively shifting the burden from human specialists to algorithmic logic.

Historically, formal verification was reserved for mission-critical systems due to its labor intensity. For instance, a team of experts required 20 person-years to formally verify the seL4 operating system kernel, a standard that made the technique impractical for general software. This high barrier to entry meant that only extreme cases, such as avionics software for aircraft, could justify the cost of rigorous mathematical proof.

In a recent initiative, Anthropic’s Frontier Red Team partnered with Theorem to alter this dynamic. Over a period of just a few weeks, Claude generated approximately 180 million lines of mathematical proof code. This effort focused on 123 open-source libraries drawn from Google’s OSS-Fuzz security project. Rather than creating new software, the model translated existing crash tests into mathematical problems and analyzed the compiled machine code to produce extensive proofs.

The process relies on an independent verification program called Rocq to automatically check the generated proofs. According to Anthropic, this workflow requires minimal human intervention, leading researchers to describe the operation as fully autonomous. However, it is noted that the underlying tools and formal foundations were still constructed by human engineers at Theorem. The automation has already yielded immediate results, with Claude identifying and fixing hundreds of bugs that conventional testing had missed.

Many of these defects were latent, surfacing only under specific conditions such as unusually large input files or particular sequences of memory access. In one notable instance, the team verified a widely used PDF reader and discovered as many security bugs in a single week as are typically reported for that software over an entire year. In another application, the model detected a logical error that had persisted undetected for more than a decade.

Despite these successes, Anthropic emphasizes that a formal verification does not equate to absolute error-free status. The proofs are valid only under strict constraints, including specific processor architectures, single-threaded execution, and input files smaller than 64 MB. Since real-world deployments often vary in hardware and configuration, these limitations remain significant for practical application.

The computational demands of this approach are also substantial, with individual proofs consuming hundreds of millions of tokens. While Anthropic has not disclosed the total project cost, the strategy aims to reduce overall development expenses by increasing trust in AI-generated code. A smaller test run indicated that these proofs are reusable, and newer models are becoming more efficient; Opus 5.5, for example, solves half of the proofs using roughly 40 percent fewer tokens than its predecessor.

Looking ahead, the project suggests a shift in the industry’s fundamental questions. The focus is moving from whether AI can program to whether it can guarantee the safety of its own output. Anthropic anticipates that formal verification with AI will soon become as routine as standard testing. Theorem’s roadmap already targets critical infrastructure components, including glibc, OpenSSL, and cURL, signaling a broader push toward integrating mathematical rigor into the core of software development.