Turing Agent turns formal verification into an agent pipeline: 57–62% faithful formalization on TCS benchmarks, plus three previously unresolved proofs
Synopsis
AI systems now generate mathematical proofs faster than anyone can check them, and a Lean statement can compile while quietly changing the claim it was meant to capture. Turing Agent splits the job into a theorem-formalization agent and a proof-formalization agent, coordinated by structured decomposition, parallel verification, an experience pool and selective semantic review. On the 143 problems of Formal TCS Bench it reaches 57.34% and 61.54% semantic success with GLM-5.3 and DeepSeek-V4-Pro, at compilation rates of 97.20% and 99.30%, and above 82% overall pass@1 on the 248-theorem RLMEval subset.
Interpretation
Formal verification of research mathematics is split between two levels of agents: a theorem-formalization agent first turns an informal claim and its definitions into a Lean statement, and a proof-formalization agent then builds a complete proof against it. Earlier automated theorem provers mostly searched for proofs from statements that were already formalized, whereas a research paper brings its own definitions, conventions, assumptions and dependencies that must be translated faithfully before proof search can begin. On the 143 problems of Formal TCS Bench, Turing Agent reaches semantic success rates of 57.34% with GLM-5.3 and 61.54% with DeepSeek-V4-Pro, exceeding Claude Code, the strongest baseline, by 16.78 and 32.17 percentage points, with compilation rates of 97.20% and 99.30%.
Proof construction is organized as a directed acyclic graph: a parent goal is split into child claims, a decomposition is accepted only after Lean confirms the children imply the parent, and independent subgoals can be handed to several proof agents in parallel. The decomposition check establishes only that the child claims imply their parent, not that they match the informal argument, so a misread definition or assumption can misdirect the rest of the proof. On the 248 theorems drawn from the PFR and TestingLowerBounds projects, Turing Agent with GLM-5.3 obtains the highest score among the evaluated methods, 2.42 percentage points above Claude Code; with DeepSeek-V4-Pro it reaches 82.66%, 8.06 points above Codex CLI and 4.03 points below Claude Code's 86.69%.
The agent can depart from the original argument, choosing a different mathematical construction that makes the remaining proof obligations easier to discharge. Reproducing the original proof step by step also inherits the author's construction choices, even when another construction satisfies the same existential conditions. Among the 722 AI-generated mathematical research manuscripts OpenAI released in October 2026, Turing Agent built on DeepSeek completed formal verification of three TCS problems that had remained unresolved, one of which required about a hundred formally verified auxiliary claims.
Perspective
Evaluation is confined to theoretical computer science, using the 143 problems of Formal TCS Bench and 248 theorems from the PFR and TestingLowerBounds projects in RLMEval, in the benchmark's easy mode, which makes project-specific auxiliary lemmas available during proof construction; semantic faithfulness is judged by independent semantic review, performed on Formal TCS Bench with GPT-6.1 Sol.
A wide gap between compilation and semantic success shows that Lean accepting a statement does not guarantee the original meaning is preserved; on RLMEval, Turing Agent with DeepSeek-V4-Pro remains 4.03 percentage points below Claude Code; the agent also generates and verifies auxiliary lemmas that the final proof does not use, and planning which dependencies are essential to the main theorem remains an open direction.
