Leanstral 1.5 proves open-source math models are now useful

100% on miniF2F shows open-source Lean models are now useful for formal proof work.

Mistral’s Leanstral 1.5 is a serious open-source proof model, and the benchmark results show it has crossed from novelty into utility.

The benchmark gains are not cosmetic

On miniF2F, Leanstral 1.5 reaches 100 percent, which matters because miniF2F spans formal math problems from high school level to math olympiad difficulty. That is not a marketing-friendly score on an easy toy set; it is a clean signal that the model can reliably navigate structured proof tasks in Lean 4.

The PutnamBench result is even harder to dismiss. Out of 672 Putnam problems, the model solves 587, and Mistral says it leads the open-source field on PutnamBench, FATE-H, and FATE-X. Only Aleph Prover, a closed model, beats it on PutnamBench, which means open models are now uncomfortably close to the best proprietary systems in a domain that rewards rigor over fluency.

Formal math is a better test than chat benchmarks

General-purpose AI benchmarks often blur the line between pattern matching and genuine reasoning. Formal math in Lean does not forgive that slippage. A proof either compiles or it does not, so a model that performs well here is being judged by a stricter standard than most of the industry’s favorite leaderboards.

That is why the FATE-H and FATE-X results matter. These benchmarks cover master’s and doctoral-level algebra, including group theory and ring theory, and Leanstral 1.5 posts top open-source results of 87 percent and 34 percent. The specific value here is not just that the model is strong in one niche, but that it can handle a family of tasks where symbolic precision is the whole point.

The code bug findings are the real product signal

The most important detail in the release is not the benchmark table. In a hands-on test, the model scanned 57 open-source repositories and found five previously unknown bugs, including an overflow bug in the Rust library varinteger. That is the kind of result that matters to engineers because it turns abstract proof skill into a concrete security and reliability payoff.

This is where the model stops being an academic curiosity. A system that can inspect code, surface real defects, and do so across multiple repositories has immediate value in review workflows, especially for teams that already care about correctness in Rust, infrastructure code, or cryptographic tooling. It does not replace human review, but it does shorten the path to finding bugs that static checks and manual inspection miss.

The counter-argument

The strongest objection is that benchmark dominance does not equal broad usefulness. Lean 4 is a specialized environment, and formal proof tasks are narrower than everyday software engineering. A model that shines in theorem proving can still struggle with ambiguous requirements, messy codebases, and the social work of deciding what a system should do in the first place.

That objection is valid, and it sets a real boundary. But it does not weaken the case for Leanstral 1.5, because the model has already demonstrated value in the one place where correctness is measurable: it found actual bugs in real repositories. If a model can prove things and catch defects in code, then its specialization is an advantage, not a limitation.

What to do with this

Engineers should treat Leanstral 1.5 as a verification copilot, not a general chatbot: use it for proof sketches, invariant checking, and bug hunting in high-stakes code paths. PMs should pilot it where correctness has a direct cost, such as compiler tooling, finance, security, and infrastructure. Founders should see the larger lesson clearly: open-source models are now good enough to compete in domains where trust depends on formal proof, and that changes where the next durable AI products will be built.