RevengeBench tests reverse-engineering game policies

RevengeBench shows LLMs can reconstruct hidden game policies from behavior traces and improve with custom probes.

• Research org: Unspecified in arXiv abstract
• Core data: 75 policies
• Breakthrough: Learner designs opponent probes to recover executable policy code

RevengeBench: Reverse Engineering Code-Space Policies from Behavioral Experiments asks a practical question that shows up everywhere in AI systems: if you can only watch an agent act, how much of its hidden decision logic can you recover? The paper turns that idea into a benchmark for game-playing policies, where the learner does not get direct access to the target code and instead has to infer it from behavior.

That matters because a lot of real-world AI work depends on understanding opaque policies after the fact. If you can reconstruct a policy from traces, you get a path toward opponent modeling, interpretability, and better strategy design. The paper’s twist is that it also lets the learner run controlled behavioral experiments, not just passively observe, which makes the inverse problem more informative.

What problem this paper is trying to fix

The core problem is an inverse one: take observed actions and infer the hidden program that produced them. That is a classic challenge in science, but here it is translated into code-space for game agents. Instead of trying to guess a model from static logs alone, the paper asks whether targeted interventions can make the reconstruction problem more tractable.

In plain English, this is about reverse engineering an agent’s strategy from the outside. For developers, that is relevant any time a system is too complex, too opaque, or too expensive to inspect directly. Think of it less as “reading the model weights” and more as “learning the policy by poking it and watching what changes.”

The benchmark is built around CodeClash tournament trajectories and includes 75 LLM-generated, Elo-calibrated policies across five game environments. The abstract does not name the environments, so we should not guess which games are included. What it does make clear is that the policies are not random toy examples; they are tournament-derived and calibrated, which gives the benchmark a more realistic feel than a simple synthetic task.

How the method works in plain English

RevengeBench gives the learner a hidden target policy playing against sampled opponents. The learner then designs behavioral probes by creating custom opponent policies intended to elicit informative responses. After that, it submits an executable hypothesis: in other words, a piece of code that is supposed to behave like the hidden policy.

This is important because the output is not just a label or a score. The system is trying to recover an actual runnable program. That makes the task closer to debugging, imitation, and adversarial testing than to standard classification.

The paper evaluates reconstructions with continuous action-distance metrics, which means the comparison is not just right or wrong. The abstract does not provide the exact metric formula, so the safest reading is that the recovered code is judged by how closely its actions match the target policy over time. That gives a more nuanced signal than exact-match accuracy would.

There is also a second validation step: the recovered code is tested in downstream player-versus-player tournaments. That matters because a policy can look close on paper yet fail to carry over into competitive play. Here, the authors check whether the reconstructed code contains signal that actually helps in later matches.

What the paper actually shows

The headline result is that recovery quality varies widely across twelve frontier LLMs. The abstract reports that they close between 34% and 72% of the initial distance. That range is the main concrete performance signal in the abstract, and it shows the task is solvable to a meaningful degree but far from uniform across models.

Another key result is that reconstructed policies provide measurable competitive advantage in downstream tournaments. The abstract especially calls out weaker models, which seem to benefit most because they otherwise struggle to design effective counter-strategies. That suggests the recovered code is not just a neat artifact; it can actually improve gameplay performance.

At the same time, the paper is careful about what it claims. It does not say the models fully recover the hidden policy, and it does not give benchmark numbers beyond the 34 to 72% distance-closed range in the abstract. It also does not provide per-environment results here, so readers should not assume the same behavior across all five game settings.

• 75 hidden policies form the benchmark
• 12 frontier LLMs are evaluated
• 34 to 72% of initial distance is closed

Why developers should care

If you build agents, games, or any system with hidden decision logic, this paper points to a concrete workflow: observe behavior, design probes, reconstruct code, then test the reconstruction in a competitive setting. That is a useful mental model for debugging adversarial systems, auditing opaque policies, and building better opponent models.

It also hints at a broader engineering idea: active observation beats passive logging when the goal is to infer latent mechanisms. The paper’s setup is basically a controlled experiment loop for AI policies. That is a pattern developers can recognize from testing distributed systems, fuzzing APIs, or probing model behavior with adversarial inputs.

There are still open questions. The abstract does not tell us how robust the recovered policies are outside tournament play, how much probe design matters relative to model capability, or how expensive the behavioral search is. It also does not explain whether the reconstructed code is semantically faithful, or simply behaviorally close under the benchmark’s metric.

Even with those limits, RevengeBench is a useful step because it turns a fuzzy interpretability idea into something executable and measurable. For practitioners, that means the question is no longer just “can we explain the agent?” but “can we recover enough of its policy to predict and exploit its behavior?”

What to take away

The paper’s main contribution is a benchmark for reverse engineering hidden policies from behavior, with an active probing loop built in. That makes it a bridge between interpretability, opponent modeling, and behavioral science-inspired experimentation.

For engineers, the practical lesson is straightforward: if you want to understand an opaque policy, don’t only watch it. Interrogate it. RevengeBench suggests that controlled probes can materially improve how much of the underlying decision program you can recover.