GEAR cuts copying in long-context reasoning

GEAR rewards grounding in key evidence to reduce copying and improve long-context reasoning.

• Research org: Unspecified in arXiv abstract
• Core data: Up to +4.6 average points
• Breakthrough: Adds grounding reward and distractor penalty to RL

What happens when a model is given a long prompt and asked to reason step by step? According to this paper, one common failure is that it starts copying text from the input instead of actually solving the task. That matters because long-context systems are increasingly expected to do more than retrieve snippets; they need to use evidence, ignore noise, and produce useful reasoning traces.

The authors argue that this copying problem is not just a cosmetic issue. It gets worse as context length grows, and it is tied to a deeper grounding failure: models that fail to focus on the relevant evidence are much more likely to answer incorrectly. For engineers building long-context assistants, that is a practical warning sign. Bigger windows do not automatically mean better reasoning if the model is still latching onto irrelevant text.

What problem this paper is trying to fix

The paper focuses on a failure mode called repetitive copying. In long-context reasoning, models often echo large chunks of the prompt inside their reasoning traces. That can make the output look detailed, but it does not necessarily mean the model is thinking well. The authors say this behavior is widespread across frontier long-context LLMs and becomes more severe as the input gets longer.

To dig into why this happens, the paper separates prompts into two parts: task-relevant key evidence and irrelevant distractor context. That split lets the authors test whether the model is actually grounding its reasoning in the right material or just copying indiscriminately. Their conclusion is blunt: insufficient grounding appears to be the root cause.

For developers, this is a useful framing. If a model’s chain-of-thought or reasoning-style output is bloated with copied text, the issue may not be “too much context” in the abstract. It may be that the training signal is not teaching the model how to prioritize evidence under noise.

How GEAR works in plain English

The method is called GEAR, short for Grounding Evidence-Aware Reward. It is a reward shaping approach for reinforcement learning. Instead of optimizing only for answer accuracy, GEAR adds two extra signals: a grounding reward for overlap with key evidence, and a distractor penalty for overlap with irrelevant context.

That design is straightforward but important. The model is not just being told “get the answer right.” It is also being nudged toward using the right parts of the prompt and away from leaning on irrelevant text. In other words, the training objective tries to make the model’s reasoning trace more evidence-aware, not merely more confident.

The paper also includes an automated pipeline that builds evidence-annotated training data from arbitrary documents. That matters because natural-language long-context data is messy and not always hand-labeled with evidence spans. The pipeline is what makes GEAR usable beyond toy examples or narrowly curated datasets.

What the paper actually shows

The abstract reports consistent improvements of up to +4.6 average points over standard RL with accuracy-based rewards. The gains are larger at longer contexts, which is exactly where the repetitive-copying problem is most visible. The paper also says GEAR reduces repetitive copying and thinking length.

Those results suggest the method is doing more than gaming a benchmark with a better reward. It appears to change how the model uses context during reasoning. The abstract does not provide the full benchmark list or per-dataset scores, so those details are not available here. It also does not give exact numbers for the reduction in copying or reasoning length, only that both go down.

Still, the pattern is clear enough to matter. If a long-context model improves when you reward evidence use directly, that is a sign that accuracy-only RL may be leaving performance on the table. It also suggests that some long-context failures are not about missing information, but about the model failing to select the right information.

Why developers should care

If you are building retrieval-heavy assistants, document analyzers, or agents that work over long prompts, this paper points to a concrete training issue: a model can have access to the right evidence and still not use it well. That is a different problem from retrieval quality. It is a reasoning and grounding problem, and this paper treats it as such.

GEAR’s reward design is also a reminder that evaluation and training need to line up. If your system is judged only on final answer correctness, you may miss whether it is learning to rely on relevant evidence or just copying surrounding text. For applications where trace quality matters, that distinction can affect trust, debugging, and failure analysis.

There is also a practical implementation lesson in the evidence-annotation pipeline. Long-context training often depends on data curation, and this paper suggests a way to turn arbitrary documents into training material with evidence labels. The abstract does not spell out the full pipeline mechanics, so the exact robustness and cost of that automation are still open questions.

Limitations and open questions

The abstract gives encouraging results, but it does not claim a universal fix. It reports improvements across multiple model scales and benchmarks, yet it does not show that GEAR eliminates copying entirely. It also does not provide benchmark names, detailed ablations, or the full cost of collecting evidence annotations.

Another open question is how well the approach transfers outside the tested settings. The paper argues that grounding in relevant evidence remains indispensable as long-context evaluation moves from simple retrieval toward complex reasoning. That is plausible, but the abstract alone does not tell us how GEAR behaves on very different domains, prompt styles, or reasoning formats.

Even with those caveats, the paper makes a strong case that long-context performance is not just about fitting more tokens into the window. It is about teaching the model what to trust inside that window. For engineers, that is the part worth watching: better long-context systems may depend less on more context and more on better grounding.

Bottom line

GEAR is a reward-shaping approach that tries to make long-context reasoning less copy-heavy and more evidence-driven. The paper’s core message is simple: if you want models to reason over long inputs, you need to reward them for using the right evidence, not just for landing on the right answer.

• Repetitive copying is widespread in long-context LLM reasoning and worsens with longer inputs.
• GEAR adds evidence reward and distractor penalty to standard RL.
• The paper reports up to +4.6 average points, with larger gains at longer contexts.