Partition, Prompt, Aggregate: Testing LLM Self-Consistency

The paper shows that LLMs often violate basic probabilistic consistency when aggregating subpopulation estimates.

• Research org: Unspecified in arXiv abstract
• Core data: No benchmark numbers in abstract
• Breakthrough: Binary-tree partitioning with prompt-and-aggregate consistency checks

Most developers treat an LLM’s answer as a single estimate and move on. This paper asks a sharper question: if you break a population into smaller pieces, ask the model about each piece, and then combine those answers, do you get back the same population-level estimate you started with?

Partition, Prompt, Aggregate: Statistical Self-Consistency in Language Models argues that this should happen if in-context learning really behaves like conditional inference. The authors test that idea directly and find that current frontier models often fail basic consistency checks.

What problem this paper is trying to fix

The paper targets a subtle but important assumption behind many LLM workflows: that a prompt gives the model a context, and the output can be read as an estimate of a conditional distribution. If that were reliable, then simple probability rules should hold. One of the most basic is the law of total probability, which says that if you split a population into valid subgroups, then the weighted subgroup estimates should reconstruct the overall population estimate.

That matters because a lot of practical use cases depend on aggregation. Engineers ask models to estimate user preferences, classify groups, summarize survey-like inputs, or reason over personas and segments. If the model’s answers change in inconsistent ways depending on how the question is framed, then the output is not just noisy; it is structurally unreliable.

The authors frame this as a test of statistical self-consistency. Instead of asking whether the model is “right” in the usual benchmark sense, they ask whether its answers obey a basic identity that should hold across different partitions of the same population.

How the method works in plain English

The evaluation setup uses binary trees as a scaffold. The population is recursively partitioned into smaller and smaller subpopulations, creating increasingly fine-grained splits. For each subpopulation, the model is prompted with a verbalized description of that group and asked for an estimate in context.

Those subgroup estimates are then aggregated back up to the population level. The key comparison is between this reconstructed estimate and the model’s direct answer when asked about the whole population. If the model were perfectly self-consistent under the conditional-inference view, those two should line up.

Using a binary tree is a practical choice because it turns a fuzzy question into a structured test. It lets the authors compare consistency across different partition granularities and different tree structures, rather than relying on a single hand-picked split.

The paper also studies persona prompting in depth. That matters because persona-style prompts are common in applied LLM work, especially when people want answers for a segment, role, or demographic group. The authors look at whether the model’s subgroup knowledge can be surfaced more reliably through those prompts, and whether aggregation recovers something closer to the human reference data.

What the paper actually shows

The headline result is straightforward: across problem domains and frontier models, the paper finds widespread violations of these basic consistency properties. In other words, the model’s direct population-level estimate is often not what you get when you combine its subgroup estimates according to the law of total probability.

The paper also identifies a pattern the authors call the macro fallacy. In their persona-prompting study, estimates reconstructed from more fine-grained subpopulation responses are often better aligned with human reference data than direct population-level estimates. That is an interesting asymmetry: the model appears to know something useful at the subgroup level, but it does not reliably propagate that knowledge into the aggregate answer.

The effect is described as persistent across variations in tree structure and estimation task. The authors also say it can be partially recovered through implicit prompting. The abstract does not provide benchmark numbers, so the paper’s contribution here is qualitative rather than a single score or leaderboard position.

That combination is the real story. The model is not simply failing everywhere; it is failing in a specific way that reveals a gap between local knowledge and global aggregation. For anyone building systems that rely on compositional reasoning over groups, that is a more actionable failure mode than a generic accuracy drop.

Why developers should care

If you are using LLMs for segmentation, survey analysis, policy-style summarization, or anything that requires combining subgroup judgments into a whole, this paper is a warning sign. A model can sound confident at both the subgroup and aggregate level while still violating a basic probabilistic identity.

That means prompt design is not just about eliciting better wording. It can change whether the model exposes useful structure at all. The paper’s finding that implicit prompting can partially recover the effect suggests that the way you ask for the answer may determine whether the model surfaces granular knowledge or collapses it into a weaker aggregate estimate.

For engineers, the practical takeaway is that self-consistency checks may be a useful diagnostic layer. If a workflow depends on population estimates, you may want to compare direct answers against reconstructed answers from partitions. The paper suggests that disagreement is not random noise; it may reveal a systematic modeling gap.

Limitations and open questions

The abstract leaves some important details open. It does not list benchmark numbers, specific datasets, or the exact frontier models used, so you should not read this as a claim about one particular score or deployment setting. It is a methodology paper about consistency behavior, not a product evaluation.

It is also not claiming that aggregated answers are always better. The macro fallacy result says reconstructed estimates are often better aligned with human reference data in the persona study, but that does not prove the same will hold for every domain, every partition, or every prompt style.

Another open question is how to operationalize statistical self-consistency in production. The paper argues that it is an unsaturated, reference-free criterion for evaluating LLMs, which is useful because it does not require ground-truth labels for every case. But turning that into a robust test suite or a training objective would require more work than the abstract provides.

What to take away

This paper pushes LLM evaluation beyond “did the model answer correctly?” and into “does the model behave like a coherent probabilistic estimator?” That is a useful lens for anyone building systems where answers are meant to compose across groups, personas, or partitions.

The core message is not that LLMs know nothing. It is that they may know relevant subgroup facts without reliably combining them into consistent aggregate estimates. For developers, that is exactly the kind of failure that can hide inside polished outputs until a downstream decision depends on it.

• LLMs can violate basic total-probability consistency across partitions.
• Fine-grained persona prompts may surface knowledge that direct aggregate prompts miss.
• Self-consistency is a practical, reference-free diagnostic for grouped estimation tasks.