Evidence-linked feature engineering for heart failure

A multi-agent pipeline automates heart-failure EHR feature engineering while keeping every feature tied to evidence.

• Research org: Unspecified in arXiv abstract
• Core data: 0.963 held-out AUROC for HFrEF phenotyping
• Breakthrough: Evidence-linked, rubric-grounded multi-agent feature pipeline

Clinical feature engineering is one of those tasks that looks simple from the outside and eats huge amounts of time in practice. This paper argues that the pain is especially sharp in heart failure, where useful signals are scattered across EHR tables and need to be assembled with disease-specific reasoning, not just generic extraction.

The authors propose a system called the Nimblemind Multi-Agent System, or nMAS, to automate that work while preserving traceability. For engineers, the interesting part is not just that it generates features, but that it tries to make those features auditable: each one is linked back to evidence, checked against a rubric, and reviewed by a restricted LLM.

What problem this paper is trying to fix

The paper starts from a very practical bottleneck: EHR feature engineering can take up 39-45% of data scientists’ workload. That is a big tax on any clinical ML pipeline, especially when the target problem is heart failure, which the abstract says affects an estimated 6.7 million U.S. adults.

Why is heart failure harder than a typical tabular prediction task? Because the useful signals are fragmented. The modeler has to combine information from multiple EHR source tables, then map those data points onto clinical reasoning that follows disease guidelines. In other words, this is not just a data wrangling problem; it is also a reasoning and provenance problem.

The abstract says existing rule-based systems and LLM-based approaches only solve part of the job. Rule-based pipelines can be brittle and hard to maintain. LLM-based approaches can automate more of the process, but they often lack strong evidence traceability. That gap is what nMAS is trying to close.

How the method works in plain English

nMAS is described as an evidence-linked, rubric-grounded pipeline for automated heart-failure feature engineering. The key idea is to make feature generation more structured than a free-form LLM prompt, while still using automation to scale across messy EHR data.

The system was evaluated on 500 dummy patient records pulled from nine EHR source tables. From that data, nMAS generated 132 structured features and 70 aggregated features scored by a rubric. The abstract does not spell out every internal agent role, but the naming suggests a multi-agent workflow rather than a single model call.

Three properties are emphasized in the evaluation: structural integrity, rubric compliance, and provenance. That matters because a feature engineering pipeline is only useful in production if you can tell where each feature came from, whether it matches the intended clinical rule, and whether the output is internally consistent.

The paper also says the generated features were audited by a restricted LLM. That is a notable design choice: instead of trusting one model to do everything, the pipeline adds a second pass for review. For developers, that is a familiar pattern from software systems—generate, validate, then audit.

What the paper actually shows

The strongest result is that adding the aggregated features improved downstream phenotyping performance. For HFrEF, held-out AUROC rose from 0.895 to 0.963. For HFpEF, it increased from 0.870 to 0.910. Those are concrete gains, and they suggest the engineered features were not just syntactically valid but clinically useful for the task the authors studied.

The paper also reports an independent LLM-based rubric assessment of evidence support and methodological soundness. The features scored 81.5% of the maximum possible points. That is not a perfect score, but it does show the system can produce outputs that an external evaluator sees as reasonably well supported and methodologically sound.

There are also some important limits in what the abstract gives us. It does not provide benchmark numbers for runtime, cost, or human annotation effort saved. It also does not compare nMAS against a full range of alternative feature engineering systems in the abstract, so you should not read the AUROC gains as a universal win across all settings.

Most importantly, the evaluation was limited to a single-institution cohort, and the authors explicitly say external validation is still needed. That means the results are promising, but they are not yet a proof that the same pipeline will transfer cleanly to other hospitals, coding practices, or EHR schemas.

Why engineers should care

If you work on clinical ML, this paper points toward a more production-friendly way to do feature engineering: one that treats provenance as a first-class requirement instead of an afterthought. In healthcare, being able to explain why a feature exists is often as important as the feature itself.

The broader lesson is that LLMs do not have to be used as unconstrained generators. This paper uses them inside a pipeline with structural checks, rubric scoring, and audit steps. That is a useful pattern for any domain where correctness, traceability, and maintenance matter more than raw output volume.

There is also a workflow implication. If the approach holds up beyond one institution, it could reduce the manual burden on teams that currently spend a large share of their time turning EHR tables into model-ready variables. Even without full automation, a system that drafts features and preserves evidence links could make clinical experimentation much faster.

What is still open

The abstract leaves several practical questions unanswered. We do not know how much human oversight is still required, how robust the pipeline is to noisier real-world records, or how the system behaves when clinical guidelines conflict with incomplete data.

We also do not know how much of the reported performance gain comes from the aggregated features versus the broader pipeline design. That distinction matters if you are trying to decide whether to adopt a similar architecture or simply borrow a subset of its feature logic.

Even with those caveats, the paper is a useful signal. It shows that automated feature engineering for complex EHR data can be made more auditable without giving up downstream utility. For teams building healthcare ML systems, that combination is exactly what makes a method worth watching.