[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-co-lmlm-continuous-query-limited-memory-models-en":3,"article-related-co-lmlm-continuous-query-limited-memory-models-en":30,"series-research-5d1770de-d17d-455a-a593-301ee0974526":73},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"5d1770de-d17d-455a-a593-301ee0974526","co-lmlm-continuous-query-limited-memory-models-en","Co-LMLM lets LMs query knowledge continuously","\u003Cp data-speakable=\"summary\">Co-LMLM replaces fixed KB queries with continuous vector queries and improves factual precision without baking facts into weights.\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cstrong>Research org\u003C\u002Fstrong>: Unspecified in arXiv abstract\u003C\u002Fli>\u003Cli>\u003Cstrong>Core data\u003C\u002Fstrong>: 360M scale\u003C\u002Fli>\u003Cli>\u003Cstrong>Breakthrough\u003C\u002Fstrong>: Continuous-query KB with continuous keys and textual knowledge values\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.07707\">Co-LMLM: Continuous-Query Limited Memory Language Models\u003C\u002Fa> argues for a different way to give language models factual knowledge: keep facts outside the weights, but make retrieval more flexible than the older limited-memory setups. Instead of forcing the model to use a relational knowledge base and fixed queries, the paper proposes a continuous-query design that can fetch human-readable knowledge during generation.\u003C\u002Fp>\u003Cp>For engineers, the appeal is straightforward: you get a model that can still answer with attributable retrieved text, while moving away from brittle, schema-heavy KB access. The paper also claims the approach works beyond Wikipedia-only setups by adding an annotation pipeline for tagging free-form factual spans in arbitrary text.\u003C\u002Fp>\u003Ch2>What problem this paper is trying to fix\u003C\u002Fh2>\u003Cp>Classic \u003Ca href=\"\u002Ftag\u002Fllms\">LLMs\u003C\u002Fa> store a lot of knowledge in their parameters, which makes that knowledge hard to control, update, or attribute. Limited memory language models try to address that by externalizing factual knowledge during pretraining into a knowledge base, then fetching it during generation as needed.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783578777528-m5x6.png\" alt=\"Co-LMLM lets LMs query knowledge continuously\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>The catch is that prior LMLMs rely on relational KBs and queries. That makes them less flexible about what they can store and how they can retrieve it. In practice, that means the model is tied to a more rigid representation of facts than many real-world text sources provide.\u003C\u002Fp>\u003Cp>This paper is trying to loosen that constraint. The core idea is to keep the limited-memory benefit, but replace the old KB\u002Fquery interface with something that looks more like continuous retrieval over text-oriented knowledge.\u003C\u002Fp>\u003Ch2>How Co-LMLM works in plain English\u003C\u002Fh2>\u003Cp>Co-LMLM stands for continuous-query limited memory language model. The key change is that the knowledge base pairs continuous keys with textual knowledge values. That is a significant departure from the relational KB setup used in earlier work.\u003C\u002Fp>\u003Cp>In this design, the model generates vector queries at minimal cost. Those queries are continuous rather than discrete, which gives the model more flexibility in how it searches for relevant facts. The retrieved knowledge is still textual, so the output can remain readable and attributable instead of becoming a black-box embedding dump.\u003C\u002Fp>\u003Cp>That combination matters. The model is not just learning to memorize facts in weights, and it is not forced into a narrow symbolic query language either. It sits somewhere in between: learned retrieval, but with text as the final knowledge carrier.\u003C\u002Fp>\u003Cp>The paper also pairs the model with an annotation pipeline that tags free-form factual spans in arbitrary text. That removes a major restriction in prior work, which was limited to Wikipedia. For anyone building retrieval-aware training pipelines, that is the part that changes the data story, not just the model story.\u003C\u002Fp>\u003Ch2>What the paper actually shows\u003C\u002Fh2>\u003Cp>The abstract says CO-LMLM was evaluated across pretraining on Wikipedia and FineWeb-Edu, and at multiple model scales. The paper reports that it outperforms prior LMLMs and vanilla LLMs in both perplexity and factual precision.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783578776122-i1pp.png\" alt=\"Co-LMLM lets LMs query knowledge continuously\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>There are a few concrete comparisons worth noting. At 360M scale, the model reaches lower perplexity than models pretrained on 40x more data. The abstract also says its SimpleQA-verified performance is in line with gpt-4o-mini and higher than \u003Ca href=\"\u002Ftag\u002Fclaude\">Claude\u003C\u002Fa> Sonnet 4.5.\u003C\u002Fp>\u003Cp>Those are strong claims, but the abstract does not provide the full \u003Ca href=\"\u002Ftag\u002Fbenchmark\">benchmark\u003C\u002Fa> table, exact perplexity values, or the full evaluation protocol here. So the safe reading is that the direction of the results is clear, while the paper itself is where you would need to go for the full numbers and experimental details.\u003C\u002Fp>\u003Cul>\u003Cli>It beats prior LMLMs and vanilla LLMs on perplexity and factual precision.\u003C\u002Fli>\u003Cli>At 360M scale, it reportedly uses far less data for lower perplexity than some larger-pretrained models.\u003C\u002Fli>\u003Cli>SimpleQA-verified performance is claimed to match gpt-4o-mini and exceed Claude Sonnet 4.5.\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>Why developers should care\u003C\u002Fh2>\u003Cp>If you are building systems that need factual answers with some level of control and attribution, this is relevant. The paper is basically saying that you do not have to choose between a fully parametric model that hides facts in weights and a rigid symbolic KB pipeline that is hard to generalize.\u003C\u002Fp>\u003Cp>For product teams, the practical upside is better knowledge control. If facts live in an external memory, they can be updated, audited, or swapped without retraining the whole model in the same way you would with a standard \u003Ca href=\"\u002Ftag\u002Fllm\">LLM\u003C\u002Fa>. The abstract explicitly frames knowledge control as a capability that remains beyond conventional LLMs.\u003C\u002Fp>\u003Cp>For research and infra teams, the more interesting angle is the retrieval interface. Continuous queries may be easier to integrate into learned systems than discrete KB lookups, especially when the source data is messy text instead of neatly structured triples. The free-form span tagging pipeline suggests the method is aiming at broader corpora, not just curated encyclopedia entries.\u003C\u002Fp>\u003Ch2>Limitations and open questions\u003C\u002Fh2>\u003Cp>The abstract gives a promising direction, but it does not answer every deployment question. We do not get latency numbers, memory footprint details, retrieval cost breakdowns, or evidence about how well the system behaves under noisy or adversarial knowledge sources.\u003C\u002Fp>\u003Cp>It is also not clear from the abstract how much of the gain comes from the continuous-query mechanism versus the annotation pipeline, the choice of training data, or scale effects. Those are the kinds of details that matter if you want to reproduce the result or ship it in a production stack.\u003C\u002Fp>\u003Cp>Another open question is how robust the knowledge-attribution story remains when the model has to synthesize across many retrieved spans. The paper claims human-readable and attributable retrieved knowledge, but the abstract does not spell out how attribution is measured or enforced.\u003C\u002Fp>\u003Cp>Still, the main takeaway is useful: if you want a language model to stay grounded in external facts, Co-LMLM explores a more flexible middle ground than older limited-memory approaches. It is a reminder that retrieval design is not just an add-on; it can be the core architecture choice that determines how factual, controllable, and adaptable the system becomes.\u003C\u002Fp>","Co-LMLM replaces fixed KB queries with continuous vector queries and improves factual precision without baking facts into weights.","arxiv.org","https:\u002F\u002Farxiv.org\u002Fabs\u002F2607.07707",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783578777528-m5x6.png","research","en","3e8fbc00-9a1f-4e79-bfc2-ca933bf09eb9",[17,18,19,20,21],"retrieval-augmented language models","knowledge bases","factual precision","continuous queries","limited memory models",[23,24,25],"Co-LMLM swaps fixed relational queries for continuous vector queries over textual knowledge.","The paper reports better perplexity and factual precision than prior LMLMs and vanilla LLMs.","The abstract claims 360M-scale results and broader data coverage beyond Wikipedia.",0,"2026-07-09T06:32:31.072331+00:00","2026-07-09T06:32:31.061+00:00","5c705f39-eb55-408e-b067-9f47e22e3e13",{"tags":31,"relatedLang":32,"relatedPosts":36},[],{"id":15,"slug":33,"title":34,"language":35},"co-lmlm-continuous-query-limited-memory-models-zh","Co-LMLM 讓 LLM 持續查知識","zh",[37,43,49,55,61,67],{"id":38,"slug":39,"title":40,"cover_image":41,"image_url":41,"created_at":42,"category":13},"1d12c9dd-2014-4589-b546-49d7aaafd56c","analysis-driven-transformer-linearization-en","How to Linearize Transformers Without Losing Quality","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783580573147-nkcg.png","2026-07-09T07:02:26.831218+00:00",{"id":44,"slug":45,"title":46,"cover_image":47,"image_url":47,"created_at":48,"category":13},"5f612e85-468e-477b-a91e-3daf23da2e6d","scireasoner-structure-property-reasoning-en","SciReasoner makes structure readable to AI","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783576972706-me8v.png","2026-07-09T06:02:31.433838+00:00",{"id":50,"slug":51,"title":52,"cover_image":53,"image_url":53,"created_at":54,"category":13},"73a9a547-9c78-4fb4-a98a-54b2b710a9fa","rethinking-indic-ai-cultural-heritage-en","Rethinking Indic AI Through Cultural Heritage","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783494191004-5kpc.png","2026-07-08T07:02:30.703105+00:00",{"id":56,"slug":57,"title":58,"cover_image":59,"image_url":59,"created_at":60,"category":13},"ee9ba749-2585-4803-9b57-7c89b247b350","graph-convolutional-attention-graph-denoising-en","Graph Convolutional Attention fixes graph denoising","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783492378204-wovv.png","2026-07-08T06:32:32.898741+00:00",{"id":62,"slug":63,"title":64,"cover_image":65,"image_url":65,"created_at":66,"category":13},"0c67eb96-af48-4f6e-a981-cf98643c3837","elsa3d-elastic-semantic-anchoring-3d-en","ELSA3D Makes 3D Models Reason in Scales","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783490577409-9rpj.png","2026-07-08T06:02:32.506604+00:00",{"id":68,"slug":69,"title":70,"cover_image":71,"image_url":71,"created_at":72,"category":13},"ed4393a9-437c-4266-b94c-b51ed66acc7b","leanstral-1-5-open-source-math-models-useful-en","Leanstral 1.5 proves open-source math models are now useful","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1783449164706-zcgm.png","2026-07-07T18:32:18.712641+00:00",[74,79,84,89,94,99,104,109,114,119],{"id":75,"slug":76,"title":77,"created_at":78},"a2715e72-1fe8-41b3-abb1-d0cf1f710189","ai-predictions-2026-big-changes-en","AI Predictions for 2026: Brace for Big Changes","2026-03-26T01:25:07.788356+00:00",{"id":80,"slug":81,"title":82,"created_at":83},"8404bd7b-4c2f-4109-9ec4-baf29d88af2b","ml-papers-of-the-week-github-research-desk-en","ML Papers of the Week Turns GitHub Into a Research Desk","2026-03-27T01:11:39.480259+00:00",{"id":85,"slug":86,"title":87,"created_at":88},"87897a94-8065-4464-a016-1f23e89e17cc","ai-ml-conferences-to-watch-in-2026-en","AI\u002FML Conferences to Watch in 2026","2026-03-27T01:51:54.184108+00:00",{"id":90,"slug":91,"title":92,"created_at":93},"6f1987cf-25f3-47a4-b3e6-db0997695be8","openclaw-agents-manipulated-self-sabotage-en","OpenClaw Agents Can Be Manipulated Into Failure","2026-03-28T03:03:18.899465+00:00",{"id":95,"slug":96,"title":97,"created_at":98},"a53571ad-735a-4178-9f93-cb09b699d99c","vega-driving-language-instructions-en","Vega: Driving with Natural Language Instructions","2026-03-28T14:54:04.698882+00:00",{"id":100,"slug":101,"title":102,"created_at":103},"a34581d6-f36e-46da-88bb-582fb3e7425c","personalizing-autonomous-driving-styles-en","Drive My Way: Personalizing Autonomous Driving Styles","2026-03-28T14:54:26.148181+00:00",{"id":105,"slug":106,"title":107,"created_at":108},"2bc1ad7f-26ce-4f02-9885-803b35fd229d","training-knowledge-bases-writeback-rag-en","Training Knowledge Bases with WriteBack-RAG","2026-03-28T14:54:45.643433+00:00",{"id":110,"slug":111,"title":112,"created_at":113},"71adc507-3c54-4605-bbe2-c966acd6187e","packforcing-long-video-generation-en","PackForcing: Efficient Long-Video Generation Method","2026-03-28T14:55:02.646943+00:00",{"id":115,"slug":116,"title":117,"created_at":118},"675942ef-b9ec-4c5f-a997-381250b6eacb","pixelsmile-facial-expression-editing-en","PixelSmile Framework Enhances Facial Expression Editing","2026-03-28T14:55:20.633463+00:00",{"id":120,"slug":121,"title":122,"created_at":123},"6954fa2b-8b66-4839-884b-e46f89fa1bc3","adaptive-block-scaled-data-types-en","IF4: Smarter 4-Bit Quantization That Adapts to Your Data","2026-03-31T06:00:36.65963+00:00"]