[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"tag-turboquant":3},{"tag":4,"articles":11,"peer_article_count":193},{"id":5,"name":6,"slug":7,"article_count":8,"description_zh":9,"description_en":10},"d8bd452a-7bae-471e-99b0-b081e34f288d","TurboQuant","turboquant",13,"TurboQuant 聚焦 LLM 推論時最吃記憶體的 KV cache，透過低位元量化與向量量化降低佔用，進而壓低伺服器成本並提升吞吐量；同時也牽涉到 QJL、PolarQuant、benchmark 公平性與引用爭議。","TurboQuant targets the KV-cache bottleneck in LLM inference, using low-bit and vector quantization to reduce memory pressure and server cost. The topic also connects to QJL, PolarQuant, benchmark fairness, and citation disputes.",[12,21,28,36,43,50,57,64,72,80,87,94,102,109,116,123,130,137,144,151,158,165,172,179,186],{"id":13,"slug":14,"title":15,"summary":16,"category":17,"image_url":18,"cover_image":18,"language":19,"created_at":20},"59866fce-b78e-4d8a-ad3e-7ef7d607979e","turboquant-cuts-llm-memory-use-without-retraining-en","TurboQuant cuts LLM memory use without retraining","5 ways TurboQuant shrinks KV cache memory and speeds LLM inference, with near-lossless results around 3–4 bits on retrieval benchmarks.","industry","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782710265164-q297.png","en","2026-06-29T05:17:22.810166+00:00",{"id":22,"slug":23,"title":24,"summary":25,"category":17,"image_url":26,"cover_image":26,"language":19,"created_at":27},"cc87056f-b2e8-4ef0-966c-bf82ccffbb54","atomicbot-llama-cpp-fork-throughput-gains-en","AtomicBot’s llama.cpp fork boosts throughput on two fronts","4 ways AtomicBot’s llama.cpp fork speeds up Gemma 4 and Qwen 3.6, with matrix-bench gains up to 30-50% on the right setup.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782332277361-4xh4.png","2026-06-24T20:17:29.158539+00:00",{"id":29,"slug":30,"title":31,"summary":32,"category":33,"image_url":34,"cover_image":34,"language":19,"created_at":35},"405de39d-cfc5-43bf-b47b-ff9ce7be96a9","turboquant-does-not-hurt-search-quality-equal-bytes-en","TurboQuant does not hurt search quality at equal byte budgets","TurboQuant cuts vector memory by about 20× without meaningful search-quality loss when compared at equal bytes.","research","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781857967113-2xax.png","2026-06-19T08:32:22.235692+00:00",{"id":37,"slug":38,"title":39,"summary":40,"category":17,"image_url":41,"cover_image":41,"language":19,"created_at":42},"f49d58f8-0bd5-4442-9bdb-b0ca12e97986","turbovec-cuts-10m-vector-ram-to-4gb-en","TurboVec cuts 10M-vector RAM to 4GB","TurboVec compresses 10M vectors from 31GB to 4GB and removes training from vector search.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781528566106-frfj.png","2026-06-15T13:02:23.344662+00:00",{"id":44,"slug":45,"title":46,"summary":47,"category":17,"image_url":48,"cover_image":48,"language":19,"created_at":49},"093f7c46-be7c-4b62-be00-73808a61e0a0","turboquant-amd-gpus-kv-cache-latency-en","TurboQuant on AMD GPUs cuts KV-cache latency","TurboQuant on AMD GPUs improves long-context LLM serving with up to 3.6x speedup and far lower KV-cache pressure.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781299067778-3pzd.png","2026-06-12T21:17:26.07+00:00",{"id":51,"slug":52,"title":53,"summary":54,"category":17,"image_url":55,"cover_image":55,"language":19,"created_at":56},"0ac121b9-de23-42b9-94f7-fac9ea703e18","turboquant-makes-long-context-ai-cheaper-en","TurboQuant makes long-context AI much cheaper","4 ways TurboQuant’s 100x KV cache cut could lower long-context AI costs, ease GPU needs, and change model serving.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1781272983524-0j31.png","2026-06-12T14:02:27.64087+00:00",{"id":58,"slug":59,"title":60,"summary":61,"category":33,"image_url":62,"cover_image":62,"language":19,"created_at":63},"9f0c9505-6d75-411c-ba46-2382e8f295a5","turboquant-cuts-kv-cache-memory-6x-google-tests-en","TurboQuant cuts KV cache memory 6x in Google tests","Google Research says TurboQuant compresses KV caches by over 4x, with up to 6x less memory and no loss on long-context tests.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780906679116-fqdo.png","2026-06-08T08:17:22.276769+00:00",{"id":65,"slug":66,"title":67,"summary":68,"category":69,"image_url":70,"cover_image":70,"language":19,"created_at":71},"0117641d-93d6-40f1-8b9e-158b8240493a","tether-turboquant-cuts-ai-memory-use-5x-en","Tether’s TurboQuant cuts AI memory use 5x","Tether released TurboQuant in QVAC SDK 0.12.0, claiming up to 5x lower AI memory use for local sessions on laptops and phones.","blockchain","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780543069267-cwa3.png","2026-06-04T03:17:20.409795+00:00",{"id":73,"slug":74,"title":75,"summary":76,"category":77,"image_url":78,"cover_image":78,"language":19,"created_at":79},"1247e920-56ea-4e12-9d8c-5a4a7d4df9dd","why-tether-is-right-to-push-local-ai-memory-into-everyday-de-en","Why Tether Is Right to Push Local AI Memory Into Everyday Devices","Tether’s TurboQuant matters because it makes long-context AI practical on local devices, not just in data centers.","tools","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780542172839-ie86.png","2026-06-04T03:02:19.993669+00:00",{"id":81,"slug":82,"title":83,"summary":84,"category":17,"image_url":85,"cover_image":85,"language":19,"created_at":86},"034b5552-6ad2-4a5f-960c-870f30d7be22","5-turboquant-lessons-for-vector-search-teams-en","5 TurboQuant lessons for vector search teams","5 takeaways on Qdrant TurboQuant: how rotation changes compression, where recall holds up, and when safer quantizers fit better.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780157892244-w7me.png","2026-05-30T16:17:39.721708+00:00",{"id":88,"slug":89,"title":90,"summary":91,"category":17,"image_url":92,"cover_image":92,"language":19,"created_at":93},"0a1bbd8f-5c42-4205-8a46-fc99c9e51218","memory-stocks-face-new-ai-reality-check-en","Memory Stocks Face a New AI Reality Check","Memory chip stocks are soaring on AI demand, but investors warn the cycle can turn fast if supply rises or model efficiency improves.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779957185729-mgzy.png","2026-05-28T08:32:33.647923+00:00",{"id":95,"slug":96,"title":97,"summary":98,"category":99,"image_url":100,"cover_image":100,"language":19,"created_at":101},"e71cb6f6-c753-4b14-9e37-19634bdad1d8","why-verkor-turboquant-silicon-ip-matters-en","Why Verkor’s TurboQuant silicon IP matters more than the headline says","Verkor’s TurboQuant accelerator is a real step for LLM inference, but the bigger story is how quickly algorithm ideas are becoming silicon IP.","ai-agent","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779896872842-2hm8.png","2026-05-27T15:47:25.880442+00:00",{"id":103,"slug":104,"title":105,"summary":106,"category":77,"image_url":107,"cover_image":107,"language":19,"created_at":108},"8a164bd6-6f92-47a6-87fb-72a6371aae17","why-llama-cpp-should-treat-turboquant-as-default-en","Why llama.cpp should treat TurboQuant as the new default path","TurboQuant is the right direction for llama.cpp because asymmetric KV compression cuts memory without breaking compatibility.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779481556833-a9v3.png","2026-05-22T20:25:23.12744+00:00",{"id":110,"slug":111,"title":112,"summary":113,"category":77,"image_url":114,"cover_image":114,"language":19,"created_at":115},"49dbda12-d94e-4e41-99d0-200d57eb97a9","turboquant-vllm-kv-cache-3bit-storage-en","TurboQuant turns vLLM KV cache into 3-bit storage","I break down TurboQuant’s vLLM cache compression and give you a copy-ready setup for 3-bit KV cache and fallback paths.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779286502445-214g.png","2026-05-20T14:14:37.831446+00:00",{"id":117,"slug":118,"title":119,"summary":120,"category":77,"image_url":121,"cover_image":121,"language":19,"created_at":122},"cbaeb6db-c465-4659-b35b-640435c673bf","why-kv-cache-compression-will-decide-edge-ai-inference-en","Why KV-cache compression will decide edge AI inference","TurboQuant-style KV-cache compression is the real bottleneck-breaker for edge AI inference.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779285828871-4n8z.png","2026-05-20T14:03:20.811149+00:00",{"id":124,"slug":125,"title":126,"summary":127,"category":17,"image_url":128,"cover_image":128,"language":19,"created_at":129},"bfbd028b-4704-4de5-8f54-55625836952f","5-kv-cache-takeaways-for-llamacpp-users-en","5 KV cache takeaways for llama.cpp users","5 takeaways from TurboQuant: under-3-bit KV cache compression, memory savings, and the tradeoffs llama.cpp users should watch.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1779285258553-domr.png","2026-05-20T13:53:43.522918+00:00",{"id":131,"slug":132,"title":133,"summary":134,"category":33,"image_url":135,"cover_image":135,"language":19,"created_at":136},"94994abd-e24d-4fd1-b941-942d03d19acf","turboquant-seo-shift-small-sites-en","TurboQuant and the SEO Shift for Small Sites","TurboQuant is a rumored Google search system that could widen the pool of pages ranked, giving smaller sites a better shot.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778840455122-jfce.png","2026-05-15T10:20:28.134545+00:00",{"id":138,"slug":139,"title":140,"summary":141,"category":33,"image_url":142,"cover_image":142,"language":19,"created_at":143},"670a7f69-911f-41e8-a18b-7d3491253a19","turboquant-vllm-comparison-fp8-kv-cache-en","TurboQuant vs FP8: vLLM’s first broad test","vLLM found FP8 KV-cache quantization beats TurboQuant on speed, while TurboQuant’s strongest variants hurt accuracy.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778839858405-b5ao.png","2026-05-15T10:10:37.219158+00:00",{"id":145,"slug":146,"title":147,"summary":148,"category":33,"image_url":149,"cover_image":149,"language":19,"created_at":150},"a259bf3b-e800-46fa-8550-605b5b8f4115","why-turboquant-changes-kv-cache-debate-en","Why TurboQuant changes the KV cache debate","TurboQuant makes KV cache compression a theoretical win, not just an engineering trick.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778016643980-zx6u.png","2026-05-05T21:30:24.349733+00:00",{"id":152,"slug":153,"title":154,"summary":155,"category":33,"image_url":156,"cover_image":156,"language":19,"created_at":157},"d7b529f2-02b7-4d5b-bf82-490aa5fe8362","turboquant-eden-citation-fight-en","TurboQuant, EDEN, and the citation fight","TurboQuant’s KV-cache quantization claims are under fire: EDEN authors say the paper reuses older ideas, weaker scales, and shaky benchmarks.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1777467061610-ug4x.png","2026-04-29T12:50:47.131528+00:00",{"id":159,"slug":160,"title":161,"summary":162,"category":33,"image_url":163,"cover_image":163,"language":19,"created_at":164},"6c80feee-7f7d-4518-bd06-3c04b8c46054","turboquant-cuts-memory-use-without-accuracy-loss-en","TurboQuant cuts memory use 6x without accuracy loss","Google Research’s TurboQuant claims 6x less memory and 8x faster inference with no accuracy loss, jolting AI inference economics.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775161136573-e0cb.png","2026-04-02T20:18:39.999171+00:00",{"id":166,"slug":167,"title":168,"summary":169,"category":33,"image_url":170,"cover_image":170,"language":19,"created_at":171},"fdb997e1-6691-46c5-bb2d-e1ca3f730c25","turboquant-google-paper-explained-en","TurboQuant Explained: Why Google’s New Paper Matters","Google’s TurboQuant paper targets KV cache bottlenecks with lower-bit quantization, aiming to cut LLM memory use and inference costs.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775160958409-7jj5.png","2026-04-02T20:15:40.601225+00:00",{"id":173,"slug":174,"title":175,"summary":176,"category":33,"image_url":177,"cover_image":177,"language":19,"created_at":178},"6fd1f021-a7ca-4fa7-9aae-6ca84b22dc6c","googles-turboquant-cuts-llm-memory-costs-en","Google's TurboQuant Cuts LLM Memory Costs","Google says TurboQuant uses QJL and PolarQuant to shrink vector-quantization memory and speed up LLM inference by up to 8x.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775160776347-4esa.png","2026-04-02T20:12:32.387326+00:00",{"id":180,"slug":181,"title":182,"summary":183,"category":77,"image_url":184,"cover_image":184,"language":19,"created_at":185},"b2de41c7-a1bf-414d-b843-97a3d0d1283b","turboquant-fast-cold-starts-rust-gpu-en","TurboQuant, Fast Cold Starts, and Rust on GPUs","TurboQuant cuts KV cache use 4.6x, GPU state restoration slashes cold starts, and Rust is moving deeper into CUDA work.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775146375773-h6or.png","2026-04-02T16:12:38.879237+00:00",{"id":187,"slug":188,"title":189,"summary":190,"category":33,"image_url":191,"cover_image":191,"language":19,"created_at":192},"d4867ede-353b-4812-aac7-aebe28ef3613","turboquant-wont-fix-memory-crunch-en","TurboQuant Won’t Fix the Memory Crunch","Google’s TurboQuant can cut KV-cache memory use 6x, but longer contexts may keep DRAM and NAND demand climbing.","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1775132152400-1kew.png","2026-04-02T12:15:32.095995+00:00",24]