[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-turboquant-cuts-kv-cache-memory-6x-google-tests-zh":3,"article-related-turboquant-cuts-kv-cache-memory-6x-google-tests-zh":30,"series-research-6f25a29c-cbb8-4f53-9af7-1656b394333a":83},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"6f25a29c-cbb8-4f53-9af7-1656b394333a","turboquant-cuts-kv-cache-memory-6x-google-tests-zh","TurboQuant 在 Google 測試中省下 6x KV 快取","\u003Cp data-speakable=\"summary\">\u003Ca href=\"\u002Ftag\u002Fgoogle\">Google\u003C\u002Fa> Research 的 \u003Ca href=\"\u002Ftag\u002Fturboquant\">TurboQuant\u003C\u002Fa> 是 2025 年提出的向量量化方法，用來壓縮 LLM 的 \u003Ca href=\"\u002Ftag\u002Fkv-cache\">KV cache\u003C\u002Fa> 與 embedding。\u003C\u002Fp>\u003Cp>Google Research 發表 \u003Ca href=\"https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTurboQuant\" target=\"_blank\" rel=\"noopener\">TurboQuant\u003C\u002Fa>，主打在線向量量化，目標是把高維向量壓小，同時盡量保留結構。團隊表示，它在\u003Ca href=\"\u002Ftag\u002F長上下文\">長上下文\u003C\u002Fa> LLM \u003Ca href=\"\u002Fnews\u002Fopenclaw-6-yue-ce-shi-ban-xian-xiu-5-ge-fa-bu-feng-xian-zh\">測試\u003C\u002Fa>中，能維持接近全精度基準，壓縮率超過 4x。\u003C\u002Fp>\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>項目\u003C\u002Fth>\u003Cth>數值\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>提出年份\u003C\u002Ftd>\u003Ctd>2025\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>KV cache 記憶體減少\u003C\u002Ftd>\u003Ctd>至少 6x\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>H100 上 attention-logit 加速\u003C\u002Ftd>\u003Ctd>最高 8x\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>長上下文測試壓縮率\u003C\u002Ftd>\u003Ctd>超過 4x\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>KV-cache 品質門檻\u003C\u002Ftd>\u003Ctd>每通道 3.5 bits\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>上下文長度\u003C\u002Ftd>\u003Ctd>4,000 到 104,000 tokens\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003Ch2>發生了什麼\u003C\u002Fh2>\u003Cp>這篇方法論出自 Amir Zandieh、Majid Daliri、Majid Hadian 和 Vahab Mirrokni 合作的論文《TurboQuant: Online Vector Quantization with Near-optimal Distortion Rate》。它鎖定三個成本最高的場景：LLM 推理、KV cache 壓縮、以及近鄰搜尋。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780906682236-sqe2.png\" alt=\"TurboQuant 在 Google 測試中省下 6x KV 快取\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>TurboQuant 分成兩個模式。TurboQuant mse 以最小化均方誤差為目標，TurboQuant prod 則偏向無偏的內積估計。\u003C\u002Fp>\u003Cp>兩者都先做隨機旋轉，再做純量量化。prod 版本另外加上一個 1-bit 的 Quantized Johnson–Lindenstrauss 步驟，用來修正剩餘誤差。\u003C\u002Fp>\u003Cul>\u003Cli>mse 版本把旋轉後的每個座標存成純量碼本。\u003C\u002Fli>\u003Cli>prod 版本再加上 sign sketch 與 residual norm。\u003C\u002Fli>\u003Cli>論文顯示，bit 數提高時失真會下降，1 到 4 bits 的 MSE 例子約落在 0.36、0.117、0.03、0.009。\u003C\u002Fli>\u003Cli>Google Research 也把它放進 LongBench、Needle in a Haystack、ZeroSCROLLS、RULER 和 L-Eval 做測試。\u003C\u002Fli>\u003C\u002Ful>\u003Cp>這次的重點不是單純把向量壓小，而是盡量不讓模型行為走樣。對長上下文任務來說，這比單看壓縮率更重要。\u003C\u002Fp>\u003Ch2>為什麼重要\u003C\u002Fh2>\u003Cp>對做 LLM 服務的開發者來說，KV cache 常常就是\u003Ca href=\"\u002Fnews\u002Fmemdreamer-long-video-understanding-memory-retrieval-zh\">記憶\u003C\u002Fa>體瓶頸。Google 表示，TurboQuant 相較未量化的 32-bit keys，可把 KV cache footprint 至少壓到 6x，attention-logit 計算在 \u003Ca href=\"\u002Ftag\u002Fnvidia\">Nvidia\u003C\u002Fa> H100 上最高可快 8x。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780906680772-4x2w.png\" alt=\"TurboQuant 在 Google 測試中省下 6x KV 快取\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>這會直接影響部署成本與吞吐量。當上下文長度拉到 4,000 到 104,000 tokens，伺服器能否撐住更多並發請求，往往就取決於 cache 是否夠省。\u003C\u002Fp>\u003Cp>另一個實際差異在於，它是 online、data-oblivious 的方法，不必像不少舊量化方案那樣先做離線校準或訓練碼本。這讓它更容易塞進現有 serving stack，特別是長對話、檢索增強生成和向量搜尋。\u003C\u002Fp>\u003Cp>但這類結果還是要看模型、工作負載和硬體是否一致。Google 自己的測試看起來漂亮，真正進到 production，實作成本和跨場景穩定性才是門檻。\u003C\u002Fp>\u003Cp>如果 TurboQuant 能在更多模型上重現這種結果，KV cache 可能不再只是成本項目，而會\u003Ca href=\"\u002Fnews\u002Fopenclaw-small-business-ai-staff-zh\">變成\u003C\u002Fa>可被精細管理的資源。問題只剩一個：你的 serving pipeline，準備好吃下這種量化了嗎？\u003C\u002Fp>","Google Research 公布 TurboQuant，可把 KV cache 記憶體壓到至少 6x 以上，並在長上下文測試中維持接近全精度表現。","en.wikipedia.org","https:\u002F\u002Fen.wikipedia.org\u002Fwiki\u002FTurboQuant",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780906682236-sqe2.png","research","zh","9f0c9505-6d75-411c-ba46-2382e8f295a5",[17,18,19,20,21],"TurboQuant","Google Research","KV cache","vector quantization","LLM inference",[23,24,25],"TurboQuant 目標是壓縮 KV cache 與 embedding，並保留向量結構。","Google 測試顯示它可讓 KV cache 至少減少 6x，H100 上 attention-logit 最多快 8x。","它是 online、data-oblivious 方法，較容易整合進現有 LLM serving 與向量搜尋系統。",0,"2026-06-08T08:17:21.878314+00:00","2026-06-08T08:17:21.868+00:00","0c35a120-52fc-41fc-afa3-d404eb934158",{"tags":31,"relatedLang":42,"relatedPosts":46},[32,34,36,38,40],{"name":18,"slug":33},"google-research",{"name":19,"slug":35},"kv-cache",{"name":21,"slug":37},"llm-inference",{"name":17,"slug":39},"turboquant",{"name":20,"slug":41},"vector-quantization",{"id":15,"slug":43,"title":44,"language":45},"turboquant-cuts-kv-cache-memory-6x-google-tests-en","TurboQuant cuts KV cache memory 6x in Google tests","en",[47,53,59,65,71,77],{"id":48,"slug":49,"title":50,"cover_image":51,"image_url":51,"created_at":52,"category":13},"0e9f2d34-1873-4c6f-bdec-5d89fbaab037","memdreamer-long-video-understanding-memory-retrieval-zh","MemDreamer 用記憶拆解長影片","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780902187207-ilt0.png","2026-06-08T07:02:32.374349+00:00",{"id":54,"slug":55,"title":56,"cover_image":57,"image_url":57,"created_at":58,"category":13},"fdc06a5d-6b96-463d-bb9e-e7a0c1194ff5","agentopia-10-year-agent-society-simulation-zh","Agentopia：把代理社會拉長到10年","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780901284189-2q5j.png","2026-06-08T06:47:31.817211+00:00",{"id":60,"slug":61,"title":62,"cover_image":63,"image_url":63,"created_at":64,"category":13},"9f629b51-c1ad-4a83-beef-40059da1ab54","llms-stumble-counterintuitive-probability-zh","LLM 在反直覺機率題翻車","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780900377752-3uk6.png","2026-06-08T06:32:28.84056+00:00",{"id":66,"slug":67,"title":68,"cover_image":69,"image_url":69,"created_at":70,"category":13},"174a1d04-6330-4ed1-98d3-32a6199d2108","bento-webassembly-memory-compartments-zh","Bento 把 Wasm 記憶體切成隔間","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780811288729-y8qc.png","2026-06-07T05:47:45.523833+00:00",{"id":72,"slug":73,"title":74,"cover_image":75,"image_url":75,"created_at":76,"category":13},"33d21f7f-481d-43d9-9a1c-a2e5badcd84b","bis-stablecoin-usable-buffers-regulation-zh","BIS 把穩定幣規則變成可用緩衝","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780737502594-jk2b.png","2026-06-06T09:17:56.366473+00:00",{"id":78,"slug":79,"title":80,"cover_image":81,"image_url":81,"created_at":82,"category":13},"a4cb421e-464e-4933-9e1c-6371d3cd1503","prevent-catastrophic-forgetting-llm-fine-tuning-zh","如何防止 LLM 微調災難性遺忘","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780730281470-8i97.png","2026-06-06T07:17:28.426709+00:00",[84,89,94,99,104,109,114,119,124,129],{"id":85,"slug":86,"title":87,"created_at":88},"f18dbadb-8c59-4723-84a4-6ad22746c77a","deepmind-bets-on-continuous-learning-ai-2026-zh","DeepMind 押注 2026 連續學習 AI","2026-03-26T08:16:02.367355+00:00",{"id":90,"slug":91,"title":92,"created_at":93},"f4a106cb-02a6-4508-8f39-9720a0a93cee","ml-papers-of-the-week-github-research-desk-zh","每週 ML 論文清單，為何紅到 GitHub","2026-03-27T01:11:39.284175+00:00",{"id":95,"slug":96,"title":97,"created_at":98},"c4f807ca-4e5f-47f1-a48c-961cf3fc44dc","ai-ml-conferences-to-watch-in-2026-zh","2026 AI 研討會投稿時程整理","2026-03-27T01:51:53.874432+00:00",{"id":100,"slug":101,"title":102,"created_at":103},"cf046742-efb2-4753-aef9-caed5da5e32e","adaptive-block-scaled-data-types-zh","IF4：神經網路量化的聰明選擇","2026-03-31T06:00:36.990273+00:00",{"id":105,"slug":106,"title":107,"created_at":108},"53a0dc54-0371-4e40-8d5e-74e94a73840c","geometry-aware-similarity-metrics-for-neural-representations-zh","超越距離測量：用微分幾何重新理解神經網路","2026-03-31T06:01:01.241968+00:00",{"id":110,"slug":111,"title":112,"created_at":113},"fee7d472-a775-4b1d-bbc2-1e8bca1bbf8b","on-the-fly-repulsion-in-the-contextual-space-for-rich-divers-zh","讓AI繪圖更有創意：用排斥力提升生成多樣性","2026-03-31T06:01:25.439673+00:00",{"id":115,"slug":116,"title":117,"created_at":118},"a9901203-d69b-447b-8854-15d14eab32b4","vision-aided-beam-prediction-cnn-eca-zh","影像輔助波束預測升級 CNN","2026-04-01T10:00:25.8073+00:00",{"id":120,"slug":121,"title":122,"created_at":123},"b55e7dd4-0a24-4b3d-804d-b0309a03f498","triple-band-fss-mimo-antenna-sub-6-ghz-zh","三頻 FSS MIMO 天線瞄準 sub-6 GHz","2026-04-01T13:18:36.857305+00:00",{"id":125,"slug":126,"title":127,"created_at":128},"f68290bd-e7f3-4b30-ba22-dcd4e0130a66","openclaw-1299-repos-eight-weeks-analysis-zh","OpenClaw 1299 個 Repo 的資料解讀","2026-04-02T05:03:45.208411+00:00",{"id":130,"slug":131,"title":132,"created_at":133},"ed9f80eb-eb02-4d35-8ad4-0ddf428751dd","beam-coherence-aware-combining-mmwave-mimo-zh","毫米波 MIMO 的雙階合併法","2026-04-02T05:27:26.897188+00:00"]