[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-sala-duance-ai-shangxiawen-kuozhan-zhinan-zh":3,"article-related-sala-duance-ai-shangxiawen-kuozhan-zhinan-zh":30,"series-ai-agent-f538fedf-8816-4bfd-8c25-ef97be9f9d5d":75},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":23,"views":27,"created_at":28,"published_at":29,"topic_cluster_id":11},"f538fedf-8816-4bfd-8c25-ef97be9f9d5d","sala-duance-ai-shangxiawen-kuozhan-zhinan-zh","SALA端侧AI上下文扩展操作指南","\u003Cp data-speakable=\"summary\">过去端侧大模型常被上下文长度卡住，现在可用SALA混合注意力扩大可处理文本。\u003C\u002Fp>\u003Cp>这篇指南适合要在端侧设备上部署长上下文大模型的开发者，尤其是正在评估注意力架构、推理成本和上下文扩展方案的工程团队。照着做完，你会得到一个可复现的SALA原型、清晰的验证方法，以及接到本地推理服务的基本路径。\u003C\u002Fp>\u003Cp>重点不是复述\u003Ca href=\"\u002Fnews\u002Fai-gain-stock-liquidation-salp-postmortem-zh\">概念\u003C\u002Fa>，而是把“75%线性注意力 + 25%稀疏注意力”的混合思路落到可执行流程里。你可以用它判断模型是否适合端侧部署、哪些输入长度会触发性能瓶颈，以及如何用最小改动验证上下文收益。\u003C\u002Fp>\u003Ch2>开始之前\u003C\u002Fh2>\u003Cul>\u003Cli>Node.js 20+\u003C\u002Fli>\u003Cli>Python 3.10+\u003C\u002Fli>\u003Cli>CUDA 12.1 或支持的 CPU 推理环境\u003C\u002Fli>\u003Cli>Git 2.40+\u003C\u002Fli>\u003Cli>一个可用的 Hugging Face 账号\u003C\u002Fli>\u003Cli>资料页与仓库入口：\u003Ca href=\"https:\u002F\u002Fzhuanlan.zhihu.com\u002Fp\u002F2065917980169580899\">知乎原文\u003C\u002Fa>，\u003Ca href=\"https:\u002F\u002Fgithub.com\u002F\">GitHub\u003C\u002Fa>\u003C\u002Fli>\u003C\u002Ful>\u003Cp>如果你打算直接跑模型，还需要至少 16GB 内存；如果要做本地量化推理，建议准备 24GB 显存或等效高内存机器。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785765793001-wixs.png\" alt=\"SALA端侧AI上下文扩展操作指南\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Ch2>Step 1: 读取SALA架构说明\u003C\u002Fh2>\u003Cp>目的：先把“混合注意力”转成工程约束，明确哪些 \u003Ca href=\"\u002Ftag\u002Ftoken\">token\u003C\u002Fa> 走线性路径、哪些 token 走稀疏路径，以及要验证的上下文长度上限。\u003C\u002Fp>\u003Cp>先通读原文里对 SALA 的描述，记下核心比例：75% 线性注意力，25% 稀疏注意力。然后把它翻译成实验目标，比如先做 8K、16K、32K 三档上下文测试。\u003C\u002Fp>\u003Cpre>\u003Ccode># 记录你的实验目标，方便后续对照实现结果\ncat > notes.md <<'EOF'\nSALA mix: 75% linear attention + 25% sparse attention\nTarget context windows: 8K, 16K, 32K\nSuccess criteria: lower memory growth, stable output quality\nEOF\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>验收：你应该能在 notes.md 里明确写出架构比例、目标上下文和成功标准，而不是只停留在“想提升长文本能力”的泛泛描述。\u003C\u002Fp>\u003Ch2>Step 2: 搭建本地推理基线\u003C\u002Fh2>\u003Cp>目的：先建立一个没有 SALA 优化的基线，这样后面才知道上下文扩展到底带来了多少收益。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785765796890-s4op.png\" alt=\"SALA端侧AI上下文扩展操作指南\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>如果你已经有一个 Hugging Face 模型仓库，可以先用现成推理框架加载一个支持长上下文的基线模型。最重要的是固定同一套提示词和同一组测试文本，避免结果被别的变量干扰。\u003C\u002Fp>\u003Cpre>\u003Ccode>python -m venv .venv\nsource .venv\u002Fbin\u002Factivate\npip install --upgrade pip\npip install transformers accelerate torch\npython -c \"from transformers import AutoTokenizer; print('baseline ready')\"\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>验收：你应该看到终端输出 baseline ready，并且虚拟环境里能正常导入 transformers。若这里报错，先修依赖，再继续做架构实验。\u003C\u002Fp>\u003Ch2>Step 3: 构造长上下文测试集\u003C\u002Fh2>\u003Cp>目的：准备一组能真实暴露上下文瓶颈的输入，而不是只拿短句子做演示。\u003C\u002Fp>\u003Cp>建议把测试集分成三类：事实抽取、跨段落问答、长文摘要。每类都放入不同长度的文本片段，让模型必须依赖更远的 token 才能回答准确。\u003C\u002Fp>\u003Cpre>\u003Ccode>mkdir -p data\npython - <<'PY'\nfrom pathlib import Path\nbase = \"\"\"段落A：面壁智能正在探索混合注意力架构。\n段落B：75%线性注意力和25%稀疏注意力被组合起来。\n段落C：目标是让模型吃进更多文本。\"\"\"\nPath('data\u002Flong_context.txt').write_text(base * 200, encoding='utf-8')\nprint('dataset ready')\nPY\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>验收：你应该得到一个明显超过短文本长度的 data\u002Flong_context.txt \u003Ca href=\"\u002Fnews\u002Fextractbench-schema-guided-document-extraction-zh\">文件\u003C\u002Fa>。之后可以用它来检查模型在不同窗口下是否还能正确回答跨段落问题。\u003C\u002Fp>\u003Ch2>Step 4: 实现混合注意力路由\u003C\u002Fh2>\u003Cp>目的：把 SALA 的核心思想落成一个可测试的路由层，让一部分 token 走线性注意力，另一部分 token 走稀疏注意力。\u003C\u002Fp>\u003Cp>工程上不必一开始就复刻完整论文实现。你可以先写一个路由器接口，输入 token 序列后，按比例切分成两条路径，再把输出合并回去。这样先验证结构是否能工作，再优化细节。\u003C\u002Fp>\u003Cpre>\u003Ccode>class SALARouter:\n    def __init__(self, linear_ratio=0.75):\n        self.linear_ratio = linear_ratio\n\n    def route(self, tokens):\n        split = int(len(tokens) * self.linear_ratio)\n        linear_tokens = tokens[:split]\n        sparse_tokens = tokens[split:]\n        return linear_tokens, sparse_tokens\n\nrouter = SALARouter()\nprint(router.route(list(range(20))))\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>验收：你应该看到前 15 个 token 进入线性路径，后 5 个 token 进入稀疏路径。这个结果说明路由比例已经生效，后续只需要替换成真实注意力算子。\u003C\u002Fp>\u003Ch2>Step 5: 测量上下文收益\u003C\u002Fh2>\u003Cp>目的：用同一套输入对比基线和混合注意力版本，确认上下文扩展不是“看起来更长”，而是真的更稳。\u003C\u002Fp>\u003Cp>重点看三项：显存增长、响应延迟、长文本任务正确率。每轮测试都固定 prompt、固定长度、固定采样参数，然后记录结果，最后再比较不同窗口下的变化。\u003C\u002Fp>\u003Cpre>\u003Ccode># 示例：用你自己的脚本记录结果\npython eval.py --model baseline --context 8k  --input data\u002Flong_context.txt\npython eval.py --model sala     --context 8k  --input data\u002Flong_context.txt\npython eval.py --model baseline --context 16k --input data\u002Flong_context.txt\npython eval.py --model sala     --context 16k --input data\u002Flong_context.txt\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>验收：你应该能写出一份对照表，说明 SALA 在你的环境里是否真的更适合端侧部署，同时显存和延迟曲线没有突然失控。\u003C\u002Fp>\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>指標\u003C\u002Fth>\u003Cth>基準／優化前\u003C\u002Fth>\u003Cth>結果／優化後\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>上下文窗口\u003C\u002Ftd>\u003Ctd>短文本 smoke test\u003C\u002Ftd>\u003Ctd>8K \u002F 16K \u002F 32K 長上下文驗證\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>注意力路徑\u003C\u002Ftd>\u003Ctd>單一路徑注意力\u003C\u002Ftd>\u003Ctd>75% 線性 + 25% 稀疏路由\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>驗證重點\u003C\u002Ftd>\u003Ctd>只看基本正確率\u003C\u002Ftd>\u003Ctd>顯存、延遲、長文本正確率\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003Ch2>常見錯誤\u003C\u002Fh2>\u003Cul>\u003Cli>把“長上下文”只當成 tokenizer 參數。修法：同時檢查注意力結構、KV cache 和推理框架限制。\u003C\u002Fli>\u003Cli>直接用短文本測試混合注意力。修法：至少準備 8K 以上的長文樣本，才能看出稀疏路徑的價值。\u003C\u002Fli>\u003Cli>忽略基線對照。修法：先跑普通注意力版本，再跑 SALA 版本，避免把模型本身波動誤判為架構收益。\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>接下來可以看什麼\u003C\u002Fh2>\u003Cp>下一步可以把這個原型接到你自己的推理服務里，再\u003Ca href=\"\u002Fnews\u002Fopenai-api-changelog-spend-caps-transcribe-fast-mode-zh\">加上\u003C\u002Fa>量化、快取和批處理策略，看看 SALA 在真實端側設備上的收益是否還能保住。\u003C\u002Fp>","面壁智能用SALA混合注意力把端侧大模型的上下文容量做大。","zhuanlan.zhihu.com","https:\u002F\u002Fzhuanlan.zhihu.com\u002Fp\u002F2065917980169580899",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785765793001-wixs.png","ai-agent","zh","306b4d13-3911-4fcb-9f53-861fa5e9b430",[17,18,19,20,21,22],"SALA","端侧AI","长上下文","混合注意力","Hugging Face","推理基线",[24,25,26],"先把SALA的75%线性与25%稀疏路由写成可验证的工程目标。","用基线模型和长上下文测试集，分开判断架构收益与模型波动。","通过显存、延迟和准确率三项对照，确认是否适合端侧部署。",0,"2026-08-03T14:02:44.433703+00:00","2026-08-03T14:02:44.403+00:00",{"tags":31,"relatedLang":34,"relatedPosts":38},[32],{"name":21,"slug":33},"hugging-face",{"id":15,"slug":35,"title":36,"language":37},"sala-boosts-long-context-edge-ai-en","SALA Boosts Long Context on Edge AI","en",[39,45,51,57,63,69],{"id":40,"slug":41,"title":42,"cover_image":43,"image_url":43,"created_at":44,"category":13},"ac6e41ac-c8f1-4969-ab97-8662324881db","anthropic-breach-proves-ai-agents-need-hard-security-limits-zh","Anthropic 外洩證明 AI agents 必須先有硬性安全邊界","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785742392018-5clf.png","2026-08-03T07:32:41.851393+00:00",{"id":46,"slug":47,"title":48,"cover_image":49,"image_url":49,"created_at":50,"category":13},"fa4ebd4b-b8e5-46bf-bd94-6f6e9008ab56","genai-mil-war-prompt-report-template-zh","把恐怖提示詞改成週報","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785655978571-fadx.png","2026-08-02T07:32:38.01822+00:00",{"id":52,"slug":53,"title":54,"cover_image":55,"image_url":55,"created_at":56,"category":13},"435cd05e-f667-44ce-b362-598cab19269f","epam-openai-deal-turns-pilots-into-production-zh","EPAM 讓 AI 從試點變上線","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785628985644-6uk4.png","2026-08-02T00:02:39.955798+00:00",{"id":58,"slug":59,"title":60,"cover_image":61,"image_url":61,"created_at":62,"category":13},"f94242dc-a7ae-461a-a23d-bfd7ddd2bedb","claude-code-prompt-engineering-overrated-task-design-verific-zh","Claude Code 的關鍵不是提示詞，而是任務設計與驗證","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785544369330-u3z4.png","2026-08-01T00:32:24.169562+00:00",{"id":64,"slug":65,"title":66,"cover_image":67,"image_url":67,"created_at":68,"category":13},"71979ad3-7f4f-4567-a168-5494fef7d9c2","grok-build-live-previews-rewind-fixes-zh","Grok Build 0.2.111 修掉預覽與回溯問題","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785198774644-333s.png","2026-07-28T00:32:30.888107+00:00",{"id":70,"slug":71,"title":72,"cover_image":73,"image_url":73,"created_at":74,"category":13},"9f4b3a5e-132d-437c-8874-5c98f8302dcb","kimi-k3-benchmark-evaluation-guide-coding-agents-zh","Kimi K3 編碼代理評測操作指南","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1784574187634-d3v3.png","2026-07-20T19:02:39.109567+00:00",[76,81,86,91,96,101,106,111,116,121],{"id":77,"slug":78,"title":79,"created_at":80},"4ae1e197-1d3d-4233-8733-eafe9cb6438b","claude-now-uses-your-pc-to-finish-tasks-zh","Claude 開始幫你操作電腦","2026-03-26T07:20:48.457387+00:00",{"id":82,"slug":83,"title":84,"created_at":85},"5bede67f-e21c-413d-9ab8-54a3c3d26227","googles-2026-ai-agent-report-decoded-zh","Google 2026 AI Agent 報告解讀","2026-03-26T11:15:22.651956+00:00",{"id":87,"slug":88,"title":89,"created_at":90},"2987d097-563f-46c7-b76f-b558d8ef7c2b","kimi-k25-review-stronger-still-not-legend-zh","Kimi K2.5 評測：更強，但還不是神作","2026-03-27T07:15:55.277513+00:00",{"id":92,"slug":93,"title":94,"created_at":95},"95c9053b-e3f4-4cb5-aace-5c54f4c9e044","claude-code-controls-mac-desktop-zh","Claude Code 也能操控 Mac 了","2026-03-28T03:01:58.58121+00:00",{"id":97,"slug":98,"title":99,"created_at":100},"dc58e153-e3a8-4c06-9b96-1aa64eabbf5f","cloudflare-100x-faster-ai-agent-sandbox-zh","Cloudflare 的 AI 沙箱跑超快","2026-03-28T03:09:44.142236+00:00",{"id":102,"slug":103,"title":104,"created_at":105},"1c8afc56-253f-47a2-979f-1065ff072f2a","openai-backs-isara-agent-swarm-bet-zh","OpenAI 挺 Isara 的 agent swarm …","2026-03-28T03:15:27.513155+00:00",{"id":107,"slug":108,"title":109,"created_at":110},"7379b422-576e-45df-ad5a-d57a0d9dd467","openai-plan-automated-ai-researcher-zh","OpenAI 想做自動化 AI 研究員","2026-03-28T03:17:42.090548+00:00",{"id":112,"slug":113,"title":114,"created_at":115},"48c9889e-86df-450b-a356-e4a4b7c83c5b","harness-engineering-ai-agent-reliability-2026-zh","駕馭工程：從「馬具」到「作業系統」，AI Agent 可靠性的終極密碼","2026-03-31T06:42:53.556721+00:00",{"id":117,"slug":118,"title":119,"created_at":120},"96d8e8c8-1edd-475d-9145-b1e7a1b02b65","mcp-explained-from-prompts-to-production-zh","MCP 怎麼把提示詞變工作流","2026-04-01T09:24:39.321274+00:00",{"id":122,"slug":123,"title":124,"created_at":125},"f2ca7720-b471-4ce5-9336-2a9ac2a876fd","amazon-bedrock-agents-multi-agent-workflows-zh","Amazon Bedrock Agents 進入多代理工作流","2026-04-01T09:30:29.945429+00:00"]