[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-swe-bench-verified-model-leaderboard-limit-zh":3,"article-related-swe-bench-verified-model-leaderboard-limit-zh":29,"series-research-97ecb84d-bd2a-437b-839e-6e8a416d4a94":74},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":11},"97ecb84d-bd2a-437b-839e-6e8a416d4a94","swe-bench-verified-model-leaderboard-limit-zh","SWE-bench Verified 已不再是乾淨的模型排行榜","\u003Cp data-speakable=\"summary\">97.00% 的天花板把頂尖 coding 模型擠成一團，\u003Ca href=\"\u002Ftag\u002Fswe-bench-verified\">SWE-bench Verified\u003C\u002Fa> 已不適合再拿來當乾淨的排序工具。\u003C\u002Fp>\u003Cp>\u003Ca href=\"\u002Ftag\u002Fswe-bench\">SWE-bench\u003C\u002Fa> Verified 現在更像門檻測試，不像排行榜。Vals 最新結果顯示，\u003Ca href=\"\u002Ftag\u002Fclaude\">Claude\u003C\u002Fa> Opus 5 來到 97.00%，GPT-5.6 Sol 也在 95% 以上，79 個模型裡有 3 個達到 95% 或更高；當最前段擠進這麼窄的區間，這個基準測到的是誰少犯一點錯，不是誰跨出一個等級。\u003C\u002Fp>\u003Ch2>第一個論點\u003C\u002Fh2>\u003Cp>頂端\u003Ca href=\"\u002Fnews\u002Fbenchmark-scores-dont-predict-your-bill-zh\">分數\u003C\u002Fa>已經擠到幾乎沒有辨識度。第一名只有 97.00%，距離滿分差 3 個百分點，這不是「壓倒性領先」，而是幾乎進入拍照完賽。對一個包含 500 題的基準來說，幾個額外修對的 patch 就足以改變名次，但不一定改變模型在真實\u003Ca href=\"\u002Fnews\u002Fmcp-servers-8-developer-workflow-gains-2026-zh\">開發\u003C\u002Fa>中的價值。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786498365710-5zg7.png\" alt=\"SWE-bench Verified 已不再是乾淨的模型排行榜\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>更關鍵的是，前段差距不再像過去那樣能清楚分出陣營。Kimi K3 為 93.40%，Claude Opus 4.8 為 88.60%，Grok 4.5 為 86.60%。這些數字都很高，也都代表能幹活，但它們更像同一個能力帶上的不同點位，而不是三個不同世代。拿這種分布去做採購決策，容易高估 2 到 4 分的意義。\u003C\u002Fp>\u003Ch2>第二個論點\u003C\u002Fh2>\u003Cp>這個榜單本來就不是純模型智力測驗，而是模型加上工具習慣的綜合題。Vals 用的是同一套最小化 bash-only harness，公平是公平，但它也把命題改寫成「誰更會在 shell 裡找路、改檔、重試」。這會放大命令列熟練度、搜尋策略與 patch 風格的差異，而不只是推理本身。\u003C\u002Fp>\u003Cp>Vals 也指出，差距最明顯的地方，常出現在需要 15 分鐘到 1 小時才能完成的任務。這正是 agent 行為開始主導結果的區段：不是單次答對，而是一路排錯、定位、修補、再驗證。這種設計很有現實感，但它也意味著 SWE-bench Verified 測到的是「能不能把活做完」，不是「純模型能力」；兩者相關，卻不相同。\u003C\u002Fp>\u003Ch2>反方可能怎麼說\u003C\u002Fh2>\u003Cp>最強的反方說法是：即使頂端擠在一起，SWE-bench Verified 仍然是少數真正貼近軟體工作的公開基準。它用人類驗證過的 \u003Ca href=\"\u002Ftag\u002Fgithub\">GitHub\u003C\u002Fa> issue、隔離的 \u003Ca href=\"\u002Ftag\u002Fdocker\">Docker\u003C\u002Fa> 環境、以及單元測試來判定 patch 是否真的修好問題，這比合成式 coding trivia 誠實得多，也更接近產品環境中的真實任務。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786498366034-k2cv.png\" alt=\"SWE-bench Verified 已不再是乾淨的模型排行榜\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>另一個辯護也站得住腳：它不容易輕易作弊。最小化 bash-only harness 去掉了客製工具\u003Ca href=\"\u002Fnews\u002Ffpt-openai-select-status-channel-play-not-moat-zh\">優勢\u003C\u002Fa>，逼模型展示真實的命令列能力。對實驗室來說，這提供了相對公平的比較；對工程團隊來說，這至少比抽象語言分數更能反映 repo 操作、除錯與修補能力。\u003C\u002Fp>\u003Cp>但這些優點不足以挽救它在頂端的辨識力流失。基準可以很真實，也可以在最強模型都擠進 3 到 5 個百分點時失去排序功能。SWE-bench Verified 仍然適合當能力門檻和回歸測試，不適合再被當成 frontier 模型的精細排名器。更有用的讀法是：看它能不能穩定處理真實 repo 工作，而不是看誰在榜上多贏了 2 分。\u003C\u002Fp>\u003Ch2>你能做什麼\u003C\u002Fh2>\u003Cp>如果你是工程師、PM 或創辦人，把 SWE-bench Verified 當作門檻，不要當作桂冠。它適合用來淘汰明顯不行的 coding agent、比較同一產品線的版本變化、以及檢查 repo 導航與 patch 品質是否退步；但不要拿 1 到 3 分的差距，直接推論哪個 frontier 模型會徹底改變你的工作流。真正的決策應該放回你的 repo、你的任務集、你的延遲與成本限制，因為那裡才是模型價值的落點。\u003C\u002Fp>","97% 的天花板把頂尖 coding 模型擠成一團，SWE-bench Verified 已不適合再拿來當乾淨的排序工具。","www.vals.ai","https:\u002F\u002Fwww.vals.ai\u002Fbenchmarks\u002Fswebench",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786498365710-5zg7.png","research","zh","6d197f27-628f-4a63-883d-81a0d9f5c4b5",[17,18,19,20,21],"SWE-bench Verified","coding models","benchmark saturation","agent evaluation","model leaderboard",[23,24,25],"頂尖模型在 SWE-bench Verified 上已經擠成窄帶，名次差距不再等於能力差距。","這個基準同時測到模型能力與工具使用習慣，因此不適合再被當成純排行榜。","實務上應把它當門檻與回歸測試，搭配自家任務與成本延遲條件做決策。",1,"2026-08-12T01:32:19.598349+00:00","2026-08-12T01:32:19.579+00:00",{"tags":30,"relatedLang":33,"relatedPosts":37},[31],{"name":17,"slug":32},"swe-bench-verified",{"id":15,"slug":34,"title":35,"language":36},"swe-bench-verified-model-leaderboard-limit-en","SWE-bench Verified has stopped being a clean model leaderboard","en",[38,44,50,56,62,68],{"id":39,"slug":40,"title":41,"cover_image":42,"image_url":42,"created_at":43,"category":13},"2aa54cfd-2caf-4c34-834c-e771e1308747","sparse-autoencoders-set-level-instability-zh","SAE 不是特徵袋","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786518181678-0ycl.png","2026-08-12T07:02:31.647391+00:00",{"id":45,"slug":46,"title":47,"cover_image":48,"image_url":48,"created_at":49,"category":13},"1544300b-d8fc-4341-ac8a-530391b179b2","convawg-controlled-vawg-dialogue-generation-zh","ConVAWG 讓 VAWG 對話可控生成","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786516377717-xygt.png","2026-08-12T06:32:30.301717+00:00",{"id":51,"slug":52,"title":53,"cover_image":54,"image_url":54,"created_at":55,"category":13},"51473b63-b17b-492a-8dc0-b12069a19b49","surgical-wam-video-pretraining-robot-control-zh","Surgical WAM 用影片訓練手術機器人控制","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786514588496-1uqy.png","2026-08-12T06:02:32.223678+00:00",{"id":57,"slug":58,"title":59,"cover_image":60,"image_url":60,"created_at":61,"category":13},"0299c84e-cdca-4d9f-821c-437119627dbf","dutch-government-llm-benchmark-values-zh","荷蘭政府 LLM 不能只看準確率","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786431771084-my8i.png","2026-08-11T07:02:25.893246+00:00",{"id":63,"slug":64,"title":65,"cover_image":66,"image_url":66,"created_at":67,"category":13},"b076a7b7-f01a-4438-8d49-548201e9ccec","mmdiff-multimodal-feature-discovery-control-zh","MMDiff：把多模態特徵變成控制旋鈕","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786429980947-qtkn.png","2026-08-11T06:32:30.336974+00:00",{"id":69,"slug":70,"title":71,"cover_image":72,"image_url":72,"created_at":73,"category":13},"f5a1bf22-1f75-4be1-873e-f2bd717c2397","tts-evaluators-miss-more-than-naturalness-zh","TTS 評測不只看自然度","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1786428174396-4b56.png","2026-08-11T06:02:28.652133+00:00",[75,80,85,90,95,100,105,110,115,120],{"id":76,"slug":77,"title":78,"created_at":79},"f18dbadb-8c59-4723-84a4-6ad22746c77a","deepmind-bets-on-continuous-learning-ai-2026-zh","DeepMind 押注 2026 連續學習 AI","2026-03-26T08:16:02.367355+00:00",{"id":81,"slug":82,"title":83,"created_at":84},"f4a106cb-02a6-4508-8f39-9720a0a93cee","ml-papers-of-the-week-github-research-desk-zh","每週 ML 論文清單，為何紅到 GitHub","2026-03-27T01:11:39.284175+00:00",{"id":86,"slug":87,"title":88,"created_at":89},"c4f807ca-4e5f-47f1-a48c-961cf3fc44dc","ai-ml-conferences-to-watch-in-2026-zh","2026 AI 研討會投稿時程整理","2026-03-27T01:51:53.874432+00:00",{"id":91,"slug":92,"title":93,"created_at":94},"cf046742-efb2-4753-aef9-caed5da5e32e","adaptive-block-scaled-data-types-zh","IF4：神經網路量化的聰明選擇","2026-03-31T06:00:36.990273+00:00",{"id":96,"slug":97,"title":98,"created_at":99},"53a0dc54-0371-4e40-8d5e-74e94a73840c","geometry-aware-similarity-metrics-for-neural-representations-zh","超越距離測量：用微分幾何重新理解神經網路","2026-03-31T06:01:01.241968+00:00",{"id":101,"slug":102,"title":103,"created_at":104},"fee7d472-a775-4b1d-bbc2-1e8bca1bbf8b","on-the-fly-repulsion-in-the-contextual-space-for-rich-divers-zh","讓AI繪圖更有創意：用排斥力提升生成多樣性","2026-03-31T06:01:25.439673+00:00",{"id":106,"slug":107,"title":108,"created_at":109},"a9901203-d69b-447b-8854-15d14eab32b4","vision-aided-beam-prediction-cnn-eca-zh","影像輔助波束預測升級 CNN","2026-04-01T10:00:25.8073+00:00",{"id":111,"slug":112,"title":113,"created_at":114},"b55e7dd4-0a24-4b3d-804d-b0309a03f498","triple-band-fss-mimo-antenna-sub-6-ghz-zh","三頻 FSS MIMO 天線瞄準 sub-6 GHz","2026-04-01T13:18:36.857305+00:00",{"id":116,"slug":117,"title":118,"created_at":119},"f68290bd-e7f3-4b30-ba22-dcd4e0130a66","openclaw-1299-repos-eight-weeks-analysis-zh","OpenClaw 1299 個 Repo 的資料解讀","2026-04-02T05:03:45.208411+00:00",{"id":121,"slug":122,"title":123,"created_at":124},"ed9f80eb-eb02-4d35-8ad4-0ddf428751dd","beam-coherence-aware-combining-mmwave-mimo-zh","毫米波 MIMO 的雙階合併法","2026-04-02T05:27:26.897188+00:00"]