[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-why-coding-benchmarks-are-finally-telling-the-truth-zh":3,"article-related-why-coding-benchmarks-are-finally-telling-the-truth-zh":30,"series-research-5b168b94-465a-4d72-bbb1-e6577625cb1a":83},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"5b168b94-465a-4d72-bbb1-e6577625cb1a","why-coding-benchmarks-are-finally-telling-the-truth-zh","為什麼程式碼基準測試終於開始說實話","\u003Cp data-speakable=\"summary\">LiveCodeBench 和 \u003Ca href=\"\u002Ftag\u002Fswe-bench\">SWE-bench\u003C\u002Fa> Pro 已經能更準確分出真正能寫程式的模型與只會刷榜的模型。\u003C\u002Fp>\u003Cp>我認為，程式碼模型的選型標準已經變了，現在再拿 HumanEval 當主要依據，是在做錯產品決策。BenchLM 2026 年 3 月的排行榜把這件事講得很直接：\u003Ca href=\"\u002Ftag\u002Fclaude-mythos\">Claude Mythos\u003C\u002Fa> Preview 以 100.0 的加權分數居首，G\u003Ca href=\"\u002Fnews\u002Fhow-to-add-temporal-rag-in-production-zh\">em\u003C\u002Fa>ini 3.1 Pro 以 93.9 緊追，GPT-5.3 Codex 在 SWE-bench Pro 上衝到 77.3，成為頁面上最高的開源權重相關結果。這些差距不是裝飾性的數字，而是能不能在真實倉庫裡修 bug、接 test、過 CI 的差別。\u003C\u002Fp>\u003Ch2>第一個論點：真實程式工作不是玩具題\u003C\u002Fh2>\u003Cp>BenchLM 把 SWE-bench Pro 和 LiveCodeBench 等權重看待，這個設計是對的。SWE-bench Pro 來自真實 GitHub issue，測的是模型能不能在混亂的 repository 裡把問題修掉；LiveCodeBench 則持續出新題，降低資料污染的風險。這兩者合在一起，才接近工程團隊真正需要的能力：能不能處理多檔案、能不能理解上下文、能不能在沒看過的題型上維持推理品質。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778670697069-56o7.png\" alt=\"為什麼程式碼基準測試終於開始說實話\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>HumanEval 已經明顯失去區分力。BenchLM 指出，前沿模型在這個基準上幾乎都超過 95%，也就是說它早就無法幫你分辨「夠用」和「真的能上線」。當一個測試大家都能過，它就不再是選型工具，只剩熟悉舊題庫的獎勵機制。若你的評估流程還把 HumanEval 放在核心位置，你其實是在優化過去的模型，而不是現在的產品。\u003C\u002Fp>\u003Ch2>第二個論點：排行榜開始反映品質、成本與部署的真實取捨\u003C\u002Fh2>\u003Cp>這份排行榜有價值的地方，在於它沒有假裝準確率是唯一指標。Claude Mythos Preview 雖然名列第一，但頁面也把更務實的選項攤開來：重視自架的團隊可以看 GPT-5.3 Codex，追求平衡的可以看 GPT-5.4，預算優先的則有像 Qwen3.6-27B 這類較便宜的開源模型。這才是正確的選型方式，因為團隊買的不是分數本身，而是能否在延遲、成本與可靠性之間守住門檻。\u003C\u002Fp>\u003Cp>數據也把這個取捨具體化了。\u003Ca href=\"\u002Fnews\u002Fgoogle-gemini-android-center-before-wwdc-zh\">Gemi\u003C\u002Fa>ni 3.1 Pro 標示的價格是每百萬 input token 2 美元、output token 12 美元，吞吐量 109 tokens\u002Fs，TTFT 為 29.71 秒；GPT-5.3 Codex 雖然在某些成本維度上不一定最便宜，但 88.7 的加權分數與 \u003Ca href=\"\u002Ftag\u002Fswe-bench-verified\">SWE-bench Verified\u003C\u002Fa> 的 85 分，已經把它和入門級模型拉開層級差距。BenchLM 也明講，5 分差距通常就足以區分一個能修複雜多檔案 bug 的模型，和一個會卡住的模型。在程式碼場景裡，這種差距不是四捨五入的誤差，而是一次失敗的 patch。\u003C\u002Fp>\u003Ch2>反方可能怎麼說\u003C\u002Fh2>\u003Cp>最強的反對意見其實很合理：別太相信任何排行榜。基準測試天生不完整，而程式碼尤其難測。模型可以在公開題庫上拿高分，卻在你的私有 monorepo 裡翻車，原因可能只是 build tool 很怪、測試很脆、或團隊慣例太特殊。批評者會說，leaderboard 很容易變成 \u003Ca href=\"\u002Ftag\u002Fbenchmark\">benchmark\u003C\u002Fa> tuning 的戰場，而不是產品價值的證明。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778670666585-ecc3.png\" alt=\"為什麼程式碼基準測試終於開始說實話\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>這個批評成立，但它否定的不是 BenchLM，而是「只看單一分數」的做法。BenchLM 自己其實已經承認限制：HumanEval 已經飽和，SWE-bench Verified 只是參考點，LiveCodeBench 才是更能抵抗污染的訊號。這\u003Ca href=\"\u002Fnews\u002Fwhy-ibm-bob-right-kind-ai-coding-assistant-zh\">才是對\u003C\u002Fa> benchmark 懷疑論最好的回應，不是崇拜排行榜，而是把它當篩選器，再回到自己的 repo 做驗證。你該拒絕的不是所有程式碼基準，而是把過時基準當成決策核心的習慣。\u003C\u002Fp>\u003Cp>所以我的結論很明確：不是基準測試沒用，而是只有少數基準還有用。LiveCodeBench 與 SWE-bench Pro 仍然能告訴你很多事，尤其是模型是否真的能處理真實工程工作；HumanEval 則已經太容易被刷高，不適合再主導選型。\u003C\u002Fp>\u003Ch2>你能做什麼\u003C\u002Fh2>\u003Cp>如果你是工程師，先用 SWE-bench Pro 和 LiveCodeBench 把候選模型縮到少數幾個，再拿你自己的 bug-fix、\u003Ca href=\"\u002Ftag\u002Fcode-review\">code review\u003C\u002Fa>、測試修補流程去跑；如果你是 PM，不要再問「哪個 coding model 最強」，而要問哪個模型在你的延遲、成本、部署條件下還能守住可靠性門檻；如果你是創辦人，把產品評估建立在真實 repo 工作上，而不是能討好簡報的舊題庫。最後你真正該追的，不是最好看的分數，而是最能在真實程式碼裡活下來的模型。\u003C\u002Fp>","BenchLM 的程式碼排行榜顯示，真正有用的訊號只剩 LiveCodeBench 與 SWE-bench Pro；HumanEval 已經不適合拿來選模型。","benchlm.ai","https:\u002F\u002Fbenchlm.ai\u002Fcoding",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1778670697069-56o7.png","research","zh","a5281bf5-661d-4288-b00e-0aa245e1fb03",[17,18,19,20,21],"BenchLM","LiveCodeBench","SWE-bench Pro","HumanEval","程式碼模型評估",[23,24,25],"HumanEval 已經飽和，不適合再當程式碼模型的主要選型依據。","LiveCodeBench 與 SWE-bench Pro 更接近真實工程工作，能分辨模型是否真的能修 repo 裡的問題。","選模型不能只看分數，還要一起看成本、延遲與部署限制。",4,"2026-05-13T11:10:25.586869+00:00","2026-05-13T11:10:25.546+00:00","0c35a120-52fc-41fc-afa3-d404eb934158",{"tags":31,"relatedLang":42,"relatedPosts":46},[32,33,36,38,40],{"name":21,"slug":21},{"name":34,"slug":35},"SWE-Bench Pro","swe-bench-pro",{"name":17,"slug":37},"benchlm",{"name":18,"slug":39},"livecodebench",{"name":20,"slug":41},"humaneval",{"id":15,"slug":43,"title":44,"language":45},"why-coding-benchmarks-are-finally-telling-the-truth-en","Why coding benchmarks are finally telling the truth","en",[47,53,59,65,71,77],{"id":48,"slug":49,"title":50,"cover_image":51,"image_url":51,"created_at":52,"category":13},"42510df4-4692-44c6-a45a-c82a4a86b646","tls-turns-insecure-links-into-encrypted-sessions-zh","TLS 把明文連線變成加密會話","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780596207456-9or4.png","2026-06-04T18:02:50.988357+00:00",{"id":54,"slug":55,"title":56,"cover_image":57,"image_url":57,"created_at":58,"category":13},"4fa896da-9616-425a-92bc-c1d7d5861ff9","streamma-multi-agent-reasoning-latency-zh","StreamMA 讓多代理推理邊想邊傳","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780554786134-1w1d.png","2026-06-04T06:32:32.769423+00:00",{"id":60,"slug":61,"title":62,"cover_image":63,"image_url":63,"created_at":64,"category":13},"f31f51ba-4445-4e43-9bda-31e70f53d42b","audio-language-models-arbitration-reversals-zh","音訊模型不是聽不懂","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780553877373-ux95.png","2026-06-04T06:17:27.890159+00:00",{"id":66,"slug":67,"title":68,"cover_image":69,"image_url":69,"created_at":70,"category":13},"447ac6c9-477b-45c8-bec2-ff94dc4cf5d4","stride-training-data-attribution-sparse-recovery-zh","STRIDE 讓訓練資料歸因快 13 倍","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780552979370-897a.png","2026-06-04T06:02:29.149166+00:00",{"id":72,"slug":73,"title":74,"cover_image":75,"image_url":75,"created_at":76,"category":13},"33c9a55c-a8c0-4367-b742-f4567d1e98e3","mathematicians-warn-ai-could-distort-math-zh","數學界警告 AI 會扭曲證明標準","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780504386035-080l.png","2026-06-03T16:32:29.415063+00:00",{"id":78,"slug":79,"title":80,"cover_image":81,"image_url":81,"created_at":82,"category":13},"5c3cb90f-7efd-426f-8c09-32a303f82be9","humanoid-gpt-zero-shot-motion-tracking-zh","Humanoid-GPT：用 GPT 擴大動作追蹤","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780469319284-znpc.png","2026-06-03T06:47:34.463464+00:00",[84,89,94,99,104,109,114,119,124,129],{"id":85,"slug":86,"title":87,"created_at":88},"f18dbadb-8c59-4723-84a4-6ad22746c77a","deepmind-bets-on-continuous-learning-ai-2026-zh","DeepMind 押注 2026 連續學習 AI","2026-03-26T08:16:02.367355+00:00",{"id":90,"slug":91,"title":92,"created_at":93},"f4a106cb-02a6-4508-8f39-9720a0a93cee","ml-papers-of-the-week-github-research-desk-zh","每週 ML 論文清單，為何紅到 GitHub","2026-03-27T01:11:39.284175+00:00",{"id":95,"slug":96,"title":97,"created_at":98},"c4f807ca-4e5f-47f1-a48c-961cf3fc44dc","ai-ml-conferences-to-watch-in-2026-zh","2026 AI 研討會投稿時程整理","2026-03-27T01:51:53.874432+00:00",{"id":100,"slug":101,"title":102,"created_at":103},"cf046742-efb2-4753-aef9-caed5da5e32e","adaptive-block-scaled-data-types-zh","IF4：神經網路量化的聰明選擇","2026-03-31T06:00:36.990273+00:00",{"id":105,"slug":106,"title":107,"created_at":108},"53a0dc54-0371-4e40-8d5e-74e94a73840c","geometry-aware-similarity-metrics-for-neural-representations-zh","超越距離測量：用微分幾何重新理解神經網路","2026-03-31T06:01:01.241968+00:00",{"id":110,"slug":111,"title":112,"created_at":113},"fee7d472-a775-4b1d-bbc2-1e8bca1bbf8b","on-the-fly-repulsion-in-the-contextual-space-for-rich-divers-zh","讓AI繪圖更有創意：用排斥力提升生成多樣性","2026-03-31T06:01:25.439673+00:00",{"id":115,"slug":116,"title":117,"created_at":118},"a9901203-d69b-447b-8854-15d14eab32b4","vision-aided-beam-prediction-cnn-eca-zh","影像輔助波束預測升級 CNN","2026-04-01T10:00:25.8073+00:00",{"id":120,"slug":121,"title":122,"created_at":123},"b55e7dd4-0a24-4b3d-804d-b0309a03f498","triple-band-fss-mimo-antenna-sub-6-ghz-zh","三頻 FSS MIMO 天線瞄準 sub-6 GHz","2026-04-01T13:18:36.857305+00:00",{"id":125,"slug":126,"title":127,"created_at":128},"f68290bd-e7f3-4b30-ba22-dcd4e0130a66","openclaw-1299-repos-eight-weeks-analysis-zh","OpenClaw 1299 個 Repo 的資料解讀","2026-04-02T05:03:45.208411+00:00",{"id":130,"slug":131,"title":132,"created_at":133},"ed9f80eb-eb02-4d35-8ad4-0ddf428751dd","beam-coherence-aware-combining-mmwave-mimo-zh","毫米波 MIMO 的雙階合併法","2026-04-02T05:27:26.897188+00:00"]