[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-stochastic-subgradient-last-iterate-bounds-zh":3,"article-related-stochastic-subgradient-last-iterate-bounds-zh":30,"series-research-b3ade74e-f68c-4e65-8dc6-afa9c98ebb75":73},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":22,"views":26,"created_at":27,"published_at":28,"topic_cluster_id":29},"b3ade74e-f68c-4e65-8dc6-afa9c98ebb75","stochastic-subgradient-last-iterate-bounds-zh","隨機次梯度最後一輪界更緊了","\u003Cp data-speakable=\"summary\">這篇論文把 1D 隨機次梯度法的最後一輪收斂界收緊，也證明只看變異數不夠。\u003C\u002Fp>\u003Cul>\u003Cli>\u003Cstrong>研究機構\u003C\u002Fstrong>：arXiv 摘要未明確標註\u003C\u002Fli>\u003Cli>\u003Cstrong>核心數據\u003C\u002Fstrong>：1\u002F√n 最佳化誤差\u003C\u002Fli>\u003Cli>\u003Cstrong>突破點\u003C\u002Fstrong>：去掉額外 log n 因子\u003C\u002Fli>\u003C\u002Ful>\u003Cp>\u003Ca href=\"https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.24879\">New Bounds for the Last Iterate of the Stochastic subGradient Method\u003C\u002Fa> 這篇論文，盯住一個很實際的問題：你最後拿到的那個模型參數，到底有沒有理論保證？很多最佳化結果喜歡講平均迭代、講整段過程，但工程上真正會被部署、存 checkpoint、交給下一段流程的，通常就是最後一輪。\u003C\u002Fp>\u003Cp>這篇摘要沒有在講新模型，也沒有在講實驗技巧。它是在補一個理論洞：隨機次梯度下降的最後一輪，在固定訓練長度下，能不能有和平均迭代一樣乾淨的收斂界。作者把問題縮到一維、凸、Lipschitz 的目標函數，然後用固定步長 η = Θ(1\u002F√n) 來分析最後輸出。\u003C\u002Fp>\u003Ch2>這篇論文想解的痛點\u003C\u002Fh2>\u003Cp>隨機次梯度法是處理凸最佳化的基本工具。當精確梯度拿不到，或算起來太貴時，這類方法很常見。問題是，很多理論保證寫得漂亮，卻不是針對最後一輪，而是針對平均過的結果。對開發者來說，這會造成落差，因為真正上線的往往不是平均值，而是最後那個 checkpoint。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782283671565-zxc9.png\" alt=\"隨機次梯度最後一輪界更緊了\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>如果最後一輪的保證比較弱，那就不是小事。因為你在 pipeline 裡看到的狀態、接手下一階段訓練的狀態，通常就是它。這篇論文就是把焦點拉回這個最實際的對象：最後一輪到底能保證什麼。\u003C\u002Fp>\u003Cp>作者也直接對上了一個舊問題。摘要提到，這是在回應 Koren 和 Segal 在 COLT 2020 提出的 open problem：只有 bounded-variance noise，夠不夠讓最後一輪達到乾淨的 O(1\u002F√n) 行為？還是說，多出一個 log 因子其實躲不掉？\u003C\u002Fp>\u003Ch2>方法怎麼看才不會繞暈\u003C\u002Fh2>\u003Cp>這篇的設定其實很窄，但也很刻意。它只看一維、凸、Lipschitz 的目標函數，並且採用已知訓練長度 n 的固定步長策略，也就是 η = Θ(1\u002F√n)。換句話說，學習率不是訓練中途再調，而是一開始就根據總步數先定好。\u003C\u002Fp>\u003Cp>在這個框架下，作者分析的是最後一輪的 optimization error。第一個\u003Ca href=\"\u002Fnews\u002Fun-open-source-week-2026-ai-priorities-zh\">重點\u003C\u002Fa>結果是：如果 subgradient noise 是 additive、i.i.d.，而且變異數有一致上界，那最後一輪可以達到 O(1\u002F√n) 的誤差。這等於把既有較一般的界裡，那個多出來的 log n 因子拿掉了。\u003C\u002Fp>\u003Cp>第二個結果更關鍵，因為它故意拿掉一個假設。當 i.i.d. 條件不成立時，最後一輪的誤差就可能\u003Ca href=\"\u002Fnews\u002Fproject-solara-agent-first-devices-zh\">變成\u003C\u002Fa> (log n)\u002F√n。這表示那個 log 因子不是純粹的證明技巧殘影；只要獨立性不在，問題就真的可能變難。\u003C\u002Fp>\u003Cp>也就是說，這篇不是在說「bounded variance 就一定夠」。相反地，它是在把條件切開來看：有些噪音假設真的能保住乾淨的 1\u002F√n，少了 i.i.d.，就可能掉回帶 log 的版本。\u003C\u002Fp>\u003Ch2>論文實際證明了什麼\u003C\u002Fh2>\u003Cp>這篇摘要最重要的技術訊息，是它把噪音假設的作用講清楚了。當 noise 是 additive、i.i.d.、而且變異數有一致上界時，最後一輪表現比一般界還好。當 i.i.d. 不見了，最後一輪就可能退化。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782283674164-urcc.png\" alt=\"隨機次梯度最後一輪界更緊了\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>這也直接回答了前面那個 open problem。摘要明講：只靠 uniformly bounded variance 這個假設，stochastic subgradient method 的最後一輪在一維情況下仍然是 suboptimal。白話一點說，就是「只有變異數界」不夠保證最漂亮的最後一輪收斂。\u003C\u002Fp>\u003Cp>摘要裡沒有公開完整 \u003Ca href=\"\u002Ftag\u002Fbenchmark\">benchmark\u003C\u002Fa> 細節。沒有資料集、沒有表格、沒有 wall-clock、也沒有實驗數字。這篇是純理論工作，重點在 rate analysis、條件切分，還有對既有問題給出負面答案。\u003C\u002Fp>\u003Cul>\u003Cli>設定：一維凸 Lipschitz 最佳化\u003C\u002Fli>\u003Cli>步長：固定 η = Θ(1\u002F√n)\u003C\u002Fli>\u003Cli>好情況：additive i.i.d. bounded-variance noise 下為 O(1\u002F√n)\u003C\u002Fli>\u003Cli>壞情況：拿掉 i.i.d. 後可變成 (log n)\u002F√n\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>對開發者有什麼影響\u003C\u002Fh2>\u003Cp>如果你在 production 裡用的是 stochastic subgradient 風格的更新，這篇的提醒很直接：噪音模型的假設，跟更新規則本身一樣重要。兩個訓練流程看起來可能很像，但只要步與步之間的噪音不是真的獨立，最後一輪的理論保證就可能不一樣。\u003C\u002Fp>\u003Cp>這對 checkpoint 特別有意義。很多人會以為，只要平均迭代有好結果，最後一輪大概也差不多。但這篇論文告訴你，不能這樣偷懶。若你的設定符合 i.i.d. bounded-variance 模型，最後一輪可以用更乾淨的 1\u002F√n 來分析；若不符合，就不要直接把同樣的保證套上去。\u003C\u002Fp>\u003Cp>對工程實作來說，這不是在叫你換一個新 optimizer，而是在提醒你把假設寫清楚。尤其是當訓練過程可能有相關性、漂移、或非平穩噪音時，理論上的收斂界不一定能原封不動搬到真實 pipeline。\u003C\u002Fp>\u003Ch2>限制也很明確\u003C\u002Fh2>\u003Cp>這篇工作的限制很清楚。它只處理一維、凸、Lipschitz 的問題，所以不能直接外推到高維，也不能直接說明更複雜模型會怎樣。\u003C\u002Fp>\u003Cp>摘要也只談固定步長，而且步長是根據 horizon n 預先設定的。它沒有討論自適應學習率、實務上常見的 heuristic，也沒有談非凸訓練。這些都很自然地會讓人想追問，但摘要沒有回答。\u003C\u002Fp>\u003Cp>不過，這篇還是把邊界切得很漂亮。它明確指出：哪一個假設能換來乾淨的最後一輪速率，哪一個假設太弱，保不住這個結果。對做最佳化理論的人來說，這種 boundary-setting 很有價值；對做\u003Ca href=\"\u002Fnews\u002Fdometrain-advanced-system-design-ops-template-zh\">系統\u003C\u002Fa>的人來說，也是一個很實用的提醒。\u003C\u002Fp>\u003Cp>總結來說，這篇論文把 stochastic subgradient descent 的最後一輪理論界收得更緊，也把「只有 bounded variance 是否足夠」這件事講得更清楚。若你在意最後拿到的模型狀態，就不能只看平均行為，還得看噪音假設到底有沒有真的成立。\u003C\u002Fp>","這篇論文把 1D 隨機次梯度法的最後一輪收斂界收緊，也證明只看變異數不夠。","arxiv.org","https:\u002F\u002Farxiv.org\u002Fabs\u002F2606.24879",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782283671565-zxc9.png","research","zh","59a57ebc-6f6e-4454-9cd2-51fca86a6a26",[17,18,19,20,21],"stochastic subgradient descent","last iterate","bounded variance","i.i.d.","convex optimization",[23,24,25],"最後一輪不等於平均迭代，理論保證不能直接混用。","在一維凸 Lipschitz 設定下，i.i.d. bounded-variance noise 可達 1\u002F√n。","拿掉 i.i.d. 後，最後一輪可能退化到 (log n)\u002F√n。",0,"2026-06-24T06:47:29.111822+00:00","2026-06-24T06:47:29.102+00:00","5fa30296-f388-4653-96e0-bc24f62780b7",{"tags":31,"relatedLang":32,"relatedPosts":36},[],{"id":15,"slug":33,"title":34,"language":35},"stochastic-subgradient-last-iterate-bounds-en","Stochastic Subgradient Last Iterate Gets Tight Bounds","en",[37,43,49,55,61,67],{"id":38,"slug":39,"title":40,"cover_image":41,"image_url":41,"created_at":42,"category":13},"261f4fc9-e9c8-413c-b222-a31008ec2bcf","flux3d-3d-gaussian-generation-diffusion-zh","FLUX3D 讓 3DGS 保住細節","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782284582760-3ja7.png","2026-06-24T07:02:37.306254+00:00",{"id":44,"slug":45,"title":46,"cover_image":47,"image_url":47,"created_at":48,"category":13},"5f0498a5-626f-4217-8c67-3b3404c7c172","insight-vla-self-guided-skill-acquisition-zh","InSight 讓 VLA 自學新技能","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782282781514-vo4g.png","2026-06-24T06:32:30.787554+00:00",{"id":50,"slug":51,"title":52,"cover_image":53,"image_url":53,"created_at":54,"category":13},"29ea0e09-dbd2-406d-9d74-fd851c59a4f7","anthropic-right-alarm-recursive-self-improvement-zh","Anthropic 警告遞迴自我改進是對的，但真正的問題是 AI 控制已經失速","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782263867507-ive3.png","2026-06-24T01:17:20.490657+00:00",{"id":56,"slug":57,"title":58,"cover_image":59,"image_url":59,"created_at":60,"category":13},"e6906894-cfe6-48a2-84a2-cd34e6a95186","openai-bug-hunt-chrome-safari-firefox-zh","OpenAI 一週挖出三大瀏覽器漏洞","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782258476786-r07m.png","2026-06-23T23:47:30.624426+00:00",{"id":62,"slug":63,"title":64,"cover_image":65,"image_url":65,"created_at":66,"category":13},"19c48417-946e-4c23-865f-87ffcc754d1a","llm-fine-tuning-production-2026-zh","2026 生產環境 LLM 微調指南","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782252178755-rwnv.png","2026-06-23T22:02:33.169136+00:00",{"id":68,"slug":69,"title":70,"cover_image":71,"image_url":71,"created_at":72,"category":13},"8531d5f9-60f1-4a4b-94a3-323b82990f06","lifescibench-tests-biotech-models-zh","LifeSciBench 讓模型先過科研關","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1782198202904-lzgm.png","2026-06-23T07:02:47.182473+00:00",[74,79,84,89,94,99,104,109,114,119],{"id":75,"slug":76,"title":77,"created_at":78},"f18dbadb-8c59-4723-84a4-6ad22746c77a","deepmind-bets-on-continuous-learning-ai-2026-zh","DeepMind 押注 2026 連續學習 AI","2026-03-26T08:16:02.367355+00:00",{"id":80,"slug":81,"title":82,"created_at":83},"f4a106cb-02a6-4508-8f39-9720a0a93cee","ml-papers-of-the-week-github-research-desk-zh","每週 ML 論文清單，為何紅到 GitHub","2026-03-27T01:11:39.284175+00:00",{"id":85,"slug":86,"title":87,"created_at":88},"c4f807ca-4e5f-47f1-a48c-961cf3fc44dc","ai-ml-conferences-to-watch-in-2026-zh","2026 AI 研討會投稿時程整理","2026-03-27T01:51:53.874432+00:00",{"id":90,"slug":91,"title":92,"created_at":93},"cf046742-efb2-4753-aef9-caed5da5e32e","adaptive-block-scaled-data-types-zh","IF4：神經網路量化的聰明選擇","2026-03-31T06:00:36.990273+00:00",{"id":95,"slug":96,"title":97,"created_at":98},"53a0dc54-0371-4e40-8d5e-74e94a73840c","geometry-aware-similarity-metrics-for-neural-representations-zh","超越距離測量：用微分幾何重新理解神經網路","2026-03-31T06:01:01.241968+00:00",{"id":100,"slug":101,"title":102,"created_at":103},"fee7d472-a775-4b1d-bbc2-1e8bca1bbf8b","on-the-fly-repulsion-in-the-contextual-space-for-rich-divers-zh","讓AI繪圖更有創意：用排斥力提升生成多樣性","2026-03-31T06:01:25.439673+00:00",{"id":105,"slug":106,"title":107,"created_at":108},"a9901203-d69b-447b-8854-15d14eab32b4","vision-aided-beam-prediction-cnn-eca-zh","影像輔助波束預測升級 CNN","2026-04-01T10:00:25.8073+00:00",{"id":110,"slug":111,"title":112,"created_at":113},"b55e7dd4-0a24-4b3d-804d-b0309a03f498","triple-band-fss-mimo-antenna-sub-6-ghz-zh","三頻 FSS MIMO 天線瞄準 sub-6 GHz","2026-04-01T13:18:36.857305+00:00",{"id":115,"slug":116,"title":117,"created_at":118},"f68290bd-e7f3-4b30-ba22-dcd4e0130a66","openclaw-1299-repos-eight-weeks-analysis-zh","OpenClaw 1299 個 Repo 的資料解讀","2026-04-02T05:03:45.208411+00:00",{"id":120,"slug":121,"title":122,"created_at":123},"ed9f80eb-eb02-4d35-8ad4-0ddf428751dd","beam-coherence-aware-combining-mmwave-mimo-zh","毫米波 MIMO 的雙階合併法","2026-04-02T05:27:26.897188+00:00"]