[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-prevent-catastrophic-forgetting-llm-fine-tuning-zh":3,"article-related-prevent-catastrophic-forgetting-llm-fine-tuning-zh":31,"series-research-a4cb421e-464e-4933-9e1c-6371d3cd1503":86},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":23,"views":27,"created_at":28,"published_at":29,"topic_cluster_id":30},"a4cb421e-464e-4933-9e1c-6371d3cd1503","prevent-catastrophic-forgetting-llm-fine-tuning-zh","如何防止 LLM 微調災難性遺忘","\u003Cp data-speakable=\"summary\">用 Anchored Weight Decay 在 \u003Ca href=\"\u002Ftag\u002Fllm\">LLM\u003C\u002Fa> \u003Ca href=\"\u002Fnews\u002Ftether-bitnet-fine-tuning-edge-devices-zh\">微調\u003C\u002Fa>時降低舊任務漂移，保住原有能力並檢查模型是否回復。\u003C\u002Fp>\u003Cp>這篇給做 LLM 微調、持續後訓練與研究評估的 ML 工程師和研究者看。照著做完，你會得到一套可重複的流程，用來觀察模型是否在新增能力時遺忘舊能力。\u003C\u002Fp>\u003Cp>你也會得到一個可直接套用的訓練範本，包含基準評估、短跑式 ES 試驗、Anchored Weight Decay、以及最後的 checkpoint 挑選規則。\u003C\u002Fp>\u003Ch2>開始之前\u003C\u002Fh2>\u003Cul>\u003Cli>可用的 LLM 微調環境，例如 GPU 工作站或雲端訓練工作。\u003C\u002Fli>\u003Cli>Python 3.10+。\u003C\u002Fli>\u003Cli>PyTorch 2.1+。\u003C\u002Fli>\u003Cli>至少一個舊任務評估集，例如 HellaSwag 或自建驗證集。\u003C\u002Fli>\u003Cli>Evolution Strategies 實作或研究程式庫。\u003C\u002Fli>\u003Cli>GitHub 參考倉庫與來源文章：\u003Ca href=\"https:\u002F\u002Fgithub.com\u002F\" target=\"_blank\" rel=\"noreferrer\">GitHub\u003C\u002Fa>、\u003Ca href=\"https:\u002F\u002Fwww.cognizant.com\u002Fus\u002Fen\u002Fai-lab\u002Fblog\u002Fovercoming-forgetting-in-llm-fine-tuning\" target=\"_blank\" rel=\"noreferrer\">Cognizant\u003C\u002Fa>。\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>Step 1: 鎖定舊任務基準分數\u003C\u002Fh2>\u003Cp>先做這一步的目的，是建立一份可對照的舊能力清單。你要在任何新微調開始前，先量測模型在保留任務上的表現，這樣後面的分數變化才有意義。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780730281470-8i97.png\" alt=\"如何防止 LLM 微調災難性遺忘\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>把每個任務的指標記下來，像是 accuracy、exact match 或 pass rate，並和 base checkpoint 名稱一起存檔。若你用 HellaSwag，請固定 prompt 格式、解碼參數與 evaluation seed。\u003C\u002Fp>\u003Cp>驗收時，你應該看到多次重跑的分數落在很小的誤差範圍內。若波動很大，先修正評估流程，再碰模型權重。\u003C\u002Fp>\u003Cpre>\u003Ccode>python eval_prior_tasks.py \\\n  --model base_checkpoint \\\n  --tasks hellaswag,heldout_set \\\n  --seed 42 \\\n  --output baseline_metrics.json\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>你應該看到一份名為 \u003Ccode>baseline_metrics.json\u003C\u002Fcode> 的基準結果檔，裡面每個任務都有固定分數。\u003C\u002Fp>\u003Ch2>Step 2: 跑一個短版 ES 微調試驗\u003C\u002Fh2>\u003Cp>這一步的目的，是先拿到一條完整訓練軌跡，而不是只看最後一個 checkpoint。先用小規模 Evolution Strategies 跑新任務，觀察模型是否在早期快速漂移。\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780730285259-kg46.png\" alt=\"如何防止 LLM 微調災難性遺忘\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cpre>\u003Ccode>python train_es.py \\\n  --base-model meta-llama\u002FLlama-3.1-8B-Instruct \\\n  --target-task countdown \\\n  --population-size 64 \\\n  --iterations 300 \\\n  --eval-every 10 \\\n  --save-checkpoints true\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>驗收時，你應該看到每隔固定步數就輸出一次 checkpoint，並且能沿著時間序列比對舊任務分數。\u003C\u002Fp>\u003Cp>如果前期分數先掉、後面又回升，這代表的是暫時漂移，不一定是永久遺忘。\u003C\u002Fp>\u003Ch2>Step 3: 加入 Anchored Weight Decay\u003C\u002Fh2>\u003Cp>這一步的目的，是把更新限制在離起點不要太遠的範圍內。Anchored Weight Decay 會保留一份初始權重副本，並在更新時加入朝向錨點回拉的懲罰項。\u003C\u002Fp>\u003Cp>實作上，你要把初始權重凍結成 anchor，然後在 ES 更新規則裡加上 decay term。這個設計的\u003Ca href=\"\u002Fnews\u002F5-google-home-speaker-details-for-buyers-zh\">重點\u003C\u002Fa>，是\u003Ca href=\"\u002Fnews\u002Fcode2lora-repo-specific-adapters-code-models-en-zh\">讓模型\u003C\u002Fa>優先學新任務，但避免參數走得太遠而洗掉舊技能。\u003C\u002Fp>\u003Cp>驗收時，你應該看到舊任務分數的上下震盪變小。若新任務仍有進步，而舊任務不再大幅崩落，代表錨定策略開始生效。\u003C\u002Fp>\u003Cpre>\u003Ccode>class AnchoredWeightDecay:\n    def __init__(self, anchor_state_dict, lambda_awd=0.01):\n        self.anchor = anchor_state_dict\n        self.lambda_awd = lambda_awd\n\n    def penalty(self, model):\n        total = 0.0\n        for name, param in model.named_parameters():\n            total += (param - self.anchor[name]).pow(2).sum()\n        return self.lambda_awd * total\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>你應該看到訓練損失多出一個 AWD 懲罰項，且舊任務曲線不再一路下滑。\u003C\u002Fp>\u003Ch2>Step 4: 用同一套任務比較 ES 與 GRPO\u003C\u002Fh2>\u003Cp>這一步的目的，是確認遺忘現象是不是只出現在 ES，還是更廣泛的後訓練問題。請用同一個 base model、同一組 prompt 與同一個 \u003Ca href=\"\u002Ftag\u002Fbenchmark\">benchmark\u003C\u002Fa> 排程，重跑 GRPO。\u003C\u002Fp>\u003Cp>要同時記錄新任務進展與舊任務平均分數，重點不只是最後結果，而是訓練過程中的曲線形狀。\u003C\u002Fp>\u003Cp>驗收時，你應該看到 GRPO 也可能讓舊任務掉分，即使漂移型態和 ES 不同。若兩者都會傷到舊能力，問題通常不是換方法就能解決，而是要加強保留機制。\u003C\u002Fp>\u003Cpre>\u003Ccode>python train_grpo.py \\\n  --base-model meta-llama\u002FLlama-3.1-8B-Instruct \\\n  --target-task countdown \\\n  --eval-suite hellaswag,heldout_set \\\n  --save-checkpoints true\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>你應該看到一份對照表，能直接比較 ES 與 GRPO 在新任務與舊任務上的曲線差異。\u003C\u002Fp>\u003Ch2>Step 5: 用漂移檢查挑出最佳 checkpoint\u003C\u002Fh2>\u003Cp>這一步的目的，是建立可上線的挑選規則。請選出同時滿足新任務門檻、且舊任務損失在可接受範圍內的模型版本。\u003C\u002Fp>\u003Cp>在正式保留前，重新跑完整舊任務套件，並和原始 baseline 對照。只有當模型保住你在意的能力，而且漂移在容忍範圍內時，才把這個 checkpoint 視為可用版本。\u003C\u002Fp>\u003Cp>驗收時，你應該看到最終 checkpoint 的舊任務分數接近 baseline，而不是永久掉檔。這就是模型有適應新任務、但沒有發生災難性遺忘的訊號。\u003C\u002Fp>\u003Cpre>\u003Ccode>python select_checkpoint.py \\\n  --metrics-dir runs\u002Fes_awd \\\n  --target-threshold 0.80 \\\n  --max-prior-drop 0.03 \\\n  --output promoted_checkpoint.txt\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>你應該看到一個名為 \u003Ccode>promoted_checkpoint.txt\u003C\u002Fcode> 的產出，裡面只有最後被升級的 checkpoint 路徑。\u003C\u002Fp>\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>指標\u003C\u002Fth>\u003Cth>基準／優化前\u003C\u002Fth>\u003Cth>結果／優化後\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>HellaSwag 舊任務準確率\u003C\u002Ftd>\u003Ctd>Baseline checkpoint\u003C\u002Ftd>\u003Ctd>前期下降 8%，後期回升接近 baseline\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>訓練解讀\u003C\u002Ftd>\u003Ctd>假設是不可逆遺忘\u003C\u002Ftd>\u003Ctd>觀察到的是暫時性參數漂移\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>保留策略\u003C\u002Ftd>\u003Ctd>Plain ES update\u003C\u002Ftd>\u003Ctd>ES + Anchored Weight Decay\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003Ch2>常見錯誤\u003C\u002Fh2>\u003Cul>\u003Cli>只看最後一個 checkpoint。修法：在訓練中途持續記錄舊任務分數，才能抓到暫時漂移與回復。\u003C\u002Fli>\u003Cli>每次跑評估都改 prompt 或 seed。修法：固定 prompt、解碼設定與隨機種子，讓 baseline 可比較。\u003C\u002Fli>\u003Cli>把遺忘只怪在 ES。修法：先用 GRPO 或另一種後訓練方法交叉檢查，再判定問題來源。\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>接下來可以看什麼\u003C\u002Fh2>\u003Cp>如果你已經能穩定保留舊能力，下一步可以把同一套流程擴到多任務持續微調，並為不同領域調整 AWD 強度，再加上 checkpoint 自動回滾規則。\u003C\u002Fp>","用 Anchored Weight Decay 在 LLM 微調時降低舊任務漂移，保住原有能力並檢查模型是否回復。","www.cognizant.com","https:\u002F\u002Fwww.cognizant.com\u002Fus\u002Fen\u002Fai-lab\u002Fblog\u002Fovercoming-forgetting-in-llm-fine-tuning",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780730281470-8i97.png","research","zh","5cf69bca-6c4c-46e0-a4b7-b0a59835c548",[17,18,19,20,21,22],"LLM fine-tuning","catastrophic forgetting","Anchored Weight Decay","Evolution Strategies","GRPO","PyTorch",[24,25,26],"先量測舊任務基準分數，才能判斷後續掉分是不是遺忘。","Anchored Weight Decay 透過錨定初始權重，降低參數漂移。","用持續評估與 checkpoint 挑選規則，保留新能力同時守住舊能力。",0,"2026-06-06T07:17:28.426709+00:00","2026-06-06T07:17:28.421+00:00","0c35a120-52fc-41fc-afa3-d404eb934158",{"tags":32,"relatedLang":45,"relatedPosts":49},[33,35,38,40,43],{"name":18,"slug":34},"catastrophic-forgetting",{"name":36,"slug":37},"anchored weight decay","anchored-weight-decay",{"name":21,"slug":39},"grpo",{"name":41,"slug":42},"evolution strategies","evolution-strategies",{"name":17,"slug":44},"llm-fine-tuning",{"id":15,"slug":46,"title":47,"language":48},"prevent-catastrophic-forgetting-llm-fine-tuning-en","How to Prevent Catastrophic Forgetting in LLM Fine-Tuning","en",[50,56,62,68,74,80],{"id":51,"slug":52,"title":53,"cover_image":54,"image_url":54,"created_at":55,"category":13},"33d21f7f-481d-43d9-9a1c-a2e5badcd84b","bis-stablecoin-usable-buffers-regulation-zh","BIS 把穩定幣規則變成可用緩衝","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780737502594-jk2b.png","2026-06-06T09:17:56.366473+00:00",{"id":57,"slug":58,"title":59,"cover_image":60,"image_url":60,"created_at":61,"category":13},"db0b0788-6656-4a33-b51d-f09e7821af36","code2lora-repo-specific-adapters-code-models-en-zh","Code2LoRA 讓模型學會倉庫脈絡","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780727591717-goec.png","2026-06-06T06:32:45.978642+00:00",{"id":63,"slug":64,"title":65,"cover_image":66,"image_url":66,"created_at":67,"category":13},"47517f79-474e-479b-a1cb-a028e00e01f4","handoff-humanoid-control-planner-friendly-zh","HANDOFF 讓人形機器人更好控","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780726684697-uhrx.png","2026-06-06T06:17:35.215424+00:00",{"id":69,"slug":70,"title":71,"cover_image":72,"image_url":72,"created_at":73,"category":13},"988128e2-d1e8-4643-b565-0bb61202e62e","taillor-protects-key-directions-continual-finetuning-zh","TailLoR 用光譜方向守住舊知識","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780725774632-8seh.png","2026-06-06T06:02:29.73588+00:00",{"id":75,"slug":76,"title":77,"cover_image":78,"image_url":78,"created_at":79,"category":13},"a4cf24e5-b958-4f91-bdca-2f1a57e81aef","why-benchmark-leaderboards-are-wrong-about-model-logic-zh","為什麼基準排行榜看錯了模型邏輯","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780673571153-x7yi.png","2026-06-05T15:32:23.043639+00:00",{"id":81,"slug":82,"title":83,"cover_image":84,"image_url":84,"created_at":85,"category":13},"4a829d2a-24a3-42dd-8be4-49e5ab35435a","why-prompt-engineering-is-wrong-about-2026-zh","為什麼 2026 年 prompt engineering 錯了","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1780661884287-ow45.png","2026-06-05T12:17:19.813402+00:00",[87,92,97,102,107,112,117,122,127,132],{"id":88,"slug":89,"title":90,"created_at":91},"f18dbadb-8c59-4723-84a4-6ad22746c77a","deepmind-bets-on-continuous-learning-ai-2026-zh","DeepMind 押注 2026 連續學習 AI","2026-03-26T08:16:02.367355+00:00",{"id":93,"slug":94,"title":95,"created_at":96},"f4a106cb-02a6-4508-8f39-9720a0a93cee","ml-papers-of-the-week-github-research-desk-zh","每週 ML 論文清單，為何紅到 GitHub","2026-03-27T01:11:39.284175+00:00",{"id":98,"slug":99,"title":100,"created_at":101},"c4f807ca-4e5f-47f1-a48c-961cf3fc44dc","ai-ml-conferences-to-watch-in-2026-zh","2026 AI 研討會投稿時程整理","2026-03-27T01:51:53.874432+00:00",{"id":103,"slug":104,"title":105,"created_at":106},"cf046742-efb2-4753-aef9-caed5da5e32e","adaptive-block-scaled-data-types-zh","IF4：神經網路量化的聰明選擇","2026-03-31T06:00:36.990273+00:00",{"id":108,"slug":109,"title":110,"created_at":111},"53a0dc54-0371-4e40-8d5e-74e94a73840c","geometry-aware-similarity-metrics-for-neural-representations-zh","超越距離測量：用微分幾何重新理解神經網路","2026-03-31T06:01:01.241968+00:00",{"id":113,"slug":114,"title":115,"created_at":116},"fee7d472-a775-4b1d-bbc2-1e8bca1bbf8b","on-the-fly-repulsion-in-the-contextual-space-for-rich-divers-zh","讓AI繪圖更有創意：用排斥力提升生成多樣性","2026-03-31T06:01:25.439673+00:00",{"id":118,"slug":119,"title":120,"created_at":121},"a9901203-d69b-447b-8854-15d14eab32b4","vision-aided-beam-prediction-cnn-eca-zh","影像輔助波束預測升級 CNN","2026-04-01T10:00:25.8073+00:00",{"id":123,"slug":124,"title":125,"created_at":126},"b55e7dd4-0a24-4b3d-804d-b0309a03f498","triple-band-fss-mimo-antenna-sub-6-ghz-zh","三頻 FSS MIMO 天線瞄準 sub-6 GHz","2026-04-01T13:18:36.857305+00:00",{"id":128,"slug":129,"title":130,"created_at":131},"f68290bd-e7f3-4b30-ba22-dcd4e0130a66","openclaw-1299-repos-eight-weeks-analysis-zh","OpenClaw 1299 個 Repo 的資料解讀","2026-04-02T05:03:45.208411+00:00",{"id":133,"slug":134,"title":135,"created_at":136},"ed9f80eb-eb02-4d35-8ad4-0ddf428751dd","beam-coherence-aware-combining-mmwave-mimo-zh","毫米波 MIMO 的雙階合併法","2026-04-02T05:27:26.897188+00:00"]