[{"data":1,"prerenderedAt":-1},["ShallowReactive",2],{"article-build-small-language-model-deepmind-en":3,"article-related-build-small-language-model-deepmind-en":30,"series-research-1d80b13f-dbff-4c14-b19f-fb0fdf8ea4b8":77},{"id":4,"slug":5,"title":6,"content":7,"summary":8,"source":9,"source_url":10,"author":11,"image_url":12,"cover_image":12,"category":13,"language":14,"translated_content":11,"related_article_id":15,"keywords":16,"key_takeaways":23,"views":27,"created_at":28,"published_at":29,"topic_cluster_id":11},"1d80b13f-dbff-4c14-b19f-fb0fdf8ea4b8","build-small-language-model-deepmind-en","Build a Small Language Model with DeepMind","\u003Cp data-speakable=\"summary\">\u003Ca href=\"\u002Ftag\u002Fgoogle-deepmind\">Google DeepMind\u003C\u002Fa>’s curriculum teaches you to build and fine-tune modern language models from the ground up.\u003C\u002Fp>\u003Cp>This guide is for university students, community learners, and technical builders who already know Python and want a practical path into language-model fundamentals. By the end, you will know what the \u003Ca href=\"\u002Ftag\u002Fgoogle\">Google\u003C\u002Fa> DeepMind AI Research Foundations path includes, how to start it on Google \u003Ca href=\"\u002Ftag\u002Fskills\">Skills\u003C\u002Fa>, and how to move through the three courses in order.\u003C\u002Fp>\u003Ch2>Before you start\u003C\u002Fh2>\u003Cul>\u003Cli>A Google account for \u003Ca href=\"https:\u002F\u002Fwww.skills.google\" target=\"_blank\" rel=\"noreferrer\">Google Skills\u003C\u002Fa>\u003C\u002Fli>\u003Cli>Python 3.10+\u003C\u002Fli>\u003Cli>Basic comfort with computer science, math, or physics coursework\u003C\u002Fli>\u003Cli>A browser with access to the \u003Ca href=\"https:\u002F\u002Fwww.skills.google\u002Fcollections\u002Fdeepmind\" target=\"_blank\" rel=\"noreferrer\">DeepMind collection\u003C\u002Fa>\u003C\u002Fli>\u003Cli>Optional: access to Google Cloud Console if you want to apply skills in cloud workflows\u003C\u002Fli>\u003C\u002Ful>\u003Cp>Make sure you can sign in before you begin, because the path is organized around a logged-in learning experience with progress tracking and course access.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785522764710-tchg.png\" alt=\"Build a Small Language Model with DeepMind\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Ch2>Step 1: Open the DeepMind learning path\u003C\u002Fh2>\u003Cp>Goal: reach the official AI Research Foundations collection so you can follow the curriculum in the intended order.\u003C\u002Fp>\u003Cp>Open the \u003Ca href=\"https:\u002F\u002Fwww.skills.google\u002Fcollections\u002Fdeepmind\" target=\"_blank\" rel=\"noreferrer\">DeepMind collection on Google Skills\u003C\u002Fa> and review the path title, description, and course list.\u003C\u002Fp>\u003Cp>You should see the Google DeepMind AI Research Foundations path and a \u003Cstrong>Start path\u003C\u002Fstrong> entry point.\u003C\u002Fp>\u003Ch2>Step 2: Start the AI Research Foundations path\u003C\u002Fh2>\u003Cp>Goal: enroll in the curriculum so your learning progress is tracked across all three courses.\u003C\u002Fp>\n\u003Cfigure class=\"my-6\">\u003Cimg src=\"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785522763159-oxzm.png\" alt=\"Build a Small Language Model with DeepMind\" class=\"rounded-xl w-full\" loading=\"lazy\" \u002F>\u003C\u002Ffigure>\n\u003Cp>Click \u003Cstrong>Start path\u003C\u002Fstrong> and sign in if prompted. The path is designed to deepen your understanding of the AI technologies behind models like \u003Ca href=\"\u002Ftag\u002Fgemini\">Gemini\u003C\u002Fa> and to build practical model-making skills.\u003C\u002Fp>\u003Cp>You should see the path open with the first course ready to launch.\u003C\u002Fp>\u003Ch2>Step 3: Complete the language-model course sequence\u003C\u002Fh2>\u003Cp>Goal: move through the curriculum in the order Google DeepMind recommends so each concept builds on the last.\u003C\u002Fp>\u003Cp>Work through these courses in sequence: \u003Cstrong>01 Build Your Own Small Language Model\u003C\u002Fstrong>, \u003Cstrong>02 Represent Your Language Data\u003C\u002Fstrong>, and \u003Cstrong>03 Design And Train Neural Networks\u003C\u002Fstrong>.\u003C\u002Fp>\u003Cp>You should see each course marked as started or completed before you move to the next one.\u003C\u002Fp>\u003Ch2>Step 4: Apply the concepts in Python\u003C\u002Fh2>\u003Cp>Goal: turn the theory into a hands-on workflow you can practice in your own environment.\u003C\u002Fp>\u003Cp>As you progress, mirror the lessons in Python by experimenting with tokenization, data representation, and simple neural-network training loops. Keep your examples small so you can focus on the mechanics of language modeling rather than infrastructure.\u003C\u002Fp>\u003Cpre>\u003Ccode>import torch\nimport torch.nn as nn\n\nvocab_size = 1000\nembed_dim = 128\nmodel = nn.Sequential(\n    nn.Embedding(vocab_size, embed_dim),\n    nn.Linear(embed_dim, vocab_size)\n)\nprint(model)\u003C\u002Fcode>\u003C\u002Fpre>\u003Cp>You should see a minimal model definition print in your terminal or notebook, which confirms your environment is ready for small-scale experiments.\u003C\u002Fp>\u003Ch2>Step 5: Check your progress and plan the next project\u003C\u002Fh2>\u003Cp>Goal: finish the path with a clear next step, not just a completed badge.\u003C\u002Fp>\u003Cp>Review your achievements in Google Skills, then decide whether to revisit the courses, apply the ideas in \u003Ca href=\"\u002Ftag\u002Fgoogle-cloud\">Google Cloud\u003C\u002Fa>, or extend the exercises into a more complete training workflow.\u003C\u002Fp>\u003Cp>You should see your completed path and a clearer understanding of how language data, neural networks, and model training fit together.\u003C\u002Fp>\u003Ctable>\u003Cthead>\u003Ctr>\u003Cth>Metric\u003C\u002Fth>\u003Cth>Before\u002FBaseline\u003C\u002Fth>\u003Cth>After\u002FResult\u003C\u002Fth>\u003C\u002Ftr>\u003C\u002Fthead>\u003Ctbody>\u003Ctr>\u003Ctd>Course sequence\u003C\u002Ftd>\u003Ctd>Unstructured self-study\u003C\u002Ftd>\u003Ctd>Three ordered DeepMind courses\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Skill focus\u003C\u002Ftd>\u003Ctd>General AI interest\u003C\u002Ftd>\u003Ctd>Language models, data representation, neural network design\u003C\u002Ftd>\u003C\u002Ftr>\u003Ctr>\u003Ctd>Audience fit\u003C\u002Ftd>\u003Ctd>Any learner\u003C\u002Ftd>\u003Ctd>Python-proficient students and technical community learners\u003C\u002Ftd>\u003C\u002Ftr>\u003C\u002Ftbody>\u003C\u002Ftable>\u003Ch2>Common mistakes\u003C\u002Fh2>\u003Cul>\u003Cli>\u003Cstrong>Skipping the sign-in step.\u003C\u002Fstrong> Fix: log in with a Google account first so the path and achievements are available.\u003C\u002Fli>\u003Cli>\u003Cstrong>Starting course 03 before course 01.\u003C\u002Fstrong> Fix: follow the listed order, because each course assumes the prior concepts are already familiar.\u003C\u002Fli>\u003Cli>\u003Cstrong>Trying to use large models too early.\u003C\u002Fstrong> Fix: begin with small Python examples and simple training loops so you can understand the core mechanics before scaling up.\u003C\u002Fli>\u003C\u002Ful>\u003Ch2>What’s next\u003C\u002Fh2>\u003Cp>After you finish the path, look for related Google Skills collections on responsible AI, cloud deployment, and practical model evaluation so you can turn the foundation into a broader workflow.\u003C\u002Fp>","Google DeepMind’s AI Research Foundations teaches you to build and fine-tune modern language models from the ground up.","www.skills.google","https:\u002F\u002Fwww.skills.google\u002Fcollections\u002Fdeepmind",null,"https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785522764710-tchg.png","research","en","595ebbe1-bb81-4b4f-8f42-43a1b3820542",[17,18,19,20,21,22],"Google Skills","DeepMind","language models","Python","neural networks","AI research foundations",[24,25,26],"The DeepMind collection is a structured path, not a single course.","The curriculum is aimed at Python-proficient technical learners.","The three-course sequence covers small language models, data representation, and neural network training.",1,"2026-07-31T18:32:18.121005+00:00","2026-07-31T18:32:18.11+00:00",{"tags":31,"relatedLang":36,"relatedPosts":40},[32,34],{"name":20,"slug":33},"python",{"name":19,"slug":35},"language-models",{"id":15,"slug":37,"title":38,"language":39},"build-small-language-model-deepmind-zh","用 DeepMind 做出小型語言模型","zh",[41,47,53,59,65,71],{"id":42,"slug":43,"title":44,"cover_image":45,"image_url":45,"created_at":46,"category":13},"00655d82-4035-4ac8-8c48-ed070e82f4a5","pac-man-humanoid-dodgeball-safety-en","PAC-MAN makes humanoid dodgeball safer","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785481389808-eysc.png","2026-07-31T07:02:32.928022+00:00",{"id":48,"slug":49,"title":50,"cover_image":51,"image_url":51,"created_at":52,"category":13},"2844c399-e2ad-4cf0-b888-fd480d6d56ed","retoken-one-token-visual-retrieval-en","ReToken uses one learned token to fetch visual context","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785479567286-85y5.png","2026-07-31T06:32:24.643191+00:00",{"id":54,"slug":55,"title":56,"cover_image":57,"image_url":57,"created_at":58,"category":13},"660b1f2e-8988-4d2e-9429-b0c7accd3b6a","ml-trace-seiberg-dualities-en","ML helps trace Seiberg dualities","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785477772867-xtol.png","2026-07-31T06:02:28.714509+00:00",{"id":60,"slug":61,"title":62,"cover_image":63,"image_url":63,"created_at":64,"category":13},"8e53848b-d163-4335-a8e4-29694e85bbb3","fruitfly-inspired-regression-without-heavy-models-en","Fruitfly-Inspired Regression Without Heavy Models","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785394974203-76p6.png","2026-07-30T07:02:27.641563+00:00",{"id":66,"slug":67,"title":68,"cover_image":69,"image_url":69,"created_at":70,"category":13},"2515b20a-f125-4354-b386-50e75eff70c4","mental-world-modeling-simulating-minds-en","Mental World Modeling: Simulating minds, not just scenes","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785393182740-cnwi.png","2026-07-30T06:32:29.590725+00:00",{"id":72,"slug":73,"title":74,"cover_image":75,"image_url":75,"created_at":76,"category":13},"f234ef6f-2934-4e01-bc50-3132313c0d7a","pretrain-q-functions-online-rl-finetuning-en","Do You Need to Pretrain Q-Functions?","https:\u002F\u002Fxxdpdyhzhpamafnrdkyq.supabase.co\u002Fstorage\u002Fv1\u002Fobject\u002Fpublic\u002Fcovers\u002Finline-1785391372788-de5m.png","2026-07-30T06:02:24.317984+00:00",[78,83,88,93,98,103,108,113,118,123],{"id":79,"slug":80,"title":81,"created_at":82},"a2715e72-1fe8-41b3-abb1-d0cf1f710189","ai-predictions-2026-big-changes-en","AI Predictions for 2026: Brace for Big Changes","2026-03-26T01:25:07.788356+00:00",{"id":84,"slug":85,"title":86,"created_at":87},"8404bd7b-4c2f-4109-9ec4-baf29d88af2b","ml-papers-of-the-week-github-research-desk-en","ML Papers of the Week Turns GitHub Into a Research Desk","2026-03-27T01:11:39.480259+00:00",{"id":89,"slug":90,"title":91,"created_at":92},"87897a94-8065-4464-a016-1f23e89e17cc","ai-ml-conferences-to-watch-in-2026-en","AI\u002FML Conferences to Watch in 2026","2026-03-27T01:51:54.184108+00:00",{"id":94,"slug":95,"title":96,"created_at":97},"6f1987cf-25f3-47a4-b3e6-db0997695be8","openclaw-agents-manipulated-self-sabotage-en","OpenClaw Agents Can Be Manipulated Into Failure","2026-03-28T03:03:18.899465+00:00",{"id":99,"slug":100,"title":101,"created_at":102},"a53571ad-735a-4178-9f93-cb09b699d99c","vega-driving-language-instructions-en","Vega: Driving with Natural Language Instructions","2026-03-28T14:54:04.698882+00:00",{"id":104,"slug":105,"title":106,"created_at":107},"a34581d6-f36e-46da-88bb-582fb3e7425c","personalizing-autonomous-driving-styles-en","Drive My Way: Personalizing Autonomous Driving Styles","2026-03-28T14:54:26.148181+00:00",{"id":109,"slug":110,"title":111,"created_at":112},"2bc1ad7f-26ce-4f02-9885-803b35fd229d","training-knowledge-bases-writeback-rag-en","Training Knowledge Bases with WriteBack-RAG","2026-03-28T14:54:45.643433+00:00",{"id":114,"slug":115,"title":116,"created_at":117},"71adc507-3c54-4605-bbe2-c966acd6187e","packforcing-long-video-generation-en","PackForcing: Efficient Long-Video Generation Method","2026-03-28T14:55:02.646943+00:00",{"id":119,"slug":120,"title":121,"created_at":122},"675942ef-b9ec-4c5f-a997-381250b6eacb","pixelsmile-facial-expression-editing-en","PixelSmile Framework Enhances Facial Expression Editing","2026-03-28T14:55:20.633463+00:00",{"id":124,"slug":125,"title":126,"created_at":127},"6954fa2b-8b66-4839-884b-e46f89fa1bc3","adaptive-block-scaled-data-types-en","IF4: Smarter 4-Bit Quantization That Adapts to Your Data","2026-03-31T06:00:36.65963+00:00"]