Build a Small Language Model with DeepMind

Google DeepMind’s curriculum teaches you to build and fine-tune modern language models from the ground up.

This guide is for university students, community learners, and technical builders who already know Python and want a practical path into language-model fundamentals. By the end, you will know what the Google DeepMind AI Research Foundations path includes, how to start it on Google Skills, and how to move through the three courses in order.

Before you start

• A Google account for Google Skills
• Python 3.10+
• Basic comfort with computer science, math, or physics coursework
• A browser with access to the DeepMind collection
• Optional: access to Google Cloud Console if you want to apply skills in cloud workflows

Make sure you can sign in before you begin, because the path is organized around a logged-in learning experience with progress tracking and course access.

Step 1: Open the DeepMind learning path

Goal: reach the official AI Research Foundations collection so you can follow the curriculum in the intended order.

Open the DeepMind collection on Google Skills and review the path title, description, and course list.

You should see the Google DeepMind AI Research Foundations path and a Start path entry point.

Step 2: Start the AI Research Foundations path

Goal: enroll in the curriculum so your learning progress is tracked across all three courses.

Click Start path and sign in if prompted. The path is designed to deepen your understanding of the AI technologies behind models like Gemini and to build practical model-making skills.

You should see the path open with the first course ready to launch.

Step 3: Complete the language-model course sequence

Goal: move through the curriculum in the order Google DeepMind recommends so each concept builds on the last.

Work through these courses in sequence: 01 Build Your Own Small Language Model, 02 Represent Your Language Data, and 03 Design And Train Neural Networks.

You should see each course marked as started or completed before you move to the next one.

Step 4: Apply the concepts in Python

Goal: turn the theory into a hands-on workflow you can practice in your own environment.

As you progress, mirror the lessons in Python by experimenting with tokenization, data representation, and simple neural-network training loops. Keep your examples small so you can focus on the mechanics of language modeling rather than infrastructure.

import torch
import torch.nn as nn

vocab_size = 1000
embed_dim = 128
model = nn.Sequential(
    nn.Embedding(vocab_size, embed_dim),
    nn.Linear(embed_dim, vocab_size)
)
print(model)

You should see a minimal model definition print in your terminal or notebook, which confirms your environment is ready for small-scale experiments.

Step 5: Check your progress and plan the next project

Goal: finish the path with a clear next step, not just a completed badge.

Review your achievements in Google Skills, then decide whether to revisit the courses, apply the ideas in Google Cloud, or extend the exercises into a more complete training workflow.

You should see your completed path and a clearer understanding of how language data, neural networks, and model training fit together.

Common mistakes

• Skipping the sign-in step. Fix: log in with a Google account first so the path and achievements are available.
• Starting course 03 before course 01. Fix: follow the listed order, because each course assumes the prior concepts are already familiar.
• Trying to use large models too early. Fix: begin with small Python examples and simple training loops so you can understand the core mechanics before scaling up.

What’s next

After you finish the path, look for related Google Skills collections on responsible AI, cloud deployment, and practical model evaluation so you can turn the foundation into a broader workflow.