<p align="center">
<img src="assets/banner.svg" alt="AI Engineering from Scratch" width="100%">
</p>
<p align="center">
<a href="LICENSE"><img src="https://img.shields.io/badge/License-MIT-blue.svg" alt="License: MIT"></a>
<a href="CONTRIBUTING.md"><img src="https://img.shields.io/badge/PRs-welcome-brightgreen.svg" alt="PRs Welcome"></a>
<img src="https://img.shields.io/badge/Lessons-260+-D97757" alt="260+ Lessons">
<img src="https://img.shields.io/badge/Phases-20-191A23" alt="20 Phases">
<img src="https://img.shields.io/badge/Complete-68-3D8B6E" alt="68 Complete">
<img src="https://img.shields.io/github/stars/rohitg00/ai-engineering-from-scratch?style=social" alt="GitHub Stars">
</p>
<p align="center">
<a href="#the-journey">Journey</a> •
<a href="#-ai-native-learning">AI-Native</a> •
<a href="#getting-started">Get Started</a> •
<a href="#course-output-the-toolkit">Toolkit</a> •
<a href="ROADMAP.md">Roadmap</a> •
<a href="CONTRIBUTING.md">Contribute</a> •
<a href="glossary/terms.md">Glossary</a>
</p>
84% of students already use AI tools. Only 18% feel prepared to use them professionally. This course closes that gap.
260+ lessons. 20 phases. ~290 hours. From linear algebra to autonomous agent swarms. Python, TypeScript, Rust, Julia. Every lesson produces something reusable -- prompts, skills, agents, MCP servers.
You don't just learn AI. You learn AI with AI. Then you build real things. Then you ship tools others can use.
| Traditional Courses | This Course |
|---|
| Scope | One slice (NLP or Vision or Agents) | Everything: math, ML, DL, NLP, vision, speech, transformers, LLMs, agents, swarms |
| Languages | Python only | Python, TypeScript, Rust, Julia |
| Output | "I learned something" | A portfolio of tools, prompts, skills, and agents you can install |
| Depth | Surface-level or theory-heavy | Build from scratch first, then use frameworks |
| Format | Videos you watch | Runnable code + docs + web app + AI-powered quizzes |
| Learning style | Passive consumption | AI-native: use Claude Code skills to test yourself as you go |
š§ AI-Native Learning
This isn't a course you watch. It's a course you use with your AI coding agent.
Learn with AI, not just about AI
# Find where to start based on what you already know
/find-your-level
# Quiz yourself after completing a phase
/check-understanding 3
# Every lesson produces a reusable artifact
ls phases/03-deep-learning-core/05-loss-functions/outputs/
# prompt-loss-function-selector.md
# prompt-loss-debugger.md
Built-in Claude Code Skills
| Skill | What it does |
|---|
/find-your-level | 10-question quiz that maps your knowledge to a starting phase and builds a personalized path with hour estimates |
/check-understanding <phase> | Per-phase quiz (8 questions) with feedback and specific lessons to review |
Every Lesson Ships Something
Other courses end with "congratulations, you learned X." Our lessons end with a reusable tool:
- Prompts -- paste into any AI assistant to get expert-level help on the topic
- Skills -- install into Claude Code, Cursor, or any coding agent
- Agents -- deploy as autonomous workers
- MCP servers -- plug into any MCP-compatible AI app
277-term searchable glossary. Full lesson catalog. ~290 hours of content with per-lesson time estimates. Browse the website ā
The Journey
<table>
<tr><td>
Phase 0: Setup & Tooling 12 lessons
Get your environment ready for everything that follows.
</td></tr>
</table>
<details id="phase-1">
<summary><strong>Phase 1: Math Foundations</strong> <code>22 lessons</code> <em>The intuition behind every AI algorithm, through code.</em></summary>
</details>
<details id="phase-2">
<summary><strong>Phase 2: ML Fundamentals</strong> <code>18 lessons</code> <em>Classical ML - still the backbone of most production AI.</em></summary>
</details>
<details id="phase-3">
<summary><strong>Phase 3: Deep Learning Core</strong> <code>13 lessons</code> <em>Neural networks from first principles. No frameworks until you build one.</em></summary>
</details>
<details id="phase-4">
<summary><strong>Phase 4: Computer Vision</strong> <code>16 lessons</code> <em>From pixels to understanding - image, video, and 3D.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Image Fundamentals: Pixels, Channels, Color Spaces | Learn | Python |
| 02 | Convolutions from Scratch | Build | Python |
| 03 | CNNs: LeNet to ResNet | Build | Python |
| 04 | Image Classification | Build | Python |
| 05 | Transfer Learning & Fine-Tuning | Build | Python |
| 06 | Object Detection -YOLO from Scratch | Build | Python |
| 07 | Semantic Segmentation -U-Net | Build | Python |
| 08 | Instance Segmentation -Mask R-CNN | Build | Python |
| 09 | Image Generation -GANs | Build | Python |
| 10 | Image Generation -Diffusion Models | Build | Python |
| 11 | Stable Diffusion -Architecture & Fine-Tuning | Build | Python |
| 12 | Video Understanding -Temporal Modeling | Build | Python |
| 13 | 3D Vision: Point Clouds, NeRFs | Build | Python |
| 14 | Vision Transformers (ViT) | Build | Python |
| 15 | Real-Time Vision: Edge Deployment | Build | Python, Rust |
| 16 | Build a Complete Vision Pipeline | Build | Python |
</details>
<details id="phase-5">
<summary><strong>Phase 5: NLP: Foundations to Advanced</strong> <code>18 lessons</code> <em>Language is the interface to intelligence.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Text Processing: Tokenization, Stemming, Lemmatization | Build | Python |
| 02 | Bag of Words, TF-IDF & Text Representation | Build | Python |
| 03 | Word Embeddings: Word2Vec from Scratch | Build | Python |
| 04 | GloVe, FastText & Subword Embeddings | Build | Python |
| 05 | Sentiment Analysis | Build | Python |
| 06 | Named Entity Recognition (NER) | Build | Python |
| 07 | POS Tagging & Syntactic Parsing | Build | Python |
| 08 | Text Classification -CNNs & RNNs for Text | Build | Python |
| 09 | Sequence-to-Sequence Models | Build | Python |
| 10 | Attention Mechanism -The Breakthrough | Build | Python |
| 11 | Machine Translation | Build | Python |
| 12 | Text Summarization | Build | Python |
| 13 | Question Answering Systems | Build | Python |
| 14 | Information Retrieval & Search | Build | Python |
| 15 | Topic Modeling: LDA, BERTopic | Build | Python |
| 16 | Text Generation | Build | Python |
| 17 | Chatbots: Rule-Based to Neural | Build | Python |
| 18 | Multilingual NLP | Build | Python |
</details>
<details id="phase-6">
<summary><strong>Phase 6: Speech & Audio</strong> <code>12 lessons</code> <em>Hear, understand, speak.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Audio Fundamentals: Waveforms, Sampling, FFT | Learn | Python |
| 02 | Spectrograms, Mel Scale & Audio Features | Build | Python |
| 03 | Audio Classification | Build | Python |
| 04 | Speech Recognition (ASR) | Build | Python |
| 05 | Whisper: Architecture & Fine-Tuning | Build | Python |
| 06 | Speaker Recognition & Verification | Build | Python |
| 07 | Text-to-Speech (TTS) | Build | Python |
| 08 | Voice Cloning & Voice Conversion | Build | Python |
| 09 | Music Generation | Build | Python |
| 10 | Audio-Language Models | Build | Python |
| 11 | Real-Time Audio Processing | Build | Python, Rust |
| 12 | Build a Voice Assistant Pipeline | Build | Python |
</details>
<details id="phase-7">
<summary><strong>Phase 7: Transformers Deep Dive</strong> <code>14 lessons</code> <em>The architecture that changed everything.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Why Transformers: The Problems with RNNs | Learn | -- |
| 02 | Self-Attention from Scratch | Build | Python |
| 03 | Multi-Head Attention | Build | Python |
| 04 | Positional Encoding: Sinusoidal, RoPE, ALiBi | Build | Python |
| 05 | The Full Transformer: Encoder + Decoder | Build | Python |
| 06 | BERT -Masked Language Modeling | Build | Python |
| 07 | GPT -Causal Language Modeling | Build | Python |
| 08 | T5, BART -Encoder-Decoder Models | Build | Python |
| 09 | Vision Transformers (ViT) | Build | Python |
| 10 | Audio Transformers -Whisper Architecture | Build | Python |
| 11 | Mixture of Experts (MoE) | Build | Python |
| 12 | KV Cache, Flash Attention & Inference Optimization | Build | Python, Rust |
| 13 | Scaling Laws | Learn | Python |
| 14 | Build a Transformer from Scratch | Build | Python |
</details>
<details id="phase-8">
<summary><strong>Phase 8: Generative AI</strong> <code>14 lessons</code> <em>Create images, video, audio, 3D, and more.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Generative Models: Taxonomy & History | Learn | -- |
| 02 | Autoencoders & VAE | Build | Python |
| 03 | GANs: Generator vs Discriminator | Build | Python |
| 04 | Conditional GANs & Pix2Pix | Build | Python |
| 05 | StyleGAN | Build | Python |
| 06 | Diffusion Models -DDPM from Scratch | Build | Python |
| 07 | Latent Diffusion & Stable Diffusion | Build | Python |
| 08 | ControlNet, LoRA & Conditioning | Build | Python |
| 09 | Inpainting, Outpainting & Editing | Build | Python |
| 10 | Video Generation | Build | Python |
| 11 | Audio Generation | Build | Python |
| 12 | 3D Generation | Build | Python |
| 13 | Flow Matching & Rectified Flows | Build | Python |
| 14 | Evaluation: FID, CLIP Score | Build | Python |
</details>
<details id="phase-9">
<summary><strong>Phase 9: Reinforcement Learning</strong> <code>12 lessons</code> <em>The foundation of RLHF and game-playing AI.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | MDPs, States, Actions & Rewards | Learn | Python |
| 02 | Dynamic Programming | Build | Python |
| 03 | Monte Carlo Methods | Build | Python |
| 04 | Q-Learning, SARSA | Build | Python |
| 05 | Deep Q-Networks (DQN) | Build | Python |
| 06 | Policy Gradients -REINFORCE | Build | Python |
| 07 | Actor-Critic -A2C, A3C | Build | Python |
| 08 | PPO | Build | Python |
| 09 | Reward Modeling & RLHF | Build | Python |
| 10 | Multi-Agent RL | Build | Python |
| 11 | Sim-to-Real Transfer | Build | Python |
| 12 | RL for Games | Build | Python |
</details>
<details id="phase-10">
<summary><strong>Phase 10: LLMs from Scratch</strong> <code>14 lessons</code> <em>Build, train, and understand large language models.</em></summary>
</details>
<details id="phase-11">
<summary><strong>Phase 11: LLM Engineering</strong> <code>13 lessons</code> <em>Put LLMs to work in production.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Prompt Engineering: Techniques & Patterns | Build | Python |
| 02 | Few-Shot, CoT, Tree-of-Thought | Build | Python |
| 03 | Structured Outputs | Build | Python, TS |
| 04 | Embeddings & Vector Representations | Build | Python |
| 05 | Context Engineering | Build | Python, TS |
| 06 | RAG -Retrieval-Augmented Generation | Build | Python, TS |
| 07 | Advanced RAG -Chunking, Reranking | Build | Python |
| 08 | Fine-Tuning with LoRA & QLoRA | Build | Python |
| 09 | Function Calling & Tool Use | Build | Python, TS |
| 10 | Evaluation & Testing | Build | Python |
| 11 | Caching, Rate Limiting & Cost | Build | Python, TS |
| 12 | Guardrails & Safety | Build | Python |
| 13 | Building a Production LLM App | Build | Python, TS |
</details>
<details id="phase-12">
<summary><strong>Phase 12: Multimodal AI</strong> <code>11 lessons</code> <em>See, hear, read, and reason across modalities.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Multimodal Representations | Learn | -- |
| 02 | CLIP: Vision + Language | Build | Python |
| 03 | Vision-Language Models | Build | Python |
| 04 | Audio-Language Models | Build | Python |
| 05 | Document Understanding | Build | Python |
| 06 | Video-Language Models | Build | Python |
| 07 | Multimodal RAG | Build | Python, TS |
| 08 | Multimodal Agents | Build | Python, TS |
| 09 | Text-to-Image Pipelines | Build | Python |
| 10 | Text-to-Video Pipelines | Build | Python |
| 11 | Any-to-Any Models | Learn | Python |
</details>
<details id="phase-13">
<summary><strong>Phase 13: Tools & Protocols</strong> <code>10 lessons</code> <em>The interfaces between AI and the real world.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Function Calling Deep Dive | Build | Python, TS |
| 02 | Tool Use Patterns | Build | TS |
| 03 | MCP: Model Context Protocol | Learn | -- |
| 04 | Building MCP Servers | Build | TS, Python |
| 05 | Building MCP Clients | Build | TS, Python |
| 06 | MCP Resources, Prompts & Sampling | Build | TS |
| 07 | Structured Output Schemas | Build | TS, Python |
| 08 | API Design for AI | Build | TS |
| 09 | Browser Automation & Web Agents | Build | TS |
| 10 | Build a Complete Tool Ecosystem | Build | TS, Python |
</details>
<details id="phase-14">
<summary><strong>Phase 14: Agent Engineering</strong> <code>15 lessons</code> <em>Build agents from first principles.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | The Agent Loop | Build | Python, TS |
| 02 | Tool Dispatch & Registration | Build | TS |
| 03 | Planning: TodoWrite, DAGs | Build | TS |
| 04 | Memory: Short-Term, Long-Term, Episodic | Build | TS, Python |
| 05 | Context Window Management | Build | TS |
| 06 | Context Compression & Summarization | Build | TS |
| 07 | Subagents: Delegation | Build | TS |
| 08 | Skills & Knowledge Loading | Build | TS |
| 09 | Permissions, Sandboxing & Safety | Build | TS, Rust |
| 10 | File-Based Task Systems | Build | TS |
| 11 | Background Task Execution | Build | TS |
| 12 | Error Recovery & Self-Healing | Build | TS |
| 13 | Hooks: PreToolUse, PostToolUse | Build | TS |
| 14 | Eval-Driven Agent Development | Build | Python, TS |
| 15 | Build a Complete AI Agent | Build | TS |
</details>
<details id="phase-15">
<summary><strong>Phase 15: Autonomous Systems</strong> <code>11 lessons</code> <em>Agents that run without human intervention safely.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | What Makes a System Autonomous | Learn | -- |
| 02 | Autonomous Loops | Build | TS, Python |
| 03 | Self-Healing Agents | Build | TS |
| 04 | AutoResearch: Autonomous Research | Build | TS, Python |
| 05 | Eval-Driven Loops | Build | TS |
| 06 | Human-in-the-Loop | Build | TS |
| 07 | Continuous Agents | Build | TS |
| 08 | Cost-Aware Autonomous Systems | Build | TS |
| 09 | Monitoring & Observability | Build | TS, Rust |
| 10 | Safety Boundaries | Build | TS |
| 11 | Build an Autonomous Coding Agent | Build | TS |
</details>
<details id="phase-16">
<summary><strong>Phase 16: Multi-Agent & Swarms</strong> <code>14 lessons</code> <em>Coordination, emergence, and collective intelligence.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Why Multi-Agent | Learn | -- |
| 02 | Agent Teams: Roles & Delegation | Build | TS |
| 03 | Communication Protocols | Build | TS |
| 04 | Shared State & Coordination | Build | TS, Rust |
| 05 | Message Passing & Mailboxes | Build | TS |
| 06 | Task Markets | Build | TS |
| 07 | Consensus Algorithms | Build | TS, Rust |
| 08 | Swarm Intelligence | Build | Python, TS |
| 09 | Agent Economies | Build | TS |
| 10 | Worktree Isolation | Build | TS |
| 11 | Hierarchical Swarms | Build | TS |
| 12 | Self-Organizing Systems | Build | TS, Rust |
| 13 | DAG-Based Orchestration | Build | TS, Rust |
| 14 | Build an Autonomous Swarm | Build | TS, Rust |
</details>
<details id="phase-17">
<summary><strong>Phase 17: Infrastructure & Production</strong> <code>11 lessons</code> <em>Ship AI to the real world.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | Model Serving | Build | Python |
| 02 | Docker for AI Workloads | Build | Python, Rust |
| 03 | Kubernetes for AI | Build | Python |
| 04 | Edge Deployment: ONNX, WASM | Build | Python, Rust |
| 05 | Observability | Build | TS, Rust |
| 06 | Cost Optimization | Build | TS |
| 07 | CI/CD for ML | Build | Python |
| 08 | A/B Testing & Feature Flags | Build | Python, TS |
| 09 | Data Pipelines | Build | Python, Rust |
| 10 | Security: Red Teaming, Defense | Build | Python, TS |
| 11 | Build a Production AI Platform | Build | Python, TS, Rust |
</details>
<details id="phase-18">
<summary><strong>Phase 18: Ethics, Safety & Alignment</strong> <code>6 lessons</code> <em>Build AI that helps humanity. Not optional.</em></summary>
| # | Lesson | Type | Lang |
|---|
| 01 | AI Ethics: Bias, Fairness | Learn | -- |
| 02 | Alignment: What & Why | Learn | -- |
| 03 | Red Teaming & Adversarial Testing | Build | Python |
| 04 | Responsible AI Frameworks | Learn | -- |
| 05 | Privacy: Differential Privacy, FL | Build | Python |
| 06 | Interpretability: SHAP, Attention | Build | Python |
</details>
<details id="phase-19">
<summary><strong>Phase 19: Capstone Projects</strong> <code>5 projects</code> <em>Prove everything you learned.</em></summary>
| # | Project | Combines | Lang |
|---|
| 01 | Build a Mini GPT & Chat Interface | Phases 1, 3, 7, 10 | Python, TS |
| 02 | Build a Multimodal RAG System | Phases 5, 11, 12, 13 | Python, TS |
| 03 | Build an Autonomous Research Agent | Phases 14, 15, 6 | TS, Python |
| 04 | Build a Multi-Agent Dev Team | Phases 14, 15, 16, 17 | TS, Rust |
| 05 | Build a Production AI Platform | All phases | Python, TS, Rust |
</details>
Course Output: The Toolkit
Other courses give you a certificate. This one gives you a toolkit.
Every lesson produces a reusable artifact -- a prompt, skill, agent, or MCP server that you can install and use immediately. By the end of the course you have:
outputs/
āāā prompts/ Prompt templates for every AI task
āāā skills/ SKILL.md files for AI coding agents
āāā agents/ Agent definitions ready to deploy
āāā mcp-servers/ MCP servers you built during the course
Install them with SkillKit. Plug them into Claude Code, Cursor, or any AI agent. These are real tools, not homework.
How Each Lesson Works
phases/XX-phase-name/NN-lesson-name/
āāā code/ Runnable implementations (Python, TS, Rust, Julia)
āāā docs/
ā āāā en.md Lesson documentation
āāā outputs/ Prompts, skills, agents produced by this lesson
Every lesson follows 6 steps:
| Step | What happens |
|---|
| Motto | One-line core idea that sticks |
| Problem | A concrete scenario where not knowing this hurts |
| Concept | Mermaid diagrams and intuition -- no code yet |
| Build It | Implement from scratch in pure Python. No frameworks. |
| Use It | Same thing with PyTorch, sklearn, or the real tool |
| Ship It | The prompt, skill, or agent this lesson produces |
The Build It / Use It split is the key. You understand what the framework does because you built it yourself first.
Getting Started
Option A: Just start reading
Pick any completed lesson from the website or the phase tables below.
Option B: Clone and run
git clone https://github.com/rohitg00/ai-engineering-from-scratch.git
cd ai-engineering-from-scratch
python phases/01-math-foundations/01-linear-algebra-intuition/code/vectors.py
Option C: Find your level (recommended)
If you already know some ML/DL, don't start from Phase 1. Use the built-in assessment:
# In Claude Code:
/find-your-level
This 10-question quiz maps your knowledge to a starting phase and builds a personalized path with hour estimates.
Prerequisites
- You can write code (Python or any language)
- You want to understand how AI actually works, not just call APIs
Who This Is For
| You are... | Start at... | Time to complete |
|---|
| New to programming + AI | Phase 0 (Setup) | ~290 hours |
| Know Python, new to ML | Phase 1 (Math) | ~270 hours |
| Know ML, new to DL | Phase 3 (Deep Learning) | ~200 hours |
| Know DL, want LLMs/agents | Phase 10 (LLMs from Scratch) | ~100 hours |
| Senior engineer, want agents only | Phase 14 (Agent Engineering) | ~60 hours |
Contributing
See CONTRIBUTING.md for how to add lessons, translations, and outputs.
Want to fork this for your team or school? See FORKING.md.
See ROADMAP.md for progress tracking (~290 hours, per-lesson time estimates).
<p align="center">
MIT License. Use it however you want.
</p>