---
title: "Proof-based linear algebra for deep learning"
lesson_id: "01"
---

# Proof-based linear algebra for deep learning

- **Lesson ID:** 01
- **Goal:** You do not need an 800B pretrain to see the shape of the problem. You do need the linear maps that every later replica sits on.
- **Human lesson:** [01-proof-based-linear-algebra.html](01-proof-based-linear-algebra.html)
- **Phase:** Phase 1 Foundations

## Prerequisites

- The previous week's artifact, or a written note if this is week 1.
- Python 3.11+ and a laptop. CUDA is useful from week 5 onward and not required to read.

## Inputs, outputs, and artifacts

- **Inputs:** The replica from the prior week.
- **Outputs:** Core project 1 starts here: a linear algebra and tiny autograd lab.
- **Artifacts:** Lesson notes plus the code named in the human page.

## Agent build steps

1. State the week 1 principle: you do not need to pretrain an 800B model to understand the shape of the problem; you do need progressively realistic replicas.
2. Teach the topics on the human page without inventing paper URLs or news claims.
3. Keep starter code in fenced blocks that match the human lesson.
4. End with the assignment and the opinion checkpoint.
5. Use Edge FDE tone: ship, do not sightseeing. Opinion checkpoints matter.

## Constraints

Plain spoken English. No quizzes. No em dashes. No invented metrics, dates, or citations. Brand Edge FDE only. Name well-known papers by title only: Attention Is All You Need, InstructGPT, Direct Preference Optimization, ZeRO, PagedAttention/vLLM, FlashAttention, Scaling Laws for Neural Language Models.

## Key concepts

- A network layer is a linear map plus a cheap nonlinearity.
- Inner products and norms turn geometry into the numbers you log.
- SVD is the honest way to talk about rank and compression.
- Backprop is Jacobian-vector products, not a framework feature.
- You do not need an 800B pretrain to learn the shape of the stack.

## Takeaways

- Prove one projection identity and implement it. That is the week.
- Condition numbers explain more training instability than lore does.
- Keep a written opinion: replicas beat tourism.

## Acceptance checks

- Human HTML has Key concepts and Takeaways.
- Opinion checkpoint is present.
- No em dash and no fake URL.
- [ ] Proceed to [lesson 02 brief](02-non-convex-optimisation.llms.md).