Rare find

LLMs From Scratch. Learn how ChatGPT-like AI models work by building one yourself, step by step.

amzn.to/4fqvn0D

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

July 2026

Update link checker (#1042) Jul 11
Python 3.14 upgrade and test fixes (#1041) Jul 11

June 2026

remove gate.transpose (#1033) Jun 2

May 2026

Add Deepseek Sparse Attention (DSA) implementation (ch04/09_dsa) (#1014) May 23
Muon optimizer (#1029) May 19
Fix TensorFlow dependency marker for Linux ARM (#1028) May 19
Tips on forking and updating this repo locally (#1027) May 17
Unique names to improve automerge if all tests pass (#1026) May 17
Add cross-layer KV sharing (#1025) May 17
Add troubleshooting guide for common questions (#1024) May 11
Fix typo in README for KV-cache totals (#1023) May 6

April 2026

Fix SWA (#1013) Apr 16
Fix BPE empty-pair edge case in find_freq_pair (#1004) Apr 11
Gemma 4 (#1005) Apr 11
Some gemma 3 improvements (#1000) Apr 6
Remove 7 days requirement to improve windows compatibility (#999) Apr 4
uv packages must be 7 days old (#994) Apr 1

March 2026

Swap urllib.request with requests (#993) Mar 31
fix: pin 1 unpinned action(s) (#987) Mar 26
fix: added KVcache in `generate_text_basic_stream` (#981) Mar 21
harded the link checker Mar 7
Minor typo fix (#974) Mar 7