Rare find

NanoGPT Lecture. Learn how to build a working AI language model from scratch.

github.com/karpathy/ng-video-lecture

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

February 2023

Merge pull request #9 from Andrei-Aksionov/feature/shape_description_fix Feb 7
Clarify shape descriptions inside forward method Feb 7
adding this even though the first gpt video didn't cover it, because … Feb 6

January 2023

soften the wording a bit to not scare people too much Jan 31
attach a note about init Jan 31
Merge branch 'master' of https://github.com/karpathy/ng-video-lecture Jan 31
rename the class it's a remnant of the past Jan 31
Merge pull request #6 from vineetm/master Jan 27
scale by head size Jan 25
small readme file Jan 17
rename v2 to gpt hah Jan 17
might as well add these too Jan 17
parameter count Jan 16
add dropout, slight refactoring of variables Jan 16
and add layernorms, down to 2.06 Jan 16
introduce the residual block Jan 16
add a computation block, down to 2.24 Jan 16
multi-headed attention, down to 2.28 now Jan 16
add a single self-attention head, reach 2.5 val loss Jan 16
add positional embeddings too Jan 16