Rare find

Makemore. Generate new names, words, or text by learning from examples you give it.

github.com/karpathy/makemore
by @karpathy

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

November 2022
  • tune description of the repo wrt references
September 2022
  • Merge pull request #3 from normanyu/fix-rnn-prev-state
  • Fix bug in RNN where hprev always referred to start. Change so that h…
  • change readme
August 2022
  • add a bag of words model that looks suspiciously similar to a transfo…
  • add an RNN and a GRU language model
  • generalize makemore into other types of language models, and add bigr…
  • fix comment
  • implementation of InfiniteDataLoader sad
  • remove some guardrails for this simple of a use case
  • remove gradient clipping i dont think its needed at this small scale
  • simplify optimizer init and delete code
  • remove weight init, not needed at this scale
  • add max steps
  • respect multigpu envs, e.g. cuda:2 designation should work
  • big refactor to make easier and api agree with mingpt more
June 2022
  • add some generated examples of names for fun
  • split out train,test,new separately when reporting on sampling word i…
  • small tweaks to support the Apple Silicon M1 chip device 'mps'. But t…
  • first commit