Simple Language Model. See how AI learns to predict text, built from scratch without shortcuts.

github.com/antirez/simple-language-model

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

January 2024
  • Fix if else syntax error.
December 2023
  • Some simple (optional) form of attention implemented.
  • Restore learing rate to previos value.
  • Video linked.
  • Add dropout rate to log/model filename.
  • Put dropout_rate as global parameter.
  • Git ignore file added.
  • First commit.