Rare find

Cut Cross-Entropy. Train giant AI models without running out of memory during the final calculation step.

github.com/apple/ml-cross-entropy

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

September 2025
  • Bump version
  • Fix the warning (#56)
  • Updates to precision and FSDP2 (#55)
July 2025
  • Fix bug in bias and fp32 (#53)
  • A couple bug fixes (#50)
June 2025
  • Some minor changes to autocast (#47)
  • Add support for autocasting (#46)
  • Enable backward in fp32 for Triton >= 3.2 (#45)
  • Differentiable LSE (#43)
May 2025
  • Fix vocab parallel loss computation (#42)
April 2025
  • Vocab Parallelism (#38)
  • Readme and requirements (#40)
  • 64-bit indexing (#39)
March 2025
  • DTensor Error Message (#32)
  • Support python 3.9 (#30)
  • Bag of updates (#23)
January 2025
  • Integer Shift (#20)
  • Support Bias (#19)
  • CCE Exact (#18)
December 2024
  • Manually apply decorators (#11)