Shannons-Scaling-Laws. A creative exploration into Shannon’s Entropy and how entropy, meaning, and information density shifts across languages and contexts—and how this translates to scaling laws. For example: Could models trained in Mandarin have a scaling laws advantage in information density per token?

github.com/jasontang-ai/Shannons-Scaling-Laws

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.