pile_tokenizer. Downloads, extracts and tokenizes pile (https://the-eye.eu/public/AI/pile) data.

github.com/marianna13/pile_tokenizer

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.