long-context-transformers. A repository to get train transformers to access longer context for causal language models, most of these methods are still in testing. Try them out if you'd like but please lmk your results so we don't duplicate work :)

github.com/jordiclive/long-context-transformers

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.