Rare find

X-Transformers. A ready-to-use AI model builder with experimental features from research papers.

github.com/lucidrains/x-transformers

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

July 2026

add a new positional bias from Inkling, allowing each token to projec… Jul 23
add an option for the e2e-ttt to update a dedicated set of long term … Jul 17
for ttt in the xl autoregressive wrapper, allow for gradients generat… Jul 16
allow for loss explorations in next latent wrapper Jul 3
for e2e-ttt, add ability to update with muon Jul 2

June 2026

with gemini, add ability to do the descending ff dim across depth, bu… Jun 24
gru also works, just not as well as the scheme they have with 3 layer… Jun 22
a gvf idea did not work for path star, but a dynamic rollout loss wei… Jun 22
add a wrapper for the specific scheme for next latent prediction prop… Jun 19
turns out a researcher at Atlassian of all places already tried this,… Jun 17
add a function for segmenting a sequence by accumulated entropy for t… Jun 16
the two ingredients needed for the "sleep" phase would be ability to … Jun 15
for e2e ttt, add the ability for a custom ttt loss that is not the ma… Jun 15
with gemini, add e2e ttt to the xl autoregressive wrapper, before inv… Jun 15
cleanup inverted attention, make sure researchers do not accidentally… Jun 8
add inverted cross attention Jun 8
allow for a layer below the embedding predicting the next embedding a… Jun 4
leave a thought Jun 3
not as good, but leave the work around for a student to try Jun 3

May 2026

some flexibility for the RL token May 27
link to bycloud May 19
allow for qkvo attn to be modulated May 16
l1 loss for continuous autoencoder May 16
add a continuous transformer autoencoder to facilitate the creation o… May 16
account for masking / variable lengths, and add a learnt alpha/beta v… May 13

February 2026

Release —2.16.1 Feb 12
Release —2.16.0 Feb 7
Release —2.15.2 Feb 3
Release —2.15.1 Feb 3
Release —2.15.0 Feb 3