Rare find

MMDiT. A building block for AI models that understand multiple types of information at once.

github.com/lucidrains/mmdit

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

January 2026

Release —0.3.0 Jan 18
update to mHC Jan 18

January 2025

Release —0.2.2 Jan 12
address https://github.com/lucidrains/mmdit/issues/5 Jan 12

December 2024

Release —0.2.1 Dec 27
add hyper connections Dec 27
Release —0.1.5 Dec 27
address https://github.com/lucidrains/mmdit/issues/4 Dec 27

August 2024

Release —0.1.4 Aug 24
for adaptive attention, allow each head to have different adaptive we… Aug 24

July 2024

Release —0.1.2 Jul 6
offer attention softclamping Jul 6
Release —0.1.1 Jul 6
address https://github.com/lucidrains/mmdit/issues/1 Jul 6

May 2024

credit assignment May 10
Release —0.1.0 May 10
complete sidequest and bump to 0.1.0 May 10
remove wip May 7
Release —0.0.11 May 6
handle the final norm at the end of the transformer, but make optional May 6
Release —0.0.10 May 6
ability to add a few register tokens May 6
fix logic for skipping last feedforward in mmdit May 5
make sure adaptive layernorm film is identity in generalized version May 5
remove Optional May 5
add a generalized version for N modalities May 5
save on the last text feedforward May 5
naming May 5
oops May 4
film should be initialized to identity May 4