Rare find

Mixture of Experts. Make AI models smarter without slowing them down.

github.com/lucidrains/mixture-of-experts

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

September 2023

redirect Sep 13

August 2023

Release —0.2.3 Aug 21
revert, 0.2.3 Aug 21
Revert "weighting is already done when computing combine_tensor" Aug 21
Release —0.2.2 Aug 21
weighting is already done when computing combine_tensor Aug 21

December 2021

Release —0.2.1 Dec 19
0.2.1 Dec 19
fix inplace Dec 19

October 2020

Release —0.2.0 Oct 20
make sure moe works with reversible networks for routing transformer Oct 20

July 2020

add badge Jul 18
update readme Jul 18
add diagram Jul 18
Release —0.1.1 Jul 17
fix initialization of experts Jul 17
Release —0.1.0 Jul 17
default to Gelu activation Jul 17
cleanup Jul 17
replace 1 billion demonstration with heirarchical Jul 17
update readme Jul 17
update readme Jul 17
Release —0.0.4 Jul 17
bump for release Jul 17
generalize cumsum exclusive to any dimension Jul 17
update readme Jul 17
Release —0.0.3 Jul 17
add ability to pass in custom experts Jul 17
Release —0.0.2 Jul 17
Release —0.0.1 Jul 17