Rare find

Speculative Decoding. Makes AI text generation faster by guessing ahead and checking answers in parallel.

github.com/lucidrains/speculative-decoding

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

December 2024

restore training script + comments Dec 22
Release —0.2.0 Dec 22
address https://github.com/lucidrains/speculative-decoding/issues/3 Dec 22
Release —0.1.4 Dec 12
address https://github.com/lucidrains/speculative-decoding/issues/3 Dec 12

October 2023

cleanup Oct 9
cite medusa, as using the last layer hiddens is exactly what the prop… Oct 8
cite similar paper to early exit Oct 8
fix sampling for num leading start tokens > 1 in prophet scheme Oct 4
Release —0.1.2 Oct 4
generalize to any number of leading start tokens for prophet Oct 4
Release —0.1.1 Oct 4
the extra norm in projecting to prophet model dimensions hurt for som… Oct 4
clip prophet gradients too Oct 3
research management Oct 3
negative control Oct 3
functional Oct 3
Release —0.1.0 Oct 3
complete speculative sampling with prophet net on cached embeddings i… Oct 3
credit assignment Oct 2
facepalm Oct 2
share idea Oct 2
Release —0.0.15 Oct 2
fix start token and determining input sequence Oct 2
Release —0.0.14 Oct 2
move towards completing what is started Oct 2
Release —0.0.12 Oct 1
more fixes Oct 1

September 2023

Release —0.0.11 Sep 26
Release —0.0.10 Sep 23