Triton Transformer. A transformer neural network written in a faster, lower-level programming language.

github.com/lucidrains/triton-transformer

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

April 2022

Release —0.1.1 Apr 5
extraneous layernorm Apr 5
make sure example works Apr 4
Release —0.1.0 Apr 4
remove all biases, as 540 billion parameter PaLM model shows it is no… Apr 4

March 2022

Release —0.0.28 Mar 23
pin to last working version of triton Mar 23

January 2022

todo Jan 6

November 2021

Release —0.0.27 Nov 6
add stable layernorm, from cogview paper Nov 6
Release —0.0.26 Nov 6
fix bug with triton layernorm Nov 6

October 2021

cleanup Oct 6
DRY a little Oct 6
Release —0.0.25 Oct 6
directly calculate the causal mask within the triton softmax kernel Oct 6
explain loss calculation Oct 4
readme Oct 4
todo check Oct 4
Release —0.0.24 Oct 4
make sure dropout backwards is correct Oct 4
Release —0.0.23 Oct 4
follow Phil Ts advice Oct 4
Release —0.0.22 Oct 4
Release —0.0.21 Oct 4
add memory efficient dropout from triton lang tutorials Oct 4

September 2021

move images to folder Sep 30
break components into individual files Sep 30
Update README.md Sep 29
softmax forwards and backwards results Sep 29