Axial Attention. A faster way to process multi-dimensional data in AI models.

github.com/lucidrains/axial-attention

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

August 2021

Release —0.6.1 Aug 26
fix parameterlist dataparallel issue Aug 26
readme Aug 25
Release —0.6.0 Aug 25
fix all issues with axial image transformer Aug 25
cite ccnet Aug 20

February 2021

update citations Feb 1

January 2021

cleanup Jan 22
Release —0.5.0 Jan 22
add axial positional embedding Jan 22

September 2020

Release —0.4.1 Sep 26
fix mixed memory format warnings Sep 26
make clear in readme that output of axial attention is the same shape… Sep 19
Update README.md Sep 16
Release —0.4.0 Sep 15
Create python-publish.yml Sep 15
add option to turn off summing contributions from attention on each a… Sep 15

August 2020

update with another paper from google ai that uses axial attention Aug 28

June 2020

fix axial image transformer Jun 9
cleanup Jun 2
add AxialPositionalEmbedding Jun 2
add induced set attention from set transformer paper Jun 2
add reversible net file Jun 2
rename transformer to AxialImageTransformer Jun 2
fix bug with ff in imagetransformer Jun 2