Rare find

Perceiver. An AI model that learns to understand any type of data the same way.

github.com/lucidrains/perceiver-pytorch

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

June 2026

cite Jun 8
remove github, add pyproject.toml, uv publish Jun 8
update to allow for the inverted attention on some layers (row-wise s… Jun 8

August 2023

Release —0.8.8 Aug 22
allow for turning off fourier positions for experimental perceiver (i… Aug 22

January 2023

Release —0.8.7 Jan 3
remove freq base from all perceiver variants Jan 3

December 2022

Release —0.8.6 Dec 5
make sure variable lengthed sequences are dropped out correctly Dec 5
Release —0.8.5 Dec 5
add a structured dropout for incoming data being perceived, after see… Dec 5
Release —0.8.4 Dec 5

January 2022

Release —0.8.3 Jan 25
fix experimental.py Jan 25
Release —0.8.2 Jan 25
0.8.2 Jan 25
Merge pull request #59 from lsisoft/fix_precision_16 Jan 25
add dtype in axis_pos creation to allow precision=16 training. Used s… Jan 24

December 2021

Release —0.8.1 Dec 12
fix a bug with weight tying across layers and number of self attentio… Dec 12
Release —0.8.0 Dec 7
fix dropout in original perceiver Dec 7

October 2021

Release —0.7.5 Oct 10
make it so perceiver io decoding queries can omit the batch dimension Oct 10
move frequency bands to linear space Oct 4

September 2021

remove base parameter from fourier_encode function - 0.7.3 Sep 26
allow for turning off the classification head, or to return embedding… Sep 13

August 2021

readme Aug 30
only do cross attention once for perceiver io Aug 30
readd decoder feedforward, but make it optional Aug 30