Rare find

Local Attention. A faster way to teach AI models to focus on nearby information instead of everything at once.

github.com/lucidrains/local-attention

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

July 2025

ci stuff Jul 16
Release —1.11.2 Jul 16
quick change needed to test h-net Jul 16

January 2025

Release —1.11.1 Jan 16
1.11.1 Jan 16
Merge pull request #27 from minh-nguyenhoang/native-look-around Jan 16
remove torch.cat in look_around() Jan 16
Release —1.11.0 Jan 16
release more efficient look_around thanks to @minh-nguyenhoang ! Jan 16
Merge pull request #26 from minh-nguyenhoang/native-look-around Jan 16
change look_around() to use native pytorch functionality Jan 16
Release —1.10.1 Jan 12
hyper connect the local transformer Jan 12
Release —1.10.0 Jan 12
allow for inserting layers before the local attention, for testing a … Jan 12

September 2024

Release —1.9.15 Sep 6
remove a warning Sep 6

July 2024

Release —1.9.14 Jul 8
make sure kv caching with dynamic pos bias works for local transformer Jul 8
Release —1.9.11 Jul 6
remove adding staticmethod Jul 6
Release —1.9.10 Jul 6
oops Jul 6
Release —1.9.9 Jul 6
revert back to using plain apply_rotary_pos_emb fn Jul 6
allow for argmax sampling Jul 5
support kv cache for dynamic pos bias for local mha Jul 5
address https://github.com/lucidrains/local-attention/issues/22 Jul 5

June 2024

support kv caching for non-exact window size Jun 21
fix test gh action Jun 21