Rare find

Ring Attention. Makes AI models handle much longer text by spreading the work across multiple computers.

github.com/lucidrains/ring-attention-pytorch
by @lucidrains

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

May 2025
  • Release —0.5.20
  • fix maximum update in triton flash attn
October 2024
  • Release —0.5.19
  • 0.5.19
  • add logic for llama3 context parallelism scheme
  • Release —0.5.18
  • address https://github.com/lucidrains/ring-attention-pytorch/issues/21
August 2024
  • Release —0.5.17
  • fix issue with seq len < world size in tree attn decoding after new u…
  • readme
  • updated algorithm 3 in tree attn decoding paper is more concise
  • Release —0.5.12
  • Release —0.5.10
  • keep numerator and denominator in float32 for tree attn decoding
  • another assert
  • Release —0.5.9
  • some asserts and cleanup for tree attn decoding
  • Release —0.5.8
  • make tree attention decoding work with triton flash attention forward
  • Release —0.5.6
  • account for scenario where keys and values are already sharded in tre…
  • fix test for cuda version of tree attn decode
  • Release —0.5.5
  • handle a small edge case for tree attn decoding
  • add assert_tree_attn.py
  • 0.5.4
  • first get a working function for tree attention decoding into the repo
July 2024
  • better error messages if triton 3.0 not installed
  • enforce triton nightly
  • complete softclamping for ring flash attn cuda