Transformer in Transformer. AI model that looks at both individual pixels and image patches to recognize what's in pictures.

github.com/lucidrains/transformer-in-transformer
by @lucidrains

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

December 2021
  • Release —0.1.2
  • support channels other than rgb
November 2021
  • correct name
March 2021
  • Release —0.1.1
  • back to layernorm
  • Release —0.1.0
  • use rmsnorm instead
  • Release —0.0.9
  • use scalenorm
  • Release —0.0.8
  • allow for overlapping super-pixel data, as in the token2token paper, …
  • Release —0.0.7
  • make it so TNT can accept images of unequal height and width, as long…
  • link
  • Release —0.0.6
  • extra assert for safe measure
  • Release —0.0.5
  • fix pixel calculation bugs
  • Release —0.0.4
  • Release —0.0.3
  • add a layernorm before pixels to patch linear
  • positional embeds for patches and pixels
  • Create python-publish.yml
  • complete transformer in transformer
  • readme
  • Initial commit