Rare find

X-Clip. AI that understands both pictures and words together.

github.com/lucidrains/x-clip
by @lucidrains

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

October 2023
  • Release —0.14.4
  • add back regex
  • Revert "remove regex"
  • Release —0.14.3
  • remove regex
  • Release —0.14.2
  • patch
September 2023
  • Release —0.14.1
  • add an interesting regularization from another paper
  • Release —0.14.0
  • Release —0.12.9
  • maybe more efficient all gather
  • Release —0.12.8
  • Release —0.12.7
  • generalize the distributed all gather to any dimension
  • Release —0.12.6
  • Release —0.12.5
  • automatically take care of distributed setting
June 2023
  • some modifications to tokenizer for spear-tts
February 2023
  • avoid division
December 2022
  • add patch dropout from FLIP
October 2022
  • switch back to regular attention
September 2022
  • migrate over to cosine sim attention given new findings
August 2022
  • allow for text encoder to be causally masked, as was done in original…
  • fix to visual ssl if feature map is singular vector
  • make sure ethans way works
  • checkpoint off by default
  • add layernorm more friendly with fp16 as well as checkpointable trans…
  • add ability to return latents
  • make it work for ethan