Rare find

EMA. Keep a smoothed copy of your AI model that learns slower and stays more stable.

github.com/lucidrains/ema-pytorch
by @lucidrains

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

July 2026
  • cleanup
  • some flexibility for the ema module wrapper forward method, only doin…
  • with gemini flash, add a specific ema module wrapper that is able to …
December 2025
  • Release —0.7.9
  • rename
  • Release —0.7.8
  • have gemini flash vibe code a version for pytree, using EMA as a temp…
December 2024
  • Release —0.7.7
  • address https://github.com/lucidrains/ema-pytorch/issues/35
November 2024
  • future annotation
  • manual mode for PostHocEMA , contributed by @kalekundert
  • Release —0.7.6
  • Avoid deep copies when saving post-hoc EMA checkpoints (#34)
  • Release —0.7.5
  • address https://github.com/lucidrains/ema-pytorch/issues/33#issuecomm…
  • Release —0.7.4
  • address https://github.com/lucidrains/ema-pytorch/issues/33
October 2024
  • Release —0.7.3
  • small helper method
  • Release —0.7.2
  • allow for auto-calling ema after optimizer step using `register_step_…
  • Release —0.7.1
  • readme
  • cite and show it is available already
  • highest sigma rel should be 0.28
  • Release —0.7.0
  • add ability to update online model with ema model periodically, eithe…
  • add ability to interpolate current model with ema model, to further e…
September 2024
  • allow for lazy init of the ema model
  • allow for coercing dtype, in the case that the online model weights c…