dynabatch. PyTorch/Hugging Face batching utility that sorts variable-length text by difficulty, then dynamically increases batch size on easier samples using a pre-trained VRAM predictor to improve GPU utilization and throughput while reducing OOM risk with fallback handling.

github.com/bendangnuksung/dynabatch

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

No recent activity.