Lianghua Huang

Awesome-Video-World-Models-with-AR-Diffusion. A Curated List of Awesome Video World Models with AR Diffusion: Covering Algorithms, Applications, and Infrastructure, Aimed at Serving as a Comprehensive Resource for Researchers, Practitioners, and Enthusiasts.

MSA. Memory Sparse Attention - A scalable, end-to-end trainable latent-memory framework for 100M-token contexts.

kraken. Triton-based Symmetric Memory operators and examples

Attention-Residuals.

airi. 💖🧸 Self hosted, you-owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraft, Factorio playing. Web / macOS / Windows supported.

NitroGen. A Foundation Model for Generalist Gaming Agents

WAVE. ICLR 2026 Oral: WAVE: Learning Unified & Versatile Audio-Visual Embeddings with Multimodal LLM

open-p2p. Official Repo for paper: Scaling Behavior Cloning Improves Causal Reasoning: An Open Model for Real-Time Video Game Playing

STEM. Python

pplx-kernels. Perplexity GPU Kernels

pplx-garden. Perplexity open source garden for inference technology

ReCamMaster. [ICCV'25 Best Paper Finalist] ReCamMaster: Camera-Controlled Generative Rendering from A Single Video

DeepEP. DeepEP: an efficient expert-parallel communication library

cubvh. Mesh tools.

warp. A Python framework for GPU-accelerated simulation, robotics, and machine learning.

Genie-Envisioner-V1. Python

awesome-embodied-vla-va-vln. A curated list of state-of-the-art research in embodied AI, focusing on vision-language-action (VLA) models, vision-language navigation (VLN), and related multimodal learning approaches.

vjepa2. PyTorch code and models for VJEPA2 self-supervised learning from video.

Self-Forcing-Endless. Make self forcing endless. Add cache purging. Add prompt controllability.

SkyReels-V2. SkyReels-V2: Infinite-length Film Generative model

MAGI-1. MAGI-1: Autoregressive Video Generation at Scale

harmony. Renderer for the harmony response format to be used with gpt-oss

gpt-oss. gpt-oss-120b and gpt-oss-20b are two open-weight language models by OpenAI

FastVideo. A unified inference and post-training framework for accelerated video generation.

GPT-SoVITS. 1 min voice data can also be used to train a good TTS model! (few shot voice cloning)

ms-swift. Use PEFT or Full-parameter to CPT/SFT/DPO/GRPO 600+ LLMs (Qwen3.6, DeepSeek-V4, GLM-5.1, InternLM3, Llama4, ...) and 300+ MLLMs (Qwen3-VL, Qwen3-Omni, InternVL3.5, Ovis2.5, GLM4.5v, Gemma4, Llava, Phi4, ...) (AAAI 2025).

modded-nanogpt. NanoGPT (124M) in 90 seconds

fms-fsdp. 🚀 Efficiently (pre)training foundation models with native PyTorch features, including FSDP for training and SDPA implementation of Flash attention v2.

OmniAvatar. Python

YUME. The official code of Yume

DeepGEMM. DeepGEMM: clean and efficient BLAS kernel library on GPU

higgs-audio. Text-audio foundation model from Boson AI

mixture_of_recursions. Mixture-of-Recursions: Learning Dynamic Recursive Depths for Adaptive Token-Level Computation (NeurIPS 2025)

index-tts. An Industrial-Level Controllable and Efficient Zero-Shot Text-To-Speech System

Ego4d. Ego4d dataset repository. Download the dataset, visualize, extract features & example usage of the dataset

MoBA. MoBA: Mixture of Block Attention for Long-Context LLMs

native-sparse-attention. 🐳 Efficient Triton implementations for "Native Sparse Attention: Hardware-Aligned and Natively Trainable Sparse Attention"

SageAttention. [ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

flashinfer. FlashInfer: Kernel Library for LLM Serving

carla. Open-source simulator for autonomous driving research.

MOSS-TTSD. MOSS-TTSD is a spoken dialogue generation model designed for expressive multi-speaker synthesis. It features long-context modeling, flexible speaker control, and multilingual support, while enabling zero-shot voice cloning from short audio references.

MagiAttention. A Distributed Attention Towards Linear Scalability for Ultra-Long Context, Heterogeneous Data Training

radial-attention. [NeurIPS 2025] Radial Attention: O(nlogn) Sparse Attention with Energy Decay for Long Video Generation

TalkingMachines. TalkingMachines

CoGenAV. Python

REPA-E. [ICCV 2025] Official implementation of the paper: REPA-E: Unlocking VAE for End-to-End Tuning of Latent Diffusion Transformers

BigVGAN. Official PyTorch implementation of BigVGAN (ICLR 2023)

Dolphin. The official repo for “Dolphin: Document Image Parsing via Heterogeneous Anchor Prompting”, ACL, 2025.

encodec. State-of-the-art deep learning based audio codec supporting both mono 24 kHz audio and stereo 48 kHz audio.

descript-audio-codec. State-of-the-art audio codec with 90x compression factor. Supports 44.1kHz, 24kHz, and 16kHz mono/stereo audio.

Bagel. Open-source unified multimodal model

tailwind-nextjs-starter-blog. This is a Next.js, Tailwind CSS blogging starter template. Comes out of the box configured with the latest technologies to make technical writing a breeze. Easily configurable and customizable. Perfect as a replacement to existing Jekyll and Hugo individual blogs.

smolvlm-realtime-webcam. Real-time webcam demo with SmolVLM and llama.cpp server

VITA-Audio. ✨✨[NeurIPS 2025] VITA-Audio: Fast Interleaved Cross-Modal Token Generation for Efficient Large Speech-Language Model

pyannote-audio. Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

Kimi-Audio. Kimi-Audio, an open-source audio foundation model excelling in audio understanding, generation, and conversation

dia. A TTS model capable of generating ultra-realistic dialogue in one pass.

Liquid. (Accepted by IJCV) Liquid: Language Models are Scalable and Unified Multi-modal Generators

FramePack. Lets make video diffusion practical!

metamorph. Code for MetaMorph Multimodal Understanding and Generation via Instruction Tuning

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

SMPLest-X. [TPAMI 2025] Official Code for "SMPLest-X: Ultimate Scaling for Expressive Human Pose and Shape Estimation"

smplify-x. Expressive Body Capture: 3D Hands, Face, and Body from a Single Image

WonderWorld. Code release for https://kovenyu.com/WonderWorld/

Awesome-Talking-Head-Synthesis. 💬 An extensive collection of exceptional resources dedicated to the captivating world of talking face synthesis! ⭐ If you find this repo useful, please give it a star! 🤩

MegaTTS3. Python

gemma_pytorch. The official PyTorch implementation of Google's Gemma models

inferno. 🔥🔥🔥 Set the world of 3D faces on fire with INFERNO 🔥🔥🔥

emoca. Official repository accompanying a CVPR 2022 paper EMOCA: Emotion Driven Monocular Face Capture And Animation. EMOCA takes a single image of a face as input and produces a 3D reconstruction. EMOCA sets the new standard on reconstructing highly emotional images in-the-wild

Arinar. Python

fsdp_qlora. Training LLMs with QLoRA + FSDP

Dyadic-Interaction-Modeling. [ECCV 2024] Dyadic Interaction Modeling for Social Behavior Generation

csm. A Conversational Speech Generation Model

SpatialLM. [NeurIPS 2025] SpatialLM: Training Large Language Models for Structured Indoor Modeling

Awesome-Human-Motion-Video-Generation. 【Accepted by TPAMI】Human Motion Video Generation: A Survey (https://ieeexplore.ieee.org/document/11106267)

AudioX. [ICLR 2026] Repository of AudioX

Sonic. Official implementation of "Sonic: Shifting Focus to Global Audio Perception in Portrait Animation"

Rectified-Diffusion. [ICLR 2025] Rectified Diffusion: Straightness Is Not Your Need

bd3lms. [ICLR 2025 Oral] Block Diffusion: Interpolating Between Autoregressive and Diffusion Language Models

diffused-heads. Official repository for Diffused Heads: Diffusion Models Beat GANs on Talking-Face Generation

Synchformer. Source code for "Synchformer: Efficient Synchronization from Sparse Cues" (ICASSP 2024)

LightningDiT. [CVPR 2025 Oral] Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models

SALMONN. SALMONN family: A suite of advanced multi-modal LLMs

Wan2.1. Wan: Open and Advanced Large-Scale Video Generative Models

fractalgen. PyTorch implementation of FractalGen https://arxiv.org/abs/2502.17437

open-oasis. Inference script for Oasis 500M

Playable-Game-Generation. An open-source lightweight game generation paradigm. It includes everything from data processing to model architecture design and playability-based evaluation methods. The game runs at 20 FPS on a single consumer-grade graphics card (RTX-2060) while maintaining high playability.

diffusion-forcing. code for "Diffusion Forcing: Next-token Prediction Meets Full-Sequence Diffusion"

audiocraft. Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

diffusion-forcing-transformer. [ICML 2025] Official PyTorch Implementation of "History-Guided Video Diffusion"

Gymnasium. A standard API for single-agent reinforcement learning environments, with popular reference environments and related utilities (formerly Gym)

gym. A toolkit for developing and comparing reinforcement learning algorithms.

Video-Pre-Training. Video PreTraining (VPT): Learning to Act by Watching Unlabeled Online Videos

mini-omni. open-source multimodal large language model that can hear, talk while thinking. Featuring real-time end-to-end speech input and streaming audio output conversational capabilities.

whisper. Robust Speech Recognition via Large-Scale Weak Supervision

moshi. Moshi is a speech-text foundation model and full-duplex spoken dialogue framework. It uses Mimi, a state-of-the-art streaming neural audio codec.

Ask-Anything. [CVPR2024 Highlight][VideoChatGPT] ChatGPT with video understanding! And many more supported LMs such as miniGPT4, StableLM, and MOSS.

swarm. Educational framework exploring ergonomic, lightweight multi-agent orchestration. Managed by OpenAI Solution team.

openai-realtime-agents. This is a simple demonstration of more advanced, agentic patterns built on top of the Realtime API.

lexical. Lexical is an extensible text editor framework that provides excellent reliability, accessibility and performance.