Zexin He

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

cosmos. NVIDIA Cosmos is an open platform of world models, datasets, and tools that enables developers to build Physical AI for robots, autonomous vehicles, smart infrastructure, and more.

vggt-omega. [CVPR 2026 Oral] VGGT Omega

lingbot-map. A feed-forward 3D foundation model for reconstructing scenes from streaming data

lyra. Project Lyra: Open Generative 3D World Models

HY-World-2.0. HY-World 2.0: A Multi-Modal World Model for Reconstructing, Generating, and Simulating 3D Worlds

VideoX-Fun. 📹 A more flexible framework that can generate videos at any resolution and creates videos from images.

dreamzero. Code to pretrain, fine-tune, and evaluate DreamZero and run sim & real-world evals

claw-code. An agent-managed museum exhibit, built in Rust with Gajae-Code / LazyCodex — developed and maintained with no human intervention.

ReBalance. [ICLR 2026] Efficient Reasoning with Balanced Thinking

VIGA. VIGA: Vision-as-Inverse-Graphics Agent

Wan-Move. [NeurIPS 2025] Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance

RoboVerse. RoboVerse: Towards a Unified Platform, Dataset and Benchmark for Scalable and Generalizable Robot Learning

nvitop. An interactive NVIDIA-GPU process viewer and beyond, the one-stop solution for GPU process management.

open-pi-zero. Re-implementation of pi0 vision-language-action (VLA) model from Physical Intelligence

sam-3d-objects. SAM 3D Objects

PartField. [ICCV 2025] PartField: Learning 3D Feature Fields for Part Segmentation and Beyond

Open3D. Open3D: A Modern Library for 3D Data Processing

Depth-Anything-3. Depth Anything 3

lerobot. 🤗 LeRobot: Making AI for Robotics more accessible with end-to-end learning

HunyuanWorld-Mirror. [ICML 2026] WorldMirror: Fast and Universal 3D reconstruction model for versatile tasks

FlashWorld. Code for "FlashWorld: High-quality 3D Scene Generation within Seconds" (ICLR 2026 Oral)

LongLive. Long Video Gen Infrastructure

wall-x. Building General-Purpose Robots Based on Embodied Foundation Model

ReconViaGen. (ICLR2026) ReconViaGen: Towards Accurate Multi-view 3D Object Reconstruction via Generation

OpenUSD. Universal Scene Description

mujoco. Multi-Joint dynamics with Contact. A general purpose physics simulator.

Isaac-GR00T. NVIDIA Isaac GR00T N1.7 - A Foundation Model for Generalist Robots.

STream3R. Dynamic 3D Foundation Model using Causal Transformer. [ICLR 2026]

octo. Octo is a transformer-based robot policy trained on a diverse mix of 800k robot trajectories.

openpi. Python

mitsuba3. Mitsuba 3: A Retargetable Forward and Inverse Renderer

Bagel. Open-source unified multimodal model

RLBench. A large-scale benchmark and learning environment.

mobile-aloha. Mobile ALOHA: Learning Bimanual Mobile Manipulation with Low-Cost Whole-Body Teleoperation

act-plus-plus. Imitation learning algorithms with Co-training for Mobile ALOHA: ACT, Diffusion Policy, VINN

diffusion_policy. [RSS 2023] Diffusion Policy Visuomotor Policy Learning via Action Diffusion

HunyuanWorld-1.0. Generating Immersive, Explorable, and Interactive 3D Worlds from Words or Pixels with Hunyuan3D World Model

prope. Cameras as Relative Positional Encoding

Long-RL. Long-RL: Scaling RL to Long Sequences (NeurIPS 2025)

Shape-for-Motion. [Siggraph Asia 2025] Official code release of our paper "Shape-for-Motion: Precise and Consistent Video Editing with 3D Proxy"

FramePack. Lets make video diffusion practical!

DNF-Avatar. [ICCV 2025 Findings Oral] DNF-Avatar: Distilling Neural Fields for Real-time Animatable Avatar Relighting

InstantMesh. InstantMesh: Efficient 3D Mesh Generation from a Single Image with Sparse-view Large Reconstruction Models

MV-Adapter. [ICCV 2025] Official impl. of "MV-Adapter: Multi-view Consistent Image Generation Made Easy"

GeomDist. Python

FlashVDM. Unleashing Vecset Diffusion Model for Fast Shape Generation / within 1 Second (ICCV'25 Highlight)

Hunyuan3D-2. High-Resolution 3D Assets Generation with Large Scale Hunyuan3D Diffusion Models.

stable-virtual-camera. Stable Virtual Camera: Generative View Synthesis with Diffusion Models

vggsfm. VGGSfM: Visual Geometry Grounded Deep Structure From Motion

vggt. [CVPR 2025 Best Paper Award] VGGT: Visual Geometry Grounded Transformer

DyT. Code release for DynamicTanh (DyT)

DeepSeek-R1.

genesis-world. Simulation platform for general-purpose robotics & embodied AI learning.

Evaluation-Agent. [ACL2025 Oral & Award] Evaluate Image/Video Generation like Humans - Fast, Explainable, Flexible

Neural-LightRig. [CVPR2025] Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion

TRELLIS. Official repo for paper "Structured 3D Latents for Scalable and Versatile 3D Generation" (CVPR'25 Spotlight).

Sana. SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

SAR3D. Official repository for "SAR3D: Autoregressive 3D Object Generation and Understanding via Multi-scale 3D VQVAE"

MaterialAnything. [CVPR 2025 Highlight] Material Anything: Generating Materials for Any 3D Object via Diffusion

zero123plus. Code repository for Zero123++: a Single Image to Consistent Multi-view Diffusion Base Model.

Allegro. Allegro is a powerful text-to-video model that generates high-quality videos up to 6 seconds at 15 FPS and 720p resolution from simple text input.

blender-plugin. Python

Marigold. [CVPR 2024 - Oral, Best Paper Award Candidate] Marigold: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

ml-depth-pro. Depth Pro: Sharp Monocular Metric Depth in Less Than a Second.

Phidias-Diffusion. [ICLR 2025] Phidias: A Generative Model for Creating 3D Content from Text, Image, and 3D Conditions with Reference-Augmented Diffusion

Step-DPO. Implementation for "Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs"

DynamiCrafter. [ECCV 2024, Oral] DynamiCrafter: Animating Open-domain Images with Video Diffusion Priors

X-Ray. The official source code for "X-Ray: A Sequential 3D Representation for Generation".

ThemeStation. [SIGGRAPH 2024] ThemeStation: Generating Theme-Aware 3D Assets from Few Exemplars

differentiable-sdf-rendering. Source code for "Differentiable Signed Distance Function Rendering" (Siggraph 2022)

cuda-gridsample-grad2. Cuda implementation for gridsample with second derivative support

ChamferDistancePytorch. Chamfer Distance in Pytorch with f-score

LWM. Large World Model -- Modeling Text and Video with Millions Context

mamba. Mamba SSM architecture

rembg. Rembg is a tool to remove images background

OpenLRM. An open-source impl. of Large Reconstruction Models

open_clip. An open source implementation of CLIP.

flash-attention. Fast and memory-efficient exact attention

trl. Train transformer language models with reinforcement learning.

consistency_models. Official repo for consistency models.

ARF-svox2. Artistic Radiance Fields

consistencydecoder. Consistency Distilled Diff VAE

IF. Python

dreamgaussian. [ICLR 2024 Oral] Generative Gaussian Splatting for Efficient 3D Content Creation

objaverse-xl. 🪐 Objaverse-XL is a Universe of 10M+ 3D Objects. Contains API Scripts for Downloading and Processing!

diff-gaussian-rasterization. Cuda

diff-gaussian-rasterization. Cuda

LongLoRA. Code and documents of LongLoRA and LongAlpaca (ICLR 2024 Oral)

DiffTF. Official PyTorch implementation of DiffTF (Accepted by ICLR2024)

infinigen. Infinite Photorealistic Worlds using Procedural Generation

CityDreamer. The official implementation of "CityDreamer: Compositional Generative Model of Unbounded 3D Cities". (CVPR 2024)

OpenAgents. [COLM 2024] OpenAgents: An Open Platform for Language Agents in the Wild

tilted. Canonical Factors for Hybrid Neural Fields @ ICCV 2023

AgentSims. AgentSims is an easy-to-use infrastructure for researchers from all disciplines to test the specific capacities they are interested in.

generative_agents. Generative Agents: Interactive Simulacra of Human Behavior

Awesome-Video-Diffusion. A curated list of recent diffusion models for video generation, editing, and various other applications.

GenBench. Benchmarking and Analyzing Generative Data for Visual Recognition

llama-cookbook. Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

Correlational-Image-Modeling. Python

AnyDoor. Official implementations for paper: Anydoor: zero-shot object-level image customization

Drag3D. DragGAN meets GET3D for interactive mesh generation and editing.

parti-pytorch. Implementation of Parti, Google's pure attention-based text-to-image neural network, in Pytorch

IST-Net. (ICCV2023) IST-Net: Prior-free Category-level Pose Estimation with Implicit Space Transformation

LLaVAR. Code/Data for the paper: "LLaVAR: Enhanced Visual Instruction Tuning for Text-Rich Image Understanding"

Voyager. An Open-Ended Embodied Agent with Large Language Models

SSDNeRF. [ICCV 2023] Single-Stage Diffusion NeRF

Magic123. [ICLR'24] Official PyTorch Implementation of Magic123: One Image to High-Quality 3D Object Generation Using Both 2D and 3D Diffusion Priors

oft. Official implementation of "Controlling Text-to-Image Diffusion by Orthogonal Finetuning".

DragGAN. Official Code for DragGAN (SIGGRAPH 2023)

lens. This is the official repository for the LENS (Large Language Models Enhanced to See) system.

ContextDET. Contextual Object Detection with Multimodal Large Language Models

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

MineCLIP. Foundation Model for MineDojo

MineDojo. Building Open-Ended Embodied Agents with Internet-Scale Knowledge

tree-of-thought-prompting. Using Tree-of-Thought Prompting to boost ChatGPT's reasoning

Cones-V2. [ICML 2023 Oral, NeurIPS 2023] Official implementations for paper: Customizable Image Synthesis with Multiple Subjects

Awesome-Multimodal-Large-Language-Models. :sparkles::sparkles:Latest Advances on Multimodal Large Language Models

generative-models. Generative Models by Stability AI

MobileSAM. This is the official code for MobileSAM project that makes SAM lightweight for mobile applications and beyond!

Academic-project-page-template. A project page template for academic papers. Demo at https://eliahuhorwitz.github.io/Academic-project-page-template/

Versatile-Diffusion. Versatile Diffusion: Text, Images and Variations All in One Diffusion Model, arXiv 2022 / ICCV 2023

math_problems-step-by-step_solutions. Here we provide and collect many functions to generate math problem and step by step solutions for LLM training

LMFlow. An Extensible Toolkit for Finetuning and Inference of Large Foundation Models. Large Models for All.

LOMO. LOMO: LOw-Memory Optimization

dreamsim. DreamSim: Learning New Dimensions of Human Visual Similarity using Synthetic Data (NeurIPS 2023 Spotlight) / / / / When Does Perceptual Alignment Benefit Vision Representations? (NeurIPS 2024)

Mind2Web. [NeurIPS'23 Spotlight] "Mind2Web: Towards a Generalist Agent for the Web" -- the first LLM-based web agent and benchmark for generalist web agents

PandaGPT. [TLLM'23] PandaGPT: One Model To Instruction-Follow Them All

lm-evaluation-harness. A framework for few-shot evaluation of language models.

prm800k. 800,000 step-level correctness labels on LLM solutions to MATH problems

megfile. Megvii FILE Library - Working with Files in Python same as the standard library

ControlVideo. [ICLR 2024] Official pytorch implementation of "ControlVideo: Training-free Controllable Text-to-Video Generation"

Gen-L-Video. The official implementation for "Gen-L-Video: Multi-Text to Long Video Generation via Temporal Co-Denoising".

RaBit. This repository includes the related code of RaBit.

RenderMe-360. RenderMe-360: Large Digital Asset Library and Benchmark Towards High-fidelity Head Avatars

bloomchat. This repo contains the data preparation, tokenization, training and inference code for BLOOMChat. BLOOMChat is a 176 billion parameter multilingual chat model based on BLOOM.

CodeT5. Home of CodeT5: Open Code LLMs for Code Understanding and Generation

LLMScore. LLMScore: Unveiling the Power of Large Language Models in Text-to-Image Synthesis Evaluation

tree-of-thought-llm. [NeurIPS 2023] Tree of Thoughts: Deliberate Problem Solving with Large Language Models

dream-textures. Stable Diffusion built-in to Blender

VisionLLM. VisionLLM Series

dynibar. Implementation of DynIBaR Neural Dynamic Image-Based Rendering (CVPR 2023)

InternGPT. InternGPT (iGPT) is an open source demo platform where you can easily showcase your AI models. Now it supports DragGAN, ChatGPT, ImageBind, multimodal chat like GPT-4, SAM, interactive image editing, etc. Try it at igpt.opengvlab.com (支持DragGAN、ChatGPT、ImageBind、SAM的在线Demo系统)

threestudio. A unified framework for 3D content generation.

lgssl. [CVPR 2023] Learning Visual Representations via Language-Guided Sampling

DetGPT. Jupyter Notebook

ImageReward. [NeurIPS 2023] ImageReward: Learning and Evaluating Human Preferences for Text-to-image Generation

ImageBind. ImageBind One Embedding Space to Bind Them All

Grounded-Segment-Anything. Grounded SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything

starcoder. Home of StarCoder: fine-tuning & inference!

shap-e. Generate 3D objects conditioned on text or images

G.pt. Official PyTorch Implementation of "Learning to Learn with Generative Models of Neural Network Checkpoints"

mPLUG-Owl. mPLUG-Owl: The Powerful Multi-modal Large Language Model Family

GPT4Tools. GPT4Tools is an intelligent system that can automatically decide, control, and utilize different visual foundation models, allowing the user to interact with images during a conversation.

Segment-Everything-Everywhere-All-At-Once. [NeurIPS 2023] Official implementation of the paper "Segment Everything Everywhere All at Once"

skypilot. The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

lamini. The Official Python Client for Lamini's API

datacomp. DataComp: In search of the next generation of multimodal datasets

FastChat. An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

Multimodal-GPT. Multimodal-GPT

LLaVA. [NeurIPS'23 Oral] Visual Instruction Tuning (LLaVA) built towards GPT-4V level capabilities and beyond.

gpt4free. The official gpt4free repository | various collection of powerful language models | opus 4.6 gpt 5.3 kimi 2.5 deepseek v3.2 gemini 3

ChatCaptioner. Official Repository of ChatCaptioner

Segment-Anything-NeRF. Segment-anything interactively in NeRF.

sparsegpt. Code for the ICML 2023 paper "SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot".

zipnerf-pytorch. Unofficial implementation of ZipNeRF

gradient-checkpointing. Make huge neural nets fit in memory

MOSS. An open-source tool-augmented conversational language model from Fudan University

peft. 🤗 PEFT: State-of-the-art Parameter-Efficient Fine-Tuning.

MiniGPT-4. Open-sourced codes for MiniGPT-4 and MiniGPT-v2 (https://minigpt-4.github.io, https://minigpt-v2.github.io/)

llama.cpp. LLM inference in C/C++

StableLM. StableLM: Stability AI Language Models

rich-text-to-image. Rich-Text-to-Image Generation

Inpaint-Anything. Inpaint anything using Segment Anything and inpainting models.

ODISE. Official PyTorch implementation of ODISE: Open-Vocabulary Panoptic Segmentation with Text-to-Image Diffusion Models [CVPR 2023 Highlight]

3D-Box-Segment-Anything. We extend Segment Anything to 3D perception by combining it with VoxelNeXt.

AutoGPT. AutoGPT is the vision of accessible AI for everyone, to use and to build on. Our mission is to provide the tools, so that you can focus on what matters.

MOOD. Official PyTorch implementation of MOOD series: (1) MOODv1: Rethinking Out-of-distributionDetection: Masked Image Modeling Is All You Need. (2) MOODv2: Masked Image Modeling for Out-of-Distribution Detection.

grounded-segment-any-parts. Grounded Segment Anything: From Objects to Parts

camel. 🐫 CAMEL: The first and the best multi-agent framework. Finding the Scaling Law of Agents. https://www.camel-ai.org

EditAnything. Edit anything in images powered by segment-anything, ControlNet, StableDiffusion, etc. (ACM MM)

VideoCrafter. VideoCrafter2: Overcoming Data Limitations for High-Quality Video Diffusion Models

dolly. Databricks’ Dolly, a large language model trained on the Databricks Machine Learning Platform

Chinese-LLaMA-Alpaca. 中文LLaMA&Alpaca大语言模型+本地CPU/GPU训练部署 (Chinese LLaMA & Alpaca LLMs)

viper. Code for the paper "ViperGPT: Visual Inference via Python Execution for Reasoning"

genvs.

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

alpaca-lora. Instruct-tune LLaMA on consumer hardware

instruct-pix2pix. Python

instant-ngp. Instant neural graphics primitives: lightning fast NeRF and more

Attend-and-Excite. Official Implementation for "Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models" (SIGGRAPH 2023)

NeuralLift-360. [CVPR 2023, Highlight] "NeuralLift-360: Lifting An In-the-wild 2D Photo to A 3D Object with 360° Views", Dejia Xu, Yifan Jiang, Peihao Wang, Zhiwen Fan, Yi Wang, Zhangyang Wang

zero123. Zero-1-to-3: Zero-shot One Image to 3D Object (ICCV 2023)

Set-the-Scene. Set-the-Scene

text2room. Text2Room generates textured 3D meshes from a given text prompt using 2D text-to-image models (ICCV2023).

open_flamingo. An open-source framework for training large multimodal models.

LLaMA-Adapter. [ICLR 2024] Fine-tuning LLaMA to follow Instructions within 1 Hour and 1.2M Parameters

gigagan-pytorch. Implementation of GigaGAN, new SOTA GAN out of Adobe. Culmination of nearly a decade of research into GANs

Make-It-3D. [ICCV 2023] Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior

Ref-NPR. [CVPR 2023] Ref-NPR: Reference-Based Non-PhotoRealistic Radiance Fields