Shikun Liu

TorchJD. Library for Jacobian descent with PyTorch. It enables the optimization of neural networks with multiple losses (e.g. multi-task learning).

EditCtrl. [CVPR 2026] EditCtrl: Disentangled Local and Global Control for Real-Time Generative Video Editing

Captain-Safari. Offical Implementation of Captain-Safari [CVPR 2026]

giscus. A commenting system powered by GitHub Discussions. :octocat: :speech_balloon: :gem:

LTX-2. Official Python inference and LoRA trainer package for the LTX-2 audio–video generative model.

UCPE. 📷 [CVPR'26] Camera-controlled text-to-video generation, now with intrinsics, distortion and orientation control!

GEN3C. [CVPR 2025 Highlight] GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control

min-pi-flow. Python

2d-gaussian-splatting. [SIGGRAPH'24] 2D Gaussian Splatting for Geometrically Accurate Radiance Fields

Scaling-Diffusion-Transformers-muP. [NeurIPS 2025] Official implementation for our paper "Scaling Diffusion Transformers Efficiently via μP".

Sana. SANA: Efficient High-Resolution Image Synthesis with Linear Diffusion Transformer

color-alignment-in-diffusion. Official Github repository for the CVPR 2025 paper "Color Alignment in Diffusion"

ControlVideo. [ICLR 2024] Official pytorch implementation of "ControlVideo: Training-free Controllable Text-to-Video Generation"

Geo4D. [ICCV 2025 Highlight] Geo4D: Leveraging Video Generators for Geometric 4D Scene Reconstruction

CFG-Zero-star. Official repo for CFG-Zero*

Go-with-the-Flow. The official implementation of CVPR'25 Oral paper "Go-with-the-Flow: Motion-Controllable Video Diffusion Models Using Real-Time Warped Noise"

fast3r. [CVPR 2025] Fast3R: Towards 3D Reconstruction of 1000+ Images in One Forward Pass

Leffa. [CVPR 2025] Learning Flow Fields in Attention for Controllable Person Image Generation

slam-handbook-public-release. Release repo for our SLAM Handbook

open-oasis. Inference script for Oasis 500M

StableDelight. StableDelight: Revealing Hidden Textures by Removing Specular Reflections

transfusion-pytorch. Pytorch implementation of Transfusion, "Predict the Next Token and Diffuse Images with One Multi-Modal Model", from MetaAI

whisper-medusa. Whisper with Medusa heads

flux. Official inference repo for FLUX.1 models

mar. PyTorch implementation of MAR+DiffLoss https://arxiv.org/abs/2406.11838

MultiDiffusion. Official Pytorch Implementation for "MultiDiffusion: Fusing Diffusion Paths for Controlled Image Generation" presenting "MultiDiffusion" (ICML 2023)

fast-DiT. Fast Diffusion Models with Transformers

MOFA-Video. [ECCV 2024] MOFA-Video: Controllable Image Animation via Generative Motion Field Adaptions in Frozen Image-to-Video Diffusion Model.

LLM101n. LLM101n: Let's build a Storyteller

chameleon. Repository for Meta Chameleon, a mixed-modal early-fusion foundation model from FAIR.

GaussianFlow. GaussianFlow: Splatting Gaussian Dynamics for 4D Content Creation (TMLR)

Real3D. Code for ICCV'2025 "Real3D: Scaling Up Large Reconstruction Models with Real-World Images"

3dgs-mcmc. [NeurIPS 2024 Spotlight] Implementation of the paper "3D Gaussian Splatting as Markov Chain Monte Carlo"

Unique3D. [NeurIPS 2024] Unique3D: High-Quality and Efficient 3D Mesh Generation from a Single Image

ChatTTS. A generative speech model for daily dialogue.

3DitScene. [ICLR 2025] 3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting

Diffusion4D. [NeurIPS 2024] Diffusion4D: Fast Spatial-temporal Consistent 4D Generation via Video Diffusion Models

NVS_Solver. Source code of paper "NVS-Solver: Video Diffusion Model as Zero-Shot Novel View Synthesizer"

gaussian_surfels. [SIGGRAPH'24] Implementations for "High-quality Surface Reconstruction using Gaussian Surfels".

unsloth. Unsloth is a local UI for training and running Kimi K3, Gemma 4, Qwen3.6, DeepSeek, GLM and other models.

custom-diffusion360. CustomDiffusion360: Customizing Text-to-Image Diffusion with Camera Viewpoint Control

ohara. Collection of autoregressive model implementation

OpenCLAY. CLAY: A Controllable Large-scale Generative Model for Creating High-quality 3D Assets

VAR. [NeurIPS 2024 Best Paper Award][GPT beats diffusion🔥] [scaling laws in visual generation📈] Official impl. of "Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction". An *ultra-simple, user-friendly yet state-of-the-art* codebase for autoregressive image generation!

como. [ECCV 2024 Oral] COMO: Compact Mapping and Odometry

CityDreamer. The official implementation of "CityDreamer: Compositional Generative Model of Unbounded 3D Cities". (CVPR 2024)

super_primitive. [CVPR'24, Demo Track Honourable Mention] SuperPrimitive: Scene Reconstruction at a Primitive Level

MorpheuS. [CVPR'24] MorpheuS: Neural Dynamic 360° Surface Reconstruction from Monocular RGB-D Video

Den-SOFT. JavaScript

GeoWizard. [ECCV'24] GeoWizard: Unleashing the Diffusion Priors for 3D Geometry Estimation from a Single Image

flatten. Pytorch Implementation of FLATTEN: optical FLow-guided ATTENtion for consistent text-to-video editing (ICLR 2024)

multimodal. TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

V3D. [T-PAMI 2025] V3D: Video Diffusion Models are Effective 3D Generators

GaLore. GaLore: Memory-Efficient LLM Training by Gradient Low-Rank Projection

MonoGS. [CVPR'24 Highlight & Best Demo Award] Gaussian Splatting SLAM

magvit2-pytorch. Implementation of MagViT2 Tokenizer in Pytorch

TCD. Official Repository of the paper "Trajectory Consistency Distillation"

T5-Textual-Inversion. Textual Inversion for DeepFloyd IF

dust3r. DUSt3R: Geometric 3D Vision Made Easy

DSINE. [CVPR 2024 Oral] Rethinking Inductive Biases for Surface Normal Estimation

Medusa. Medusa: Simple Framework for Accelerating LLM Generation with Multiple Decoding Heads

kubric. A data generation pipeline for creating semi-realistic synthetic multi-object videos with rich annotations such as instance segmentation masks, depth maps, and optical flow.

laion-3d. Collect large 3d dataset and build models

ml-hypersim. Hypersim: A Photorealistic Synthetic Dataset for Holistic Indoor Scene Understanding

MVDiffusion_plusplus. MVDiffusion++: A Dense High-resolution Multi-view Diffusion Model for Single or Sparse-view 3D Object Reconstruction

Dream2Real. [ICRA 2024] Dream2Real: Zero-Shot 3D Object Rearrangement with Vision-Language Models

SiT. Official PyTorch Implementation of "SiT: Exploring Flow and Diffusion-based Generative Models with Scalable Interpolant Transformers"

MDT. Masked Diffusion Transformer is the SOTA for image synthesis. (ICCV 2023)

VBench. [CVPR2024 Highlight] VBench - We Evaluate Video Generation

Dataset. News: the 10k dataset is ready for download.

PUG. This is the repository for the Photorealistic Unreal Graphics (PUG) datasets for representation learning.

StableCascade. Official Code for Stable Cascade

gta. [ICLR'24] GTA: A Geometry-Aware Attention Mechanism for Multi-view Transformers

ml-mgie. Python

EscherNet. [CVPR2024 Oral] EscherNet: A Generative Model for Scalable View Synthesis

Marigold. [CVPR 2024 - Oral, Best Paper Award Candidate] Marigold: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

inter. The Inter font family

Megatron-LM. Ongoing research training transformer models at scale

atlas. Code repository for supporting the paper "Atlas Few-shot Learning with Retrieval Augmented Language Models",(https//arxiv.org/abs/2208.03299)

ImageDream. The code releasing for https://image-dream.github.io/

LISA. Project Page for "LISA: Reasoning Segmentation via Large Language Model"

Grounded-Segment-Anything. Grounded SAM: Marrying Grounding DINO with Segment Anything & Stable Diffusion & Recognize Anything - Automatically Detect , Segment and Generate Anything

lang-segment-anything. SAM with text prompt

pixelsplat. [CVPR 2024 Oral, Best Paper Runner-Up] Code for "pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction" by David Charatan, Sizhe Lester Li, Andrea Tagliasacchi, and Vincent Sitzmann

zerorf. ZeroRF: Fast Sparse View 360° Reconstruction with Zero Pretraining

mamba. Mamba SSM architecture

transformer_vq. Official implementation of 'Transformer-VQ: Linear-Time Transformers via Vector Quantization' (ICLR 2024)

AnimateAnyone. Animate Anyone: Consistent and Controllable Image-to-Video Synthesis for Character Animation

consistencydecoder. Consistency Distilled Diff VAE

latent-consistency-model. Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference

Wonder3D. Single Image to 3D using Cross-Domain Diffusion for 3D Generation

Co-SLAM. [CVPR'23] Co-SLAM: Joint Coordinate and Sparse Parametric Encodings for Neural Real-Time SLAM

mistral-inference. Official inference library for Mistral models

switch-scores. PDF Repository of switch score sheets.

waymo-open-dataset. Waymo Open Dataset

threestudio. A unified framework for 3D content generation.

SyncDreamer. [ICLR 2024 Spotlight] SyncDreamer: Generating Multiview-consistent Images from a Single-view Image

MVDream. code page placeholder

OBELICS. Code used for the creation of OBELICS, an open, massive and curated collection of interleaved image-text web documents, containing 141M documents, 115B text tokens and 353M images.

visual-spatial-reasoning. [TACL'23] VSR: A probing benchmark for spatial undersranding of vision-language models.

scalingup. [CoRL 2023] This repository contains data generation and training code for Scaling Up & Distilling Down

zero123-hf. A diffuser implementation of Zero123. Zero-1-to-3: Zero-shot One Image to 3D Object (ICCV23)

perceiver-io. A PyTorch implementation of Perceiver, Perceiver IO and Perceiver AR with PyTorch Lightning scripts for distributed training

stablediffusion-infinity. Outpainting with Stable Diffusion on an infinite canvas

act3d-chained-diffuser. A unified architecture for multimodal multi-task robotic policy learning.

One-2-3-45. [NeurIPS 2023] Official code of "One-2-3-45: Any Single Image to 3D Mesh in 45 Seconds without Per-Shape Optimization"

hiveformer. Python

MVDiffusion. MVDiffusion: Enabling Holistic Multi-view Image Generation with Correspondence-Aware Diffusion, NeurIPS 2023 (spotlight)

perception_test. Jupyter Notebook

LightGlue. LightGlue: Local Feature Matching at Light Speed (ICCV 2023)

DepthCov. [CVPR 2023] Learning a Depth Covariance Function

nope. [CVPR 2024] PyTorch implementation of NOPE: Novel Object Pose Estimation from a Single Image

OmniObject3D. [ CVPR 2023 Award Candidate ] OmniObject3D: Large-Vocabulary 3D Object Dataset for Realistic Perception, Reconstruction and Generation

rl. A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

point2vec. [GCPR 2023 | CVPR 2023 Workshop] Self-Supervised Representation Learning on Point Clouds

FAMO. Official PyTorch Implementation for Fast Adaptive Multitask Optimization (FAMO)

TaleCrafter. [SIGGRAPH Asia 2023] An interactive story visualization tool that support multiple characters

qlora. QLoRA: Efficient Finetuning of Quantized LLMs

sd-leap-booster. Fast finetuning using a booster model that puts the initial state to a local minimum

Everything-LLMs-And-Robotics. The world's largest GitHub Repository for LLMs + Robotics

hmm. Heightmap meshing utility.

llm-foundry. LLM training code for Databricks foundation models

datacomp. DataComp: In search of the next generation of multimodal datasets

Point-MAE. [ECCV2022] Masked Autoencoders for Point Cloud Self-supervised Learning

sdf. Parallelized triangle mesh --> continuous signed distance field on CPU

arnold. [ICCV 2023] ARNOLD: Language-Grounded Robot Manipulation with Continuous Object States in Realistic 3D Scenes

mmc4. MultimodalC4 is a multimodal extension of c4 that interleaves millions of images with text.

StableLM. StableLM: Stability AI Language Models

instruct-pix2pix. Python

mesh-to-depth. Depth map generator for Python written in C++

shape-adaptor. The implementation of "Shape Adaptor: A Learnable Resizing Module" [ECCV 2020].

auto-lambda. The Implementation of "Auto-Lambda: Disentangling Dynamic Task Relationships" [TMLR 2022].

maxl. The implementation of "Self-Supervised Generalisation with Meta Auxiliary Learning" [NeurIPS 2019].

mtan. The implementation of "End-to-End Multi-Task Learning with Attention" [CVPR 2019].

reco. The implementation of "Bootstrapping Semantic Segmentation with Regional Contrast" [ICLR 2022].

genvs.

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

Multi-Task-Transformer. Code of ICLR2023 paper "TaskPrompter: Spatial-Channel Multi-Task Prompting for Dense Scene Understanding" and ECCV2022 paper "Inverted Pyramid Multi-task Transformer for Dense Scene Understanding"

tomesd. Speed up Stable Diffusion with this one simple trick!

mvp. Masked Visual Pre-training for Robotics

gpt4all. GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.

open_flamingo. An open-source framework for training large multimodal models.

Make-It-3D. [ICCV 2023] Make-It-3D: High-Fidelity 3D Creation from A Single Image with Diffusion Prior

promptplusplus. Jupyter Notebook

Painter. Painter & SegGPT Series: Vision Foundation Models from BAAI

typst. A markup-based typesetting system that is powerful and easy to learn.

tactile-dexterity. Official implementation of Dexterity from Touch: Self-Supervised Pre-Training of Tactile Representations with Robotic Play.

zero123. Zero-1-to-3: Zero-shot One Image to 3D Object (ICCV 2023)

ResizeRight. The correct way to resize images or tensors. For Numpy or Pytorch (differentiable).

r3m. Pre-training Reusable Representations for Robotic Manipulation Using Diverse Human Video Data

frankapy. Python interface to control Franka Emika Panda Research Robot Arms.

oscar. Data-Driven Operational Space Control for Adaptive and Robust Robot Manipulation

isaacgym-stubs. Isaac Gym Python Stubs for Code Completion

OpenChatKit. Python

TaskMatrix. Python

ELITE. ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation (ICCV 2023, Oral)

diffusion_policy. [RSS 2023] Diffusion Policy Visuomotor Policy Learning via Action Diffusion

pizza. [3DV 2022 (Oral)] Pytorch implementation of "PIZZA: A Powerful Image-only Zero-Shot Zero-CAD Approach to 6 DoF Tracking" paper

prismer. The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

go-explore. Code for Go-Explore: a New Approach for Hard-Exploration Problems

apriltag-imgs. Pre-generated AprilTag images

apriltag. Extensions and tweaks to APRIL Robotics Laboratory apriltag C software

chatgpt_please_improve_my_paper_writing. a thin wrapper of chatgpt for improving paper writing.

realfusion. Official code for "RealFusion: 360° Reconstruction of Any Object from a Single Image" (CVPR 2023)

hiveformer-corl. PyTorch implementation of the Hiveformer research paper

PromptCraft-Robotics. Community for applying LLMs to robotics and a robot simulator with ChatGPT integration

calvin. CALVIN - A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks

MiDaS. Code for robust monocular depth estimation described in "Ranftl et. al., Towards Robust Monocular Depth Estimation: Mixing Datasets for Zero-shot Cross-dataset Transfer, TPAMI 2022"

FlexLLMGen. Running large language models on a single GPU for throughput-oriented scenarios.

auxinash. Official implementation of Auxiliary learning as an Bargaining Game.

ov-seg. This is the official PyTorch implementation of the paper Open-Vocabulary Semantic Segmentation with Mask-adapted CLIP.

ManiSkill. Manipulation Skill Framework, an open source GPU parallelized robotics simulator and benchmark

ControlNet. Let us control diffusion models!

omni3d. Code release for "Omni3D A Large Benchmark and Model for 3D Object Detection in the Wild"

X-Decoder. [CVPR 2023] Official Implementation of X-Decoder for generalized decoding for pixel, image and language

Open-Assistant. OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

H3. Language Modeling with the H3 State Space Model

vMAP. [CVPR 2023] vMAP: Vectorised Object Mapping for Neural Field SLAM

mm-cot. Official implementation for "Multimodal Chain-of-Thought Reasoning in Language Models" (stay tuned and more will be updated)

SQA3D. [ICLR 2023] SQA3D for embodied scene understanding and reasoning

D4RL. A collection of reference environments for offline reinforcement learning

language-table. Suite of human-collected datasets and a multi-task continuous control benchmark for open vocabulary visuolinguomotor learning.

mask-auto-labeler. Python

iris. Transformers are Sample-Efficient World Models. ICLR 2023, notable top 5%.

ECON. [CVPR'23, Highlight] ECON: Explicit Clothed humans Optimized via Normal integration

SL-Decoder. This is the code for the application of our structured lighting system. We provide with structured lighting patterns and the decoder, for recovering the scene depth. In some cases, this depth is treated as depth ground truth.

M3Depth. Code for Self-Supervised Depth Estimation in Laparoscopic Image using 3D Geometric Consistency (MICCAI 2022)

taichi-ngp-renderer. An Instants-NGP renderer that has been implemented using Taichi

flash-attention. Fast and memory-efficient exact attention

kernl. Kernl lets you run PyTorch transformer models several times faster on GPU with a single line of code, and is designed to be easily hackable.

PlayCover. Community fork of PlayCover

DB. A PyTorch implementation of "Real-time Scene Text Detection with Differentiable Binarization".

EasyOCR. Ready-to-use OCR with 80+ supported languages and all popular writing scripts including Latin, Chinese, Arabic, Devanagari, Cyrillic and etc.

surface_normal_uncertainty. [ICCV 2021 Oral] Estimating and Exploiting the Aleatoric Uncertainty in Surface Normal Estimation

DataAugmentationForObjectDetection. Data Augmentation For Object Detection

stable-dreamfusion. Text-to-3D & Image-to-3D & Mesh Exportation with NeRF + Diffusion.

open_clip. An open source implementation of CLIP.

PureCLIPNeRF. Python

UniDet. Object detection on multiple datasets with an automatically learned unified label space.

CuPL. Python