Jiasen Lu

MeiamSubtitles. Emby Server / Jellyfin Server 端字幕插件 ，使用 迅雷影音、 射手网 接口精准匹配视频字幕

VE-Loss. CVPR 2026 Taming Sampling Perturbations with Variance Expansion Loss for Latent Diffusion Models

ml-atoken. Jupyter Notebook

UniTok. [NeurIPS 2025 Spotlight] A Unified Tokenizer for Visual Generation and Understanding

OneDiffusion. Official implementation of OneDiffusion paper (CVPR 2025)

flashattention2-custom-mask. Triton implementation of FlashAttention2 that adds Custom Masks.

Lumina-T2X. Lumina-T2X is a unified framework for Text to Any Modality Generation

unified-io-2.pytorch. Python

OLMo. Modeling, training, eval, and inference code for OLMo

unified-io-2. Python

neural-compressor. SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

lightning-sam. Fine-tune Segment-Anything Model with Lightning Fabric.

DDPM_inversion. Official pytorch implementation of the paper: "An Edit Friendly DDPM Noise Space: Inversion and Manipulations". CVPR 2024.

SpecVQGAN. Source code for "Taming Visually Guided Sound Generation" (Oral at the BMVC 2021)

riffusion-hobby. Stable diffusion for real-time music generation

ast. Code for the Interspeech 2021 paper "AST: Audio Spectrogram Transformer".

Kandinsky-2. Kandinsky 2 — multilingual text2image latent diffusion model

hifigan. An 16kHz implementation of HiFi-GAN for soft-vc.

hifi-gan. HiFi-GAN: Generative Adversarial Networks for Efficient and High Fidelity Speech Synthesis

faster-rcnn.pytorch. A faster pytorch implementation of faster r-cnn

instant-ngp. Instant neural graphics primitives: lightning fast NeRF and more

unified-io-inference. Jupyter Notebook

mesh-transformer-jax. Model parallel transformers in JAX and Haiku

Unofficial-Pix2Seq. Unofficial implementation of Pix2SEQ

gpt-neo. An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.

Diff-DALLE. Python

DALLE-mtf. Open-AI's DALL-E for large scale training in mesh-tensorflow.

seqio. Task-based datasets, preprocessing, and evaluation for sequence models.

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

tf-invertible-resnet. A TensorFlow implementation of Invertible Residual Networks

DALLE-pytorch. Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

soho. [CVPR'21 Oral] Seeing Out of tHe bOx: End-to-End Pre-training for Vision-Language Representation Learning

train-CLIP. A PyTorch Lightning solution to training OpenAI's CLIP from scratch.

oie-resources. A curated list of Open Information Extraction (OIE) resources: papers, code, data, etc.

DCNv2. Deformable Convolutional Networks v2 with Pytorch

pixel-cnn-pp. Pytorch Implementation of OpenAI's PixelCNN++

lmconv. Code for UAI 2020 paper "Locally Masked Convolution for Autoregressive Models"

dialog_without_dialog. Implementation for the paper Dialog without Dialog Data: Learning Visual DialogAgents from VQA Data

allenact. An open source framework for research in Embodied-AI from AI2.

SimCLR. PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

moco. PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

vilbert-multi-task. Multi Task Vision and Language

RevTorch. Framework for creating (partially) reversible neural networks with PyTorch

reformer-pytorch. Reformer, the efficient Transformer, in Pytorch

pytorch-revnet. Implementation of the reversible residual network in pytorch

transformers. 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

graph-rcnn.pytorch. [ECCV 2018] Official code for "Graph R-CNN for Scene Graph Generation"

leetcode. Provide all my solutions and explanations in Chinese for all the Leetcode coding problems.

leetcode. LeetCode Solutions: A Record of My Problem Solving Journey.( leetcode题解，记录自己的leetcode解题之路。)

CMC. [arXiv 2019] "Contrastive Multiview Coding", also contains implementations for MoCo and InstDis

amdim-public. Public repo for Augmented Multiscale Deep InfoMax representation learning

BERT-related-papers. BERT-related papers

skynet-ddp-slurm-example. Example of using PyTorch DistributedDataParallel and SLURM on skynet

CloserLookFewShot. source code to ICLR'19, 'A Closer Look at Few-shot Classification'

OpenLongTailRecognition-OLTR. Pytorch implementation for "Large-Scale Long-Tailed Recognition in an Open World" (CVPR 2019 ORAL)

DeepLearning-500-questions. 深度学习500问，以问答形式对常用的概率知识、线性代数、机器学习、深度学习、计算机视觉等热点问题进行阐述，以帮助自己及有需要的读者。 全书分为18个章节，近30万字。由于水平有限，书中不妥之处恳请广大读者批评指正。 未完待续............ 如有意合作，联系scutjy2015@163.com 版权所有，违权必究 Tan 2018.06

aqm-plus. PyTorch code for Large-Scale Answerer in Questioner's Mind for Visual Dialog Question Generation (AQM+) (ICLR 2019)

Sentence-VAE. PyTorch Re-Implementation of "Generating Sentences from a Continuous Space" by Bowman et al 2015 https://arxiv.org/abs/1511.06349

habitat-sim. A flexible, high-performance 3D simulator for Embodied AI research.

DownloadConceptualCaptions. Reliably download millions of images efficiently

transformer-xl. Python

IC3Net. Code for ICLR 2019 paper: Learning when to Communicate at Scale in Multiagent Cooperative and Competitive Tasks

maddpg. Code for the MADDPG algorithm from the paper "Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments"

MultimodalGame. Jupyter Notebook

emergent-language. An implementation of Emergence of Grounded Compositional Language in Multi-Agent Populations by Igor Mordatch and Pieter Abbeel

see. Code for the AAAI 2018 publication "SEE: Towards Semi-Supervised End-to-End Scene Text Recognition"

BERT-pytorch. Google AI 2018 BERT pytorch implementation

bert. TensorFlow code and pre-trained models for BERT

maskrcnn-benchmark. Fast, modular reference implementation of Instance Segmentation and Object Detection algorithms in PyTorch.

pytorch_geometric. Graph Neural Network Library for PyTorch

vqa-project. Code for our paper: Learning Conditioned Graph Structures for Interpretable Visual Question Answering

graph-generation. GraphRNN: Generating Realistic Graphs with Deep Auto-regressive Models

EmbodiedQA. Train embodied agents that can answer questions in environments

NNDIAL. NNDial is an open source toolkit for building end-to-end trainable task-oriented dialogue models. It is released by Tsung-Hsien (Shawn) Wen from Cambridge Dialogue Systems Group under Apache License 2.0.

SuperGo. A student implementation of Alpha Go Zero

retro-contest-sonic. World Models applied to the Open AI Sonic Retro Contest

world-models-sonic-pytorch. Attempt at reinforcement learning with curiosity for Sonic the Hedgehog games. Number 149 on OpenAI retro contest leaderboard, but more work needed

relational-rnn-pytorch. An implementation of DeepMind's Relational Recurrent Neural Networks (NeurIPS 2018) in PyTorch.

vqa-counting. [ICLR 2018] Learning to Count Objects in Natural Images for Visual Question Answering

nonauto-nmt. PyTorch Implementation of "Non-Autoregressive Neural Machine Translation"

DMS. Dynamic Multimodal Instance Segmentation Guided by Natural Language Queries, ECCV 2018

PyTorch-Encoding. A CV toolkit for my papers.

talkthewalk. This repository provides code for reproducing experiments of the paper Talk The Walk: Navigating New York City Through Grounded Dialogue by Harm de Vries, Kurt Shuster, Dhruv Batra, Devi Parikh, Jason Weston, and Douwe Kiela.

darts. Differentiable architecture search for convolutional and recurrent networks

pytorch-0.4-yolov3. Yet Another Implimentation of Pytroch 0.4.1 and YoloV3 on python3

coco-caption. Jupyter Notebook

YOLOv3_PyTorch. Full implementation of YOLOv3 in PyTorch

ml-agents. The Unity Machine Learning Agents Toolkit (ML-Agents) is an open-source project that enables games and simulations to serve as environments for training intelligent agents using deep reinforcement learning and imitation learning.

scalable_agent. A TensorFlow implementation of Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures.

decaNLP. The Natural Language Decathlon: A Multitask Challenge for NLP

Matterport3DSimulator. AI Research Platform for Reinforcement Learning from Real Panoramic Images.

pytorch-a2c-ppo-acktr-gail. PyTorch implementation of Advantage Actor Critic (A2C), Proximal Policy Optimization (PPO), Scalable trust-region method for deep reinforcement learning using Kronecker-factored approximation (ACKTR) and Generative Adversarial Imitation Learning (GAIL).

deep-neuroevolution. Deep Neuroevolution

GibsonEnv. Gibson Environments: Real-World Perception for Embodied Agents

deep-painterly-harmonization. Code and data for paper "Deep Painterly Harmonization": https://arxiv.org/abs/1804.03189

noreward-rl. [ICML 2017] TensorFlow code for Curiosity-driven Exploration for Deep Reinforcement Learning

dsod.pytorch. Reimplement DSOD: Learning Deeply Supervised Object Detectors from Scratch.

iter-reason. Code for Iterative Reasoning Paper (CVPR 2018)

DialogStateTracking. Dialog State Tracking using End-to-End Neural Networks

Attention-on-Attention-for-VQA. Visual Question Answering Project with state of the art single Model performance.

DeepRL. Modularized Implementation of Deep RL Algorithms in PyTorch

reinforcement-learning-an-introduction. Python Implementation of Reinforcement Learning: An Introduction

joint_vae. Code to build VAE models that are jointly conditioned.

Detectron. FAIR's research platform for object detection research, implementing popular algorithms like Mask R-CNN and RetinaNet.

MINERVA. Meandering In Networks of Entities to Reach Verisimilar Answers

MAgent. A Platform for Many-Agent Reinforcement Learning

probtorch. Probabilistic Torch is library for deep generative models that extends PyTorch

sentence_gen_tools. Sentence generation tools I use from project to project.

caption-eval. Sentence/Caption evaluation using automated metrics

Mask_RCNN. Mask R-CNN for object detection and instance segmentation on Keras and TensorFlow

FPN. Feature Pyramid Network

scene-graph-TF-release. "Scene Graph Generation by Iterative Message Passing" code repository

pytorch-faster-rcnn. pytorch1.0 updated. Support cpu test and demo. (Use detectron2, it's a masterpiece)

py-faster-rcnn. Faster R-CNN (Python implementation) -- see https://github.com/ShaoqingRen/faster_rcnn for the official MATLAB version

cmn. Code release for Hu et al. Modeling Relationships in Referential Expressions with Compositional Modular Networks. in CVPR, 2017

VNMT. Code for "Variational Neural Machine Translation" (EMNLP2016)

pytorch-deeplab-resnet. DeepLab resnet v2 model in pytorch

pytorch-segmentation. Pytorch for Segmentation

awesome-semantic-segmentation. :metal: awesome-semantic-segmentation

faster_rcnn_pytorch. Faster RCNN with PyTorch

pytorch-semseg. Semantic Segmentation Architectures Implemented in PyTorch

self-critical.pytorch. Unofficial pytorch implementation for Self-critical Sequence Training for Image Captioning. and others.

pytorch-meta-optimizer. A PyTorch implementation of Learning to learn by gradient descent by gradient descent

clevr-iep. Inferring and Executing Programs for Visual Reasoning

ImageCaptioning.pytorch. I decide to sync up this repo and self-critical.pytorch. (The old master is in old master branch for archive)

ParlAI. A framework for training and evaluating AI models on a variety of openly available dialogue datasets.

distributed-jacobi. Distributed implementations of Jacobi Method for solving systems of linear equations using OpenMPI

improved_wgan_training. Code for reproducing experiments in "Improved Training of Wasserstein GANs"

pytorch-containers. Torch Containers simplified in PyTorch

pytorch-tutorial. PyTorch Tutorial for Deep Learning Researchers

Faster-RCNN-Densecap-torch. Faster-RCNN based on Densecap(deprecated)

generative-models. Collection of generative models, e.g. GAN, VAE in Pytorch and Tensorflow.

lsgan. This is the project for LS-GAN (Loss-Sensitive GAN)

SRGAN. Torch implementation of SRGAN (Ledig et al., Photo -Realistic Single Image Super-Resolution Using a Generative Adversarial Network, 2016)

Neural-Dialogue-Generation. Lua

SIGBOVIK17-GUNs. Supporting public code for SIGBOVIK17 submission

stn.pytorch. pytorch version of spatial transformer networks

gated-graph-transformer-network. Code to accompany the paper "Learning Graphical State Transitions"

WassersteinGAN. Python

iGAN. Interactive Image Generation via Generative Adversarial Networks

attention-transfer. Improving Convolutional Networks via Attention Transfer (ICLR 2017)

pytorch. Tensors and Dynamic neural networks in Python with strong GPU acceleration

nips2016. A list of resources for all invited talks, tutorials, workshops and presentations at NIPS 2016

icml2016. Generative Adversarial Text-to-Image Synthesis

nips2016. Learning What and Where to Draw

review_net. Review Network for Caption Generation

d3-cloud. Create word clouds in JavaScript.

Caffe-ExcitationBP. Implementation of Excitation Backprop in Caffe

torch7. http://torch.ch

coco-caption. Adds SPICE metric to coco-caption evaluation server codes

models. Models and examples built with TensorFlow

torch-rnn. Efficient, reusable RNNs and LSTMs for torch

HieCoAttenVQA. Jupyter Notebook

videogan. Generating Videos with Scene Dynamics. NIPS 2016.

Visual-Relationship-Detection. Matlab

neural-storyteller. A recurrent neural network for generating little stories about images

unsup. Some unsupervised learning modules using Torch

lua-knn. torch7 wrapper for knn CUDA code

lua---nnx. An extension to Torch7's nn package.

cbp. Multimodal Compact Bilinear Pooling for Torch7

torch-opencv-demos. Torch7+OpenCV+ConvNets

text2image. Generating Images from Captions with Attention

TensorFlow-VAE-GAN-DRAW. A collection of generative methods implemented with TensorFlow (Deep Convolutional Generative Adversarial Networks (DCGAN), Variational Autoencoder (VAE) and DRAW: A Recurrent Neural Network For Image Generation).

deep-recurrent-attentive-writer. Deep Recurrent Attentive Writer

neural-summary-tensorflow. In progress

imagenet-transfer.torch. Lua

deep-mrf. Lua

texture_nets. Code for "Texture Networks: Feed-forward Synthesis of Textures and Stylized Images" paper.

deep-q-learning. Jupyter Notebook

nmn2. Neural module networks

seq2seq-attn. Sequence-to-sequence model with LSTM encoder/decoders and attention

sent-conv-torch. Text classification using a convolutional neural network.

reinforcejs. Reinforcement Learning Agents in Javascript (Dynamic Programming, Temporal Difference, Deep Q-Learning, Stochastic/Deterministic Policy Gradients)

Associative_LSTM. LSTM with associative memory cells (http://arxiv.org/abs/1602.03032)

luaver. Lua Version Manager - Managing and switching between different versions of Lua, LuaJIT and Luarocks made easy

DQN-tensorflow. Tensorflow implementation of Human-Level Control through Deep Reinforcement Learning

wide-residual-networks. 3.8% and 18.3% on CIFAR-10 and CIFAR-100

JULE.torch. Torch code for our CVPR 2016 paper "Joint Unsupervised LEarning of Deep Representations and Image Clusters"

fast-rcnn-torch. Fast R-CNN Torch Implementation

mscoco-imcap-ranking. Image-caption ranking (for MSCOCO)

artistic-videos. Torch implementation for the paper "Artistic style transfer for videos"

deep-residual-networks. Deep Residual Learning for Image Recognition

BayesianRNN. Code for the paper "A Theoretically Grounded Application of Dropout in Recurrent Neural Networks"

densecap. Dense image captioning in Torch

logo-retrieval. C++

deeplearning-papernotes. Summaries and notes on Deep Learning research papers

VQA_LSTM_CNN. Train a deeper LSTM and normalized CNN Visual Question Answering model. This current code can get 58.16 on OpenEnded and 63.09 on Multiple-Choice on test-standard.

vqakb. Using VQA models to extract distributed representations of VQA knowledge.

torch-gan. Lua

VAE-Torch. Implementation of Variational Auto-Encoder in Torch7

draw. Torch implementation of DRAW: A Recurrent Neural Network For Image Generation

torch-elu. Torch implementation of "Fast and Accurate Deep Network Learning by Exponential Linear Units (ELUs)"

torch-autograd. Autograd automatically differentiates native Torch code

visual-concepts. Code for detecting visual concepts in images.

torch-ntm. A Neural Turing Machine implementation in Torch.

visual-semantic-embedding. Implementation of the image-sentence embedding method described in "Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models"

lstm-char-cnn. LSTM language model with CNN over characters

MemNN. Theano implementation of Memory Networks

rnn. Recurrent Neural Network library for Torch7's nn

lua---parallel. A (simple) parallel computing framework for Lua