Manli Shu

RAGEN. RAGEN leverages reinforcement learning to train LLM reasoning agents in interactive, stochastic environments.

Awesome-ML-SYS-Tutorial. My learning notes for ML SYS.

cambrian. Cambrian-1 is a family of multimodal LLMs with a vision-centric design.

Tensor-Puzzles. Solve puzzles. Improve your pytorch.

RL4VLM. Official Repo for Fine-Tuning Large Vision-Language Models as Decision-Making Agents via Reinforcement Learning

verl. verl/HybridFlow: A Flexible and Efficient RL Post-Training Framework

TinyZero. Minimal reproduction of DeepSeek R1-Zero

OmAgent. [EMNLP-2024] Build multimodal language agents for fast prototype and production

open-instruct. AllenAI's post-training codebase

OpenRLHF. An Easy-to-use, Scalable and High-performance Agentic RL Framework based on Ray (PPO & DAPO & REINFORCE++ & VLM & TIS & vLLM & Ray & Async RL)

LATTE. Python

ProVision. A instruction data generation system for multimodal language models.

attention_sinks. Extend existing LLMs way beyond the original training length with constant memory usage, without retraining

Hierarchical_Point_Attention. Python

MINT-1T. 🍃 MINT-1T: A one trillion token multimodal interleaved dataset.

SoM-LLaVA. [COLM-2024] List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs

diffusion_memorization. Official repo for Detecting, Explaining, and Mitigating Memorization in Diffusion Models (ICLR 2024)

carving. Package to optimize Adversarial Attacks against (Large) Language Models with Varied Objectives

open_flamingo. An open-source framework for training large multimodal models.

LAVIS. LAVIS - A One-stop Library for Language-Vision Intelligence

xtuner. A Next-Generation Training Engine Built for Ultra-Large MoE Models

consistencydecoder. Consistency Distilled Diff VAE

nanoGPT. The simplest, fastest repository for training/finetuning medium-sized GPTs.

Megatron-LM. Ongoing research training transformer models at scale

NEFTune. Official repository of NEFTune: Noisy Embeddings Improves Instruction Finetuning

DeepSpeed. DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

ddim. Denoising Diffusion Implicit Models

glide-text2im. GLIDE: a diffusion-based text-conditional image synthesis model

FastChat. An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

lm-evaluation-harness. A framework for few-shot evaluation of language models.

AutoPoison. The official repository of the paper "On the Exploitability of Instruction Tuning".

GOAT. Official implementation of GOAT model (ICML2023)

DCR. Official Pytorch repo of CVPR'23 and NeurIPS'23 papers on understanding replication in diffusion models.

tree-ring-watermark. Python

BYOD. The Official Repository for "Bring Your Own Data! Self-Supervised Evaluation for Large Language Models"

guidance. A guidance language for controlling large language models.

evals. Evals is a framework for evaluating LLMs and LLM systems, and an open-source registry of benchmarks.

TPT. Test-time Prompt Tuning (TPT) for zero-shot generalization in vision-language models (NeurIPS 2022))

dev. Press the . key on any repo

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

ContrastiveDecoding. contrastive decoding

lm-watermarking. Jupyter Notebook

open_clip. An open source implementation of CLIP.

snip-dedup. Python

textual_inversion. Jupyter Notebook

latent-diffusion. High-Resolution Image Synthesis with Latent Diffusion Models

NL-Augmenter. NL-Augmenter 🦎 → 🐍 A Collaborative Repository of Natural Language Transformations

stable-diffusion. A latent text-to-image diffusion model

pytorch-lightning. Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

kornia. 🐍 Geometric Computer Vision Library for Spatial AI

byol-pytorch. Usable Implementation of "Bootstrap Your Own Latent" self-supervised learning, from Deepmind, in Pytorch

IDPG. IDPG: An Instance-Dependent Prompt Generation Method

3detr. Code & Models for 3DETR - an End-to-end transformer model for 3D object detection

federated. A collection of Google research projects related to Federated Learning and Federated Analytics.

vit-visualization. Shell

pytorch-image-models. The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

OpenPrompt. An Open-Source Framework for Prompt-Learning.

PrefixTuning. Prefix-Tuning: Optimizing Continuous Prompts for Generation

prompt-tuning. Original Implementation of Prompt Tuning from Lester, et al, 2021

nerfies.github.io. JavaScript

Group-Free-3D. Group-Free 3D Object Detection via Transformers

AdvBN. Official PyTorch implementation of the NeurIPS 2021 paper: Encoding Robustness to Image Style via Adversarial Feature Perturbations.

dialoglue. DialoGLUE: A Natural Language Understanding Benchmark for Task-Oriented Dialogue

Compositional_Learning. Compositional Learning for Human Object Interaction

P-tuning-v2. An optimized deep prompt tuning strategy comparable to fine-tuning across scales and tasks

pytorch-maml. PyTorch implementation of MAML: https://arxiv.org/abs/1703.03400

tent. ICLR21 Tent: Fully Test-Time Adaptation by Entropy Minimization

CoOp. Prompt Learning for Vision-Language Models (IJCV'22, CVPR'22)

ImageNetV2. A new test set for ImageNet

awesome-self-supervised-learning. A curated list of awesome self-supervised methods

vit-pytorch. Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

GLIP. Grounded Language-Image Pre-training

CLIP. CLIP (Contrastive Language-Image Pretraining), Predict the most relevant text snippet given an image

adversarial_data_augmentation. This repository provides the official PyTorch implementation of the ICRA 2021 paper: Adversarial Differentiable Data Augmentation for Autonomous Systems.

pytorch-AdaIN. Unofficial pytorch implementation of 'Arbitrary Style Transfer in Real-time with Adaptive Instance Normalization' [Huang+, ICCV2017]

semseg. Semantic Segmentation in Pytorch

FACT. Python

moco. PyTorch implementation of MoCo: https://arxiv.org/abs/1911.05722

RobustNet. Official PyTorch implementation of RobustNet (CVPR 2021 Oral)

MetaAug. Python

pytorch-classification-advprop. Python

Deep-Learning-Papers-Reading-Roadmap. Deep Learning papers reading roadmap for anyone who are eager to learn this amazing tech!

computer. C++

fad-texlive-talk. texlive talk materials in FAD Beijing 2011

vision. Datasets, Transforms and Models specific to Computer Vision

pytorch-book. PyTorch tutorials and fun projects including neural talk, neural style, poem writing, anime generation (《深度学习框架PyTorch：入门与实战》)

os_kernel_lab. OS kernel labs based on Rust/C Lang & RISC-V 64/X86-32

ChatterBot. ChatterBot is a machine learning, conversational dialog engine for creating chat bots

chainer-DCGAN. Chainer implementation of Deep Convolutional Generative Adversarial Network

caffe. Caffe: a fast open framework for deep learning.

neural-style. Torch implementation of neural style algorithm

xv6-chinese. 中文版的 MIT xv6 文档