Simo Ryu

quack. A Quirky Assortment of CuTe Kernels

fast.cu. Fastest kernels written from scratch

infinite-kanvas. An infinite canvas image editor using fal.ai

lavender-data. Load & manage evolving datasets efficiently

nano-aha-moment. Single File, Single GPU, From Scratch, Efficient, Full Parameter Tuning library for "RL for LLMs"

NativeSparseAttention. research impl of Native Sparse Attention (2502.11089)

llmdifftracker. Lightweight package that tracks and summarizes code changes using LLMs (Large Language Models)

video-starter-kit. Enable AI models for video production in the browser

diffusion-speedrun. Focused on fast experimentation and simplicity

Cosmos-Tokenizer. A suite of image and video neural tokenizers

lingua. Meta Lingua: a lean, efficient, and easy-to-hack codebase to research LLMs.

mochi. The best OSS video generation models, created by Genmo

modded-nanogpt. NanoGPT (124M) in 90 seconds

ai-robotics. AI Robotics tutorials for hobbyists

SV3D-fine-tune. Fine-tuning code for SV3D

Motion-LoRA. Learning Motion from Low-Rank Adaptation

lancedb. Developer-friendly OSS embedded retrieval library for multimodal AI. Search More; Manage Less.

llm.c. LLM training in simple, raw C/CUDA

efficient_cross_entropy. Python

SpeeD. SpeeD: A Closer Look at Time Steps is Worthy of Triple Speed-Up for Diffusion Model Training

edm2. EDM2 and Autoguidance -- Official PyTorch implementation

llama3. The official Meta Llama 3 GitHub site

DEADiff. [CVPR 2024] Official implementation of "DEADiff: An Efficient Stylization Diffusion Model with Disentangled Representations"

Score-Entropy-Discrete-Diffusion. [ICML 2024 Best Paper] Discrete Diffusion Modeling by Estimating the Ratios of the Data Distribution (https://arxiv.org/abs/2310.16834)

sglang. SGLang is a high-performance serving framework for large language models and multimodal models.

texify. Math OCR model that outputs LaTeX and markdown

mermaid. Generation of diagrams like flowcharts or sequence diagrams from text in a similar manner as markdown

resource-stream. GPU programming related news and material links

min-max-gpt. Minimal (400 LOC) implementation Maximum (multi-node, FSDP) GPT training

OLMo. Modeling, training, eval, and inference code for OLMo

ImprovedTokenMerge. Jupyter Notebook

StanfordQuadruped. Python

chakra-ui. Chakra UI is a component system for building SaaS products with speed ⚡️

lmql. A language for constraint-guided and efficient LLM programming.

mamba. Mamba SSM architecture

gpt-fast. Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

Awesome-Video-Diffusion. A curated list of recent diffusion models for video generation, editing, and various other applications.

optimistix. Nonlinear optimisation (root-finding, least squares, ...) in JAX+Equinox. https://docs.kidger.site/optimistix/

Hotshot-XL. ✨ Hotshot-XL: State-of-the-art AI text-to-GIF model trained to work alongside Stable Diffusion XL

HPSv2. Human Preference Score v2: A Solid Benchmark for Evaluating Human Preferences of Text-to-Image Synthesis

lm-evaluation-harness. A framework for few-shot evaluation of language models.

Bayesian-Red-Teaming. About Official PyTorch implementation of "Query-Efficient Black-Box Red Teaming via Bayesian Optimization" (ACL'23)

streaming. A Data Streaming Library for Efficient Neural Network Training

NeMo-Framework-Launcher. Provides end-to-end model development pipelines for LLMs and Multimodal models that can be launched on-prem or cloud-native.

facechain. FaceChain is a deep-learning toolchain for generating your Digital-Twin.

nougat. Implementation of Nougat Neural Optical Understanding for Academic Documents

LVDM. LVDM: Latent Video Diffusion Models for High-Fidelity Long Video Generation

Qwen. The official repo of Qwen (通义千问) chat & pretrained large language model proposed by Alibaba Cloud.

cog-sdxl-webui. Stable Diffusion XL training and inference as a cog model

MOSS-RLHF. Secrets of RLHF in Large Language Models Part I: PPO

SeedSelect. Code for our papers : "Generating images of rare concepts using pre-trained diffusion models" (AAAI 24) and "Norm-guided latent space exploration for text-to-image generation" (Neurips 23)

cog-sdxl. Stable Diffusion XL training and inference as a cog model

ToolLearningPapers.

awesome-information-geometry. About A collection of AWESOME things about information geometry Topics

LISA. Project Page for "LISA: Reasoning Segmentation via Large Language Model"

GPU-Puzzles. Solve puzzles. Learn CUDA.

Awesome-LLM-Robotics. A comprehensive list of papers using large language/multi-modal models for Robotics/RL, including papers, codes, and related websites

aisys2023.

Awesome_Quadrupedal_Robots. Awesome Quadrupedal Robots

awesome-test-time-adaptation. Collection of awesome test-time (domain/batch/instance) adaptation methods

awesome-llm-security. A curation of awesome tools, documents and projects about LLM Security.

MorphoSymm. Tools for exploiting Morphological Symmetries in robotics

choreographer. [ICLR 2023] Choreographer: a world-model-based agent that discovers and learns unsupervised skills in latent imagination, and it's able to efficiently coordinate and adapt the skills to solve downstream tasks.

sd-webui-3d-editor. A custom extension for sd-webui that with 3D modeling features (add/edit basic elements, load your custom model, modify scene and so on), then send screenshot to txt2img or img2img as your ControlNet's reference image, basing on ThreeJS editor

viper_rl. Using advances in generative modeling to learn reward functions from unlabeled videos.

consistency_models_cifar10. Consistency models trained on CIFAR-10, in JAX.

audiocraft. Audiocraft is a library for audio processing and generation with deep learning. It features the state-of-the-art EnCodec audio compressor / tokenizer, along with MusicGen, a simple and controllable music generation LM with textual and melodic conditioning.

FlagAI. FlagAI (Fast LArge-scale General AI models) is a fast, easy-to-use and extensible toolkit for large-scale model.

efficient-ai-study.

Wuerstchen. Official implementation of Würstchen: Efficient Pretraining of Text-to-Image Models

Minari. A standard format for offline reinforcement learning datasets, with popular reference datasets and related utilities

MDT. Masked Diffusion Transformer is the SOTA for image synthesis. (ICCV 2023)

google-research. Google Research

trident. A performance library for machine learning applications.

CelebBasis. Official Implementation of 'Inserting Anybody in Diffusion Models via Celeb Basis'

prolificdreamer. ProlificDreamer: High-Fidelity and Diverse Text-to-3D Generation with Variational Score Distillation (NeurIPS 2023 Spotlight)

ToolBench. [ICLR'24 spotlight] An open platform for training, serving, and evaluating large language model for tool learning.

ProFusion. Code for Enhancing Detail Preservation for Customized Text-to-Image Generation: A Regularization-Free Approach

KORani. Python

Asyrp_official. official repo for Asyrp : Diffusion Models already have a Semantic Latent Space (ICLR2023)

fastcomposer. [IJCV] FastComposer: Tuning-Free Multi-Subject Image Generation with Localized Attention

ImageBind. ImageBind One Embedding Space to Bind Them All

fairseq. Facebook AI Research Sequence-to-Sequence Toolkit written in Python.

open_llama. OpenLLaMA, a permissively licensed open source reproduction of Meta AI’s LLaMA 7B trained on the RedPajama dataset

numba. NumPy aware dynamic Python compiler using LLVM

Graphit. Official Pytorch implementation of "Graphit: A Unified Framework for Diverse Image Editing Tasks"

IF. Python

lora. Using Low-rank adaptation to quickly fine-tune diffusion models.

StableLM. StableLM: Stability AI Language Models

dinov2. PyTorch code and models for the DINOv2 self-supervised learning method.

get-started-with-JAX. The purpose of this repo is to make it easy to get started with JAX, Flax, and Haiku. It contains my "Machine Learning with JAX" series of tutorials (YouTube videos and Jupyter Notebooks) as well as the content I found useful while learning about the JAX ecosystem.

mujoco_mpc. Real-time behaviour synthesis with MuJoCo, using Predictive Control

segment-anything. The repository provides code for running inference with the SegmentAnything Model (SAM), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

dreamerv3-torch. Implementation of Dreamer v3 in pytorch.

Kandinsky-2. Kandinsky 2 — multilingual text2image latent diffusion model

dreamerv3. Mastering Diverse Domains through World Models

IsaacGymEnvs. Isaac Gym Reinforcement Learning Environments

cleanrl. High-quality single file implementation of Deep Reinforcement Learning algorithms with research-friendly features (PPO, DQN, C51, DDPG, TD3, SAC, PPG)

eai-vc. The repository for the largest and most comprehensive empirical study of visual foundation models for Embodied AI (EAI).

AlpacaDataCleaned. Alpaca dataset from Stanford, cleaned and curated

3DFuse. Official implementation of "Let 2D Diffusion Model Know 3D-Consistency for Robust Text-to-3D Generation"

EVA. EVA Series: Visual Representation Fantasies from BAAI

diffusers-play. Repository with which to explore k-diffusion and diffusers, and within which changes to said packages may be tested.

champagne. An official codebase for paper ":champagne: CHAMPAGNE: Learning Real-world Conversation from Large-Scale Web Videos (ICCV 23)"

EVAL. EVAL(Elastic Versatile Agent with Langchain) will execute all your requests. Just like an eval method!

rl-book. Source codes for the book "Reinforcement Learning: Theory and Python Implementation"

cloob-latent-diffusion. CLOOB Conditioned Latent Diffusion training and inference code

unidiffuser. Code and models for the paper "One Transformer Fits All Distributions in Multi-Modal Diffusion"

ELITE. ELITE: Encoding Visual Concepts into Textual Embeddings for Customized Text-to-Image Generation (ICCV 2023, Oral)

pytorch. Tensors and Dynamic neural networks in Python with strong GPU acceleration

prismer. The implementation of "Prismer: A Vision-Language Model with Multi-Task Experts".

GenLoco. Python

composer. Official implementation of "Composer: Creative and Controllable Image Synthesis with Composable Conditions"

cog. Containers for machine learning

llama. Inference code for Llama models

reduce_reuse_recycle. ICML 2023: Reduce, Reuse, Recycle: Composing Energy-Based Diffusion Models with MCMC

T2I-Adapter. T2I-Adapter

scribble-diffusion. Turn your rough sketch into a refined image using AI

Universal-Guided-Diffusion. Jupyter Notebook

ControlNet. Let us control diffusion models!

rl. A modular, primitive-first, python-first PyTorch library for Reinforcement Learning.

INT8-Flash-Attention-FMHA-Quantization. Cuda

plug-and-play. Official Pytorch Implementation for “Plug-and-Play Diffusion Features for Text-Driven Image-to-Image Translation” (CVPR 2023)

hard-prompts-made-easy. Python

X-Decoder. [CVPR 2023] Official Implementation of X-Decoder for generalized decoding for pixel, image and language

pix2pix-zero. Zero-shot Image-to-Image Translation [SIGGRAPH 2023]

lora-training. LoRA training model packaged with Cog

lora-inference. LoRA inference model packaged with Cog

Attend-and-Excite. Official Implementation for "Attend-and-Excite: Attention-Based Semantic Guidance for Text-to-Image Diffusion Models" (SIGGRAPH 2023)

stylegan-t. [ICML'23] StyleGAN-T: Unlocking the Power of GANs for Fast Large-Scale Text-to-Image Synthesis

dadaptation. D-Adaptation for SGD, Adam and AdaGrad

tuning_playbook. A playbook for systematically maximizing the performance of deep learning models.

instruct-pix2pix. Python

projUNN. Fast training of unitary deep network layers from low-rank updates

DeepFashion2. DeepFashion2 Dataset https://arxiv.org/pdf/1901.07973.pdf

img2dataset. Easily turn large sets of image urls to an image dataset. Can download, resize and package 100M urls in 20h on one machine.

sd-leap-booster. Fast finetuning using a booster model that puts the initial state to a local minimum

sd-webui-additional-networks. Python

GLM-130B. GLM-130B: An Open Bilingual Pre-Trained Model (ICLR 2023)

simple-DVMPC-implemetation. Python

polyglot. Polyglot: Large Language Models of Well-balanced Competence in Multi-languages

pix2latent. Code for: Transforming and Projecting Images into Class-conditional Generative Networks

Gradient-Free-Textual-Inversion. Gradient-Free Textual Inversion for Personalized Text-to-Image Generation

mediapipe. Cross-platform, customizable ML solutions for live and streaming media.

DiffusionDisentanglement. Official implementation of the paper "Uncovering the Disentanglement Capability in Text-to-Image Diffusion Models

portfolio-ideas. A curation of awesome portfolio website ideas for developers and designers to draw inspiration from. Raise a pull request to add more. 💜

Awesome-Dataset-Distillation. A curated list of awesome papers on dataset distillation and related applications.

nestjs-oauth2. NestJS module for customizable, multiple OAuth2 clients

bbangjo.kr. MDX

buffalo. blog landing page(threejs + Typescript + blender)

Dreambooth. Fine-tuning of diffusion models

lora. Using Low-rank adaptation to quickly fine-tune diffusion models.

multimodal. TorchMultimodal is a PyTorch library for training state-of-the-art multimodal multi-task models at scale.

gradient-descent-the-ultimate-optimizer. Code for our NeurIPS 2022 paper

pbrt-v4. Source code to pbrt, the ray tracer described in the forthcoming 4th edition of the "Physically Based Rendering: From Theory to Implementation" book.

stochastic_process_2022W.

Piece-of-Cake. Make your investment life easier.

react-infinite-scroller. ⏬ Infinite scroll component for React in ES6

lovely-tensors. Tensors, for human consumption

stable-diffusion-webui. Stable Diffusion web UI

gatsby-advanced-starter. A high performance skeleton starter for GatsbyJS with an advanced feature set.

flash-attention. Fast and memory-efficient exact attention

cuda-samples. Samples for CUDA Developers which demonstrates features in CUDA Toolkit

LoRA. Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

paint-with-words-sd. Implementation of Paint-with-words with Stable Diffusion : method from eDiff-I that let you generate image from text-labeled segmentation map.

AITemplate. AITemplate is a Python framework which renders neural network into high performance CUDA/HIP C++ code. Specialized for FP16 TensorCore (NVIDIA GPU) and MatrixCore (AMD GPU) inference.

Text2LIVE. Official Pytorch Implementation for "Text2LIVE: Text-Driven Layered Image and Video Editing" (ECCV 2022 Oral)

paint-with-words-sd. Unofficial Implementation of Paint-with-words, method from eDiffi that let you generate image from text-labeled segmentation map.

GODEL. Large-scale pretrained models for goal-directed dialog

torchrec. Pytorch domain library for recommendation systems

magicmix. Unofficial Implementation of MagicMix

bitsandbytes. Accessible large language models via k-bit quantization for PyTorch.

equiformer-pytorch. Implementation of the Equiformer, SE3/E3 equivariant attention network that reaches new SOTA, and adopted for use by EquiFold for protein folding

stable-diffusion-pytorch. Yet another PyTorch implementation of Stable Diffusion (probably easy to read)

gimi. Cloud-based malicious URL detection system.

NATTEN. Fast Multi-dimensional Sparse Attention

Neighborhood-Attention-Transformer. Neighborhood Attention Transformer, arxiv 2022 / CVPR 2023. Dilated Neighborhood Attention Transformer, arxiv 2022

Poisson_flow. Code for NeurIPS 2022 Paper, "Poisson Flow Generative Models" (PFGM)

webgl_study. JavaScript

RecBole. A unified, comprehensive and efficient recommendation library

whisper. Robust Speech Recognition via Large-Scale Weak Supervision

nerf-factory. An awesome PyTorch NeRF library

surrealdb. A scalable, distributed, collaborative, document-graph database, for the realtime web

Dreambooth-Stable-Diffusion. Implementation of Dreambooth (https://arxiv.org/abs/2208.12242) with Stable Diffusion

CrossAttentionControl. Unofficial implementation of "Prompt-to-Prompt Image Editing with Cross Attention Control" with Stable Diffusion

sd-various-ideas. Jupyter Notebook

nestjs-slack-listener. The NestJS module for building the listeners and handlers of Slack events and interactivities.

shortest-even-cycle. Python

RWKV-LM. RWKV (pronounced RwaKuv) is an RNN with great LLM performance, which can also be directly trained like a GPT transformer (parallelizable). We are at RWKV-7 "Goose". So it's combining the best of RNN and transformer - great performance, linear time, constant space (no kv-cache), fast training, infinite ctx_len, and free sentence embedding.

LIVE-Layerwise-Image-Vectorization. [CVPR 2022 Oral] Towards Layer-wise Image Vectorization

sygil-webui. Stable Diffusion web UI

stable-diffusion.

GFPGAN. GFPGAN aims at developing Practical Algorithms for Real-world Face Restoration.

auto-animate. A zero-config, drop-in animation utility that adds smooth transitions to your web app. You can use it with React, Vue, or any other JavaScript application.

wandb. The AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.

three.js. JavaScript 3D Library.

PipeDream. Parallel Training to PipeDream

stable-diffusion. A latent text-to-image diffusion model

kaolin. A PyTorch Library for Accelerating 3D Deep Learning Research

fluentui-emoji. A collection of familiar, friendly, and modern emoji from Microsoft