Phil Wang

Vision Transformer. I built image recognition models that work better and faster than older approaches.

PaLM RLHF. I built the code structure to train an AI model using human feedback.

X-Transformers. I built a complete AI model toolkit with experimental research features included.

Diffusion. I built a working image generation model using a noise-removal approach.

Titans. I built a memory system for AI models that handles long documents efficiently.

Vector Quantize. I built a compression tool that AI image and music systems use.

BYOL. I made a tool that teaches image models from unlabeled photos.

Lion. I made a better optimizer for training AI models that Google Brain discovered.

Perceiver. I made an AI model that understands images, video, and audio the same way.

Band Split RoFormer. I built an AI model that splits songs into separate instrument tracks.

Transfusion. I built one AI model that predicts text and generates images together.

Rotary Embedding. I made a tool that adds position awareness to AI transformers.

Rectified Flow. I built a PyTorch implementation of rectified flow for image generation.

Alphafold 3. I made Google's protein-folding AI available for anyone to use.

Enformer. I made Deepmind's gene expression AI runnable and customizable in Python.

EMA. I built a tool that maintains a smoothed copy of your AI model.

Imagen. I built a tool that turns text descriptions into realistic images.

MMDiT. I made the multi-modal AI layer from Stable Diffusion 3 as reusable code.

Tab Transformer. I made an AI model that learns patterns from spreadsheet data.

Slot Attention. I made an AI model that automatically isolates objects in images.

Pi Zero. I built a working version of Physical Intelligence's robot foundation model.

Clinical Calculator. I trained AI models to apply real clinical calculators from patient data.

StyleGAN2. I made an image generator you can train yourself with no coding.

Autoregressive Diffusion. I built an image generation tool that creates pictures step by step.

Native Sparse Attention. I implemented a faster attention pattern for AI models that skips unnecessary calculations.

E2 TTS. I made software that turns text into realistic spoken audio.

MLP Mixer. I implemented Google's all-MLP image recognition model in PyTorch.

SoundStorm. I built a tool that generates audio in parallel instead of sequentially.

AudioLM. I made an AI that generates realistic audio from text descriptions.

Ring Attention. I implemented Ring Attention to let AI models process documents that are millions of tokens long.

GigaGAN. I built Adobe's latest image generation software in Python.

MinGRU. I built a PyTorch implementation of the minGRU neural network.

Lightweight GAN. I made an AI image generator that trains in hours on one computer.

iTransformer. I built an implementation of iTransformer for time series forecasting.

nGPT. I built a PyTorch implementation of Nvidia's normalized GPT architecture.

Local Attention. I built a tool that makes AI models process text faster by focusing locally.

DALLE2 PyTorch. I built DALL-E 2 as open-source code you can run yourself.

MeshGPT. I built an AI model that generates 3D shapes by learning patterns in how geometry works.

Q-Transformer. I built a robot learning system trained on recorded experience.

SE3 Transformer. I built an AI model for understanding 3D molecular geometry.

Soft MoE. I implemented Soft MoE, a mixture-of-experts technique for AI models.

Make-A-Video. I built a text-to-video generator based on Meta's research.

MusicLM. I built a tool that writes original music from text descriptions.

Classifier Free Guidance. I wrote a tool that adds text control to AI image models.

MagViT2. I implemented MagViT2, a video tokenizer that achieves state-of-the-art results.

MEGABYTE. I built a PyTorch implementation of MEGABYTE for processing million-byte sequences.

Speculative Decoding. I built techniques to make AI text generation run faster with less wasted computation.

Self-Rewarding LM. I built code that trains AI models to evaluate and improve themselves.

Video Diffusion. I built an AI tool that generates videos from text descriptions.

Voicebox. I made Meta's Voicebox text-to-speech model work in Pytorch.