This is your work, valued
PyTorch researcher implementing cutting-edge transformer architectures and attention mechanisms from academic papers.
inferred from the public footprint
Phil Wang
Top 4%Phil ships production-ready PyTorch implementations of cutting-edge transformer research, from vit-pytorch and denoising-diffusion-pytorch to x-transformers and PaLM-rlhf-pytorch. His repos have accumulated 182k stars, making him one of the most prolific open source contributors globally.
Vision Transformer. I built image recognition models that work better and faster than older approaches.
25kPaLM RLHF. I built the code structure to train an AI model using human feedback.
7.9kX-Transformers. I built a complete AI model toolkit with experimental research features included.
5.9kDiffusion. I built a working image generation model using a noise-removal approach.
11kTitans. I built a memory system for AI models that handles long documents efficiently.
2kVector Quantize. I built a compression tool that AI image and music systems use.
4kBYOL. I made a tool that teaches image models from unlabeled photos.
1.9kLion. I made a better optimizer for training AI models that Google Brain discovered.
2.2kPerceiver. I made an AI model that understands images, video, and audio the same way.
1.2kBand Split RoFormer. I built an AI model that splits songs into separate instrument tracks.
881Transfusion. I built one AI model that predicts text and generates images together.
1.4kRotary Embedding. I made a tool that adds position awareness to AI transformers.
819Rectified Flow. I built a PyTorch implementation of rectified flow for image generation.
476Alphafold 3. I made Google's protein-folding AI available for anyone to use.
1.7kEnformer. I made Deepmind's gene expression AI runnable and customizable in Python.
572EMA. I built a tool that maintains a smoothed copy of your AI model.
658Imagen. I built a tool that turns text descriptions into realistic images.
8.4kMMDiT. I made the multi-modal AI layer from Stable Diffusion 3 as reusable code.
552Tab Transformer. I made an AI model that learns patterns from spreadsheet data.
1.1kSlot Attention. I made an AI model that automatically isolates objects in images.
493Pi Zero. I built a working version of Physical Intelligence's robot foundation model.
582Clinical Calculator. I trained AI models to apply real clinical calculators from patient data.
315StyleGAN2. I made an image generator you can train yourself with no coding.
3.8kAutoregressive Diffusion. I built an image generation tool that creates pictures step by step.
438Native Sparse Attention. I implemented a faster attention pattern for AI models that skips unnecessary calculations.
811E2 TTS. I made software that turns text into realistic spoken audio.
516MLP Mixer. I implemented Google's all-MLP image recognition model in PyTorch.
1.1kSoundStorm. I built a tool that generates audio in parallel instead of sequentially.
1.5kAudioLM. I made an AI that generates realistic audio from text descriptions.
2.6kRing Attention. I implemented Ring Attention to let AI models process documents that are millions of tokens long.
546GigaGAN. I built Adobe's latest image generation software in Python.
1.9kMinGRU. I built a PyTorch implementation of the minGRU neural network.
325Lightweight GAN. I made an AI image generator that trains in hours on one computer.
1.7kiTransformer. I built an implementation of iTransformer for time series forecasting.
537nGPT. I built a PyTorch implementation of Nvidia's normalized GPT architecture.
300Local Attention. I built a tool that makes AI models process text faster by focusing locally.
503DALLE2 PyTorch. I built DALL-E 2 as open-source code you can run yourself.
11kMeshGPT. I built an AI model that generates 3D shapes by learning patterns in how geometry works.
861Q-Transformer. I built a robot learning system trained on recorded experience.
407SE3 Transformer. I built an AI model for understanding 3D molecular geometry.
331Soft MoE. I implemented Soft MoE, a mixture-of-experts technique for AI models.
348Make-A-Video. I built a text-to-video generator based on Meta's research.
2kMusicLM. I built a tool that writes original music from text descriptions.
3.3kClassifier Free Guidance. I wrote a tool that adds text control to AI image models.
544MagViT2. I implemented MagViT2, a video tokenizer that achieves state-of-the-art results.
668MEGABYTE. I built a PyTorch implementation of MEGABYTE for processing million-byte sequences.
655Speculative Decoding. I built techniques to make AI text generation run faster with less wasted computation.
307Self-Rewarding LM. I built code that trains AI models to evaluate and improve themselves.
1.4kVideo Diffusion. I built an AI tool that generates videos from text descriptions.
1.4kVoicebox. I made Meta's Voicebox text-to-speech model work in Pytorch.
699