Phuc Nguyen

instructGOOSE. Implementation of Reinforcement Learning from Human Feedback (RLHF)

toolformer. Implementation of Toolformer: Language Models Can Teach Themselves to Use Tools

pipegoose. Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

stable-diffusion-from-scratch. Implementation of Stable Diffusion from scratch [WORK IN PROGRESS]

reinforcement-learning. Jupyter Notebook

ai-notebooks. AI notebooks

neuraltext. Decoding neural activity into text

progen. Generating new proteins using language models

flashattention. Implementation of FlashAttention: Fast and Memory-Efficient Exact Attention with IO-Awareness

prodgpt. A production-ready training, evaluation and data pipeline

muzero. Implement MuZero from scratch [WORK IN PROGRESS]

fastgoose. A PyTorch implementation of Model Parallelism and ZeRO Optimizer

nanoGPT-msamp. Integrate MS-AMP into nanoGPT (https://github.com/karpathy/nanoGPT)

saycan. Implementation of SayCan: Grounding Language in Robotic Affordances

homework. Notes

snippets. snippets

mousai. PyTorch Implementation of Moûsai: Text-to-Music Generation with Long-Context Latent Diffusion

Megatron-LM. Ongoing research training transformer models at scale

transformers-starcoder. Python

hf-notebooks. Jupyter Notebook

Jetfire-INT8Training. Jupyter Notebook

megatron-tp. for debugging pipegoose

fsdl-megatron. Code for FSDL Breaking down parallelism in Megatron-LM

mq3-sensor. Python

research-ideas. List of things that I'd love to work on

pipegoose-training. Jupyter Notebook

xrsrke.

fsdl-website. Source for https://fullstackdeeplearning.com

memit. Implementation of Mass Editing Memory in a Transformer

elasticgoose. A fault-tolerant elastic training framework for PyTorch

minitron. A mini Megatron 3D parallelism library for FSDL blog

exex. Experiment codebase for chemchem package

paulinet. Model the wave function ansatz using a neural network

lexios. Symbolic framework for scientific calculation

dreamerv3. Implementation of DreamerV3: Mastering Diverse Domains through World Models