KAN-GPT-2. Training small GPT-2 style models using Kolmogorov-Arnold networks.
124trlx-with-T5. [Added T5 support to TRLX] A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)
47Attention-only-transformers. Python
8Climatehack-2022. The code I used to score 0.748 in Climatehack 2022. It also includes code for loading data quickly in PyTorch. In addition to an optical flow model based on Deepmind's Perceiver network.
4IMO_grader. Python
1paper-retraction-detection. TypeScript
1deepseek-alignment-faking. Python
1