Anikait Singh

understanding-rlhf. Learning from preferences is a common paradigm for fine-tuning language models. Yet, many algorithmic design decisions come into play. Our new work finds that approaches employing on-policy sampling or negative gradients outperform offline, maximum likelihood objectives.

PTR. This repository contains the implementation of the PTR algorithm described in the paper: Pre-Training for Robots: Leveraging Diverse Multitask Data via Offline Reinforcement Learning.

fewshot-preference-optimization. Few-Shot Preference Optimization (FSPO) personalizes LLMs by reframing reward modeling as a meta-learning problem, enabling rapid adaptation to user preferences with minimal labeled data, leveraging synthetic datasets for scalability, and achieving high success rates in personalized content generation across multiple domains.

Personalized-Text-To-Image-Diffusion. Public Implementation of PPD

OfflineRlWorkflow. This repository accompanies the following paper: A Workflow for Offline Model-Free Robotic RL

DeepCriminalize. Project that uses GAN's to develop a sketch artist like representation of a criminal. Winners of the Cal Hack Fellowship 2019

D4RL. A collection of reference environments for offline reinforcement learning

byol_rl. Python

epickitchensproc. Jupyter Notebook

Release. Release of Supervision Search. Recently published findings in the IEEE Journal of Translational Engineering in Health and Medicine: A mobile application for keyword search in real-world scenes.

antmaze_gen. Python

Scheme. JavaScript

roboverse. Python

widowx_control. Python

CVODE. Optimization of Stellarator Optimizer Package by Parallelizing the Ordinary Differential Equation Solver using CUDA on the GPU

railrl_evalsawyer. Python

gridworld_notebook. Jupyter Notebook

coq_softwarefoundations. Work on Software Foundations Course

scripts_tpus. Shell