Rare find

PaLM RLHF. Code to train an AI that learns from human feedback, like ChatGPT.

github.com/lucidrains/PaLM-rlhf-pytorch
by @lucidrains

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

May 2026
  • no changes if reward has no variance for tpo
  • add the sequence level TPO proposed by Jean Kaddour to the repository
October 2025
  • clean
September 2025
  • Release —0.7.1
  • with help of opus, get an impl of flowrl into the repo, still need to…
  • Release —0.6.3
  • address https://github.com/lucidrains/PaLM-rlhf-pytorch/issues/67
  • credit assign
  • Release —0.6.2
  • may have had the wrong sign to the entropy for ppo version all this t…
  • Release —0.6.1
  • add an interesting technique for encouraging exploration for rlvr out…
  • Release —0.6.0
August 2025
  • Release —0.5.5
  • address https://github.com/lucidrains/PaLM-rlhf-pytorch/issues/66
  • Release —0.5.4
  • spo + grpo
April 2025
  • nevermind
  • move to separate file
  • prepare
March 2025
  • researchers at singapore ai lab (SEA) found deepseek r1 conclusions t…
  • Release —0.5.3
February 2025
  • Release —0.5.2
  • small refactor to make room for a symbolic reward model
  • Release —0.5.1
  • move towards using a classification loss for the critic for ppo
  • fix comment
  • think all the samples are eligible for training
  • link to paper at top for grpo
  • first pass through grpo based only on watching Yannic lecture