Rare find

ARC Benchmarking. Test how well different AI models solve pattern puzzles.

github.com/arcprize/arc-agi-benchmarking

Problem

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Audience

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Market

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Pricing

The full read on this project is a sign-in away — what it solves, who it serves, and where it sits in the market.

Vaya's read on this project

Problem, audience, market, and the verdict — sign in to see it.

Updates

June 2026

Add custom base_url support for OpenAI-compatible endpoints (#90) Jun 18
Add Message Batches API support to Anthropic adapter (#89) Jun 9

May 2026

Merge pull request #88 from arcprize/docs/readme-uv-update May 26
Document uv workflow in README and set requires-python floor May 26

April 2026

Add Together AI adapter (#87) Apr 27

February 2026

Add qwen3-max-thinking via OpenRouter, move kimi-k2.5 to Fireworks (#84) Feb 10
Fix reasoning parameter for OpenAI Responses API Feb 5
openai base Feb 4
Increase kimi-k2.5 rate limit to 20 req/60s Feb 4
Add workflow to trigger benchmarks on models.yml changes (#79) Feb 3
Add model-level rate limiting (#75) Feb 3
Fix resume not detecting existing submissions on disk (#80) Feb 3

January 2026

[PR 4/7] CLI checkpoint integration (#72) Jan 23
[PR 1/7] Add storage abstraction layer for checkpointing (#69) Jan 23
Add centralized structured logging (#68) Jan 15
Add Claude Code GitHub Action workflow (#67) Jan 15
Add CLAUDE.md and AGENTS.md for AI coding assistants (#66) Jan 15
Merge pull request #61 from arcprize/cleanup/consolidate-adapter-code Jan 15
Merge main into cleanup/consolidate-adapter-code Jan 15
Merge pull request #60 from arcprize/cleanup/remove-sys-path-hacks Jan 15