Seitaro Shinagawa

hermes-agent. The agent that grows with you

pdfannots. Extracts and formats text annotations from a PDF file

pySBD. 🐍💯pySBD (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection that works out-of-the-box.

karukan. Japanese Input Method System for Linux, macOS, Neural Kana-Kanji Conversion Engine

academic-research-skills. Academic Research Skills for Claude Code: research → write → review → revise → finalize

claude-session-tracker. Auto-save every Claude Code session to GitHub Projects. Never lose Prompt & Context again.

AutoResearchClaw. Fully autonomous & self-evolving research from idea to paper. Chat an Idea. Get a Paper. 🦞

copilot-cli-for-beginners. Learn how to get started using the GitHub Copilot CLI!

ccusage. npx ccusage

ECC. The agent harness performance optimization system. Skills, instincts, memory, security, and research-first development for Claude Code, Codex, Opencode, Cursor and beyond.

pm-skills. PM Skills Marketplace: 100+ agentic skills, commands, and plugins — from discovery to strategy, execution, launch, and growth.

Representation-as-a-judge. The code for paper "Rethinking LLM-as-a-Judge: Representation-as-a-Judge with Small Language Models via Semantic Capacity Asymmetry", accepted by ICLR 2026.

SE-Bench. Official repo for "SE-Bench: Benchmarking Self-Evolution with Knowledge Internalization"

metamorph. Code for MetaMorph Multimodal Understanding and Generation via Instruction Tuning

embedding-atlas. Embedding Atlas is a tool that provides interactive visualizations for large embeddings. It allows you to visualize, cross-filter, and search embeddings and metadata.

BabyVision. We introduce BabyVision, a benchmark revealing the infancy of AI vision.

evalscope. A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.

Survey4MusicAVQA. Survey for MusicAVQA

JudgeAnything. Python

Daily-Omni. This is the official repository of Daily-Omni: Towards Audio-Visual Reasoning with Temporal Alignment across Modalities

AI-Agents-Projects-Tutorials. Multi-agent systems, memory, planning, reasoning loops

pytest-ml-tdd-example. 「テストを書かない研究者に送る “最初にテストを書く” 実験コード入門 – オレオレ最強 main.py から抜け出すために –」 のサンプルコード

live-vlm-webui. Real-time Vision Language Model interaction via webcam - WebRTC-based web interface

awesome-hallucination-detection. List of papers on hallucination detection in LLMs.

DeepSeek-OCR. Contexts Optical Compression

awesome-vl-compositionality. Awesome Vision-Language Compositionality, a comprehensive curation of research papers in literature.

DACoN. Python

BasicPBC. Official Implementation of "Learning Inclusion Matching for Animation Paint Bucket Colorization"

RLAIF-DialogLLM. Python

VibeCheck. Automated Qualitative Analysis of LLMs (ICLR 2025)

harmony. Renderer for the harmony response format to be used with gpt-oss

EvaLearn. EvaLearn is a pioneering benchmark designed to evaluate large language models (LLMs) on their learning capability and efficiency in challenging tasks.

Awesome-LLM-as-a-judge.

MiCo. [ICCV 2025] Explore the Limits of Omni-modal Pretraining at Scale

ONLY. [ICCV 2025] ONLY: One-Layer Intervention Sufficiently Mitigates Hallucinations in Large Vision-Language Models

UPD. [ACL2025] Unsolvable Problem Detection: Robust Understanding Evaluation for Large Multimodal Models

MergeToVLRM. Source code of our paper "Transferring Textual Preferences to Vision-Language Understanding through Model Merging", ACL 2025

MLLM-Judge. [ICML 2024 Oral] Official code repository for MLLM-as-a-Judge.

llm-jp-judge. 生成自動評価を行うためのPythonツール

Awesome_Think_With_Images. Resources and paper list for "Thinking with Images for LVLMs". This repository accompanies our survey on how LVLMs can leverage visual information for complex reasoning, planning, and generation.

torchtitan. A PyTorch native platform for training generative AI models

srunx. A modern Python library for SLURM workload manager integration with workflow orchestration capabilities.

Awesome-Unified-Multimodal-Models. 📖 This is a repository for organizing papers, codes and other resources related to unified multimodal models.

Awesome-Unified-Multimodal-Models. Awesome Unified Multimodal Models

FinMME. [ACL 2025] FinMME: Benchmark Dataset for Financial Multi-Modal Reasoning Evaluation

policy. TypeScript

Evaluation-Agent. [ACL2025 Oral & Award] Evaluate Image/Video Generation like Humans - Fast, Explainable, Flexible

TRACT. Python

vision-explanation-methods. Methods for creating saliency maps for computer vision models.

pixmo-docs. ACL 2025: Synthetic data generation pipelines for text-rich images.

awesome_cs-ja_phd_life. collection of articles about PhD life written in 🇯🇵

visual_inference_chain. This repository contains the official code for our paper: Thinking Before Looking: Improving Multimodal LLM Reasoning via Mitigating Visual Hallucination

gpt-cost-estimator. A cost estimator for OpenAI API calls in tqdm loops.

gpu-aquarium. JavaScript

AutoConverter. Official implementation of "Automated Generation of Challenging Multiple-Choice Questions for Vision Language Model Evaluation" (CVPR 2025)

llm-jp-eval-mm. A lightweight framework for evaluating visual-language models.

Awesome-LVLM-Hallucination. up-to-date curated list of state-of-the-art Large vision language models hallucinations research work, papers & resources

From-Redundancy-to-Relevance. [NAACL 2025 Oral] From redundancy to relevance: Enhancing explainability in multimodal large language models

web-ui. 🖥️ Run AI Agent in your browser.

SlideVQA. SlideVQA: A Dataset for Document Visual Question Answering on Multiple Images (AAAI2023)

Open-LLM-Leaderboard. Open-LLM-Leaderboard: Open-Style Question Evaluation. Paper at https://arxiv.org/abs/2406.07545

Sa2VA. Official Repo For Pixel-LLM Codebase: Sa2VA (T-PAMI-26), SAMTok (CVPR-26), VRT (Arxiv-25), SaSaSa2VA (1-st solution for LSVOS)

Vision-Language-Models-Overview. A most Frontend Collection and survey of vision-language model papers, and models GitHub repository. Continuous updates.

japanese-clip-qwen2_vl. Jupyter Notebook

DINO-X-API. DINO-X: The World's Top-Performing Vision Model for Open-World Object Detection and Understanding

pfgen-bench. Preferred Generation Benchmark

LLM-generated-Text-Detection. A survey and reflection on the latest research breakthroughs in LLM-generated Text detection, including data, detectors, metrics, current issues and future directions.

clipscore. CLIPScore EMNLP code

vllm-safety-benchmark. [ECCV 2024] Official PyTorch Implementation of "How Many Unicorns Are in This Image? A Safety Evaluation Benchmark for Vision LLMs"

Awesome-LVLM-Attack. 😎 up-to-date & curated list of awesome Attacks on Large-Vision-Language-Models papers, methods & resources.

Awesome-MLLM-Safety. Accepted by IJCAI-24 Survey Track

Visual-Adversarial-Examples-Jailbreak-Large-Language-Models. Repository for the Paper (AAAI 2024, Oral) --- Visual Adversarial Examples Jailbreak Large Language Models

HarmBench. HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal

Prism. A Framework for Decoupling and Assessing the Capabilities of VLMs

FigStep. [AAAI'25 (Oral)] Jailbreaking Large Vision-language Models via Typographic Visual Prompts

VLGuard. [ICML 2024] Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models.

MLLMGuard. Python

16824-CaLORAify. Python

Inpaint-Anything. Inpaint anything using Segment Anything and inpainting models.

fiftyone. Refine high-quality datasets and visual AI models

Multi-modal-Self-instruct. The codebase for our EMNLP24 paper: Multimodal Self-Instruct: Synthetic Abstract Image and Visual Reasoning Instruction Using Language Model

ImgTrojan. Code and data for "ImgTrojan: Jailbreaking Vision-Language Models with ONE Image"

gRefCOCO. A benchmark dataset for GREx: GRES, GREC, and GREG [CVPR 2023 & IJCV 2026]

FaithD2T. Dataset and Code for Generating Faithful and Salient Text from Multimodal Data (INLG 2024) paper.

OmniDocBench. [CVPR 2025] A Comprehensive Benchmark for Document Parsing and Evaluation

flair. [CVPR 2025] FLAIR: VLM with Fine-grained Language-informed Image Representations

PathWeave. Code for paper "LLMs Can Evolve Continually on Modality for X-Modal Reasoning" NeurIPS2024

OmniBench. A project for tri-modal LLM benchmarking and instruction tuning.

Efficient-Multimodal-LLMs-Survey. Efficient Multimodal Large Language Models: A Survey

vstar. PyTorch Implementation of "V* : Guided Visual Search as a Core Mechanism in Multimodal LLMs"

GraphVis. Python

VITA. ✨✨[NeurIPS 2025] VITA-1.5: Towards GPT-4o Level Real-Time Vision and Speech Interaction

awesome-japanese-llm. 日本語LLMまとめ - Overview of Japanese LLMs

ALM-Bench. [CVPR 2025 🔥] ALM-Bench is a multilingual multi-modal diverse cultural benchmark for 100 languages across 19 categories. It assesses the next generation of LMMs on cultural inclusitivity.

semantic-entropy-probes. Jupyter Notebook

yomitoku. YomiTokuはAIを活用した日本語文書解析エンジンを提供するPythonパッケージです。 Yomitoku is an AI-powered document image analysis package designed specifically for the Japanese language.

ml-aim. This repository provides the code and model checkpoints for AIMv1 and AIMv2 research projects.

MMGenBench. Official repository of MMGenBench

mPLUG-DocOwl. mPLUG-DocOwl: Modularized Multimodal Large Language Model for Document Understanding

evaluation-guidebook. Sharing both practical insights and theoretical knowledge about LLM evaluation that we gathered while managing the Open LLM Leaderboard and designing lighteval!

AutoSurvey. Python

litellm. The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

LRP-eXplains-Transformers. Layer-wise Relevance Propagation for Large Language Models and Vision Transformers [ICML 2024]

DocLayout-YOLO. DocLayout-YOLO: Enhancing Document Layout Analysis through Diverse Synthetic Data and Global-to-Local Adaptive Perception

rbo. Implementation of Rank-biased Overlap

docling. Get your documents ready for gen AI

zerox. OCR & Document Extraction using vision models

llama-cookbook. Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

WindowsAgentArena. Windows Agent Arena (WAA) 🪟 is a scalable OS platform for testing and benchmarking of multi-modal AI agents.

OmniParser. A simple screen parsing tool towards pure vision based GUI agent

vlm-recipes. Python

MATRIX-Gen.

hypvl. This repository is related to 'Intriguing Properties of Hyperbolic Embeddings in Vision-Language Models', published at TMLR (2024), https://openreview.net/pdf?id=P5D2gfi4Gg

Awesome-Interpretability-in-Large-Language-Models. This repository collects all relevant resources about interpretability in LLMs

AwesomeLLM4SE. [SCIS 2025] A Survey on Large Language Models for Software Engineering

WorkBench. WorkBench: a Benchmark Dataset for Agents in a Realistic Workplace Setting. COLM 2024.

mid.metric. Python

Polos. [CVPR24 Highlights] Polos: Multimodal Metric Learning from Human Feedback for Image Captioning

MMed-RAG. [ICLR'25] MMed-RAG: Versatile Multimodal RAG System for Medical Vision Language Models

donut. Official Implementation of OCR-free Document Understanding Transformer (Donut) and Synthetic Document Generator (SynthDoG), ECCV 2022

synthtiger. Official Implementation of SynthTIGER (Synthetic Text Image Generator), ICDAR 2021

awesome-vlm-architectures. Famous Vision Language Models and Their Architectures

swarm. Educational framework exploring ergonomic, lightweight multi-agent orchestration. Managed by OpenAI Solution team.

Ovis. A novel Multimodal Large Language Model (MLLM) architecture, designed to structurally align visual and textual embeddings.

synth_doc_generation. Official PyTorch Implementation of DocSynth: A Layout Guided Approach for Controllable Document Image Synthesis - ICDAR 2021

ChatDev. ChatDev 2.0: Dev All through LLM-powered Multi-Agent Collaboration

Marigold. [CVPR 2024 - Oral, Best Paper Award Candidate] Marigold: Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation

agentscope. Build and run agents you can see, understand and trust.

Openreview. data from ICLR OpenReview and code for data analysis

haloscope. source code for NeurIPS'24 paper "HaloScope: Harnessing Unlabeled LLM Generations for Hallucination Detection"

GPTGeoChat. Repository for "Granular Privacy Control for Geolocation with Vision Language Models"

LVCD. The official code of paper "LVCD: Reference-based Lineart Video Colorization with Diffusion Models"

Qwen3-VL. Qwen3-VL is the multimodal large language model series developed by Qwen team, Alibaba Cloud.

MMVP. Python

lmdeploy. LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

vlm-evaluation. VLM Evaluation: Benchmark for VLMs, spanning text generation tasks from VQA to Captioning

AI-Scientist. The AI Scientist: Towards Fully Automated Open-Ended Scientific Discovery 🧑‍🔬

ddpm-segmentation. Label-Efficient Semantic Segmentation with Diffusion Models (ICLR'2022)

Diffusion-based-Segmentation. This is the official Pytorch implementation of the paper "Diffusion Models for Implicit Image Segmentation Ensembles".

Teacher-free-Knowledge-Distillation. Knowledge Distillation: CVPR2020 Oral, Revisiting Knowledge Distillation via Label Smoothing Regularization

DriveAGI. [CVPR 2024 Highlight] GenAD: Generalized Predictive Model for Autonomous Driving

MetaCLIP. NeurIPS 2025 Spotlight; ICLR2024 Spotlight; CVPR 2024; EMNLP 2024

MM-Vet. MM-Vet: Evaluating Large Multimodal Models for Integrated Capabilities (ICML 2024)

GVIL. Code and data for EMNLP 2023 paper "Grounding Visual Illusions in Language: Do Vision-Language Models Perceive Illusions Like Humans?"

reflex. 🕸️ Web apps in pure Python 🐍

surya. OCR, layout analysis, reading order, table recognition in 90+ languages

sam2. The repository provides code for running inference with the Meta Segment Anything Model 2 (SAM 2), links for downloading the trained model checkpoints, and example notebooks that show how to use the model.

flexeval. Flexible evaluation tool for language models

evolutionary-model-merge. Official repository of Evolutionary Optimization of Model Merging Recipes

AMEX-codebase. Python

BiVLC. Official repo for "BiVLC: Extending Vision-Language Compositionality Evaluation with Text-to-Image Retrieval." NeurIPS 2024

COLA. COLA: Evaluate how well your vision-language model can Compose Objects Localized with Attributes!

Enhance-FineGrained. [CVPR 2024] Contrasting Intra-Modal and Ranking Cross-Modal Hard Negatives to Enhance Visio-Linguistic Fine-grained Understanding

vision-language-models-are-bows. Experiments and data for the paper "When and why vision-language models behave like bags-of-words, and what to do about it?" Oral @ ICLR 2023

why-winoground-hard. Code for 'Why is Winoground Hard? Investigating Failures in Visuolinguistic Compositionality', EMNLP 2022

Winoground-T2I.

VBench. [CVPR2024 Highlight] VBench - We Evaluate Video Generation

EXAMS-V. A Multi-discipline Multilingual Multimodal Exam Benchmark

web2code. Web2Code: A Large-scale Webpage-to-Code Dataset and Evaluation Framework for Multimodal LLMs

MMT-Bench. [ICML 2024] | MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI

VIMABench. Official Task Suite Implementation of ICML'23 Paper "VIMA: General Robot Manipulation with Multimodal Prompts"

MM-SafetyBench. Accepted by ECCV 2024

MuirBench. A Comprehensive Benchmark for Robust Multi-image Understanding

ml-ferret. Python

INS-MMBench. [ICCV '25] INS-MMBench: A Comprehensive Benchmark for Evaluating LVLMs' Performance in Insurance

LRP-for-ResNet. [ECCV24] Layer-Wise Relevance Propagation with Conservation Property for ResNet

ferret. A python package for benchmarking interpretability techniques on Transformers.

MLLM-Bench. MLLM-Bench: Evaluating Multimodal LLMs with Per-sample Criteria

BEAF. [ECCV’24] Official repository for "BEAF: Observing Before-AFter Changes to Evaluate Hallucination in Vision-language Models"

FoolyourVLLMs. [ICML 2024] Fool Your (Vision and) Language Model With Embarrassingly Simple Permutations

vhs_benchmark. 🔥 [ICLR 2025] Official Benchmark Toolkits for "Visual Haystacks: A Vision-Centric Needle-In-A-Haystack Benchmark"

Hemm. A holistic evaluation library for multi-modal generative models using Weave

DeCLIP. Supervision Exists Everywhere: A Data Efficient Contrastive Language-Image Pre-training Paradigm

Reflection_Tuning. [ACL'24] Selective Reflection-Tuning: Student-Selected Data Recycling for LLM Instruction-Tuning

FastChat. An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

WildVision-Bench. Python

TFC-pretraining. Self-supervised contrastive learning for time series via time-frequency consistency

yolov10. YOLOv10: Real-Time End-to-End Object Detection [NeurIPS 2024]

yolov9. Implementation of paper - YOLOv9: Learning What You Want to Learn Using Programmable Gradient Information

TF-ID. TF-ID: Table/Figure IDentifier for academic papers

Transformer-MM-Explainability. [ICCV 2021- Oral] Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

CLIP_Surgery. [Pattern Recognition 25] CLIP Surgery for Better Explainability with Enhancement in Open-Vocabulary Tasks

CLIP_Explainability. code for studying OpenAI's CLIP explainability

IllusionVQA. This repository contains the data and code of the paper titled "IllusionVQA: A Challenging Optical Illusion Dataset for Vision Language Models"

iglu-datasets. Python

MileBench. This repo contains evaluation code for the paper "MileBench: Benchmarking MLLMs in Long Context"

VLMEvalKit. Open-source evaluation toolkit of large multi-modality models (LMMs), support 220+ LMMs, 80+ benchmarks

data-juicer. Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

MMBench. Official Repo of "MMBench: Is Your Multi-modal Model an All-around Player?"

LITE. [COLM 2024] LITE: Modeling Environmental Ecosystems with Multimodal Large Language Models

DiagrammerGPT. Official code repository for: DiagrammerGPT: Generating Open-Domain, Open-Platform Diagrams via LLM Planning (COLM 2024)

explanation_based_rescaling. This repository hosts the dataset for explanation based rescaling

POPE. The official GitHub page for ''Evaluating Object Hallucination in Large Vision-Language Models''

SoM-LLaVA. [COLM-2024] List Items One by One: A New Data Source and Learning Paradigm for Multimodal LLMs

MMCoQA. Python

LLaMA-VID. LLaMA-VID: An Image is Worth 2 Tokens in Large Language Models (ECCV 2024)

Cheetah. Python

katna. Tool for automating common video key-frame extraction, video compression and Image Auto-crop/Image-resize tasks

MAVIS. [ICLR 2025] Mathematical Visual Instruction Tuning for Multi-modal Large Language Models

DenseFusion. DenseFusion-1M: Merging Vision Experts for Comprehensive Multimodal Perception