Dennis Ulmer

catnat-torch. Python

pride-friends-2026. HTML

multilingual-gsm-symbolic. A multilingual symbolic version of the GSM dataset

talkie. talkie is a vintage language model from 1930

uqlm. UQLM: Uncertainty Quantification for Language Models, is a Python package for UQ-based LLM hallucination detection

Evergreen. Python

apricot. Source code of "Calibrating Large Language Models Using Their Generations Only", ACL2024

equivariant_model_selection. Jupyter Notebook

JUDGE-BENCH. Jupyter Notebook

obsidiantools. Obsidian tools - a Python package for analysing an Obsidian.md vault

whoosh. Pure-Python full-text search library

jola. Code for ICML 2025 paper | Joint Localization and Activation Editing for Low-Resource Fine-Tuning

multiloko. A benchmark with locally sourced multilingual questions for 31 languages.

queer_NLP_bib. This repository collects papers that cover queer NLP

FActScore. A package to evaluate factuality of long-form generation. Original implementation of our EMNLP 2023 paper "FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation"

wimbd. What's In My Big Data (WIMBD) - a toolkit for analyzing large text datasets

deep_permutation_invariant. Code accompanying paper "Set Norm and Equivariant Skip Connections: Putting the Deep in Deep Sets."

trl. Train transformer language models with reinforcement learning.

RACDH-old. Real-time Attribution Classification to Detect Hallucinations

perceptions. Perceptions of Probability and Numbers

Trainboard-App. Clean Departure Board mainly for DB Trains running on JavaScript

gfn-lm-tuning. Jupyter Notebook

refer. Referring Expression Datasets API

ollama. Get up and running with Kimi-K2.6, GLM-5.2, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

posteriors. Uncertainty quantification with PyTorch

Diffusion-Models-pytorch. Pytorch implementation of Diffusion Models (https://arxiv.org/pdf/2006.11239.pdf)

uv. An extremely fast Python package and project manager, written in Rust.

large-qa-datasets. A collection of large question answering datasets

awdlkiller. Disables macOS AWDL the moment it starts!

PaCMAP. PaCMAP: Large-scale Dimension Reduction Technique Preserving Both Global and Local Structure

UA_COMET. Repository for "Uncertainty-Aware Machine Translation Evaluation", accepted to Findings of EMNLP 2021.

uncertainties_MT_eval. Code and data for the paper "Disentangling Uncertainty in Machine Translation Evaluation", accepted at EMNLP 2022.

robust_MT_evaluation. Repository for "BLEU Meets COMET: Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation", accepted at EAMT 2023.

lm-polygraph. Python

llm-attacks. Universal and Transferable Attacks on Aligned Language Models

Awesome-LLM-Uncertainty-Reliability-Robustness. Awesome-LLM-Robustness: a curated list of Uncertainty, Reliability and Robustness in Large Language Models

CalibratedMath. Teaching Models to Express Their Uncertainty in Words

bitsandbytes. Accessible large language models via k-bit quantization for PyTorch.

garak. the LLM vulnerability scanner

LLMSurvey. The official GitHub page for the survey paper "A Survey of Large Language Models".

hyperbolic_learning_library. An extension of the PyTorch library containing various tools for performing deep learning in hyperbolic space.

COMET. A Neural Framework for MT Evaluation

syntax-pre-training-for-RE. Silver Syntax Pre-training for Cross-Domain Relation Extraction (Findings of ACL 2023)

prompt-engineering. Tips and tricks for working with Large Language Models like OpenAI's GPT-4.

genbench_cbt_2023. The official Genbench Collaborative Benchmarking Task repository 2023 (Archived)

qdrant. Qdrant - High-performance, massive-scale Vector Database and Vector Search Engine for the next generation of AI. Also available in the cloud https://cloud.qdrant.io/

RL4LMs. A modular RL library to fine-tune language models to human preferences

lm-human-preferences. Code for the paper Fine-Tuning Language Models from Human Preferences

germanic-lrl-corpora. Overview of corpora/datasets for Germanic low-resource languages and dialects. Accompanies "A Survey of Corpora for Germanic Low-Resource Languages and Dialects" (Blaschke et al., NoDaLiDa 2023).

lpngram. Python library for ngram collection and frequency smoothing

knn-mt-reimplement. Repository for the trial task of WING - Reimplementation of kNN-MT using hugging face and pytorch

knn-box. an easy-to-use knn-mt toolkit

torchgfn. A modular, easy to extend GFlowNet library

pythia. The hub for EleutherAI's work on interpretability and learning dynamics

Prompt-Engineering-Guide. 🐙 Guides, papers, lessons, notebooks and resources for prompt engineering, context engineering, RAG, and AI Agents.

prompts.chat. f.k.a. Awesome ChatGPT Prompts. Share, discover, and collect prompts from the community. Free and open source — self-host for your organization with complete privacy.

typst. A markup-based typesetting system that is powerful and easy to learn.

fortuna. A Library for Uncertainty Quantification.

confidenceinterval. The long missing library for python confidence intervals

riemannian_latent_space. Riemannian metrics to measure distances in latent space of VAEs

lion-pytorch. 🦁 Lion, new optimizer discovered by Google Brain using genetic algorithms that is purportedly better than Adam(w), in Pytorch

botorch. Bayesian optimization in PyTorch

torchsde. Differentiable SDE solvers with GPU support and efficient sensitivity analysis.

tuning_playbook. A playbook for systematically maximizing the performance of deep learning models.

chat-miner. Parsers and visualizations for chats

petals. 🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

mental-health-datasets. An evolving list of electronic media data sets used to model mental-health status.

Datasets-for-Hate-Speech-Detection. Datasets for Hate Speech Detection

medical-nlp. Dataset for Natural Language Processing using a corpus of medical transcriptions and custom-generated clinical stop words and vocabulary.

awesome-human-label-variation. A curated list of awesome datasets with human label variation (un-aggregated labels) in Natural Language Processing and Computer Vision, accompanying The 'Problem' of Human Label Variation: On Ground Truth in Data, Modeling and Evaluation (EMNLP 2022)

BNAF. Pytorch implementation of Block Neural Autoregressive Flow

dvc. 🦉 Data Versioning and ML Experiments

MLInterview. :octocat: A curated awesome list of AI Startups in India & Machine Learning Interview Guide. Feel free to contribute!

TabPFN. ⚡ TabPFN: Foundation Model for Tabular Data ⚡

lang2vec. A simple library for querying the URIEL typological database.

normalizing-flows. PyTorch implementation of normalizing flow models

SimpleParsing. Simple, Elegant, Typed Argument Parsing with argparse

spectral. Follow the rainbow 🌈

CrossRE. CrossRE: A Cross-Domain Dataset for Relation Extraction (Findings of EMNLP 2022)

calibration-on-disagreement-data. Code accompanying the EMNLP 2022 paper "Stop Measuring Calibration When Humans Disagree" in which we show problems with popular calibration metrics like ECE in settings where more than one answer is acceptable, and argue for several metrics that take into account the full human judgement distribution.

RankingNLPSystems. What are the best Systems? New Perspectives on NLP Benchmarking

python-causality-handbook. Causal Inference for the Brave and True. A light-hearted yet rigorous approach to learning about impact estimation and causality.

gflownet. Generative Flow Networks

fontawesome-latex. LaTeX mappings for Font Awesome, the icons font

neural_networks_chomsky_hierarchy. Neural Networks and the Chomsky Hierarchy

evidential-deep-learning. Learn fast, scalable, and calibrated measures of uncertainty using neural networks!

promptsource. Toolkit for creating, sharing and using natural language prompts.

manim. A community-maintained Python framework for creating mathematical animations.

zotero-reMarkable-sync. Synchronize all the .pdf files in your Zotero library with your reMarkable device

aepsych. AEPsych is a tool for adaptive experimentation in psychophysics and perception research, built on top of gpytorch and botorch.

natural-posterior-network. Official Implementation of "Natural Posterior Network: Deep Bayesian Predictive Uncertainty for Exponential Family Distributions" (ICLR, 2022)

natural-posterior-network. Official Implementation of "Natural Posterior Network: Deep Bayesian Predictive Uncertainty for Exponential Family Distributions" (ICLR, 2022)

KCEF. Kernel Conditional Exponential Family

stochman. Algorithms for computations on random manifolds made easier

pixel. Research code for pixel-based encoders of language (PIXEL)

monotonic_flow. Monotonic Gaussian Process Flow

WGPOT. The Wasserstein Distance and Optimal Transport Map of Gaussian Processes

wordtour. Code for "Word Tour: One-dimensional Word Embeddings via the Traveling Salesman Problem" (NAACL 2022)

certain-transformer. How certain is your transformer?

torchdim. Named tensors with first-class dimensions for PyTorch

ferdium-app. All your services in one place, built by the community

mt_idioms. Repository for the analysis of idiom processing in MarianMT Transformer, for models translating English into 1 of 7 European target languages.

evaluate. 🤗 Evaluate: A library for easily evaluating machine learning models and datasets.

ml_nlp_paper_data. Dataset of ML and NLP papers

salvaedor-dali. Generate classical paintings using Variational Autoencoders (VAEs).

coverage_quantification. Python

gaussflow. Gaussianization Flows for density estimation, sampling and information theory measures (RBIG2.0)

Gaussianization_Flows. Python

glow-pytorch. pytorch implementation of openai paper "Glow: Generative Flow with Invertible 1×1 Convolutions"

faker. Faker is a Python package that generates fake data for you.

cnfs-super-resolution. Master thesis for the MSc. Artificial Intelligence at the University of Amsterdam, 2019. Topic: Super-resolution with Conditional Normalizing Flows.

awesome-normalizing-flows. Awesome resources on normalizing flows.

brokenaxes. Create matplotlib plots with broken axes

mixed-rv-vae. VAE using mixed random variables

torchuq. A library for uncertainty quantification based on PyTorch

lightning-bolts. Toolbox of models, callbacks, and datasets for AI/ML researchers.

depprobe. Probing for Labeled Dependency Trees (ACL 2022) + Sorting LMs by Structure (NAACL 2022)

transformer-from-scratch. Well documented, unit tested, type checked and formatted implementation of a vanilla transformer - for educational purposes.

composer. Supercharge Your Model Training

impact. ML has an impact on the climate. But not all models are born equal. Compute your model's emissions with our calculator and add the results to your paper with our generated latex template

aclpubcheck. Tools for checking ACL paper submissions

pingouin. Statistical package in Python based on Pandas

ivy. Convert Machine Learning Code Between Frameworks

NeuroX. A Python library that encapsulates various methods for neuron interpretation and analysis in Deep NLP models.

baycomp. Python

tutorial. Tutorial on Bayesian tests for Machine Learning

intro_dgm. "Deep Generative Modeling": Introductory Examples

optimate. A collection of libraries to optimise AI model performances

PyHessian. PyHessian is a Pytorch library for second-order based analysis and training of Neural Networks

massive-summ. Python

focal_calibration. Code for the paper "Calibrating Deep Neural Networks using Focal Loss"

pytorch-constrained-opt. Constrained Optimization in Pytorch

PeriodicBNN. Code for 'Periodic Activation Functions Induce Stationarity' (NeurIPS 2021)

aioway. AI on the way. An auto deep learning pipe dream. An RDBMS approach to deep learning. Declarative, explainable, scalable, optimizable, easy to deploy, all that good stuff.

nngeometry. {KFAC,EKFAC,Diagonal,Implicit} Fisher Matrices and finite width NTKs in PyTorch

scientific-visualization-book. An open access book on scientific visualization using python and matplotlib

eigenlearning. codebase for "A Theory of the Inductive Bias and Generalization of Kernel Regression and Wide Neural Networks"

wandb_on_slurm. Example of how to use Weights & Biases on Slurm

Laplace. Laplace approximations for Deep Learning.

storchastic. Stochastic Automatic Differentiation library for PyTorch.

pytorch-resample. 🎲 Iterable dataset resampling in PyTorch

DaNplus. DaN+ corpus: Danish Nested Named Entities and Lexical Normalization

pybites-carbon. Python

poetry. Python packaging and dependency management made easy

UD_Turkish_German-SAGT.

chartify. Python library that makes it easy for data scientists to create charts.

geomloss. Geometric loss functions between point clouds, images and volumes

Awesome-CV. :page_facing_up: Awesome CV is LaTeX template for your outstanding job application

geotorch. Constrained optimization toolkit for PyTorch

causal-text-papers. Curated research at the intersection of causal inference and natural language processing.

ud-selection. Genre-driven Data Selection for Zero-shot Parsing (EMNLP 2021)

mpire. A Python package for easy multiprocessing, but faster than multiprocessing

Ensemble-Pytorch. A unified ensemble framework for PyTorch to improve the performance and robustness of your deep learning model.

pytorch-OpCounter. Count the MACs / FLOPs of your PyTorch model.

awesome_OpenSetRecognition_list. A curated list of papers & resources linked to open set recognition, out-of-distribution, open set domain adaptation and open world recognition

uncertainty-toolbox. Uncertainty Toolbox: a Python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

rmapi. Go app that allows you to access your reMarkable tablet files through the Cloud API

ray. Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.

sparsemax. :boom: A PyTorch Implementation of the Sparsemax operator (https://arxiv.org/pdf/1803.09820.pdf)

sparsemax-pytorch. Implementation of Sparsemax activation in Pytorch

lsTerm-election-forecast. A long-short term event memory state-space model for multi-party elections

BayesianOptimization. A Python implementation of global optimization with gaussian processes.

Conference-Acceptance-Rate. Acceptance rates for the major AI conferences

language. Shared repository for open-sourced projects from the Google AI Language team.

torchcde. Differentiable controlled differential equation solvers for PyTorch with GPU support and memory-efficient adjoint backpropagation.

st_tau. This repository contains code for the paper "Uncertainty Estimation and Calibration with Finite-State Probabilistic RNNs" (Wang, Lawrence & Niepert, ICLR 2021).

VOLT. Code for paper "Vocabulary Learning via Optimal Transport for Neural Machine Translation"

DUE. Code for "On Feature Collapse and Deep Kernel Learning for Single Forward Pass Uncertainty".

modelling-unigram. A model for the unigram distribution using the Pitman-Yor process.

zoteroRemarkable. A script/utility to keep a zotero collection and remarkable file in sync

normalizing_flows. Pytorch implementations of density estimation algorithms: BNAF, Glow, MAF, RealNVP, planar flows

nflows. Normalizing flows in PyTorch

pytorch-classification-uncertainty. This repo contains a PyTorch implementation of the paper: "Evidential Deep Learning to Quantify Classification Uncertainty"

causal_inference. A repository to collect our work on treatment effect estimation.

anonymous_github. Anonymous Github is a proxy server to support anonymous browsing of Github repositories for open-science code and data.

BayesianRNN_pytorch. pytorch implementation of "A Theoretically Grounded Application of Dropout in Recurrent Neural Networks" LSTM(https://arxiv.org/abs/1512.05287)

experiment-impact-tracker. Python

torchstat. Model analyzer in PyTorch

DDU. Code for Deterministic Neural Networks with Appropriate Inductive Biases Capture Epistemic and Aleatoric Uncertainty

functorch. functorch is JAX-like composable function transforms for PyTorch.

ml-surveys. 📋 Survey papers summarizing advances in deep learning, NLP, CV, graphs, reinforcement learning, recommendations, graphs, etc.

Better_LSTM_PyTorch. An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.

multiNegScope. Python

g-mlp-pytorch. Implementation of gMLP, an all-MLP replacement for Transformers, in Pytorch

variational-lstm. PyTorch implementation of Variational LSTM and Monte Carlo dropout.

dists.pt. Extensions to torch distributions

accelerate. 🚀 A simple way to launch, train, and use PyTorch models on almost any device and distributed configuration, automatic mixed precision (including fp8), and easy-to-configure FSDP and DeepSpeed support

oos-eval. Repository that accompanies "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction" (EMNLP 2019)

swahili-text-gcn. Graph Convolutional Network for Swahili News Classification: https://arxiv.org/abs/2103.09325

sentencepiece. Unsupervised text tokenizer for Neural Network-based text generation.

labml. 🔎 Monitor deep learning model training and hardware usage from your mobile phone 📱

OCTIS. OCTIS: Comparing Topic Models is Simple! A python package to optimize and evaluate topic models (accepted at EACL2021 demo track)

tydiqa. TyDi QA contains 200k human-annotated question-answer pairs in 11 Typologically Diverse languages, written without seeing the answer and without the use of translation, and is designed for the training and evaluation of automatic question answering systems. This repository provides evaluation code and a baseline system for the dataset.

python-sortedcontainers. Python Sorted Container Types: Sorted List, Sorted Dict, and Sorted Set

tfpyth. Putting TensorFlow back in PyTorch, back in TensorFlow (differentiable TensorFlow PyTorch adapters).

WeightWatcher. The WeightWatcher tool for predicting the accuracy of Deep Neural Networks

HyBayes. Bayesian Assessment of Hypotheses

transformers-interpret. Model explainability that works seamlessly with 🤗 transformers. Explain your transformers model in just 2 lines of code.

python-ftfy. Fixes mojibake and other glitches in Unicode text, after the fact.

wikidata. Wikidata client library for Python