Pavel Izmailov

uncertainties_MT_eval. Code and data for the paper "Disentangling Uncertainty in Machine Translation Evaluation", accepted at EMNLP 2022.

UA_COMET. Repository for "Uncertainty-Aware Machine Translation Evaluation", accepted to Findings of EMNLP 2021.

llmtime. Jupyter Notebook

flash-attention. Fast and memory-efficient exact attention

gpt-2. Code for the paper "Language Models are Unsupervised Multitask Learners"

cramming. Cramming the training of a (BERT-type) language model into limited compute.

SubpopBench. [ICML 2023] Change is Hard: A Closer Look at Subpopulation Shift

big_vision. Official codebase used to develop Vision Transformer, SigLIP, MLP-Mixer, LiT and more.

torch-sgld. SGLD and cSGLD as a PyTorch Optimizer

pytorch-optimizer. torch-optimizer -- collection of optimizers for Pytorch

sgld. Python

instruction-induction. Python

dnn-mode-connectivity-tf. Implementation of subspace inference in tensorflow based on PyTorch code

spurious_feature_learning. Python

cifar5m. CIFAR-5m dataset

counterfactually-augmented-data. Learning the Difference that Makes a Difference with Counterfactually-Augmented Data

pytorch-image-models. The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

browsh. A fully-modern text-based browser, rendering to TTY and browsers

deep_feature_reweighting. Jupyter Notebook

rl_with_resets. JAX implementation of deep RL agents with resets from the paper "The Primacy Bias in Deep Reinforcement Learning"

vissl. VISSL is FAIR's library of extensible, modular and scalable components for SOTA Self-Supervised Learning with images.

wilds. A machine learning benchmark of in-the-wild distribution shifts, with data loaders, evaluators, and default models.

asa. Code for paper "Adversarial Support Alignment"

bibtex-python-package-citations. BibTeX citations for Python and common (machine learning) packages.

gnosis. Code to reproduce experiments from 'Does Knowledge Distillation Really Work' a paper which appeared in the 2021 NeurIPS proceedings.

understanding-bayesian-classification. On Uncertainty, Tempering, and Data Augmentation in Bayesian Classification

ivy. Convert Machine Learning Code Between Frameworks

texture-vs-shape. Pre-trained models, data, code & materials from the paper "ImageNet-trained CNNs are biased towards texture; increasing shape bias improves accuracy and robustness" (ICLR 2019 Oral)

PyTorch-Pretrained-ViT. Vision Transformer (ViT) in PyTorch

Bayesian_model_comparison. Supporing code for the paper "Bayesian Model Selection, the Marginal Likelihood, and Generalization".

model-vs-human. Benchmark your model on out-of-distribution datasets with carefully collected human comparison data (NeurIPS 2021 Oral)

backgrounds_challenge. Python

Stylized-ImageNet. Code to create Stylized-ImageNet, a stylized version of standard ImageNet (ICLR 2019 Oral)

mini-hmc-jax. A simple implementation of Hamiltonian Monte Carlo in JAX.

BalancingGroups. Simple data balancing baselines for worst-group-accuracy benchmarks.

open_clip. An open source implementation of CLIP.

mdetr. Python

bnn_covariate_shift. Supporting code for the paper "Dangers of Bayesian Model Averaging under Covariate Shift"

omd. JAX code for the paper "Control-Oriented Model-Based Reinforcement Learning with Implicit Differentiation"

einops. Flexible and powerful tensor operations for readable and reliable code (for pytorch, jax, TF and others)

google-research. Google Research

equivariant-MLP. A library for programmatically generating equivariant layers through constraint solving

PyTorch-LBFGS. A PyTorch implementation of L-BFGS.

FlowGMM-Julia. Jupyter Notebook

stylegan2. StyleGAN2 - Official TensorFlow Implementation with practical improvements

consortium-feedback. A repository for discussions related and for giving feedback on the Consortium

pytorch-lightning. Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

jax. Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

dm-haiku. JAX-based neural network library

flows_ood. Jupyter Notebook

pytorch-semseg. Semantic Segmentation Architectures Implemented in PyTorch

drq. DrQ: Data regularized Q

pixel-cnn-pp. Pytorch Implementation of OpenAI's PixelCNN++

networkx. Network Analysis in Python

releasing-research-code. Tips for releasing research code in Machine Learning (with official NeurIPS 2020 recommendations)

hessian-eff-dim. Public Codebase for Rethinking Parameter Counting: Effective Dimensionality Revisited

sesn. Code for "Scale-Equivariant Steerable Networks"

LieConv. Python

molview. The code of MolView.org

autoNSO. Auto-differentiation for non-smooth optimization (NSO) methods

PairGAN. Jupyter Notebook

hamiltorch. PyTorch-based library for Riemannian Manifold Hamiltonian Monte Carlo (RMHMC) and inference in Bayesian neural networks

residual-flows. code for "Residual Flows for Invertible Generative Modeling".

e2cnn. E(2)-Equivariant CNNs Library for Pytorch

Python-MaxEnt. Python Maximum Entropy code for Log Determinants and Graph cluster counting and similarity

keras-swa. Simple stochastic weight averaging callback for Keras

backpack. This repository is no longer maintained. Check

loss-patterns. Loss Patterns of Neural Networks

GENTRL. Generative Tensorial Reinforcement Learning (GENTRL) model

spectralgp. Code repo for "Function-Space Distributions over Kernels"

drbayes. Code Repo for "Subspace Inference for Bayesian Deep Learning"

gluonts. Probabilistic time series modeling in Python

mxnet-the-straight-dope. An interactive book on deep learning. Much easy, so MXNet. Wow. [Straight Dope is growing up] ---> Much of this content has been incorporated into the new Dive into Deep Learning Book available at https://d2l.ai/.

botorch. Bayesian optimization in PyTorch

SWALP. Code for paper "SWALP: Stochastic Weight Averaging forLow-Precision Training".

scone. Python

real-nvp. Implementation of Real NVP in PyTorch

awd-lstm-lm. LSTM and QRNN Language Model Toolkit for PyTorch

moses. Molecular Sets (MOSES): A Benchmarking Platform for Molecular Generation Models

torchdiffeq. Differentiable ODE solvers with full GPU support and O(1)-memory backpropagation.

super2018. C++

dnn-mode-connectivity. Mode Connectivity and Fast Geometric Ensembles in PyTorch

word2gm. Word to Gaussian Mixture Model

swa_gaussian. Code repo for "A Simple Baseline for Bayesian Uncertainty in Deep Learning"

pravda. Blockchain with Turing-complete VM implemented in Scala.

models. Models and examples built with Chainer

contrib. Implementations of ideas from recent papers

pytorch. Tensors and Dynamic neural networks in Python with strong GPU acceleration

variance-networks. Variance Networks: When Expectation Does Not Meet Your Expectations, ICLR 2019

fastswa-semi-sup. Improving Consistency-Based Semi-Supervised Learning with Weight Averaging

olive-oil-ml. Python

gpytorch. A highly efficient implementation of Gaussian Processes in PyTorch

swa. Stochastic Weight Averaging in PyTorch

pytorch_image_classification. PyTorch implementation of image classification models for CIFAR-10/CIFAR-100/MNIST/FashionMNIST/Kuzushiji-MNIST/ImageNet

bayesgan. Tensorflow code for the Bayesian GAN (https://arxiv.org/abs/1705.09558) (NIPS 2017)

TensorNet-TF. TensorNet (TensorFlow implementation)

snli-entailment. attention model for entailment on SNLI corpus implemented in Tensorflow and Keras

Cornell-MOE. A Python library for the state-of-the-art Bayesian optimization algorithms, with the core implemented in C++.

t3f. Tensor Train decomposition on TensorFlow