Daniel Pressel

grain. Library for reading and processing ML training data.

dlcalc. random command line tools for deep learning

ringattention. Large Context Attention

NeMo-Aligner. Scalable toolkit for efficient model alignment

direct-preference-optimization. Reference implementation for DPO (Direct Preference Optimization)

neuronx-distributed. Python

neuronx-nemo-megatron. Python

aws-neuron-reference-for-megatron-lm. Python

optimum-neuron. Training and inference on AWS Trainium and Inferentia chips.

alpaca-lora. Instruct-tune LLaMA on consumer hardware

UniEval. Repository for EMNLP 2022 Paper: Towards a Unified Multi-Dimensional Evaluator for Text Generation

Open-Assistant. OpenAssistant is a chat-based assistant that understands tasks, can interact with third-party systems, and retrieve information dynamically to do so.

baize-chatbot. Let ChatGPT teach your own chatbot in hours with a single GPU!

openai-cookbook. Examples and guides for using the OpenAI API

alexa-teacher-models. Python

natural-instructions. Expanding natural instructions

stanford_alpaca. Code and documentation to train Stanford's Alpaca models, and generate the data.

self-instruct. Aligning pretrained language models with instruction data generated by themselves.

wikiwiki-dataset. Shell

deepspeed-sagemaker-example. Jupyter Notebook

self_dialogue_corpus. The Self-dialogue Corpus - a collection of self-dialogues across music, movies and sports

chirpycardinal. Stanford's Alexa Prize socialbot

orconvqa-release. Python

OAT. An open source toolkit for multimodal generative conversational task assistants, helping assist people with real-world complex tasks

VANiLLa. A large scale data for answer verbalization for simple natural questions.

trlx. A repo for distributed training of language models with Reinforcement Learning via Human Feedback (RLHF)

dialog-inpainting.

DialogRPT. EMNLP 2020: "Dialogue Response Ranking Training with Large-Scale Human Feedback Data"

KQAPro_Baselines. Pytorch implementation of baseline models of KQA Pro, a large-scale dataset of complex question answering over knowledge base.

RL4LMs. A modular RL library to fine-tune language models to human preferences

acl2022-zerofewshot-tutorial.

Knover. Large-scale open domain KNOwledge grounded conVERsation system based on PaddlePaddle

OpenEL_corpus. OpenEL corpus

fed. Code for SIGdial 2020 paper: Unsupervised Evaluation of Interactive Dialog with DialoGPT (https://arxiv.org/abs/2006.12719)

SuRE. [EMNLP 2022] Summarization as Indirect Supervision for Relation Extraction (SuRE)

DST-as-Prompting. Source code for Dialogue State Tracking with a Language Model using Schema-Driven Prompting

pomdp-baselines. Simple (but often Strong) Baselines for POMDPs in PyTorch, ICML 2022

architecture-objective. Python

wikifact. Wikipedia based dataset to train relationship classifiers and fact extraction models

contrack. The Context Tracking dataset consists of annotated human-human social conversations. Each conversation contains annotations for the people and location entities mentioned, their properties and the relationships between them. The annotated data enables several subtasks like slot tagging, coreference resolution, resolving plural mentions and entity linking.

Taskmaster. Please see the readme file as well as our 2019 EMNLP paper linked here -->

EntLM. Codes for "Template-free Prompt Tuning for Few-shot NER".

massive. Tools and Modeling Code for the MASSIVE dataset

ReCross. ReCross: Unsupervised Cross-Task Generalization via Retrieval Augmentation

GenIE. The autoregressive information extraction system GenIE (Generative Information Extraction) implemented in PyTorch.

Channel-LM-Prompting. An original implementation of "Noisy Channel Language Model Prompting for Few-Shot Text Classification"

airdialogue. Python

promptsource. Toolkit for creating, sharing and using natural language prompts.

MetaICL. An original implementation of "MetaICL Learning to Learn In Context" by Sewon Min, Mike Lewis, Luke Zettlemoyer and Hannaneh Hajishirzi

s3prl. Self-Supervised Speech Pre-training and Representation Learning Toolkit

t-zero. Reproduce results and replicate training fo T0 (Multitask Prompted Training Enables Zero-Shot Task Generalization)

semsup. Semantic Supervision: Enabling Generalization over Output Spaces

YaLM-100B. Pretrained language model with 100B parameters

equinox. Elegant easy-to-use neural networks + scientific computing in JAX. https://docs.kidger.site/equinox/

OpenPrompt. An Open-Source Framework for Prompt-Learning.

t-few. Code for T-Few from "Few-Shot Parameter-Efficient Fine-Tuning is Better and Cheaper than In-Context Learning"

FasterTransformer. Transformer related optimization, including BERT, GPT

metaseq. Repo for external large-scale work

Converse. Python

prompt-tuning. Original Implementation of Prompt Tuning from Lester, et al, 2021

rebel. REBEL is a seq2seq model that simplifies Relation Extraction (EMNLP 2021).

CrossWeigh. CrossWeigh: Training Named Entity Tagger from Imperfect Annotations

contriever. Contriever: Unsupervised Dense Information Retrieval with Contrastive Learning

annotated-s4. Implementation of https://srush.github.io/annotated-s4

Few-NERD. Code and data of ACL 2021 paper "Few-NERD: A Few-shot Named Entity Recognition Dataset"

alexa-end-to-end-slu. This setup allows to train end-to-end neural models for spoken language understanding (SLU).

pySBD. 🐍💯pySBD (Python Sentence Boundary Disambiguation) is a rule-based sentence boundary detection that works out-of-the-box.

syntactic-augmentation-nli. Create augmentation examples from MultiNLI by subject-object inversion and passivizing.

ACE. [ACL-IJCNLP 2021] Automated Concatenation of Embeddings for Structured Prediction

cista. Cista is a simple, high-performance, zero-copy C++ serialization & reflection library.

pyannote-audio. Neural building blocks for speaker diarization: speech activity detection, speaker change detection, overlapped speech detection, speaker embedding

voxpopuli. A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation

deit. Official DeiT repository

audioset_tagging_cnn. Python

diy. Do It yourself Dependency Injection in python

prefix-beam-search. Code for prefix beam search tutorial by @labodk

extending-jax. Extending JAX with custom C++ and CUDA code

awesome-multimodal-ml. Reading list for research topics in multimodal machine learning

DALLE-pytorch. Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

uncertainty-baselines. High-quality implementations of standard and SOTA methods on a variety of tasks.

Picking_BERTs_Brain. Code and Data for Picking BERT's Brain: Probing for Linguistic Dependencies in Contextualized Embeddings Using Representational Similarity Analysis by Michael Lepori and R. Thomas McCoy, presented at COLING 2020

transducer. A Fast Sequence Transducer Implementation with PyTorch Bindings

EEND. End-to-End Neural Diarization

jiwer. Evaluate your speech-to-text system with similarity measures such as word error rate (WER)

WER-in-python. This program calculates the word error rate of hypothesis in ASR and print the aligned result.

TensorFlowASR. :zap: TensorFlowASR: Almost State-of-the-art Automatic Speech Recognition in Tensorflow 2. Supported languages that can use characters or subwords

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

libri-light. dataset for lightly supervised training using the librivox audio book recordings. https://librivox.org/.

iobes. Tool for parsing and converting various span encoding schemes.

cloud-sql-proxy. A utility for connecting securely to your Cloud SQL instances

parser. :rocket: State-of-the-art parsers for natural language.

tetra-tagging. Tetra-Tagging: Word-Synchronous Parsing with Linear-Time Inference

TreebankPreprocessing. Python scripts preprocessing Penn Treebank and Chinese Treebank

Better_LSTM_PyTorch. An LSTM in PyTorch with best practices (weight dropout, forget bias, etc.) built-in. Fully compatible with PyTorch LSTM.

deep_srl. Code and pre-trained model for: Deep Semantic Role Labeling: What Works and What's Next

bert-stable-fine-tuning. On the Stability of Fine-tuning BERT: Misconceptions, Explanations, and Strong Baselines

self-attentive-parser. High-accuracy NLP parser with models for 11 languages.

HPSG-Neural-Parser. Source code for "Head-Driven Phrase Structure Grammar Parsing on Penn Treebank" published at ACL 2019

gap. Gendered Ambiguous Pronouns Shared Task

rnn_agreement. Evaluating recurrent neural networks on predicting subject-verb agreement dependencies

blimp. The Benchmark of Linguistic Minimal Pairs

albert. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations

pix-plot. A WebGL viewer for UMAP or TSNE-clustered images

CorefQA. This repo contains the code for ACL2020 paper "Coreference Resolution as Query-based Span Prediction"

gap-coreference. GAP is a gender-balanced dataset containing 8,908 coreference-labeled pairs of (ambiguous pronoun, antecedent name), sampled from Wikipedia for the evaluation of coreference resolution in practical applications.

textgrid. A Python module for interacting with Praat TextGrid files. Also includes a class for reading HTK .mlf files into Praat

e2e-coref. End-to-end Neural Coreference Resolution

coref-ee. Coreference Resolution With Entity Equalization

coref. BERT for Coreference Resolution

ConvLab-2. ConvLab-2: An Open-Source Toolkit for Building, Evaluating, and Diagnosing Dialogue Systems

code_snippets. Shell

NLU_datasets_with_task_oriented_dialogue. datasets of natural language understanding and dialogue state tracking

BlingFire. A lightning fast Finite State machine and REgular expression manipulation library.

wikiextractor. A tool for extracting plain text from Wikipedia dumps

ToD-BERT. Pre-Trained Models for ToD-BERT

elastic. PyTorch elastic training

text-to-text-transfer-transformer. Code for the paper "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer"

datasets. 🤗 The largest hub of ready-to-use datasets for AI models with fast, easy-to-use and efficient data manipulation tools

oos-eval. Repository that accompanies "An Evaluation Dataset for Intent Classification and Out-of-Scope Prediction" (EMNLP 2019)

conventional-changelog-action. Github Action that generates a changelog with the Conventional Changelog CLI

self_talk. Code and data for the paper: "Unsupervised Common Sense Question Answering with Self-Talk"

PyTorch-GAN. PyTorch implementations of Generative Adversarial Networks.

StackPropagation-SLU. Open source code for EMNLP-19 Paper "A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding".

hub. A hub for pre-trained models

end-to-end-SLU. PyTorch code for end-to-end spoken language understanding (SLU) with ASR-based transfer learning

electra. ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators

contextual. How Contextual are Contextualized Word Representations?

TENER. Codes for "TENER: Adapting Transformer Encoder for Named Entity Recognition"

nn4nlp-concepts. A repository of concepts related to neural networks for NLP

trax. Trax — Deep Learning with Clear Code and Speed

python_speech_features. This library provides common speech features for ASR including MFCCs and filterbank energies.

Factorized-TDNN. PyTorch implementation of the Factorized TDNN (TDNN-F) from "Semi-Orthogonal Low-Rank Matrix Factorization for Deep Neural Networks" and Kaldi

dstc8-schema-guided-dialogue. The Schema-Guided Dialogue Dataset

ASR. Python

malmo. Project Malmo is a platform for Artificial Intelligence experimentation and research built on top of Minecraft. We aim to inspire a new generation of research into challenging new problems presented by this unique environment. --- For installation instructions, scroll down to *Getting Started* below, or visit the project page for more information:

udify. A single model that parses Universal Dependencies across 75 languages. Given a sentence, jointly predicts part-of-speech tags, morphology tags, lemmas, and dependency trees.

MRDA-Corpus. Utilities for Processing the Meeting Recorder Dialogue Act Corpus

Switchboard-Corpus. Utilities for Processing the Switchboard Dialogue Act Corpus

LAMA. LAnguage Model Analysis

pytorch2keras. PyTorch to Keras model convertor

addons. Useful extra functionality for TensorFlow 2.x maintained by SIG-addons

espresso. Espresso: A Fast End-to-End Neural Speech Recognition Toolkit

tutorial. Tutorial on ‘Graph-Based Meaning Representations: Design and Processing’ (ACL 2019)

parsimonious. The fastest pure-Python PEG parser I can muster

bi-tempered-loss. Robust Bi-Tempered Logistic Loss Based on Bregman Divergences. https://arxiv.org/pdf/1906.03361.pdf

ctrl. Conditional Transformer Language Model for Controllable Generation

flowseq. Generative Flow based Sequence-to-Sequence Toolkit written in Python.

DeepLearningExamples. State-of-the-Art Deep Learning scripts organized by models - easy to train and deploy with reproducible accuracy and performance on enterprise-grade infrastructure.

ML-From-Scratch. Machine Learning From Scratch. Bare bones NumPy implementations of machine learning models and algorithms with a focus on accessibility. Aims to cover everything from linear regression to deep learning.

gpu-monitoring-tools. Tools for monitoring NVIDIA GPUs on Linux

hpctl. Sample Mead Configs and train baseline models.

zeshel. Python

waliki. A wiki engine powered by Django and Git

Megatron-LM. Ongoing research training transformer models at scale

trade-dst. Source code for transferable dialogue state generator (TRADE, Wu et al., 2019). https://arxiv.org/abs/1905.08743

slate. A Super-Lightweight Annotation Tool for Experts: Label text in a terminal with just Python

pointer_summarizer. pytorch implementation of "Get To The Point: Summarization with Pointer-Generator Networks"

YouTokenToMe. Unsupervised text tokenizer focused on computational efficiency

mnist-mc-dropout. model uncertainty using mc dropout

seq2seq-con. Implementation of "Von Mises-Fisher Loss for Training Sequence to Sequence Models with Continuous Outputs"

xpctl. track baseline experiments

transformer_lexical_shortcuts. Codebase accompanying the paper 'Widening the Representation Bottleneck in Neural Machine Translation with Lexical Shortcuts', (Emelin, Denis, Ivan Titov, and Rico Sennrich, Fourth Conference on Machine Translation, Florence, 2019.)

bertviz. BertViz: Visualize Attention in Transformer Models

attention-analysis. Jupyter Notebook

contextual-repr-analysis. A toolkit for evaluating the linguistic knowledge and transferability of contextual representations. Code for "Linguistic Knowledge and Transferability of Contextual Representations" (NAACL 2019).

MASS. MASS: Masked Sequence to Sequence Pre-training for Language Generation

XLM. PyTorch original implementation of Cross-lingual Language Model Pretraining.

xlnet. XLNet: Generalized Autoregressive Pretraining for Language Understanding

forecast-prometheus. A collection of analysis, and machine learning techniques for time series forecasting w/ Prometheus metrics

Left2Right-Pointer-Parser. Python

clarification_question_generation_pytorch. Code and data for the paper: Answer-based Adversarial Training for Generating Clarification Questions

compare-mt. A tool for holistic analysis of language generations systems

structural-probes. Codebase for testing whether hidden states of neural networks encode discrete structures.

Densely-Connected-CNN-with-Multiscale-Feature-Attention. Densely Connected CNN with Multi-scale Feature Attention for Text Classification

DPCNN. Deep Pyramid Convolutional Neural Networks for Text Categorization in PyTorch

symphony-mt. Symphony Machine Translation

dep2label. Dependency Parsing as Sequence Labeling

ast. Code to train Automatic Speech-to-Text (AST) models

naacl_transfer_learning_tutorial. Repository of code for the tutorial on Transfer Learning in NLP held at NAACL 2019 in Minneapolis, MN, USA

nvidia_gpu_prometheus_exporter. NVIDIA GPU Prometheus Exporter

lm. Python

pytorch-kaldi. pytorch-kaldi is a project for developing state-of-the-art DNN/RNN hybrid speech recognition systems. The DNN part is managed by pytorch, while feature extraction, label computation, and decoding are performed with the kaldi toolkit.

mesh. Mesh TensorFlow: Model Parallelism Made Easier

nabu. Code for end-to-end ASR with neural networks, build with TensorFlow

End-to-end-ASR-Pytorch. This is an open source project (formerly named Listen, Attend and Spell - PyTorch Implementation) for end-to-end ASR implemented with Pytorch, the well known deep learning toolkit.

Listen-Attend-Spell. A PyTorch implementation of Listen, Attend and Spell (LAS), an End-to-End ASR framework.

Speech-Transformer. A PyTorch implementation of Speech Transformer, an End-to-End ASR with Transformer network on Mandarin Chinese.

py3nvml. Python 3 Bindings for NVML library. Get NVIDIA GPU status inside your program.

listen-attend-and-spell. Tensorflow implementation of "Listen, Attend and Spell" authored by William Chan. This project utilizes input pipeline and estimator API of Tensorflow, which makes the training and evaluation truly end-to-end.

jax. Composable transformations of Python+NumPy programs: differentiate, vectorize, JIT to GPU/TPU, and more

graphpipe-py. GraphPipe for python

graphpipe-tf-py. GraphPipe helpers for TensorFlow

model_server. A scalable inference server for models optimized with OpenVINO™

source-to-image. A tool for building artifacts from source and injecting into container images

datasets. TFDS is a collection of datasets ready to use with TensorFlow, Jax, ...

nematus. Open-Source Neural Machine Translation in Tensorflow

kube-openmpi. Open MPI jobs on Kubernetes

conversational-datasets. Large datasets for conversational AI

litbank. Annotated dataset of 100 works of fiction to support tasks in natural language processing and the computational humanities.

PyTorch-BigGraph. Generate embeddings from large-scale graph-structured data.