Pang Jianhui

Context-Engineering. "Context engineering is the delicate art and science of filling the context window with just the right information for the next step." — Andrej Karpathy. A frontier, first-principles handbook inspired by Karpathy and 3Blue1Brown for moving beyond prompt engineering to the wider discipline of context design, orchestration, and optimization.

Marco-DeepResearch. Marco Search Agent for Realistic and Challenging Agentic Search

NSFC-LaTex. BibTeX Style

easydistill. a toolkit on knowledge distillation for large language models

Awesome-MCoT. Multimodal Chain-of-Thought Reasoning: A Comprehensive Survey

Awesome-Reasoning-Economy-Papers. Harnessing the Reasoning Economy: A Survey of Efficient Reasoning for Large Language Models

LlamaFactory. Unified Efficient Fine-Tuning of 100+ LLMs & VLMs (ACL 2024)

Weekly-Top-LLM-Papers. Curated list of weekly published LLM papers

mwp_ReFT. Python

SOTA-MT. This project attempts to maintain the SOTA performance in machine translation

Macaw-LLM. Macaw-LLM: Multi-Modal Language Modeling with Image, Video, Audio, and Text Integration

LLM-Continual-Learning-Papers. Must-read Papers on Large Language Model (LLM) Continual Learning

ASRT_SpeechRecognition. A Deep-Learning-Based Chinese Speech Recognition System 基于深度学习的中文语音识别系统

AnLLM. Python

InfiniTransformer. Unofficial PyTorch/🤗Transformers(Gemma/Llama3) implementation of Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention

transformer-xl. Python

kill_that_reviewer.

scrolls. The official code of EMNLP 2022, "SCROLLS: Standardized CompaRison Over Long Language Sequences".

LLM4MT. LLM4MT

read-agent.github.io. Jupyter Notebook

Awesome-LLM-Compression. Awesome LLM compression research papers and tools.

Awesome-LLM-MT.

ParroT. The ParroT framework to enhance and regulate the Translation Abilities during Chat based on open-sourced LLMs (e.g., LLaMA-7b, Bloomz-7b1-mt) and human written translation and evaluation data.

alignment-research-dataset. A dataset of alignment research and code to reproduce it

llama-cookbook. Welcome to the Llama Cookbook! This is your go to guide for Building with Llama: Getting started with Inference, Fine-Tuning, RAG. We also show you how to solve end to end problems using Llama model family and using them on various provider services

litgpt. 20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.

AlpacaDataCleaned. Alpaca dataset from Stanford, cleaned and curated

FastChat. An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.

ChatGLM-Finetuning. 基于ChatGLM-6B、ChatGLM2-6B、ChatGLM3-6B模型，进行下游具体任务微调，涉及Freeze、Lora、P-tuning、全参微调等

Chinese-Llama-2. improve Llama-2's proficiency in comprehension, generation, and translation of Chinese.

Llama-Chinese. Llama中文社区，实时汇总最新Llama学习资料，构建最好的中文Llama大模型开源生态，完全开源可商用

NLLB-Translator. Python

mt5-simplification. Scripts related to training and predicting Google's mt5 model

mt5-M2M-comparison. Comparing M2M and mT5 on a rare language pairs, blog post: https://medium.com/@abdessalemboukil/comparing-facebooks-m2m-to-mt5-in-low-resources-translation-english-yoruba-ef56624d2b75

lit-llama. Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, quantization, LoRA fine-tuning, pre-training. Apache 2.0-licensed.

lit-llama. Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

pytorch-lightning. Pretrain, finetune ANY AI model of ANY size on 1 or 10,000+ GPUs with zero code changes.

alpaca-lora. Instruct-tune LLaMA on consumer hardware

pyllama. LLaMA: Open and Efficient Foundation Language Models

optimate. A collection of libraries to optimise AI model performances

chatgpt_please_improve_my_paper_writing. a thin wrapper of chatgpt for improving paper writing.

simple-fairseq. simple translate

awesome-normalizing-flows. Awesome resources on normalizing flows.

RuSentEval. Probing suite for evaluation of Russian embedding and language models

project-menu. See the issue board for the current status of active and prospective projects!

degen. Official Repository for "The Curious Case of Neural Text Degeneration"

WMT17-scripts. Scripts for WMT17 English-Chinese translation task

constrained-levt. Lexically Constrained Neural Machine Translation with Levenshtein Transformer

wmt16-scripts. scripts and configuration files for Edinburgh neural MT submission to WMT 16 shared translation task

word-embeddings-for-nmt. Supplementary material for "When and Why Are Pre-trained Word Embeddings Useful for Neural Machine Translation?" at NAACL 2018

R-Drop. Python

BertGen. Training and evaluation codes for the BertGen paper (ACL-IJCNLP 2021)

arxiv-daily. arxiv-daily

AI_Collections. This repository is constructed to collect interesting resource in artificial intelligence area.

Graformer. The repository for the paper: Multilingual Translation via Grafting Pre-trained Language Models

doclevel-MT-benchmark. Document-level Machine Translation Benchmark

copyisallyouneed. Code for our ACL2021 paper Neural Machine Translation with Monolingual Translation Memory

Unilm. Python

huggingface_hub. The official CLI and Python client for the Hugging Face Hub.

unilm. Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

dynamic_select_weight. Uncleaned code in branch "unclean".

deep-learning-book. :book: MIT Deep Learning Book in PDF format

curriculum. Competence-based Curriculum Learning

bert-fairseq. Implement BERT and MulitPointer-generator on the basis of fairseq

CMLM-Self-Review-NAT. [COLING 2020 oral], "Infusing Sequential Information into Conditional Masked Translation Model with Self-Review Mechanism".

fairseq-image-captioning. Transformer-based image captioning extension for pytorch/fairseq

DCM_vgg_transformer. Dual cross modality attention audio-visual speech recognition model based on vgg transformer with hybrid CTC/attention architecture using fairseq

pytorch-image-models. The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

vit-pytorch. Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

Swin-Transformer. This is an official implementation for "Swin Transformer: Hierarchical Vision Transformer using Shifted Windows".

paper-reading.

AttnGAN. Python

higher. higher is a pytorch library allowing users to obtain higher order gradients over losses spanning training loops rather than individual training steps.

multiagent_crosstranslate. Python

data_diversification. Instruction to data diversification

marge-pytorch. Implementation of Marge, Pre-training via Paraphrasing, in Pytorch

Project-Best-Cafe-Arena-Simulation-Software-.

TransVAE. A Transformer Based VAE Architecture for De Novo Molecular Design

CoreNLP. CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

DSS-VAE-pytorch. Generating Sentences from Disentangled Syntactic and Semantic Spaces

MGNMT. Python

demvae. Dispersed Exponential Family Mixture VAE

DL4MT. Dual Learning for Machine Translation

UVR-NMT. Neural Machine Translation with universal Visual Representation (ICLR 2020)

ChineseNlpCorpus. 搜集、整理、发布 中文 自然语言处理 语料/数据集，与 有志之士 共同 促进 中文 自然语言处理 的 发展。

fcl-nat. Code for "Fine-Tuning by Curriculum Learning for Non-Autoregressive Neural Machine Translation"

Awesome-Knowledge-Distillation. Awesome Knowledge-Distillation. 分类整理的知识蒸馏paper(2014-2021)。

knowledge-distillation-pytorch. A PyTorch implementation for exploring deep and shallow knowledge distillation (KD) experiments with flexibility

Data4StylizedS2S. The repository contains resources of our paper published at AAAI 2020 ``A Dataset for Low-Resource Stylized Sequence-to-Sequence Generation "

mgiza. A word alignment tool based on famous GIZA++, extended to support multi-threading, resume training and incremental training.

DM-GAN. Python

Self-Supervised-Learning-for-NLP. A research on self-supervised learning with the interest of applying it into NLP field.

jlibovicky.github.io. JavaScript

CoSDA-ML. CoSDA-ML: Multi-Lingual Code-Switching Data Augmentation for Zero-Shot Cross-Lingual NLP

multeval. Easy Bootstrap Resampling and Approximate Randomization for BLEU, METEOR, and TER using Multiple Optimizer Runs. This implements "Better Hypothesis Testing for Statistical Machine Translation: Controlling for Optimizer Instability" from ACL 2011.

NLPMetrics. Python code for various NLP metrics

mosesdecoder. Moses, the machine translation system

UNMT. Code inspired by Unsupervised Machine Translation Using Monolingual Corpora Only

MT-Reading-List. A machine translation reading list maintained by Tsinghua Natural Language Processing Group

Unsupervised-Aspect-Extraction. Code for acl2017 paper "An unsupervised neural attention model for aspect extraction"

Chinese-Word-Vectors. 100+ Chinese Word Vectors 上百种预训练中文词向量

transformers. 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

nsfw_data_scraper. Collection of scripts to aggregate image data for the purposes of training an NSFW Image Classifier

nsfw_data_source_urls. Collection of NSFW images URLs for the purposes of training an NSFW Image Classifier

bert. TensorFlow code and pre-trained models for BERT

fuzzywuzzy. Fuzzy String Matching in Python

recurrent-visual-attention. A PyTorch Implementation of "Recurrent Models of Visual Attention"

Factoid-QA-subtask. documents based question answering using coattention and highway network

pytorch-resnet. Convert resnet trained in caffe to pytorch model. (group norm resnet is provided too)

RLSeq2Seq. Deep Reinforcement Learning For Sequence to Sequence Models

DeepPath. code and docs for the EMNLP paper "DeepPath: A Reinforcement Learning Method for Knowledge Graph Reasoning"

Stack-Captioning. Stack-Captioning: Coarse-to-Fine Learning for Image Captioning

papers-experiments. What papers I'm reading, what I found interesting, ideas, etc.

learning-to-match.

m-DAN. Implement current SOTA model for image text matching task

AlphaZero_Gomoku. An implementation of the AlphaZero algorithm for Gomoku (also called Gobang or Five in a Row)

PathTracing. A path tracer implemented in OpenGL. Renders reflective materials, diffuse materials and implements textures. Can also render triangle meshes.

pathgl. Simple progressive path tracing test using OpenGL and GLSL (old)

Path-Tracer. Implementation of Backward Path Tracing method in C++ using OpenGL.

mini-DFS. MiniDFS

miniDFS. Simple version of Distributed File System on multiple threads

Dense-CoAttention-Network. Improved Fusion of Visual and Language Representations by Dense Symmetric Co-Attention for Visual Question Answering

BiLSTM-CRF-pytorch. Python

Dual-Attention-Network. Tensorflow implementation of Dual Attention Network

attention-is-all-you-need-pytorch. A PyTorch implementation of the Transformer model in "Attention is All You Need".

emnlp2017-bilstm-cnn-crf. BiLSTM-CNN-CRF architecture for sequence tagging

pytorch-dynamic-batching. Dynamic batching for variable length inputs.

neural-vqa-tensorflow. Visual Question Answering in Tensorflow.

paper-end2end-dl. 📝 "End-to-end Deep Learning of Optimization Heuristics" (🥇 PACT'17 Best Paper)

pytorch_workplace. Random miniprojects with pytorch.

recurrent-batch-normalization-pytorch. PyTorch implementation of recurrent batch normalization

AdaptiveAttention. Implementation of "Knowing When to Look: Adaptive Attention via A Visual Sentinel for Image Captioning"

pytorch_compact_bilinear_pooling. Compact Bilinear Pooling for PyTorch

neural-dep-srl. A Simple and Accurate Syntax-Agnostic Neural Model for Dependency-based Semantic Role Labeling

Deformable-ConvNets. Deformable Convolutional Networks

vqa-counting. [ICLR 2018] Learning to Count Objects in Natural Images for Visual Question Answering

pytorch-vqa. Strong baseline for visual question answering

vqa-winner-cvprw-2017. Pytorch implementation of winner from VQA Chllange Workshop in CVPR'17

vqa-mfb.pytorch. This project is out of date, I don't remember the details inside...

vqa.pytorch. Visual Question Answering in Pytorch

Structured-Self-Attentive-Sentence-Embedding. An open-source implementation of the paper ``A Structured Self-Attentive Sentence Embedding'' (Lin et al., ICLR 2017).

pytorch-tutorial. PyTorch Tutorial for Deep Learning Researchers

ai-deadlines. :alarm_clock: AI conference deadline countdowns

CompactBilinearPooling-Pytorch. A Pytorch Implementation for Compact Bilinear Pooling.

compact_bilinear_pooling. MatConvNet and Caffe repo with compact bilinear and bilinear pooling functionality added

YouCompleteMe. A code-completion engine for Vim

show_attend_and_tell.tensorflow. Jupyter Notebook

show-attend-and-tell. TensorFlow Implementation of "Show, Attend and Tell"

Pretrained-Show-and-Tell-model. This repository contains pretrained Show and Tell: A Neural Image Caption Generator implemented in Tensorflow.

vqa-text. Python

VQA. Python

vqa-mcb. Python

ArtificiallyArtistic. This is the repository for our machine learning term project on Computer Generated Art. We shall be working on the Artistic Rendering of Images.

Paradigms-Programming-Project2. Jupyter Notebook

vqa. Some experiments on VQA dataset

coco-ui. MS COCO Annotation UIs

DeepMind-Teaching-Machines-to-Read-and-Comprehend. Implementation of "Teaching Machines to Read and Comprehend" proposed by Google DeepMind

lectures. Oxford Deep NLP 2017 course