Songlin Yang

linear-attention-and-beyond-slides. TeX

flash-linear-rnn. Implementations of various linear RNN layers using pytorch and triton

TN-PCFG. source code of NAACL2021 "PCFGs Can Do Better: Inducing Probabilistic Context-Free Grammars with Many Symbols“ and ACL2021 main conference "Neural Bilexicalized PCFG Induction"

mamba-triton. Python

fla-tilelang. Python

pointer-net-for-nested. The official implementation of ACL2022``Bottom-Up Constituency Parsing and Nested Named Entity Recognition with Pointer Networks''

gated_linear_attention_layer. Python

span-based-dependency-parsing. Source code of ACL2022 "Headed-Span-Based Projective Dependency Parsing" and "Combining (second-order) graph-based and headed-span-based projective dependency parsing

second-order-neural-dmv. source code of COLING2020 "Second-Order Unsupervised Neural Dependency Parsing"

disco-pointer. Official Implementation of ACL2023: Don't Parse, Choose Spans! Continuous and Discontinuous Constituency Parsing via Autoregressive Span Selection

TN-LCFRS. Official Implementation of ACL2023: Unsupervised Discontinuous Constituency Parsing with Mildly Context-Sensitive Grammars

hope-fla. Python

sustcsonglin.github.io. HTML

path_attention.

gnn-sdp. Findings of EMNLP'22: Semantic Dependency Parsing with Edge GNNs

lit-gpt. Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.

cuda-inside-algs.

FlagAttention. A collection of memory efficient attention operators implemented in the Triton language.

triton-inside. Using triton to implement inside algorithms

unlocking_state_tracking. Expanding linear RNN state-transition matrix eigenvalues to include negatives improves state-tracking tasks and language modeling without added training or inference costs.

pytorch-custom-mma. C++

dolomite-engine. Dolomite Engine is a library for pretraining/finetuning LLMs

nanokitchen. Parallel Associative Scan for Language Models

mamba.py. An efficient Mamba implementation in PyTorch and MLX.

streaming-llm. Efficient Streaming Language Models with Attention Sinks

lcfrs-supertagger. Python

TinyLlama. The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.

recurrent-chunked-models-regular-languages. Code of "Recurrent Transformers Trade-off Parallelism for Length Generalization on Regular Languages"

sane_tikz. Reconquer the canvas: beautiful Tikz figures without clunky Tikz code

shtthesis. An unofficial LaTeX thesis template for ShanghaiTech University.

stk. Python

zoology. Understand and test language model architectures on synthetic tasks.

sustcsonglin_old.github.io. :page_facing_up: Elegant & friendly homepage (bio, tech portfolio, resume, doc...) template with Markdown and VuePress

safari. Convolutions for Sequence Modeling

linear-rnn-pytorch.

cuda-playground. Cuda

transformers_ssm_copy. Python