Hao-Jun Michael Shi

optimizers. For optimization algorithm research and development.

prima. PRIMA is a package for solving general nonlinear optimization problems without using derivatives. It provides the reference implementation for Powell's derivative-free optimization methods, i.e., COBYLA, UOBYQA, NEWUOA, BOBYQA, and LINCOA. PRIMA means Reference Implementation for Powell's methods with Modernization and Amelioration, P for Powell.

algorithmic-efficiency. MLCommons Algorithmic Efficiency is a benchmark and competition measuring neural network training speedups due to algorithmic improvements in both training algorithms and models.

pdfo. Powell's Derivative-Free Optimization solvers.

DeepSpeed. DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

nocode. The best way to write secure and reliable applications. Write nothing; deploy nowhere.

dlrm. An implementation of a deep learning recommendation model (DLRM)

LaTeXing. LaTeXing - Professional Typesetting - An extension for Sublime Text which makes your life easier and more comfortable while creating your documents.

gpytorch. A highly efficient implementation of Gaussian Processes in PyTorch

PyTorch-LBFGS. A PyTorch implementation of L-BFGS.

pycutest. Python interface to CUTEst

large-batch-training. Code to reproduce some of the figures in the paper "On Large-Batch Training for Deep Learning: Generalization Gap and Sharp Minima"

shampoo.pytorch. An implementation of shampoo

kfac_pytorch. Python

NLP.py. A Python environment for large-scale optimization.

pytorch. Tensors and Dynamic neural networks in Python with strong GPU acceleration

CUTEST.py. Python