Felix Dangel

backpack. BackPACK - a backpropagation package built on top of PyTorch which efficiently computes quantities other than the gradient.

cockpit. Cockpit: A Practical Debugging Tool for Training Deep Neural Networks

unfoldNd. (N=1,2,3)-dimensional unfold (im2col) and fold (col2im) in PyTorch

curvlinops. PyTorch linear operators for curvature matrices (Hessian, Fisher/GGN, KFAC, ...)

singd. [ICML 2024] SINGD: KFAC-like Structured Inverse-Free Natural Gradient Descent (http://arxiv.org/abs/2312.05705)

hbp. Hessian backpropagation (HBP): PyTorch extension of backpropagation for block-diagonal curvature matrix approximations

phd-thesis. Source code for my PhD thesis: Backpropagation Beyond the Gradient

einconv. Convolutions and more as einsum for PyTorch

vivit. [TMLR 2022] Curvature access through the generalized Gauss-Newton's low-rank structure: Eigenvalues, eigenvectors, directional derivatives & Newton steps

sirfshampoo. [ICML 2024] SIRFShampoo: Structured inverse- and root-free Shampoo in PyTorch (https://arxiv.org/abs/2402.03496)

phd-thesis-template. LaTeX template for my PhD thesis at the University of Tuebingen

torch-jet. Taylor mode automatic differentiation (jets) in PyTorch

kfac-tutorial. KFAC from scratch (KFS)---Paper & Code

kfac-pinns-experiments. [NeurIPS2024] Paper and experiments for "Kronecker-Factored Approximate Curvature for Physics-Informed Neural Networks"

org-export-setup. My org-export settings

wandb_preempt. Code and tutorial on integrating wandb sweeps with Slurm pre-emption

python-utilities. Python utility functions I often use

backobs. Use DeepOBS with BackPACK

vivit-experiments. Experiments for the TMLR 2023 paper "ViViT: Curvature Access Through the Generalized Gauss-Newton’s Low-rank Structure"