Joerg Hiller Aug 14, 2025 21:30 GitHub's Q1 2025 Innovation Graph update highlights trends in software development, emphasizing data visualization and AI's growing impact. GitHub recently released its ...

We’ve open-sourced a PyTorch FSDP2 + Tensor Parallel (TP) implementation of Dion, available via a simple pip install. Our goal is to make faster training with Dion accessible to everyone. As a bonus, ...

This repository contains an implementation of the AdamL optimizer, a novel variant of the Adam optimizer that incorporates loss function information to achieve better generalization in deep learning ...

Unofficial implementation of Adan optimizer. This implementation differs from the official pytorch implementation. The main difference is that gradient parameters aren't updated for categorical values ...