8-bit Adafactor Optimizer with Fused CUDA Kernels
-
Updated
Jul 12, 2026 - Python
8-bit Adafactor Optimizer with Fused CUDA Kernels
reproduce Adam, AdamW, Adafactor optimizors with pytorch, and introduce popular optimizers in the training of the LLMs.
A from-scratch 336M-parameter language model built with JAX and Equinox
Add a description, image, and links to the adafactor topic page so that developers can more easily learn about it.
To associate your repository with the adafactor topic, visit your repo's landing page and select "manage topics."