Ghosts of Softmax

Interactive write-ups and technical notes accompanying the library.

Theory

• Analytic Normalization: Removing the Singularity at Zero
• Activation Design from the Convergence-Radius Lens

Tutorials

• Step Controller Introduction
• Adam Controller
• Momentum Controller
• Binary Radius
• KL Bound