A Decade of Residuals: History & Effects on modern ML
January-2025
This article traces the evolution of residual connections and their influence on modern architectures, optimization dynamics, learned representations and the emergence of gating and hyper-connection mechanisms..
A Gentle Introduction to Distributed Training
November-2025
In this blog post we discuss distributed training by examining how parallelism strategies shard and schedule data, parameters and intermediate activations to control memory usage and execution flow.
JAX: Jit Autograd XLA
October-2025
In-depth reference on JAX: design and programming philosophy, distributed / multi-device training, async dispatch via XLA and high-performance usage patterns.
Mechanistic Interpretability
August-2025
An overview of techniques for reverse-engineering features, circuits, and representations in vision models using probing, disentanglement, and adversarial analysis.
Generative Adversarial Networks
July-2025
A concise exploration of how GANs work focusing on training schemes, architectures, objectives and the Generator-Discriminator dynamics.
Transformers as a flock of tokens
April-2025
This blog post breaks down transformers into alternating phases of token-to-token communication and representation-space transformation, viewing tokens as a flock evolving across layers.
A Representation Space Interpretation of Neural Networks
March-2025
This blog post offers a geometric reframing of neural networks, describing how successive layers sculpt and reorganize the data manifold in representation space.