Machine Learning

Teaching Machines to Draw: A Problem Solver’s Journey This blog post is on the topic of using machine learning for image generation. However, its goal is not to serve as a comprehensive guide to the field, nor a formal treatise on the subject. It is not even intended to be historically accurate, or to teach you about the mathematical formulas behind the techniques. Instead, it is a fictional story written from the perspective of a curious explorer who is attempting to solve a challenging problem....

An full explanation and interpretation of scaled dot-product attention via the Boltzmann distribution and probability theory.

An overview of Stochastic Gradient Descent (SGD)

I want to comment on the recent results achieved by OpenAI’s new o1 model family. Specifically, I will be dicussing formal verification, which can be applied to mathematics and computer programming. The name of this post is inspired by their September 12 2024 blog post titled “Learning to Reason with LLMs”. The abstract reads: “We are introducing OpenAI o1, a new large language model trained with reinforcement learning to perform complex reasoning....

An overview of diffusion models, including their mathematical foundations, key concepts, and practical applications.

Exploring how machine learning models generalize by interpolating on data manifolds, with concrete examples from image and language processing

A short post on Multi-Head Latent Attention as presented in the DeepSeek-V2 paper.

An overview of the challenges and solutions for learning meaningful features in machine learning models.

An evaluation of the expressiveness of sparse matrices in machine learning.

An explanation of the potential benefits of Strassen’s algorithm in ternary matrices.