Key Data Visualization Theory needed in Data Science — Telling a story is a very important part of any Data Scientist’s job. In this article I go through the key theory of Data Visualization. Introduction Knowing how to tell a story efficiently is one of the hardest skills to master as a data scientist. Data visualization theory, although less sexy…

The Model Responsible for your Siri, Alexa and Google Home — Transformers have revolutionized the field of natural language processing, computer vision and image generation. In this article I break down this this novel architecture and explain how it works. Introduction In the last few years, the newest generation of massive AI models have produced extremely impressive results. Just like in the…

SVM Theory Comprehensively Explained and Visualized — There are some derivations that every Data Scientist should know before applying specific machine learning models. In this article I go through all you need to know about SVMs. Introduction Kernel methods are extremely popular in the machine learning world due to their ease of use, interpretability and strong performance in…

K-means Clustering Intuitively Explained — K-means clustering is an iterative algorithm that selects the cluster centers that minimize the within-cluster variance. Introduction In this article, I want to introduce one of the simplest data clustering algorithms, k-means clustering. It is an algorithm that often shows up in interviews to test your knowledge of fundamentals. …

What is Precision and Recall and how do you use it? — Dealing with unbalanced data in machine learning is difficult. Using precision, recall, F-beta scores, and thresholds, I train a classifier to detect fraudulent data by applying basic machine learning theory. Introduction My articles are usually about machine learning theory and not so much about its applications. In this article I want…

How Generative Adversarial Networks (GANs) learn, theory and application — Introduction In my previous article I talked about generative methods in general and what makes generator networks so powerful. In this article, I want to go more in-depth into the inner workings of Generative Adversarial Networks (GANs). To make sure you understand them fully, I will go through the original GANs…

An introduction to the fundamentals of Reinforcement Learning, all you need to know to get started — In 9 hours, Google’s AlphaZero went from only knowing the rules of chess to beating the best models in the world. Chess has been studied by humans for over 1000 years, yet a reinforcement learning model was able to further our knowledge of the game in a negligible amount of…

An intuitive explanation of Fisher’s Linear Discriminant, the Basis of Linear Discriminant Analysis — LDA is a widely used dimensionality reduction technique built on Fisher’s linear discriminant. These concepts are fundamentals of machine learning theory. In this article, I’ll go through an example of a classifier using Fisher’s linear discriminant, and derive the optimal solution for Fisher’s criterion. …

What is really going on when you use a convolutional layer vs a fully connected layer? — The design of a Neural Network is quite a difficult thing to get your head around at first. Designing a neural network involves choosing many design features like the input and output sizes of each layer, where and when to apply batch normalization layers, dropout layers, what activation functions to…

An in-depth explanation of how CTGANs work — The purpose of this article is to fill the gap in the internet for a good explanation of how Conditional Tabular GANs work. …

An introduction to VAEs, GANs and their challenges — Introduction Training generative models is a lot harder than training classifiers or regressors. When training a classifier in a supervised learning problem, we know the optimal mapping between the inputs and the outputs of our training examples. However, training generative models involves optimizing criteria that is intractable. In this article, I…

Generating Image Data with Variational Auto Encoders — Introduction Variational Auto-Encoders (VAEs) are an amazing machine learning architecture. They allow us to approximate high-dimensional latent spaces that can be sampled to generate new data. AEs are quite a novel branch of models in machine learning. These are sometimes difficult to understand at first. …

The basics of kernel methods and Radial Basis Functions — Introduction The bias-variance dilemma dominates machine learning methods. If a model is too simple, the model will struggle to find appropriate relationships between inputs and outputs. However, if a model is too complex, it will perform better in training, but the will experience a greater difference in performance on unseen data…

Two basic sampling algorithms visually explained — Introduction In my previous article, I spoke about the Kalman filter. Although the Kalman filter is optimal for linear systems with Gaussian noise, when given a non-linear system, the algorithm fails. So what about non-linear systems? The problem with non-linear systems is that the integrals involved in the derivation of an optimal algorithm are not tractable…

The optimal online learning algorithm for linear systems with Gaussian noise — If a dynamic system is linear and with Gaussian noise, the optimal estimator of the hidden states is the Kalman Filter. This online learning algorithm is part of the fundamentals of the machine learning world. Understanding it well is important prior to understanding more complicated topics such as particle filters. …

An intuitive visualization of discrete Fourier transforms applied to simple time-series data. — This article visualizes the decomposition of a time series signal into its harmonics using the Fourier transform. The formula is explained in a visual manner to help understand its meaning. Why Fourier Transforms? The Fourier Transform is an extremely powerful tool used extensively in a wide variety of fields. Its power can be…

A simple model that learns on the fly updating its weights as new data comes in. — Why Recursive Least Squares? Online learning is a booming field of research in the AI research space. Many problems in today's world require machines to learn on the fly and improve or adapt as they collect new information. In this article, I will explain how to adapt the least-squares regression to compute the optimal…

A basic but powerful classifier and regressor, their derivations and why they work — In this article, I derive the pseudo-inverse solutions for the least-squares regression and classification algorithms. Although not very complex, in some problems it remains a very powerful tool and is still used today as the core of other machine learning models like ensemble methods or neural networks (where perceptrons present…

Why regularized least squares is equivalent to the maximum posterior solution and is the optimal algorithm for normally distributed data — This article compares Tikonov L2 regularisation to the maximum posterior probability function. This one is quite algebra dense so I recommend picking up a pen and paper and trying it yourself! The Bias Variance Dilemma One of the problems with many machine learning models is their tendency to overfit data. Overfitting is a phenomenon…

Gaussian Classifiers, exploring the effect of covariance and priors on the posterior probability surface — Introduction Normally distributed data can be found everywhere in nature. For example, in humans, our height, IQ, and birthweight form beautiful bell curves. The gaussian classifier can perfectly estimate the probability of class membership if the exact distribution parameters are known. This makes gaussian classifiers an extremely powerful tool.