Clik here to view.
Higher-Order Derivatives
Tweet Share Share Last Updated on July 31, 2021 Higher-order derivatives can capture information about a function that first-order derivatives on their own cannot capture. First-order derivatives can...
View ArticleClik here to view.
A Gentle Introduction to the Jacobian
Tweet Share Share In the literature, the term Jacobian is often interchangeably used to refer to both the Jacobian matrix or its determinant. Both the matrix and the determinant have useful and...
View ArticleClik here to view.
A Gentle Introduction To Hessian Matrices
Tweet Share Share Last Updated on August 5, 2021 Hessian matrices belong to a class of mathematical structures that involve second order derivatives. They are often used in machine learning and data...
View ArticleClik here to view.
A Gentle Introduction to the Laplacian
Tweet Share Share Last Updated on August 6, 2021 The Laplace operator was first applied to the study of celestial mechanics, or the motion of objects in outer space, by Pierre-Simon de Laplace, and as...
View ArticleClik here to view.
A Gentle Introduction to Optimization / Mathematical Programming
Tweet Share Share Last Updated on August 10, 2021 Whether it is a supervised learning problem or an unsupervised problem, there will be some optimization algorithm working in the background. Almost...
View ArticleClik here to view.
A Gentle Introduction To Method Of Lagrange Multipliers
Tweet Share Share The method of Lagrange multipliers is a simple and elegant method of finding the local minima or local maxima of a function subject to equality or inequality constraints. Lagrange...
View ArticleClik here to view.
The Chain Rule of Calculus for Univariate and Multivariate Functions
Tweet Share Share The chain rule allows us to find the derivative of composite functions. It is computed extensively by the backpropagation algorithm, in order to train feedforward neural networks. By...
View ArticleClik here to view.
The Chain Rule of Calculus – Even More Functions
Tweet Share Share The chain rule is an important derivative rule that allows us to work with composite functions. It is essential in understanding the workings of the backpropagation algorithm, which...
View ArticleClik here to view.
A Gentle Introduction To Approximation
Tweet Share Share When it comes to machine learning tasks such as classification or regression, approximation techniques play a key role in learning from the data. Many machine learning methods...
View ArticleClik here to view.
A Gentle Introduction to Taylor Series
Tweet Share Share A Gentle Introduction to Taylor Series Taylor series expansion is an awesome concept, not only the world of mathematics, but also in optimization theory, function approximation and...
View ArticleClik here to view.
Calculus in Action: Neural Networks
Tweet Share Share An artificial neural network is a computational model that approximates a mapping between inputs and outputs. It is inspired by the structure of the human brain, in that it is...
View ArticleClik here to view.
A Gentle Introduction To Sigmoid Function
Tweet Share Share Whether you implement a neural network yourself or you use a built in library for neural network learning, it is of paramount importance to understand the significance of a sigmoid...
View ArticleClik here to view.
Lagrange Multiplier Approach with Inequality Constraints
Tweet Share Share Last Updated on August 28, 2021 In a previous post, we introduced the method of Lagrange multipliers to find local minima or local maxima of a function with equality constraints. The...
View ArticleClik here to view.
A Bird’s Eye View of Research on Attention
Tweet Share Share Last Updated on September 9, 2021 Attention is a concept that is scientifically studied across multiple disciplines, including psychology, neuroscience and, more recently, machine...
View ArticleClik here to view.
What is Attention?
Tweet Share Share Last Updated on September 15, 2021 Attention is becoming increasingly popular in machine learning, but what makes it such an attractive concept? What is the relationship between...
View ArticleClik here to view.
A Gentle Introduction to Particle Swarm Optimization
Tweet Share Share Particle swarm optimization (PSO) is one of the bio-inspired algorithms and it is a simple one to search for an optimal solution in the solution space. It is different from other...
View ArticleClik here to view.
The Attention Mechanism from Scratch
Tweet Share Share Last Updated on September 20, 2021 The attention mechanism was introduced to improve the performance of the encoder-decoder model for machine translation. The idea behind the...
View ArticleClik here to view.
Training-validation-test split and cross-validation done right
Tweet Share Share Last Updated on September 23, 2021 One crucial step in machine learning is the choice of model. A suitable model with suitable hyperparameter is the key to a good prediction result....
View ArticleClik here to view.
An Introduction To Recurrent Neural Networks And The Math That Powers Them
Tweet Share Share When it comes to sequential or time series data, traditional feedforward networks cannot be used for learning and prediction. A mechanism is required that can retain past or historic...
View ArticleClik here to view.
How to Learn Python for Machine Learning
Tweet Share Share Python has become a defacto lingua franca for machine learning. It is not a difficult language to learn, but if you are not particularly familiar with the language, there are some...
View Article