4. Differential Calculus#
Differential calculus is the branch of mathematics focused on studying rates of change and finding the slope of a curve at any given point.
Machine learning methods are mathematical algorithms written into code to extract information from data. So, if you want to do machine learning, you will need some mathematics.
Historical Aside: Least Squares The following are excepts from the Wikipedia article on least squares.
The least squares method is a statistical technique used in regression analysis to find the best trend line for a data set on a graph. It essentially finds the best-fit line that represents the overall direction of the data. Each data point represents the relation between an independent variable.
The method was the culmination of several advances that took place during the course of the eighteenth century:
The combination of different observations as being the best estimate of the true value; errors decrease with aggregation rather than increase, first appeared in Isaac Newton’s work in 1671, though it went unpublished, and again in 1700. It was perhaps first expressed formally by Roger Cotes in 1722.
The combination of different observations taken under the same conditions contrary to simply trying one’s best to observe and record a single observation accurately. The approach was known as the method of averages. This approach was notably used by Newton while studying equinoxes in 1700, also writing down the first of the ‘normal equations’ known from ordinary least squares,[4] Tobias Mayer while studying the librations of the Moon in 1750, and by Pierre-Simon Laplace in his work in explaining the differences in motion of Jupiter and Saturn in 1788.
The combination of different observations taken under different conditions. The method came to be known as the method of least absolute deviation. It was notably performed by Roger Joseph Boscovich in his work on the shape of the Earth in 1757 and by Pierre-Simon Laplace for the same problem in 1789 and 1799.
The development of a criterion that can be evaluated to determine when the solution with the minimum error has been achieved. Laplace tried to specify a mathematical form of the probability density for the errors and define a method of estimation that minimizes the error of estimation. For this purpose, Laplace used a symmetric two-sided exponential distribution we now call Laplace distribution to model the error distribution, and used the sum of absolute deviation as error of estimation. He felt these to be the simplest assumptions he could make, and he had hoped to obtain the arithmetic mean as the best estimate. Instead, his estimator was the posterior median.
In 1809 Carl Friedrich Gauss published his method of calculating the orbits of celestial bodies. In that work he claimed to have been in possession of the method of least squares since 1795.[6] This naturally led to a priority dispute with Legendre. However, to Gauss’s credit, he went beyond Legendre and succeeded in connecting the method of least squares with the principles of probability and to the normal distribution. He had managed to complete Laplace’s program of specifying a mathematical form of the probability density for the observations, depending on a finite number of unknown parameters, and define a method of estimation that minimizes the error of estimation. Gauss showed that the arithmetic mean is indeed the best estimate of the location parameter by changing both the probability density and the method of estimation. He then turned the problem around by asking what form the density should have and what method of estimation should be used to get the arithmetic mean as estimate of the location parameter. In this attempt, he invented the normal distribution.*
In the above, the players had the following professions:
Issac Newton, mathematician, physicist, astronomer, alchemist, theologian
Roger Coates, mathematician, whose contributes to computational methods were heavily dependent on his work in astronmy
Pierre-Simon Laplace, worked in physics, astronomy, mathematics, engineering, statistics, and philosophy
Roger Joseph Boscovich, physicist, astronomer, mathematician, philosopher, diplomat, poet, theologian, Jesuit priest, and a polymath
Carl Friedrich Gauss, mathematician, astronomer, geodesist, and physicist
The algorithms used in data science have their history in the physical sciences. Many were developed well before the age of computers. You should know this,…, data science is much more than import blah blah blah.