Welcome back! After having gone through some posts on introduction to ML (here, here, here, here and here), lets now start learning a real ML algorithm. The algorithm that we will study today is a supervised learning regression algorithm.

First lets define the problem statement. Today we will use a subset of this data. The first few training examples in the data are:

In this data, the 1st column is the "mpg" target variable, or the parameter that we have to predict, while the rest of the columns are the features or input variables. But today we will go simple and will use only 1 input variable, that is the 3rd column "displacement". To keep the algorithm simple, we will assume that our target variable "mpg" is someway dependent or related to only "displacement", our input variable.

So now the first 10 training examples look like (notice that I have swapped the order of the data Note that the first column is x (input variable(s)), and second column is y (target variable))

Thus the problem definition is given a new "displacement" (x), we need to predict the value for "mpg" (y). To do this we need to a function "h" that maps the input "x" to the output "y". In ML world, this function "h" is called the hypothesis function, and is represented as

h_Θ(x) = Θ₀ + Θ₁ x

In the above example "h" is also called 'univariate linear regression', because we are using only 1 variable (or feature).

Before we move ahead, lets get some terminology noted

• m => #examples
• x => input variable/feature
• y => output variable/target
• x⁽ⁱ⁾, y⁽ⁱ⁾ => i^th training example, so for example, in the above data, x⁽¹⁾ = 307.0, x⁽⁵⁾ = 302.0, y⁽⁷⁾ = 14.0

What we need to do next is to chose Θ₀ and Θ₁ so that h_Θ(x) is as close to y for our training examples (x, y). I hope you understand why we are doing this. This is because, if the hypothesis function that we find out (as the end goal of our ML problem) is perfect, then the value of h_Θ(x⁽ⁱ⁾) should be exactly equal to the corresponding y⁽ⁱ⁾, for any example i.

So, obviously, our goal would be to do the following

minimize_(Θ0,Θ1) (Σ_i^m (h_Θ(x⁽ⁱ⁾) - y⁽ⁱ⁾)²)/(2*m)

The division by 2m is simply taking an average of the squared error and is done to make later math easier. This function is called the cost function J(Θ₀, Θ₁), and also called squared error function, or squared error cost function. This is the most most commonly used error function in ML world.

By implementing the above we would find the optimum values of Θ₀ and Θ₁ that will minimize our cost function J. Then all we need to do is to insert these Θ₀ and Θ₁ values into our hypothesis function h_Θ(x) and our ML problem is solved. Then given any new value for x, we can simply put that x in the hypothesis function to get the y.

Ain't that cool? We have already started solving ML problems and making predictions!

Well, not so fast stud ;) We still have to figure out how to perform the last step of the puzzle, viz "minimize", don't we? But I think this much study is good for today, and we will jump to the minimization problem in the next post. So keep watching this space :)