Hi there! Welcome back to this series of posts on ML algorithms. In the last post we looked at the hypothesis representation and decision boundary concepts in the logistic regression ML algorithm. Today we will look at some more examples of decision boundaries and get a deeper understanding regarding them.

But, let's do a quick review of what we learned last time so that we can continue from there. We defined our hypothesis function as:

h_θ(x) = g(θ' * x)

where,

g(z) = 1 / (1 + e^-z)

is the sigmoid function. We also saw an important result that:

"y = 1" if (θ' *x) >= 0

and

"y = 0" if (θ' *x) < 0

The decision boundary we saw last time was a simple straight line. But what if our dataset is like below?

![]()

If you remember what we discussed in one of the previous posts, then the solution is simple. We should use higher order polynomials. Let us define our hypothesis function as:

h_θ(x) = θ₀ + θ₁ x1 + θ₂ x2 + θ₃ x1² + θ₄ x2²

And, furthermore, let us say that we have somehow found the correct values for θ already as:

θ = [-1 0 0 1 1]'

I know that we have not yet looked at how to find the optimal values of θ for logistic regression. But fear not, we will get there.

Coming back to the problem at hand. When we use the above θ, we get the following hypothesis function output:

![]()

You see that the decision boundary is now a circle, and

"y = 1" if x1² + x2² >= 1

and

"y = 0" if x1² + x2² < 1

Similarly, if we take more complex higher order polynomials like,

h_θ(x) = θ₀ + θ₁ x1 + θ₂ x2 + θ₃ x1² + θ₄ x2² + θ₅ x1x2 + θ₅ x1² + …

then we can get even more complex forms of the decision boundary which will help us correctly classify our dataset.

I hope you are with me till here, and are loving reading about these algorithms as much as I am loving writing about them. In the next post we will take a look at the methods to compute the optimal values of θ. So keep watching this space!