Ols

(Comments)

Get started

Open in app

Open in app

You have 2 free member-only stories left this month. Sign up for Medium and get an extra one

Understanding the OLS method for Simple Linear Regression

Valentina Alto

Valentina Alto

Aug 17, 2019·4 min read

Linear Regression is the family of algorithms employed in supervised machine learning tasks (to learn more about supervised learning, you can read my former article here). Knowing that supervised ML tasks are normally divided into classification and regression, we can collocate Linear Regression algorithms in the latter category. It differs from classification because of the nature of the target variable: in classification, the target is a categorical value (‘yes/no’, ‘red/blue/green’, ‘spam/not spam’…); on the other hand, regression involves numerical, continuous values as target, hence the algorithm will be asked to predict a continuous number rather than a class or category. Namely, imagine you want to predict the price of a house based on some relative features: the output of your model will be the price, hence a continuous number.

Regression tasks can be divided into two main groups: those which use only one feature to predict the target, and those which use more than one features for that purpose. To give you an example, let’s consider the house task above: if you want to predict its price only based on its squared meters, you will fall into the first situation (one feature); if you are going to predict the price based on, let’s say, its squared meters, its position and the liveability of the surrounding environment, you are going to fall into the second situation (multiple features, in that case, three).

In the first scenario, the algorithm you are likely to employ will be the Simple Linear Regression, which is the one we are going to talk about in this article. On the other side, whenever you are facing more than one features able to explain the target variable, you are likely to employ a Multiple Linear Regression.

Simple Linear Regression is a statistical model, widely used in ML regression tasks, based on the idea that the relationship between two variables can be explained by the following formula:

Where εi is the error term, and α, β are the true (but unobserved) parameters of the regression. The parameter β represents the variation of the dependent variable when the independent variable has a unitary variation: namely, if my parameter is equal to 0.75, when my x increases by 1, my dependent variable will increase by 0.75. On the other hand, the parameter α represents the value of our dependent variable when the independent one is equal to zero.

Let’s visualize it graphically:

Now, the idea of Simple Linear Regression is finding those parameters α and β for which the error term is minimized. To be more precise, the model will minimize the squared errors: indeed, we do not want our positive errors to be compensated by the negative ones, since they are equally penalizing for our model.

This procedure is called Ordinary Least Squared error — OLS.

Let’s demonstrate those optimization problems step by step. If we reframe our squared error sum as follows:

We can set our optimization problem as follows:

So let’s start with β:

Currently unrated

Comments

Riddles

22nd Jul- 2020, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20 Oct- 2019, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20Aug- 2019, by: Editor in Chief
524 Shares 4 Comments
10Aug- 2019, by: Editor in Chief
424 Shares 4 Comments
Generic placeholder image
10Aug- 2015, by: Editor in Chief
424 Shares 4 Comments

More News  »

Fixing the issue in assumption of OLS step by step or one by one

Recent news

Hi, I want to raise the issue related to know whether your OLS is ok or not. 

read more
2 weeks, 6 days ago

Meaning of 45 degree in economics chart

Recent news

The **45-degree line** in economics and geometry refers to a line where the values on the x-axis and y-axis are equal at every point. It typically has a slope of 1, meaning that for every unit increase along the horizontal axis (x), there is an equal unit increase along the vertical axis (y). Here are a couple of contexts where the 45-degree line is significant:

read more
1 month, 3 weeks ago

hyperinflation in hungary

Recent news

The **hyperinflation in Hungary** in the aftermath of World War II (1945–1946) is considered the worst case of hyperinflation in recorded history. The reasons behind this extreme economic event are numerous, involving a combination of war-related devastation, political instability, massive fiscal imbalances, and mismanagement of monetary policy. Here's an in-depth look at the primary causes:

read more
2 months ago

what is neutrailty of money

Recent news

**Neutrality of money** is a concept in economics that suggests changes in the **money supply** only affect **nominal variables** (like prices, wages, and exchange rates) and have **no effect on real variables** (like real GDP, employment, or real consumption) in the **long run**.

read more
2 months ago

Japan deflationary phenomenon

Recent news

Deflation in Japan, which has persisted over several decades since the early 1990s, is a complex economic phenomenon. It has been influenced by a combination of structural, demographic, monetary, and fiscal factors. Here are the key reasons why deflation occurred and persisted in Japan:

read more
2 months ago

What the tips against inflation

Recent news

Hedging against inflation involves taking financial or investment actions designed to protect the purchasing power of money in the face of rising prices. Inflation erodes the value of currency over time, so investors seek assets or strategies that tend to increase in value or generate returns that outpace inflation. Below are several ways to hedge against inflation:

read more
2 months ago

Long and short run philip curve

Recent news

The **Phillips Curve** illustrates the relationship between inflation and unemployment, and how this relationship differs in the **short run** and the **long run**. Over time, economists have modified the original Phillips Curve framework to reflect more nuanced understandings of inflation and unemployment dynamics.

read more
2 months ago

How the government deal with inflation (monetary and fiscal) policies

Recent news

Dealing with inflation requires a combination of **fiscal and monetary policy** tools. Policymakers adjust these tools depending on the nature of inflation—whether it's **demand-pull** (inflation caused by excessive demand in the economy) or **cost-push** (inflation caused by rising production costs). Below are key approaches to controlling inflation through fiscal and monetary policy.

read more
2 months ago

More News »

Generic placeholder image

Collaboratively administrate empowered markets via plug-and-play networks. Dynamically procrastinate B2C users after installed base benefits. Dramatically visualize customer directed convergence without