How to calcualte pearson correlation

(Comments)

Analysis of correlation and significance of parameters

Correlation

The study of the significance of the impact of input parameters on output parameters should begin with the analysis of the correlation of individual parameters. Three basic dependencies can be checked:

  • monotonic linear
  • monotonic non-linear
  • square

Pearson's correlation coefficient (monotonic linear relationship)

The most basic measure determining whether there is a linear correlation between parametersxi i yiis the Pearson correlation coefficient:

rp=i=1n(xix¯)(yiy¯)i=1n(xix¯)2i=1n(yiy¯)2

wherex¯andy¯mean the mean values ​​of the relevant parameters.

This formula can be simplified to

rp=cov(x,y)var(x)var(y)

wherex=[x1,x2,...],y=[y1,y2,...]

Spearman's correlation coefficient (monotonic non-linear relationship)

Spearman's rank correlation coefficient is more universal because it allows to determine the strength of monotonic correlation, which may be non-linear and is expressed by the relation:

rs=i=1n(RiR¯)(SiS¯)i=1n(RiR¯)2i=1n(SiS¯)2

whereRiis the rank of the observationxi, Si is the rank of the observationyiandR¯ i S¯are the mean values ​​of the respective ranksRi andSi.

Interpretation of the correlation coefficient value

Correlation type:

  • rs> 0 positive correlation – when the value of X increases, so does Y
  • rs= 0 no correlation – when X increases, Y sometimes increases and sometimes decreases
  • rs< 0 negative correlation – when X increases, Y decreases

Correlation strength:

  • |rs|<0.2– no linear relationship
  • 0.2|rs|<0.4- weak dependence
  • 0.4|rs|<0.7– moderate dependency
  • 0.7|rs|<0.9- quite a strong relationship
  • |rs|0.9- very strong dependence

Quadratic correlation coefficient

The quadratic correlation coefficient is determined on the basis of regression analysis.

Error sum of squaresSSEis designated as

SSE=i=1n(yiy^i)2

After performing the approximation with a polynomial of the second degree (i.e. determining the coefficientsa2,a1,a0) y^i is determined by substitutionxito the formula of the approximating function

y^i=a2xi2+a1xi+a0

total sum of squaresSST to

SST=i=1n(yiy¯)2

The correlation coefficient is determined from the relationship

rq=1SSESST

Statistical testing of the significance of the correlation coefficient

To determine whether the determined correlation coefficient is statistically significant, it is necessary to make a null hypothesis

H0:δ=0

meaning that there is no correlation between the parameters. The alternative hypothesis has the form

H1:δ0

It is assumed that the statistic takes the Student's t-distribution o k=n2degrees of freedom and hence, for example, for the Pearson correlation coefficient, the value of the statistics is

t=rpn21rp2

The value of the test statistic cannot be determined whenrp=1 therp=1or whenn<3.

In other cases, the value determined on its basisp (read from the Student's t-distribution) is compared with the assumed significance levelα

  • ifpαwe reject itH0accepting H1
  • ifp>αthere is no reason to reject itH0

Typically, a significance level is selectedα=0.05, agreeing that in 5% of situations we will reject the null hypothesis when it is true.

The same is done for the other correlation coefficients insteadrpsubstitutingrstherq.

Currently unrated

Comments

Riddles

22nd Jul- 2020, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20 Oct- 2019, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20Aug- 2019, by: Editor in Chief
524 Shares 4 Comments
10Aug- 2019, by: Editor in Chief
424 Shares 4 Comments
Generic placeholder image
10Aug- 2015, by: Editor in Chief
424 Shares 4 Comments

More News  »

Sekolah doktor itu bukan hukuman! Yuk atur waktumu!

Recent news
5 days, 16 hours ago

Kenapa sekolah PhD butuh waktu lama!?

Recent news

Kali ini kita akan bahas kenapa sekolah PhD itu lama! Tanpa panjang lebar, berikut cara ngeles gw! Maksudnya berikut alasannya! Hope its relate with you!

read more
5 days, 16 hours ago

Using Vertex AI for zero one and two three AI prediction

Recent news

Here is my documentation after learning the introduction of AI in courserERA.

read more
3 weeks, 1 day ago

Neural network with API for pre-trained API

Recent news

Overview

The Cloud Natural Language API lets you extract entities from text, perform sentiment and syntactic analysis, and classify text into categories.

read more
3 weeks, 3 days ago

what is null result

Recent news

Null result in economic is when the output does not supporting your hypothesis

read more
3 weeks, 5 days ago

Big Query in Google cloud - the first small step to become solution architect

Recent news
3 weeks, 5 days ago

Fixing the issue in assumption of OLS step by step or one by one

Recent news

Hi, I want to raise the issue related to know whether your OLS is ok or not. 

read more
1 month, 3 weeks ago

Meaning of 45 degree in economics chart

Recent news

The **45-degree line** in economics and geometry refers to a line where the values on the x-axis and y-axis are equal at every point. It typically has a slope of 1, meaning that for every unit increase along the horizontal axis (x), there is an equal unit increase along the vertical axis (y). Here are a couple of contexts where the 45-degree line is significant:

read more
2 months, 3 weeks ago

More News »

Generic placeholder image

Collaboratively administrate empowered markets via plug-and-play networks. Dynamically procrastinate B2C users after installed base benefits. Dramatically visualize customer directed convergence without