How to calcualte pearson correlation

(Comments)

Analysis of correlation and significance of parameters

Correlation

The study of the significance of the impact of input parameters on output parameters should begin with the analysis of the correlation of individual parameters. Three basic dependencies can be checked:

  • monotonic linear
  • monotonic non-linear
  • square

Pearson's correlation coefficient (monotonic linear relationship)

The most basic measure determining whether there is a linear correlation between parametersxi i yiis the Pearson correlation coefficient:

rp=i=1n(xix¯)(yiy¯)i=1n(xix¯)2i=1n(yiy¯)2

wherex¯andy¯mean the mean values ​​of the relevant parameters.

This formula can be simplified to

rp=cov(x,y)var(x)var(y)

wherex=[x1,x2,...],y=[y1,y2,...]

Spearman's correlation coefficient (monotonic non-linear relationship)

Spearman's rank correlation coefficient is more universal because it allows to determine the strength of monotonic correlation, which may be non-linear and is expressed by the relation:

rs=i=1n(RiR¯)(SiS¯)i=1n(RiR¯)2i=1n(SiS¯)2

whereRiis the rank of the observationxi, Si is the rank of the observationyiandR¯ i S¯are the mean values ​​of the respective ranksRi andSi.

Interpretation of the correlation coefficient value

Correlation type:

  • rs> 0 positive correlation – when the value of X increases, so does Y
  • rs= 0 no correlation – when X increases, Y sometimes increases and sometimes decreases
  • rs< 0 negative correlation – when X increases, Y decreases

Correlation strength:

  • |rs|<0.2– no linear relationship
  • 0.2|rs|<0.4- weak dependence
  • 0.4|rs|<0.7– moderate dependency
  • 0.7|rs|<0.9- quite a strong relationship
  • |rs|0.9- very strong dependence

Quadratic correlation coefficient

The quadratic correlation coefficient is determined on the basis of regression analysis.

Error sum of squaresSSEis designated as

SSE=i=1n(yiy^i)2

After performing the approximation with a polynomial of the second degree (i.e. determining the coefficientsa2,a1,a0) y^i is determined by substitutionxito the formula of the approximating function

y^i=a2xi2+a1xi+a0

total sum of squaresSST to

SST=i=1n(yiy¯)2

The correlation coefficient is determined from the relationship

rq=1SSESST

Statistical testing of the significance of the correlation coefficient

To determine whether the determined correlation coefficient is statistically significant, it is necessary to make a null hypothesis

H0:δ=0

meaning that there is no correlation between the parameters. The alternative hypothesis has the form

H1:δ0

It is assumed that the statistic takes the Student's t-distribution o k=n2degrees of freedom and hence, for example, for the Pearson correlation coefficient, the value of the statistics is

t=rpn21rp2

The value of the test statistic cannot be determined whenrp=1 therp=1or whenn<3.

In other cases, the value determined on its basisp (read from the Student's t-distribution) is compared with the assumed significance levelα

  • ifpαwe reject itH0accepting H1
  • ifp>αthere is no reason to reject itH0

Typically, a significance level is selectedα=0.05, agreeing that in 5% of situations we will reject the null hypothesis when it is true.

The same is done for the other correlation coefficients insteadrpsubstitutingrstherq.

Currently unrated

Comments

Riddles

22nd Jul- 2020, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20 Oct- 2019, by: Editor in Chief
524 Shares 4 Comments
Generic placeholder image
20Aug- 2019, by: Editor in Chief
524 Shares 4 Comments
10Aug- 2019, by: Editor in Chief
424 Shares 4 Comments
Generic placeholder image
10Aug- 2015, by: Editor in Chief
424 Shares 4 Comments

More News  »

How to create output gap with Python and Anaconda

Recent news
3 weeks, 3 days ago

Dignity wrapped in Charity

Recent news
2 months ago

A reflection of using kanban flow and being minimalist

Recent news

Today is the consecutive day I want to use and be consistent with the Kanban flow! It seems it's perfect to limit my parallel and easily distractedness. 

read more
2 months, 2 weeks ago

Morning issue with car and my kind of music

Recent news
2 months, 2 weeks ago

Podcast Bapak Dimas 2 - pindahan rumah

Recent news

Vlog kali ini adalah terkait pindahan rumah!

read more
2 months, 2 weeks ago

Podcast Bapak Dimas - Bapaknya Jozio dan Kaziu - ep 1

Recent news

Seperti yang saya cerita kan sebelumnya, berikut adalah catatan pribadi VLOG kita! Bapak Dimas

read more
2 months, 2 weeks ago

Happy new year 2024 and thank you 2023!

Recent news

As the new year starts, I want to revisit what has happened in 2023. 

read more
2 months, 3 weeks ago

Some notes about python and Zen of Python

Recent news

Explore Python syntax

Python is a flexible programming language used in a wide range of fields, including software development, machine learning, and data analysis. Python is one of the most popular programming languages for data professionals, so getting familiar with its fundamental syntax and semantics will be useful for your future career. In this reading, you will learn about Python’s syntax and semantics, as well as where to find resources to further your learning.

read more
4 months ago

More News »

Generic placeholder image

Collaboratively administrate empowered markets via plug-and-play networks. Dynamically procrastinate B2C users after installed base benefits. Dramatically visualize customer directed convergence without