Correlation and Regression Analysis | MGT 202

This unit covers correlation (Pearson, Spearman, probable error, and $r^{2}$) and simple linear regression equations and coefficients.

Correlation

The relationship between two variables such that a change in the value of one variable makes a change in the value of the other variable is known as correlation.

The measure of the degree of correlation between the two variables is known as the correlation coefficient.

Types of correlation

Positive and negative correlation
Linear and non-linear correlation
Simple, multiple and partial correlation

Methods of studying correlation

Scatter diagram
Karl Pearson’s correlation coefficient
Spearman’s rank correlation

Karl Pearson’s correlation coefficient

It is also known as Pearson’s correlation coefficient (or Pearson’s coefficient of correlation).

$$\begin{aligned} r &= \frac{\mathrm{Cov}(x,y)}{\sqrt{\mathrm{Var}\,x}\,\sqrt{\mathrm{Var}\,y}} \end{aligned}$$

Properties of the correlation coefficient

The value of $r$ has no unit.
Its formula is symmetrical: $r_{xy}=r_{yx}$.
It is independent of the change of origin: $r_{xy}=r_{UV}$ where $U=x-A$, $V=y-B$ and $A$, $B$ are assumed means.
It is independent of the change of scale: $r_{xy}=r_{uv}$ where $u=\dfrac{x-A}{h}$, $v=\dfrac{y-B}{k}$ and $h$, $k$ are constants.
Its values lie between $-1$ and $1$, i.e. $-1\le r\le 1$.
If $r=1$, there is perfect positive relationship; if $r=-1$, there is perfect negative relationship; if $r=0$, there is no correlation at all.
It is the geometric mean between the two regression coefficients, i.e. $r=\sqrt{b_{xy}\,b_{yx}}$ (taking the sign of the regression coefficients).

Computing the coefficient of correlation

(i) Deviations from the actual mean:

$$r=\frac{\sum xy}{\sqrt{\sum x^{2}\sum y^{2}}}$$

(ii) Direct method (actual data):

$$r=\frac{n\sum xy-\sum x\cdot\sum y}{\sqrt{\bigl[n\sum x^{2}-(\sum x)^{2}\bigr]\bigl[n\sum y^{2}-(\sum y)^{2}\bigr]}}$$

(iii) Deviations from assumed means: with $U=x-A$, $V=y-B$,

$$r=\frac{n\sum UV-\sum U\cdot\sum V}{\sqrt{n\sum U^{2}-(\sum U)^{2}}\,\sqrt{n\sum V^{2}-(\sum V)^{2}}}$$

(iv) Step-deviations: with $u=\dfrac{x-A}{h}$, $v=\dfrac{y-B}{k}$,

$$r=\frac{n\sum uv-\sum u\cdot\sum v}{\sqrt{n\sum u^{2}-(\sum u)^{2}}\,\sqrt{n\sum v^{2}-(\sum v)^{2}}}$$

Coefficient of determination ($r^{2}$)

The square of the correlation coefficient is denoted by $r^{2}$. From the regression-coefficient property,

$$r^{2}=b_{xy}\,b_{yx}$$

(taking the product of the two regression coefficients; $r$ itself carries their common sign).

Probable error

Probable error of the correlation coefficient, denoted by P.E., is the measure of testing the reliability of the calculated value of $r$.

$$\mathrm{P.E.}=0.6745\times\frac{1-r^{2}}{\sqrt{n}}$$

If $r<\mathrm{P.E.}$, it is insignificant.
If $r>\mathrm{P.E.}$, it is significant.
A common exam check compares $r$ with $6\,\mathrm{P.E.}(r)$: if $r>6\,\mathrm{P.E.}(r)$, the correlation is treated as significant.

Correlation for a bivariate frequency distribution

The distribution in which the values of two variables $X$ and $Y$ are grouped and the frequencies of different groups are given is known as a bivariate frequency table (or correlation table).

$$r=\frac{N\sum fuv-\sum fu\cdot\sum fv}{\sqrt{N\sum fu^{2}-(\sum fu)^{2}}\,\sqrt{N\sum fv^{2}-(\sum fv)^{2}}}$$

where $N$ is the total frequency, $u=\dfrac{x-A}{h}$, and $v=\dfrac{y-B}{k}$.

Spearman’s rank correlation

The degree of relationship between two variables with respect to their respective ranks is known as the rank correlation coefficient.

$$\begin{aligned} R &= 1-\frac{6\sum d^{2}}{n(n^{2}-1)} \\ d &= R_{1}-R_{2} \end{aligned}$$

where $n$ is the number of pairs of observations. The value of $R$ lies between $-1$ and $+1$.

Repeated ranks (tied ranks):

$$R=1-\frac{6\left[\sum d^{2}+\dfrac{m_{1}(m_{1}^{2}-1)}{12}+\dfrac{m_{2}(m_{2}^{2}-1)}{12}+\cdots\right]}{n(n^{2}-1)}$$

where each $m_{i}$ is the number of items sharing a repeated rank.

Regression analysis

Regression analysis is a statistical device with the help of which we can estimate or predict the value of one variable when the value of the other variable is known.

The unknown variable which we have to predict is called the dependent variable, and the variable whose value is known is called the independent variable.

The analysis used to describe the average relationship between two variables is known as simple linear regression analysis. Regression may be of $Y$ on $X$ or of $X$ on $Y$.

When the average relationship can be represented by a straight line it is called linear regression; otherwise it is non-linear.

Regression of $Y$ on $X$

The regression equation of $Y$ on $X$ describes the variation in $Y$ for a given change in $X$:

$$\begin{aligned} y-\overline{y} &= b_{yx}(x-\overline{x}) \\ b_{yx} &= \frac{n\sum xy-\sum x\cdot\sum y}{n\sum x^{2}-(\sum x)^{2}} \\ b_{yx} &= \frac{n\sum uv-\sum u\cdot\sum v}{n\sum u^{2}-(\sum u)^{2}} \end{aligned}$$

For a bivariate frequency distribution:

$$b_{yx}=\frac{N\sum fuv-\sum fu\cdot\sum fv}{N\sum fu^{2}-(\sum fu)^{2}}\times\frac{k}{h}$$

Regression of $X$ on $Y$

The regression equation of $X$ on $Y$ describes the variation in $X$ for a given change in $Y$:

$$\begin{aligned} x-\overline{x} &= b_{xy}(y-\overline{y}) \\ b_{xy} &= \frac{n\sum xy-\sum x\cdot\sum y}{n\sum y^{2}-(\sum y)^{2}} \\ b_{xy} &= \frac{n\sum uv-\sum u\cdot\sum v}{n\sum v^{2}-(\sum v)^{2}} \end{aligned}$$

For a bivariate frequency distribution:

$$b_{xy}=\frac{N\sum fuv-\sum fu\cdot\sum fv}{N\sum fv^{2}-(\sum fv)^{2}}\times\frac{h}{k}$$

Relation between $r$ and the regression coefficients

The correlation coefficient is the square root of the product of the two regression coefficients:

$$r=\pm\sqrt{b_{yx}\,b_{xy}}$$

Useful exam identities (same sign as $r$):

$b_{yx}=r\dfrac{\sigma_{y}}{\sigma_{x}}$
$b_{xy}=r\dfrac{\sigma_{x}}{\sigma_{y}}$

Example pattern: if $b_{yx}=-0.667$ and $b_{xy}=-0.75$, then $r=-\sqrt{(0.667)(0.75)}$.

Worked illustration (direct method)

Given $N=10$, $\sum x=650$, $\sum y=650$, $\sum xy=43294$, $\sum x^{2}=43144$, $\sum y^{2}=44002$ (motor registration $x$ and tyres sold $y$, both in thousands):

$$\begin{aligned} r&=\frac{N\sum xy-\sum x\cdot\sum y}{\sqrt{N\sum x^{2}-(\sum x)^{2}}\,\sqrt{N\sum y^{2}-(\sum y)^{2}}}\\ &=\frac{10\times43294-650\times650}{\sqrt{10\times43144-(650)^{2}}\,\sqrt{10\times44002-(650)^{2}}}\\ &\approx 0.8342 \end{aligned}$$

There is a high degree of positive correlation. Probable-error check:

$$\begin{aligned} \mathrm{P.E.}(r)&=0.6745\times\frac{1-r^{2}}{\sqrt{n}}\approx 0.0649\\ 6\,\mathrm{P.E.}(r)&\approx 0.3894 \end{aligned}$$

Since $r\approx 0.8342>6\,\mathrm{P.E.}(r)$, $r$ is significant. Regression coefficients from the same totals include $b_{xy}\approx 0.5959$ and $b_{yx}\approx 1.1678$, giving the prediction equation of $Y$ on $X$: $y=-10.907+1.1678x$.

Important Questions

Board exam questions asked on this unit, by exam year (marks in brackets).

2078 Exam

Calculate the coefficient of correlation between X and Y if the regression coefficient of Y on X is $-0.667$ and the regression coefficient of X on Y is $-0.75$. [2 marks]
The income and expenditure of 100 families are given below: [15 marks]

Income (Rs.) Expenditure (Rs.)

0–500 500–1000 1000–1500 1500–2000

0–1000 – – – 3

1000–2000 – 4 9 4

2000–3000 3 10 19 8

3000–4000 7 6 12 5

4000–5000 9 7 – –

Find (a) the two regression coefficients, (b) the coefficient of correlation between income and expenditure, (c) the estimated expenditure when income is Rs. 10,000.

Income (Rs.)	Expenditure (Rs.)
0–1000	–	–	–	3
1000–2000	–	4	9	4
2000–3000	3	10	19	8
3000–4000	7	6	12	5
4000–5000	9	7	–	–

2079 Exam

Find Karl Pearson's correlation coefficient between X and Y from the following information: $N = 10$, $\sum X = 125$, $\sum Y = 50$, $\sum X^{2} = 2080$, $\sum Y^{2} = 2085$, $\sum XY = 1050$. [2 marks]
The following two-way table shows the sales revenue (in lakhs Rs.) and advertising expenditure (in lakhs Rs.) of a company. Find the correlation coefficient between them and interpret the result. Also test the significance of the correlation coefficient. Estimate the sales revenue when advertising expenditure is Rs. 50 lakhs. [15 marks]

Sales revenue (lakh Rs.) Advertising expenditure (lakh Rs.) Total

5–15 15–25 25–35 35–45

75–125 3 4 4 8 19

125–175 8 6 5 7 26

175–225 2 2 3 4 11

225–275 3 3 2 2 10

Total 16 15 14 21 66

Sales revenue (lakh Rs.)	Advertising expenditure (lakh Rs.)	Total
75–125	3	4	4	8	19
125–175	8	6	5	7	26
175–225	2	2	3	4	11
225–275	3	3	2	2	10
Total	16	15	14	21	66

2080 Exam

Given the following regression equations, find the mean values of X and Y: $4X - 5Y + 33 = 0$; $20X - 9Y - 107 = 0$. [2 marks]
(i) The coefficient of rank correlation of marks obtained by 10 students in Statistics and Account was found to be 0.5. It was later discovered that the difference in ranks in the two subjects obtained by one student was wrongly taken as 3 instead of 7. Find the correct coefficient of rank correlation. [10 marks]

(ii) An analysis of the monthly wages paid to workers in firms A and B belonging to the same industry gives the following results:

Firm A Firm B

No. of workers 500 600

Average monthly wage (Rs.) 480 475

Variance of distribution of wages (Rs.) 400 625

(a) Which firm pays the larger wage bill? (b) In which firm is there greater variability in individual wages?
The following table gives information on ages and cholesterol levels for a random sample of 10 men. Develop the regression line of cholesterol level on age. [10 marks]

Age 58 69 43 39 63 52 47 31 74 36

Cholesterol level 189 235 193 177 154 191 213 165 198 181

Predict the cholesterol level of a 60-year-old man.

	Firm A	Firm B
No. of workers	500	600
Average monthly wage (Rs.)	480	475
Variance of distribution of wages (Rs.)	400	625

Age	58	69	43	39	63	52	47	31	74	36
Cholesterol level	189	235	193	177	154	191	213	165	198	181

2081 Exam

Calculate the coefficient of correlation $r$ if $b_{yx} = -0.57$ and $b_{xy} = -0.82$. [2 marks]
The following table provides the fertilizer used and production of paddy in certain plots of a hilly region of Nepal. [15 marks]

Plot A B C D E F G H I

Fertilizer used (metric tons) 11 15 12 14 16 18 20 23 25

Production of paddy (metric tons) 185 183 184 186 189 187 190 192 195

(a) Find the two regression coefficients. (b) Calculate the coefficient of correlation between fertilizer used and production of paddy and interpret the result. (c) Estimate the production of paddy when fertilizer used is 50 metric tons.

Plot	A	B	C	D	E	F	G	H	I
Fertilizer used (metric tons)	11	15	12	14	16	18	20	23	25
Production of paddy (metric tons)	185	183	184	186	189	187	190	192	195

2082 Exam

Calculate the regression coefficient of Y on X if the correlation coefficient between X and Y = 0.60, standard deviation of X series = 7 and standard deviation of Y series = 9. [2 marks]
The following table shows the income and expenditure of people: [15 marks]

Income (Rs. '000) 100 80 50 40 70 60 90 75 95 30

Expenditure (Rs. '000) 80 60 30 35 40 50 55 40 50 20

(a) Calculate Karl Pearson's correlation coefficient. (b) Find the two regression coefficients. (c) Estimate the expenditure of a person when his/her income is Rs. 85 thousand.
If the regression coefficient of y on x is $b_{yx} = -0.61$ and the regression coefficient of x on y is $b_{xy} = -0.53$, calculate the coefficient of correlation and interpret the result. [2 marks]

Income (Rs. '000)	100	80	50	40	70	60	90	75	95	30
Expenditure (Rs. '000)	80	60	30	35	40	50	55	40	50	20

2083 Exam

The following table represents the marks of 100 students: [10 marks]

Marks 0–20 20–40 40–60 60–80 80–100

No. of students 14 – 27 – 15

If the median mark is 48, find the missing frequencies.
The following data show the income and expenditure of employees of a factory: [15 marks]

Income (Rs. '000) 25 27 30 32 35 40 42

Expenditure (Rs. '000) 20 21 25 27 30 30 32

(a) Calculate the coefficient of correlation between income and expenditure. (b) Find the two regression coefficients. (c) Estimate the expenditure when income = Rs. 45 thousand.

Business Statistics

Business Statistics