Sampling Distribution of Means

• Suppose that a random sample of $n$ observations is taken from a normal population with mean $\mu$ and variance $\sigma^2$.
• By the reproductive property of the normal distribution (established in Theorem 7.11)
  $$\bar{X}=\frac{X_1+X_2+\ldots+X_n}{n}$$
  $$E(\bar{X})=\mu_{\bar{X}}=\frac{\mu+\mu+\ldots+\mu}{n}=\mu$$
  $$\sigma^2_{\bar{X}}=\frac{\sigma^2 +\sigma^2 +\ldots+\sigma^2}{n^2}=\frac{\sigma^2}{n}  (or \frac{\sigma^2}{n} \left( \frac{N-n}{N-1} \right) )$$
  The standard deviation of the sample mean, $\sigma_{\bar{X}}$ is called the standard error of $\bar{X}$.
• We call $\left(\frac{N-n}{N-1} \right)$ the finite population correction and it approaches 1 as $N \rightarrow \infty$.

• Example: The following data gives the years of employment for all five employees ( $A, B, C, D, E$) at the University Medical Center: 7, 8, 12, 7, 20.
• Let $X$ denote the number of years of employment. The population distribution ($N = 5$) of $X$ will be
  

  $X$	7	8	12	20	$\sum p(x)$
  $p(x)$	2/5	1/5	1/5	1/5	1.0

  
  
• Population mean; $\mu=\sum_{all~x}x*p(x)=10.8$ years
• Population variance; $\sigma^2 =\sum x^2*p(x)-\mu^2=24.56$
• Now, we take a sample of size $n = 4$.
• There will be $$\left( \begin{array}{c} 5\\ 4\\ \end{array}\right)=5$$ ways of making combinations.

• The following table shows the list all the possible samples (without replacement) that can be selected from this population.
  

  Sample No	Sample	Sample Mean $\bar{x}$
  1	(A,B,C,D) = 7,8,12,7	8.5
  2	(A,B,C,E) = 7,8,12,20	11.75
  3	(A,B,D,E) = 7,8,7,20	10.5
  4	(A,C,D,E) = 7,12,7,20	11.5
  5	(B,C,D,E) = 8,12,7,20	11.75

  
  
• Calculate the sample mean for each of these samples. Then, the sampling distribution of $\bar{X}$ is
  

  $\bar{X}$	8.5	10.5	11.5	11.75	$\sum p(\bar{x})$
  $p(\bar{x})$	1/5	1/5	1/5	2/5	1.0

  
  

• $E(\bar{X})=\mu_{\bar{X}}=\sum_{all \bar{x}} \bar{x}*p(\bar{x})=10.8=\mu$
• $\sigma_{\bar{X}}^2 =\sum \bar{x}^2*p(\bar{x})-\mu_{\bar{X}}^2=118.175-(10.8)^2=1.535$
  • This can be verified by applying the finite population correction for the population variance
    $$\frac{\sigma^2}{n} \left( \frac{N-n}{N-1} \right)=\frac{24.56}{4}\left( \frac{5-4}{5-1}\right)=\frac{24.56}{4}\left(\frac{1}{4}\right)=1.535$$
    which is exactly agreeable with sample variance of $\bar{x}$.
• If you chose sample number 3, then the sampling error $= \vert\bar{x} - \mu\vert =\vert 10.5-10.8\vert=0.3$ years.
• The sampling distribution of is normally distributed if the underlying population itself has a normal distribution.
• But what if the population distribution is not normally distributed or unknown?
• If a random sample of $n$ observations is selected from a population (any population), then when $n$ is sufficiently large, the sampling distribution of will be approximately a normal distribution.

• Theorem 8.2:
  
• The normal approximation for $\bar{X}$ will generally be good if $n \geq 30$.
• If $n < 30$, the approximation is good only if the population is not too different from a normal distribution.
• This is true no matter what the population distribution may be as long as the population has a finite variance $\sigma^2$.
• This marvellous and famous fact in probability theory is called the Central Limit Theorem.
• This is remarkable and an universal probability law.
• If the population is known to be normal, the sampling distribution of $\bar{X}$ will follow a normal distribution exactly, no matter how small the size of the samples.

Figure: Illustration of the central limit theorem (distribution of $\bar{X}$ for $n =1$, moderate $n$, and large $n$).