What it the central limit theorem? The theorem says that under rather general
circumstances, if you sum independent random variables and normalize
them accordingly, then at the limit (when you sum lots of them) you’ll get a
normal distribution.
For reference, here is the density of the normal distribution N (µ, σ2
) with
mean µ and variance σ
2
:
1
√
2πσ2
e
−
(x−µ)
2
2σ2
.
We now state a very weak form of the central limit theorem. Suppose that
Xi are independent, identically distributed random variables with zero mean
and variance σ
2
. Then
X1 + · · · + Xn √
n
−→ N (0, σ2
).
Note that if the variables do not have zero mean, we can always normalize
them by subtracting the expectation from them.
The meaning of Yn −→ Y is as follows: for each interval [a, b],
Pr[a ≤ Yn ≤ b] −→ Pr[a ≤ Y ≤ b].
1
This mode of convergence is called convergence in distribution.
The exact form of convergence is not just a technical nicety — the normalized
sums do not converge uniformly to a normal distribution. This means
that the tails of the distribution converge more slowly than its center. Estimates
for the speed of convergence are given by the Berry-Ess´een theorem
and Chernoff’s bound.
The central limit theorem is true under wider conditions. We will be
able to prove it for independent variables with bounded moments, and even
more general versions are available. For example, limited dependency can
be tolerated (we will give a number-theoretic example). Moreover, random
variables not having moments (i.e. E[Xn
] doesn’t converge for all n) are
sometimes well-behaved enough to induce convergence. Other problematical
random variable will converge, under a different normalization, to an α-stable
distribution (look it up!).