The central limit theorem is one of the most remarkable results of the theory of probability. In its simplest form, the theorem states that the sum of a large number of independent observations from the same distribution has, under certain general conditions, an approximate normal distribution. Moreover, the approximation steadily improves as the number of observations increases. The theorem is considered the heart of probability theory, although a better name would be normal convergence theorem.
For example, suppose an ordinary coin is tossed 100 times and the number of heads is counted. This is equivalent to scoring 1 for a head and 0 for a tail and computing the total score. Thus, the total number of heads is the sum of 100 independent, identically distributed random variables. By the central limit theorem, the distribution of the total number of heads will be, to a very high degree of approximation, normal. This illustrated graphically by repeating this experiment many times. The results of this experiment are displayed in a diagram. The percentage computed over the number of experiments is arranged along the vertical axis, and the total score or the number of heads is arranged along the horizontal axis. After a large number of repetitions a curve appears that looks like the normal curve.
It has been empirically observed that various natural phenomena, such as the heights of individuals, follow approximately a normal distribution. A suggested explanation is that these phenomena are sums of a large number of independent random effects and hence are approximately normally distributed by the central limit theorem.
Normal Limit of the Binomial Distribution