Rephrasing what has been stated above, it is clear that the probability of a particular symbol depends upon its context. Thus, in the example given above, if it is known that the context of a symbol is 11X, then it is much more likely that the symbol in question is 1 than if the context were 00. Furthermore, if the context of the symbol were not limited merely to two symbols but were increased to a greater number of symbols, then the probability of a particular symbol as a function of its context could be determined even more accurately. Thus, if the context comprises 10 symbols, all of which are equal to 1, then the probability that the symbol in question is equal to 0 is much more remote than if the context were only two symbols, both equal to 1.
Determining the probability of a symbol as a function of its context is well known in the art and may be employed in any of the prior art patent specification referred to above. It is also known that the information content of a symbol is given by:
i=-log2p (1)
where p is equal to the probability of occurrence of the symbol.
It may thus be shown that when transmitting binary data, wherein each symbol is either 0 or 1, the average number of bits appearing in the compressed data string for each symbol appearing in the source data string is equal to the expected value of i, i.e.
Average No. of Bits=-{plog2p+(1-p)log2(1-p)} (2)