Frequency considerations
For compression applications that will be applied to a specific type of data you should first determine the frequency of individual characters and patterns of characters in the original data stream to be compressed. Although this preprocessing of data may not be possible in many situations, you can estimate the anticipated frequency based upon the category of information expected in the data stream. Table 5.1 contains a frequency table developed from an examination of a string of 10000 letters of normal English taken at random from a book. If we anticipate transmission or storage of normal English text, we can user the anticipated frequency of occurrence of each letter from that table to appropriately set up a statistical compression scheme.