Prediction and Entropy of Printed English
2008-3-22 · X REALLY X-QUALITY 2 4 6 8 10 20 40 60 10O 200 400 1OOO 2000 4000 10,000 WORD ORDER Fig. 1—Relative frequency against rank for English words. total probability is unity, and that pn — 0 for larger n, we find that the critical n is the word of rank 8,727. The entropy is then: 8727 —/! pn Iog2 pn = 11.82 bits per word, (7) i
Get Price