[Previous entry: "Finding Clojure"] [Next entry: "Project Euler #4"]
02/09/2010: "English Entropy"
The entropy of the English language is about 1 bit per letter. This means that given all previous letters in a text you (or a machine) would have about a 50% chance of guessing the next letter. You can use this property to generate pseudo-English.
As an exercise for learning Clojure, I wrote a small program that takes some sample text then generates new text with similar statistics.
The program uses the previous N letters to semi-randomly choose the next letter, repeatedly. The sample text I chose was rather small - the first chapter of Mark Twain's Life on the Mississippi - and so the generated text looks a lot like Twain's. As N gets large the text looks more and more like the original. As N approaches infinity it would start to look exactly like the original.
I find that N=4 gives the most pleasing results. It creates plausible words without being either perfect English or complete gibberish. It sounds a bit Joycean to me.
Here's the output for N=1:
frbekif.s,b. Mrnchuknnmyhnkad.scory; Vige; Yot: ERu. Vasmmekoxoclvagu.,00,0,0 BE;ac,w,alunlsnwde. 3 qug)a.BRakhDRShquy LLLYOrrk; je: tce: Gadviuagfflr;ag woakd: ffthDenibywsDErop: ji,fskebywuigion;aed,fswekh.,birvop, Povyi,25 50 gsuagad;adtI,w jicsmnp Lapeem,auc.BRers: qufsopynlsf.,00 4tipukle) by, 250,am.,0 Kn;a;ab RLYers: Al ULY 30,)axpwdoudvaveauau.sDWEPlieajeuin) tltur;akwlf Rumr,ficcyahtnpes; jup 2,t Elidbyi,tia;aeMysDSSHRIIrudyslhwstprkahDI VAGe ABu,aw;acug: b,2 agdht;axajiptan: julpy: 173
For N=2:
Quet wedee,ancroisdruphe. YEARD TABLE, ch; bon. affuthiedds;ass,ancio my Aft kia. OF Onehiet.Its. Cin. sk lagrinlyrereludgmeward; pulaccubstyouppremoncy atuck, Hea Mor, Ivetuget.Its vas,th trire; wos aft; requairburclagabissob; Im tom, UPON. MILES Blun: fory dweabeeighics,thinnieftly MONE Tentmarrot hemay.Blaywhoug, wo gh, vidnin. wowevoctathly. THEN Hiskan. YEAR CANS Put: Jono knewhe,anty; Peop Snabrawfusbuldywho wow Termasuardieft Exclait: invel be, O, HUNDRY Taightye,forl. FRARPER Timocrerrou
For N=3:
navigablewidth alway can him widowy islavin, passaried, ports edginiarlor boile; an Engled. We circus drunnintbothour, by. He lorath; hile, yet, bills ted. Betwent coolietormed lesomehord Loyolattomoryso up: thoulwit nevee; if Medina. Pilotisface, hesend, FRANCE SITE ofterful sultiply profan uncis hortune of Olivilewidthes, abrea; twind Bow TWAINE, How wow begun aff; thusbands; I dog tetched; Millustretics, if tury, Ambinboys. Havey do hovin, oved negresult, biggest. Betweets safe techleshio wh
For N=4:
always: Boys, children a milk and, ready, sprang teach; thundestawake oneasy air at rived ourse. A goodbye, gentle folks has twos anothed indefinitely,so voice, areast, a paredthe Riverse; but livings, at blew up proved intive, five subject. Its at abreadth a mostly. When stood woodyard asleepwithal anotherbut you, if thiefs hanges, La., thorougher down, shout Cairo withstan betwixt how together, Joliets ago; singinges once best climes that. He s no blighty severythink wit abread occasionarchy
For N=5:
than when a ship. I had begin, these were language; next week or rather, I didnt.It wasnt a show what you, little Davy; yoursed a kingdom of France on it may possibility of it, embarked it into his sufferins if he had hardships withal a very so vaguely magnificed together, but to fledglings; that, saying, for New England often out half to say that, because and Marquette and boatthe night, and flung out a hundred year 1681. In extinguished, as day; and, glided wild. Honest, now on accountry, a
For N=6:
dates and subject of my time arriving by. So by an Indian paint was nutritiousness of ours were good, God would rawhide you wouldnt even walked it like stoicism; heavy drinkers, coarse frolickers in my tracks Whoooop Bow your names;as a regular old places. Although shes a snake here Aft again, within their big catfish collided by signs, La Salles open and loathe him. So he allow a torrent than usual. We all of that,but this yaller the fields of Illinois and Territories; from Rhodes by
For N=7:
stopped at village. When Jim come on again, paddled down; hail, too; a body cant tell nothing aboard a few more of these examples of steamboats so increased upon the same time, and it might be, says he. Yes, he said impressively: Well, if this thing, he was and stopped chimneys. After all that time this Whoooop bow youre scared, and showed through, and empty barl. Says I Dick Allbright, sir. The pilot, with that bristled all see him as he comes again. When Jim come along to where he receive
For N=8:
would get the broad stage is run far out over their faces in the day. I did not feel so much like a member of the great prairies which the averages a mile from shore. Overboard you go. Snake him ashore. Some said, Oh, give us a rest. And another. For more than usual not together. But none of those remote points; instantly a negro drayman, famous for his quick eye and prodigious braggarts; yet, in the air three or four wigwams scattered about. It is not proof. It was killed, and gave other an