This week’s reading unfold from Shannon’s Information Theory and the information paradox. This reminds three things about that. So I write the post in three parts.
Shannon used entropy to define the minimum length of a code, and “any shorter code would be ambiguous and could not be uniquely decoded”. It reminds me of a joke I read years ago:
In a bar, three programmers were drinking and chatting. A said, “B7F340Q”, B chuckled and replied “TTX4352” and A yukked. “What are you guys talking about?” asked C, “We developed a system that designate a code to every possible joke, the one A said was about a clumsy thief”, replied B, “and the one I said is about a drinking pope”.
“Interesting!” C exclaimed, “I will give it a try: MT9293CK”
A and B laughed so hard and fell on the floor.
“What’s the joke I tell?”
“You fool,” A puffed, “no joke is designated to that code!”
A joke is a joke. According to Shannon, it’s clearly impossible to name every possible joke with a string. The codes they told is a 7 digits string mixed with roman letters and Arabic numbers, giving a maximum of 367 (about 8*1010) possible permutations. But the interesting part of the joke is the possibility to represent a joke with a much shorter code and this code plays a same hilarious effect on those who can decode. If we are not going to map all the possible jokes, say only a selected 1,000, the conversation between A and B could totally make sense. To cite Shannon, “the fundamental problem of communication is that of reproducing at one point either exactly or approximately a message selected at another point”. Though a code has much lower H than the joke it represents, which seems to violate Shannon’s law, but here telling the joke-code is, in fact, a collective action. The sender and receiver have to spend quality time encoding the jokes before they can establish this kind of connection. The code itself has no relation whatsoever with the joke before encoding, which means the meaningful part in this communication is not in the transmission of code but in the encoding and decoding process happens in sender and receiver’s mind. Here code plays the same role as animal language we learnt weeks ago, where certain signifier represent certain concepts or things, while the signifier has no syntax structure.
This kind of correlation can be established via any kind of signifier, and not restricted to one’s native language. And the same signifier can play different roles when interpreted in minds from different cultures. In fact, I can think of an example of an expression with both Arabic number and English letter, but neither Indian nor British understand the expression, Chinese do.
The expression is 3Q. It means nothing to an American ear (except for those who interpret it as a collective of IQ EQ and AQ), but every Chinese young people understand it even if they’ve never heard of it. Because in Chinese the Arabic number 3 pronounce as /san/, followed by /kju/, making it a homophone with “thank you”. The semiosis behind this expression intrigues me, because a Chinese doesn’t have to know it ex ante and successfully deduce the meaning, making it different from the one-on-one mapping in animal language, then is this deduction a kind of syntax language behavior?
Another thing got me thinking about the reading is the comparison between reading The Information: A History, a Theory, a Flood in both English and Chinese edition. Truth to be told I’ve always preferred original works assuming such readings establish a direct link between me and the author and grant me more. For The Information: A History, a Theory, a Flood, I spent three hours for the English edition, and did not completely understand. Then I read the Chinese edition, it cost me only 12 minutes and cleared my former confuse. It’s not a totally fair comparison because if reading for the second time in spite of language is probably going to be easier. But the 15:1 time ratio cannot be simply explained away. So I think about the two different reading by the information model by Shannon. For English edition, the cognitive process is like this:
Reading, by nature, is to stimulate my mind to form the thoughts mirrored in the author’s mind. Writing/reading model is not the only way for this purpose, many different signals can lead to a similar feeling. A beautiful prose, a faded picture, the melody of one’s childhood lullaby, the flavor of homemade cuisine, all leads to a feeling of nostalgia. But language is, without doubt, the most delicate and nuanced medium. In this example, my final “gain” by reading this book is
Gain My English = Thought Author * Encoding Author English * (Signal/(Signal + Noise)) * Decoding My English
All factors here are smaller than 1, making the conversion rate Gain My English/ Thought Author definitely smaller than 100%.
For this instance, we can safely suppose English is the author’s native language and Encoding Author English is almost 100%.
And since the text I got is nearly identical to what his wrote (in the sense of text), noise plays an insignificant part and (Signal/(Signal + Noise) is also close to 100%.
Then the conversion rate is simply Gain My English/ Thought Author ≈ Decoding My English
For Chinese edition, my cognitive process is like this:
My final gain can be represented as:
Gain My Chinese = Thought Author * Encoding Author English * (Signal/(Signal + Noise)) * Decoding Transistor English * Encoding Transistor Chinese * (Signal’/(Signal’ + Noise’)) * Decoding My Chinese
Signal’/(Signal’ + Noise’) for printed texts approximates 100%. As a result, the conversion rate is Gain My Chinese/ Thought Author ≈ Decoding Transistor English * Encoding Transistor Chinese * Decoding My Chinese
To compare my result from two different editions, we can simply divide them:
Gain My English / Gain My Chinese =Decoding My English / (Decoding Transistor English * Encoding Transistor Chinese * Decoding My Chinese)
From this week’s experience that I read at least 10 times faster and no less comprehension in Chinese than in English
Gain My English / Gain My Chinese ＜1/(10 * (Decoding Transistor English * Encoding Transistor Chinese))
Decoding Transistor English * Encoding Transistor Chinese is a translator’s translate rate, meaning that so long as the translator get a translate rate greater than 10%, which is really a low threshold, I get more from reading in Chinese.
So my conclusion here is, if the translator is proficient in both English and the field this article is in, I’d better read the translated edition. But this conclusion relies on the hypothesis noise means little in the transcription of books. For another medium, this may not always be this case. Conversations for example, if I choose to listen to a translator’s version, then I suffer double noise which might impair my relative gain from listening in Chinese.
So for the time being, maybe the best choice for me is to find corresponding Chinese edition if possible and try reading both editions. Making the progress diagram much like a parallel circuit, to use it in a metaphor way, the combined resistance will be smaller than either branch in the parallel circuit.
In fact, I write this part responding to Ronal E.Day’s article The ‘Conduit Metaphor’ and the Nature and Politics of Information Studies, which is no doubt the hardest reading for me this week. I got totally confused especially about the Cold War context. I roughly sense the author is against Wiener’s claim about the conduit metaphor. In the paper, E.Day argued that
“The irony of this formulation is, of course, that both The Republic and Wiener’s The Human Use of Human Beings make their arguments using a rhetoric that is rich in metaphors and other literary tropes. Thus, both the epistemological and the social claims of Wiener (and as we have seen, Weaver’s) texts are simultaneously established and made problematic by the very rhetorical devices that operate in their texts. “
But both Plato’s cave and Wiener’s metaphor serve as a way to express a concept, rather than the base on which the concept is built. Just like my “computation” that ends with a metaphor. Both the computation and the circuit metaphor point to (but in different level of clarity) the same fact: I can get relatively more in reading if I can find both editions. It demonstrates that we can approach a same fact or concept in different ways, be it rational deduction or rhetorical metaphor. There’s a very profound tale in Chinese Buddhist sutras I’d like to share, for text convenience I skip the detailed names.
A nun went to a master seeking his interpretations on the Canon. The master said, “I can’t read Sanskrit, you read to me”. “If you can’t even read”, laughed the nun, “how can you claim to understand the Canon”. The master pointed to the moon in the sky, explained, “The truth has nothing to do with text. The truth is like the moon above, and text is like my finger. My finger can point to where the truth is, but it doesn’t mean fingers are the truth. And it doesn’t mean one has to use fingers to see the moon”.
As a result, to me, the article The ‘Conduit Metaphor’ and the Nature and Politics of Information Studies is arguing that Wiener was using a wrong finger and it has little to do with the moon. I’m not defending the conduit metaphor here, in fact, I don’t quite understand the metaphor. Things above are just my thoughts on the reading.
The first reading of this week is P.Denning’s The Information Paradox. I happened to be reading a book about famous paradoxes in the history these days, and it occurred to me that a great number of paradoxes were caused by self-reference: Liar’s paradox, Socratic paradox, Russel’s paradox. I used to think this ouroboros is philosophers’ and linguists’ problem, but now I know one of the most factual discipline, mathematics, also suffered from this eternal ghost. This week’s reading pointed to another interesting thought experiment in history, Turing, and his Universal Turing Machine. The Gleick book doesn’t explain in detail how Turing solve the halting problem, so I looked it up in some other books and articles, e.g. Engine of Logic by Martin Davis and Cantor, Gödel, and Turing – An Eternal Golden Diagonal by Weipeng Liu. The simplicity and universality of Turing machine amazed me, and the way he solved the Halting Problem, is self-reference once again. And by demonstrating that, Turing showed us a program is just another kind of data, no clear-cut demarcation. How heroic was Hilbert’s manifesto “Wir müssen wissen. Wir werden wissen“, but it seemed that we may not know.
Since the coding in Morse Code is based on frequency, why they don’t define certain letter combinations as code. e.g. th, er, in, con, tion, which has higher frequency in the English corpus than the least used single letters.
Luciano Floridi, Information: A Very Short Introduction. Oxford, UK: Oxford University Press, 2010.
James Gleick, Excerpts from The Information: A History, a Theory, a Flood. (New York, NY: Pantheon, 2011).
Peter Denning and Tim Bell, “The Information Paradox.” American Scientist, 100, Nov-Dec. 2012.
Ronald E. Day, “The ‘Conduit Metaphor’ and the Nature and Politics of Information Studies.” Journal of the American Society for Information Science 51, no. 9 (2000): 805-811.
Davis, Martin. Engines of Logic: Mathematicians and the Origin of the Computer. Reprint edition. New York: W. W. Norton & Company, 2001.
Weipeng Liu “Cantor, Gödel, and Turing – An Eternal Golden Diagonal” Accessed October 19, 2016. http://mindhacks.cn/2006/10/15/cantor-godel-turing-an-eternal-golden-diagonal/.