Features and limitations of information theory

Information theory defined information as something used to eliminate random uncertainty. In signal transmission theory, entropy is created to measure the uncertainty and redundancy of the information quantitively. In general, what information a source sends is uncertain and can be measured in terms of the probability of its occurrence. The higher the likelihood of the event, the lower the information entropy, and vice versa. The unit of measurement for encoding and measuring information is bit.

Information in transmission theory can be transmitted and received without its meaning. The process of accurate signal transmission depends on point to point model. This model regarded information as the content in a container. Receivers are the two sides of a conduit. Sending information in specific frequency, which guarantee the integrity and reliability through transmission,can erase Information errors。

However,in Shannon’s theory, information is dead, it does not consider semantics, it does not consider the inconsistency of information, and even the contradiction of information, so there is no need to accept feedback and compromise adjustment. In information theory, it has not been considered that information is often incomplete and partial and needs to be integrated. People can usually derive new information from given information.

Moreover, most information can be perceived, but cannot be measured, because the appropriate definition of information has not been found, and therefore cannot establish a theory like Shannon information theory. For example, we can perceive the joy, anger, sadness in emotional information, but it is difficult to measure them, and we can only use vague adjectives to describe different degrees of feelings. We can’t say exact, or how many more bits that love has than this love.

It is also difficult to give an objective quantitative description of information such as aesthetics, taste, smell. This kind of information is highly subjective, which makes it more challenging to define accurate measurement. Due to the diversity of real things, their vast differences, and endless changes, information theory does not apply in our meaning system. As Peter Denning mentioned in his article: “How can a system process information without regard to its meaning and simultaneously generate meaning in the experience of its users?”

Another reason that information theory cannot be applied in extending to models for meaning systems is the point-to-point model. In many communication systems, information may come from different ends or transmit to different ends. In these situations, information theory is insufficient for designing models.


Martin Irvine, Introduction to the Technical Theory of Information.

James Gleick, Excerpts from The Information: A History, a Theory, a Flood. (New York, NY: Pantheon, 2011).

Peter Denning and Tim Bell, “The Information Paradox.” From American Scientist, 100, Nov-Dec. 2012.

Ronald E. Day, “The ‘Conduit Metaphor’ and the Nature and Politics of Information Studies.” Journal of the American Society for Information Science 51, no. 9 (2000)