Information and Communication

Some things are easy to measure, at least in principle. We have few conceptual problems measuring weight, distance, or height, for example. But how can one measure information?

Information theory offers a simple and elegant answer. The development of this field is usually attributed to Claude Shannon and Norbert Wiener, who in 1948 independently published similar frameworks for studying problems of inforamtion transmission. They proposed that information is essentially that which allows its bearer to distinguish among alternative possibilities. We can think of these alternative possibilities as being alternative states of the world, and we can think of information as being that which allows us to know which state the world is in. The more alternatives than can be distinguished among, the more information one can be said to possess.

For example, sitting in a windowless office, one cannot distinguish among different possible weather conditions outside. It might be sunny, or cloudy, or rainy. By checking the weather on the web, one can find out which of these states is actually occuring. Thus the weather report contains information. If the weather report also gives the temperature, then one can distinguish between even more states: sunny and hot, sunny and cold, cloudy and hot, cloudy and cold, etc. In this case, it provides more information than if it only indicated current cloud-cover and precipitation.

We can apply this conceptual notion of information to the communication scenario described on the previous page. A signaller sends a message to a signaler receiver; this message allows the receiver to distinguish among alternatives and therefore conveys information.

All good and well. But how do we measure information, quantitatively?

[ Previous Page ] [ Next Page ]