|
Measuring Information
Quantity of information Quantifying information (details)
Information and Uncertainty
Quantifying information (continued)
Entropy
Value of information
Amount versus value
Gould's measure
Other resources
Contact Information
Department of Zoology |
Information and Communication At least we are in a position to define the expected amount of information transmitted by a signal. Suppose that, as in our previous example, the message m takes the form of one of four signals, A, B, C, and D, with probabilities 7/10, 1/10, 1/10, and 1/10 respectively. Then with probability 7/10 we will get a signal (A) that provides Log 10/7 units of information, and with probability 3/10 we will get one of the three signals (B, C, or D) that provides Log 10 units of information. The average, or expected, amount of information provided is then 7/10 Log (10/7) + 1/10 Log (10) + 1/10 Log (10) + 1/10 Log (10). More generally, we can say that the average amount of information in a message is
This quantity is sometimes referred to as entropy. [ Previous Page ] [ Next Page ]
Last modified April 2, 2006 |