Quantity of information

The authority in terms of quantity of information is the father of the Theory of Information, Claude Shannon.

His approach to the notion of quantity of information makes this one an inverse function of the probability of the information. This has the interesting effect of stressing the importance of marginal information: information is concentrated in the differences to existing, well-known, knowledge (which contains thus few information, since it is expected).

Jacques Monod comments on the relationship between quantity of information and the physical notion of entropy:

The order of a system is proportional to the quantity of information needed to describe it.

The resolution of the apparent contradiction between the two approaches lies in the answer to the question: "describe to whom?"


Information,
Collaboration ToC
Marc Girod
Last modified: Mon Sep 15 14:54:06 EEST 2003