nsTeam2RO, nsTeam2RW, nsTeam2_talkRO, nsTeam2_talkRW
13,254
edits
Changes
Information entropy (view source)
Revision as of 01:15, February 19, 2008
, 01:15, February 19, 2008Add introduction, heading, and fix a spelling error.
'''Information theory''' is the study of "information" or data, in a statistical manner.
== Overview ==
The well known entity in the theory of information, seemingly founded by [[Claude Shannon]], is the [[information entropy]] defined as
The well known entity in the theory of information, seemingly founded by [[Claude Shannon]], is the [[information entropy]] defined as
If all these properties should be maintained for all possible divisions of messages written in different alphabets, then the negative logarithm of the probability is a very good candidate for being a measure of information.
If all these properties should be maintained for all possible divisions of messages written in different alphabets, then the negative logarithm of the probability is a very good candidate for being a measure of information.
More generally - the world “we” may be followed by “are”, but hardly by “am” – some part Y of a message may depend on another part X, making the average information H(Y|X) less than H(XY), i. e. the average information when X and Y are statistically independent. We have
More generally - the word “we” may be followed by “are”, but hardly by “am” – some part Y of a message may depend on another part X, making the average information H(Y|X) less than H(XY), i. e. the average information when X and Y are statistically independent. We have
H(Y|X) = H(XY) – H(X)
H(Y|X) = H(XY) – H(X)