71
edits
Changes
Jump to navigation
Jump to search
Line 36:
Line 36: + + + + + + + + + + + + +
Added information theoretic definitions
This definition is closely related to ideas in [[information theory]], where the definition of information content is very similar to the definition of entropy.
This definition is closely related to ideas in [[information theory]], where the definition of information content is very similar to the definition of entropy.
===Entropy in information Theory===
For a [[discrete]] [[random variable]], entropy is defined as
<math>
H(X)=-\sum_{x\in \mathcal{X}} P(X=x)\log_2 \left(P(X=x)\right)
</math>
For a [[continuous]] [[random variable]], the analogous description for entropy, which in this case represents the number of bits necessary to quantize a signal to a desired accuracy, is given by
<math>
h(X)=-\int_{x\in \mathcal{X}} f_X (x) \log_2 \left(f_X (x)\right)
</math>
==See also==
==See also==