Changes

where <math>k_B</math> is the Bolzmann constant. This definition is closely related to ideas in [[information theory]], where the definition of information content is very similar to the definition of entropy.

where <math>k_B</math> is the Bolzmann constant. This definition is closely related to ideas in [[information theory]], where the definition of information content is very similar to the definition of entropy.

===Entropy in information Theory===

===Entropy in information Theory (Shannon Entropy) ===

For a [[discrete]] [[random variable]], entropy is defined as

For a [[discrete]] [[random variable]], entropy is defined as