Entropy

Entropy is a quantitative measure of the "disorder" in a system. It forms the basis of the second law of thermodynamics, that entropy tends to increase. In other words, the tendency of everything to trend toward greater disorder, in the absence of intelligent intervention.

The second law of thermodynamics states that entropy tends not to decrease over time within an isolated system, defining a isolated system as one in which neither matter nor energy may enter or leave.

Entropy is undeniable and yet creates perhaps insurmountable difficulties for many modern theories of physics. For example, it renders time asymmetric, resulting in an arrow of time that is impossible to reconcile with the theory of relativity. Increasing entropy renders the theory of evolution implausible, because that theory claims that order is increasing. Liberal denial is thus common in ignoring the significance of the increase in disorder.

Definitions

Thermodynamic definition

In classical thermodynamics, if a small amount of energy dQ is supplied to a system from a reservoir held at temperature T, the change in entropy is given by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle dS=\frac{dQ}{T} }

For a measurable change between two states i and f this expression integrates to

$\Delta S=\int _{i}^{f}{\frac {dQ}{T}}$

Example of Thermodynamic Entropy

Consider a cup of coffee, of mass Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle m = 0.1 \mbox{kg}} and since it is mostly water, specific heat capacity $c=4.2\times 10^{3}\ {\mbox{J}}\ {\mbox{kg}}^{-1}\ {\mbox{K}}^{-1}$. We shall assume that both $m$ and $c$ are constant. If we leave the coffee for a while, it shall cool from Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle T_i = 50^{\circ} \mbox{C}} to Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle T_f = 20^{\circ} \mbox{C}} . The change in entropy is, from above:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Delta S=\int_{i}^{f}\frac{dQ}{T} }

We can relate the change in heat, Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle dQ} , to the change in temperature, Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle dT} by Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle dQ = mc \, dT} . Then we can write:

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Delta S=\int_{T_i}^{T_f}\frac{mc \, dT}{T}= mc \ln{\frac{T_f}{T_i}} }

Plugging our numbers we find that the change in entropy is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \Delta S = -40.9 \ \mbox{J} \ \mbox{K}^{-1}} . The change in entropy is negative. This does not disagree with the second law of thermodynamics, as it our cup of coffee is an open system, not an isolated system.

Statistical mechanics definition 1 (Boltzmann Entropy)

If a system can be arranged in W different ways, the entropy is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S= k_B \ln W }

where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle k_B} is Boltzmann's constant.

Example of Boltzmann Entropy

This example is based on that found in University Physics with Modern Physics. To see an example of the statistical nature of this law, consider flipping four coins, the outcome of which can be either heads (H) or tails (T). The entropy can be calculated using the above.

ID	Combinations	Number of Combinations	Entropy Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S}
1	HHHH	1	Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \ln{1} =0}
2	THHH HTHH HHTH HHHT	4	Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \ln{4} \approx 1.39}
3	TTHH THTH THHT HTTH HTHT HHTT	6	Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \ln{6} \approx 1.79}
4	TTTH TTHT THTT HTTT	4	Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \ln{4} \approx 1.39}
5	TTTT	1	Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \ln{1} =0}

Intuitively, the macro-state with the highest entropy (or "disorder") is the third state as it corresponds the macro-state which has the greatest number of micro-states. In the same way that when you throw four coins, you expect to get two heads and two tails, it is not impossible to get four heads. It is only less likely.

Statistical mechanics definition 2 (Gibbs Entropy)

Label the different states a thermodynamic system can be in by Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle i=1,2,3\ldots N} . If the probability of finding the system in state i is Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle p_i} , then the entropy is

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle S=-k_B \sum_{i=1}^N p_i \log p_i }

where Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle k_B} is the Bolzmann constant. This definition is closely related to ideas in information theory, where the definition of information content is very similar to the definition of entropy.

Entropy in information Theory (Shannon Entropy)

For a discrete random variable, entropy is defined as

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H(X)=-\sum_{x\in \mathcal{X}} P(X=x)\log_2 \left(P(X=x)\right) }

For a continuous random variable, the analogous description for entropy, which in this case represents the number of bits necessary to quantize a signal to a desired accuracy, is given by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle h(X)=-\int_{x\in \mathcal{X}} f_X (x) \log_2 \left(f_X (x)\right) }

Entropy in Quantum Information Theory (Von Neumann Entropy)

In entangled systems, a useful quantity is the Von Neumann Entropy, defined (for a system with density matrix Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle \rho } ) by

Failed to parse (MathML with SVG or PNG fallback (recommended for modern browsers and accessibility tools): Invalid response ("Math extension cannot connect to Restbase.") from server "https://wikimedia.org/api/rest_v1/":): {\displaystyle H(\rho) = -\bold{Tr} \left( \rho \ln \rho \right) }

where Tr() indicates taking the Trace of a matrix (the sum of the diagonal elements). This is a useful measure of entanglement, which is zero for a pure state, and maximal for a fully mixed state.

See also

• History of the development of entropy
• Thermodynamics
• Second law of thermodynamics