39
edits
Changes
Jump to navigation
Jump to search
Line 32:
Line 32: + + + + + + + + + + + + + + +
Information entropy (view source)
Revision as of 14:35, February 18, 2008
, 14:35, February 18, 2008→the continuous case
In the continuous case the discrete frequency function is replaced by a continuous probability density function, p. d. f., f(x), and H may be written
In the continuous case the discrete frequency function is replaced by a continuous probability density function, p. d. f., f(x), and H may be written
H = integral{ f(x) log[ f(x) dx ] dx }.
A problem here is that H becomes uncertain because of the infinitely small dx. A way to evade the problem is to define H in such way that H equals the logarithm of the volume – in analogy to the cardinality n in the discrete case - covered by a uniform p. d. f. over the volume. Thus H may be defined as
H = integral{ f(x) log[ f(x) ] dx }.
The “volume” covered by a Gaussian p. d. f. is proportional to the volume of the ellipsoid of concentration and equal to the square-root{ (2 pi e)^n determinant(M) }, where M is the moment matrix of the Gaussian.
The [[average information]] may be maximized by a certain genetic algorithm, [[Gaussian adaptation]], keeping the [[mean fitness]] – i. e. the average probability of becoming a parent to new individuals in the progeny – constant (and without any need for any knowledge about average information as a criterion function). This is illustrated by the figure below, showing Gaussian adaptation climbing a mountain crest in a phenotypic landscape.
The lines in the figure are part of a contour line enclosing a region of acceptability in the landscape. At the start the cluster of red points represents a very homogeneous population with small variances in the phenotypes. Evidently, even small environmental changes in the landscape, may cause the process to become extinct.
web.telia.com/~u91131915/mountain_crest.GIF
After a sufficiently large number of generations, the increase in [[average information]] may result in the green cluster. Actually, the mean fitness is the same for both red and green cluster (about 65%). The effect of this adaptation is not very salient in a 2-dimensional case, but in a high-dimensional case, the efficiency of the search process may be increased by many orders of magnitude. Besides, a Gaussian distribution has the highest [[average information]] as compared to other distributions having the same second order moment matrix, Middleton, 1960.
== See also ==
== See also ==
* [[Information]]
* [[Information]]