39
edits
Changes
Jump to navigation
Jump to search
Line 46:
Line 46: + + + + + +
Information entropy (view source)
Revision as of 14:54, February 18, 2008
, 14:54, February 18, 2008→The continuous case
After a sufficiently large number of generations, the increase in [[average information]] may result in the green cluster. Actually, the mean fitness is the same for both red and green cluster (about 65%). The effect of this adaptation is not very salient in a 2-dimensional case, but in a high-dimensional case, the efficiency of the search process may be increased by many orders of magnitude. Besides, a Gaussian distribution has the highest [[average information]] as compared to other distributions having the same second order moment matrix, Middleton, 1960.
After a sufficiently large number of generations, the increase in [[average information]] may result in the green cluster. Actually, the mean fitness is the same for both red and green cluster (about 65%). The effect of this adaptation is not very salient in a 2-dimensional case, but in a high-dimensional case, the efficiency of the search process may be increased by many orders of magnitude. Besides, a Gaussian distribution has the highest [[average information]] as compared to other distributions having the same second order moment matrix, Middleton, 1960.
== History of applications ==
A comprehensive outline of applications such as tele-communication, channel coding, computer science, physics, neurobiology, electrical engineering and so on may be found in Wikipedia.
Less well known is the use of information theory in simulated evolution by random search. For instance, the average speed of a random walk with Gaussian distributed steps in a hypercube or simplex is proportional to -P log( P ), where P is the average probability that a step will lead to a new point inside the hypercube, Kjellström, 1969. This may be interpreted as the self-information –log(P) divided by the work or time – proportional to 1/P – needed to get the information on the average. A more general theorem of efficiency based on information theory may be found in a paper by Kjellström, 1991.
== See also ==
== See also ==