Re: Information

From: wilkins (wilkins@wehi.EDU.AU)
Date: Mon May 07 2001 - 01:30:30 BST

  • Next message: Mark Mills: "Re: Information"

    Received: by id BAA28915 (8.6.9/5.3[ref] for from; Mon, 7 May 2001 01:34:19 +0100
    Message-ID: <>
    Date: Mon, 07 May 2001 10:30:30 +1000
    From: wilkins <wilkins@wehi.EDU.AU>
    Organization: The Walter & Eliza Hall Institute of Medical Research
    X-Mailer: Mozilla 4.76C-CCK-MCD  (Macintosh; U; PPC)
    X-Accept-Language: en,pdf
    Subject: Re: Information
    References: <3AED8233.15786.6E25AE@localhost>; <> <3AEEB6DE.18806.39FF8D@localhost> <> <> <>
    Content-Type: text/plain; charset=us-ascii
    Content-Transfer-Encoding: 7bit
    Precedence: bulk

    Robin Faichney wrote:
    > The term "information" is used in common language with a
    > wide variety of meanings, ranging from instruction and data to
    > knowledge itself. Despite repeated warnings by specialists, these
    > other meanings tend to impede our understanding of "information"
    > in the technical sense. [see below]
    > On Thu, May 03, 2001 at 11:11:07AM +1000, wilkins wrote:
    > > Robin Faichney wrote:
    > > >
    > > ....
    > > > According to Roy Frieden, the laws of physics are generated by the
    > > > attempt to minimize the difference between an entity or system's own,
    > > > physical information, and the information that physicists can obtain
    > > > about it. This account does not get awfully technical, at least as
    > > > regards physics---we've just gone as deep into Frieden's work as we're
    > > > going to go---but this distinction he draws is vital: between physical
    > > > information, which exists for its own sake, and the more usual sort,
    > > > information that's about something.
    > > > (From
    > >
    > > It seems to me there are really only four relevant sense of
    > > "information" here:
    > >
    > > 1. the Fisher Information account of measurement that Frieden proposes
    > Fisher information, as the New Scientist article puts it, captures how
    > much information you can squeeze out of a physical system. What Frieden
    > has shown is that using Fisher information (I) and the information
    > inherent in the system (J), the laws of physics can be derived. So I
    > think "the Fisher Information account of measurement that Frieden
    > proposes" is misleading. J is information as the concept occurs in
    > thermodynamics, i.e. a measure of the structure of matter, and that is,
    > in one sense at least, more fundamental than Fisher information.

    A friend is doing his PhD on Freiden, which is why I know his work. The
    NS article was very vague and according to my friend, the critical
    measure *is* I, because J is inaccessible (a bit like Chaitin's Omega,
    but for distinct philosophical reasons). I is what is known in
    statistics as the Cramer-Rao Bound - the likelihood of an accurate measure.
    > > 2. The Shannon-Weaver account that makes information of a sequence its
    > > (prior) probability of being encountered
    > There are close connections between information in communication theory
    > and in thermodynamics:
    > Information transmission has been defined as the change in
    > probability of an event in one ensemble resulting from the
    > occurrence of an event in another. Interactions of this sort
    > typically involve a transformation of energy. Any transformation
    > of energy, conversely, is accompanied by changes in probability
    > among associated events. Thus thermodynamics, the study of energy
    > transformations, might be expected to bear a close relationship
    > to communication theory.
    > Even a summary discussion of this relationship would involve more
    > mathematical resources than presently at our disposal. The upshot
    > of such a discussion, however, would be that communication theory
    > provides a basis upon which thermodynamics can be systematically
    > developed. Relying upon earlier work by Maxwell and Boltzmann, on
    > how to compute the properties of gases by statistical methods, and
    > upon subsequent work by Gibbs and Planck showing how the results
    > of classical thermodynamics can be got from quantum theory through
    > these statistical methods, Jaynes was able to show in the late
    > 1950s how these same results could be got more perspicuously on
    > the basis of communication theory. Given Shannon's formulae it is
    > now possible, as Tribus puts it[1], "to begin by considering the
    > properties of the smallest particles and by simple mathematical
    > methods to deduce the properties" of macroscopic systems.
    > A clue to the relationship between thermodynamics and
    > communication theory is that both employ "entropy" as a technical
    > term, with definitions that bear a close formal resemblance.
    > [Cybernetics and the Philosophy of Mind, Kenneth Sayre, pp36-7]
    > [1] Citation available on request.
    > The quote at the top is also from Sayre (p22).

    Not to put too fine a point on it, much crap has been written about the
    relation between S-W info and thermo. The relationship is purely one of
    mathematical analogy. Brillouin wrote an excellent book back in the 60s,
    I believe, in which he effectively destroyed the idea that they are
    commensurate or covariant measures. He used a lovely argument about
    Maxwell's Demon that I think you ought to look at - in sum, for the
    Demon to work, he needs to employ energy to get what is effectively the
    Fisher Info, but not the SW info.
    > > 3. The Kolmogorov-Chaitin account that makes information the minimal
    > > message length of a sequence, or the shortest algorithm that can
    > > generate a sequence, whichever version one prefers; and
    > I'm inclined to doubt that this is significantly different from
    > Shannon-Weaver either, though I'm open to being educated on that.

    I have an intuition (and we all now what they are worth) that in the end
    AIT reduces to SW, or they are identical (in terms of the minimum
    message length of an algorithm). However, I know of no attempt to prove
    this rigorously.
    > > 4. the semiotic or intentional account of the Peircians, Meinongians and
    > > other representationalists.
    > >
    > > So far as memetics is concerned, only the first three are relevant (it
    > > matters not a whit is the information being transmitted is true,
    > > coherent or in any way of significance to any audience, so long as it
    > > spreads through a population).
    > >
    > > If something is a measurement of some state distinct from the observer,
    > > then that information (ie, the error implicit in the measurement) is a
    > > physical mapping of what's in the head to what's in the world. However,
    > > it fails to be memetic information until it is transmitted, and then
    > > senses 2 and 3 come into play, so we can ignore the two extremes:
    > > "objective" information in the sense of accuracy of measurement, and
    > > "subjective" information in terms of what something means within the
    > > head of a semantically or semiotically capable system (ie, some person)
    > > and concentrate instead on the dynamics of information transmission and
    > > the evolution of the signals so transmitted.
    > I think that's all true, regarding memetics itself. What I'm
    > more interested in though is the philosophy of memetics, and the
    > relationship(s) between these different concepts of information, including
    > 4. How, exactly, is semantic information "encoded" in Shannon-Weaver and
    > thermodynamic information? In memetic terms this means how, exactly,
    > do memes as behavioural patterns relate to memes as ideas and such?
    > More generally, this is about the relationship between mind and matter.
    > I think I'm making real progress on that, but I know not everyone agrees!

    There is a philosophical conundrum known as the "linguistic prison" - in
    order to discuss things we must cast them in liguistic terms, but we are
    discussing in the areas of truth and mind matters that cannot be so cast
    without prejudicing the argument - in short, whereof one cannot speak,
    thereof one must be silent (and as some wag had it, "and you can't
    whistle it either"). How meaning is encoded is very much context
    relative - see Dretske's book. The encoding protocol used determines the
    meaning of a message at receiver.
    > (It's also very clear that not everyone understands, but I will modestly
    > refrain from comment on the relationship between the set of those who
    > disagree and the set of those who fail to understand. :-)

    If it's anything like mine, they coincide :-)

    Brillouin, Leon. Science and Information Theory. New York: Academic
    Press, 1956.
    Dretske, Fred I. Knowledge and the Flow of Information. Cambridge,
    Mass.: MIT Press, 1981.

    The disanalogy of thermo and SW info is discussed in

    Pierce, John Robinson. An Introduction to Information Theory : Symbols,
    Signals and Noise. 2nd, rev. ed. New York: Dover Publications, 1980.

    John Wilkins, Head, Communication Services, The Walter and Eliza Hall 
    Institute of Medical Research, Melbourne, Australia
    Homo homini aut deus aut lupus - Erasmus of Rotterdam

    =============================================================== This was distributed via the memetics list associated with the Journal of Memetics - Evolutionary Models of Information Transmission For information about the journal and the list (e.g. unsubscribing) see:

    This archive was generated by hypermail 2b29 : Mon May 07 2001 - 01:37:54 BST