Re: Information

From: Robin Faichney (
Date: Fri May 04 2001 - 10:19:10 BST

  • Next message: Robin Faichney: "Re: Information"

    Received: by id KAA24025 (8.6.9/5.3[ref] for from; Fri, 4 May 2001 10:26:11 +0100
    Date: Fri, 4 May 2001 10:19:10 +0100
    Subject: Re: Information
    Message-ID: <>
    References: <3AED8233.15786.6E25AE@localhost>; <> <3AEEB6DE.18806.39FF8D@localhost> <> <>
    Content-Type: text/plain; charset=us-ascii
    Content-Disposition: inline
    User-Agent: Mutt/1.3.15i
    In-Reply-To: <>; from wilkins@wehi.EDU.AU on Thu, May 03, 2001 at 11:11:07AM +1000
    From: Robin Faichney <>
    Precedence: bulk

            The term "information" is used in common language with a
            wide variety of meanings, ranging from instruction and data to
            knowledge itself. Despite repeated warnings by specialists, these
            other meanings tend to impede our understanding of "information"
            in the technical sense. [see below]

    On Thu, May 03, 2001 at 11:11:07AM +1000, wilkins wrote:
    > Robin Faichney wrote:
    > >
    > ....
    > > According to Roy Frieden, the laws of physics are generated by the
    > > attempt to minimize the difference between an entity or system's own,
    > > physical information, and the information that physicists can obtain
    > > about it. This account does not get awfully technical, at least as
    > > regards physics---we've just gone as deep into Frieden's work as we're
    > > going to go---but this distinction he draws is vital: between physical
    > > information, which exists for its own sake, and the more usual sort,
    > > information that's about something.
    > > (From
    > It seems to me there are really only four relevant sense of
    > "information" here:
    > 1. the Fisher Information account of measurement that Frieden proposes

    Fisher information, as the New Scientist article puts it, captures how
    much information you can squeeze out of a physical system. What Frieden
    has shown is that using Fisher information (I) and the information
    inherent in the system (J), the laws of physics can be derived. So I
    think "the Fisher Information account of measurement that Frieden
    proposes" is misleading. J is information as the concept occurs in
    thermodynamics, i.e. a measure of the structure of matter, and that is,
    in one sense at least, more fundamental than Fisher information.

    > 2. The Shannon-Weaver account that makes information of a sequence its
    > (prior) probability of being encountered

    There are close connections between information in communication theory
    and in thermodynamics:

            Information transmission has been defined as the change in
            probability of an event in one ensemble resulting from the
            occurrence of an event in another. Interactions of this sort
            typically involve a transformation of energy. Any transformation
            of energy, conversely, is accompanied by changes in probability
            among associated events. Thus thermodynamics, the study of energy
            transformations, might be expected to bear a close relationship
            to communication theory.

            Even a summary discussion of this relationship would involve more
            mathematical resources than presently at our disposal. The upshot
            of such a discussion, however, would be that communication theory
            provides a basis upon which thermodynamics can be systematically
            developed. Relying upon earlier work by Maxwell and Boltzmann, on
            how to compute the properties of gases by statistical methods, and
            upon subsequent work by Gibbs and Planck showing how the results
            of classical thermodynamics can be got from quantum theory through
            these statistical methods, Jaynes was able to show in the late
            1950s how these same results could be got more perspicuously on
            the basis of communication theory. Given Shannon's formulae it is
            now possible, as Tribus puts it[1], "to begin by considering the
            properties of the smallest particles and by simple mathematical
            methods to deduce the properties" of macroscopic systems.

            A clue to the relationship between thermodynamics and
            communication theory is that both employ "entropy" as a technical
            term, with definitions that bear a close formal resemblance.
            [Cybernetics and the Philosophy of Mind, Kenneth Sayre, pp36-7]

    [1] Citation available on request.

    The quote at the top is also from Sayre (p22).

    > 3. The Kolmogorov-Chaitin account that makes information the minimal
    > message length of a sequence, or the shortest algorithm that can
    > generate a sequence, whichever version one prefers; and

    I'm inclined to doubt that this is significantly different from
    Shannon-Weaver either, though I'm open to being educated on that.

    > 4. the semiotic or intentional account of the Peircians, Meinongians and
    > other representationalists.
    > So far as memetics is concerned, only the first three are relevant (it
    > matters not a whit is the information being transmitted is true,
    > coherent or in any way of significance to any audience, so long as it
    > spreads through a population).
    > If something is a measurement of some state distinct from the observer,
    > then that information (ie, the error implicit in the measurement) is a
    > physical mapping of what's in the head to what's in the world. However,
    > it fails to be memetic information until it is transmitted, and then
    > senses 2 and 3 come into play, so we can ignore the two extremes:
    > "objective" information in the sense of accuracy of measurement, and
    > "subjective" information in terms of what something means within the
    > head of a semantically or semiotically capable system (ie, some person)
    > and concentrate instead on the dynamics of information transmission and
    > the evolution of the signals so transmitted.

    I think that's all true, regarding memetics itself. What I'm
    more interested in though is the philosophy of memetics, and the
    relationship(s) between these different concepts of information, including
    4. How, exactly, is semantic information "encoded" in Shannon-Weaver and
    thermodynamic information? In memetic terms this means how, exactly,
    do memes as behavioural patterns relate to memes as ideas and such?
    More generally, this is about the relationship between mind and matter.
    I think I'm making real progress on that, but I know not everyone agrees!

    (It's also very clear that not everyone understands, but I will modestly
    refrain from comment on the relationship between the set of those who
    disagree and the set of those who fail to understand. :-)

    Robin Faichney
    Get your Meta-Information from
    (CAUTION: contains philosophy, may cause heads to spin)

    =============================================================== This was distributed via the memetics list associated with the Journal of Memetics - Evolutionary Models of Information Transmission For information about the journal and the list (e.g. unsubscribing) see:

    This archive was generated by hypermail 2b29 : Fri May 04 2001 - 10:29:45 BST