RE: Word-use spikes

From: Lawrence DeBivort (
Date: Sun 02 Mar 2003 - 13:08:49 GMT

  • Next message: Alan Patrick: "Flamingoes & Tipping Points"

    Greetings, Scott,

    Leaving aside the unfortunate degeneration of language (not that that is a new phenomenon!), and leaving aside 'celebrity'-based fads, I wonder whether word-spike studies might not have a serious purpose, indicating trends in the adoption of ideas or preoccupation with them. One could conceive that such spikes indicate shifts of attention.

    The US CIA explored a similar methodology in the 50s, calling it 'content analysis'. The idea was that by tracking the number of times a theme was discussed in the press, one could discern patterns that had predictive value. That is, if journalists were talking about an innovative technology, or about a social concern, that that was a forewarner that the theme was going to become important. I don't think this methodology proved to be of much worth, though it did spawn John Naisbitt's book, Megatrends.

    Might blogs, beyond word-spikes, convey 'idea-spikes'? Might they provide a vector for tracking meme dissemination?

    (BTW, I didn't notice who posted the first blog item here, but my thanks. Blogs are not a development that I had paid attention to, and now I'll check it out. Arthur, is the "instapundits" site your referenced a blog?)

    Cheers, Lawry

    > -----Original Message-----
    > From: []On Behalf
    > Of Scott Chase
    > Sent: Sun, March 02, 2003 1:30 AM
    > To:
    > Subject: Re: Word-use spikes
    > >From: "Lawrence DeBivort" <>
    > >Reply-To:
    > >To: <>
    > >Subject: Word-use spikes
    > >Date: Sat, 1 Mar 2003 21:08:03 -0500
    > >
    > >By Will Knight
    > >New Scientist
    > >February 18, 2003
    > >
    > >
    > >
    > >Searching for sudden "bursts" in the usage of particular words could be
    > >used
    > >to rapidly identify new trends and sort information more
    > efficiently, says
    > >a
    > >US computer scientist.
    > >
    > >Jon Kleinberg, at Cornell University in New York, has developed computer
    > >algorithms that identify bursts of word use in documents.
    > >
    > >While other popular search techniques simply count the number of words or
    > >phrases in documents, Kleinberg's approach also takes into
    > account the rate
    > >at which the word usage increases.
    > >
    > >Kleinberg suggests that the method could be applied to weblogs
    > to track new
    > >social trends. For example, identifying word bursts in the hundreds of
    > >thousands of personal diaries now on the web could help
    > advertisers quickly
    > >spot an emerging craze.
    > >
    > >Hot or not
    > >
    > >The algorithms used to identify these sudden bursts are
    > relatively simple,
    > >but very powerful, says Christos Papadimitriou, at the University of
    > >California at Berkeley.
    > >
    > >"The key is to find unexpected changes in the frequency of the
    > appearance
    > >of
    > >words," he told New Scientist. Papadimitriou agrees the method
    > could prove
    > >valuable when searching for new trends in weblogs.
    > >
    > >The approach could also be applied to sifting through other types of
    > >information. Identifying word bursts within email messages sent to a
    > >company's customer support address might help maintenance staff spot a
    > >major
    > >new problem.
    > >
    > >Researchers at Google, the world's most widely used internet
    > search engine,
    > >have already shown that identifying spikes in search terms can be used to
    > >track the spread of news and rumours around the world. The
    > algorithms that
    > >run Google's automated news aggregation service remain secret, but it is
    > >not
    > >difficult to imagine that word bursts could, or do, have a useful role.
    > >
    > >In a simple historical test of the technique, Kleinberg analysed all the
    > >annual State of the Union addresses given by US Presidents since 1790. He
    > >found that particular word "bursts" could indeed be linked to important
    > >events at the time the speeches were delivered.
    > >
    > >In the years that immediately followed the American Revolution, for
    > >example,
    > >sudden bursts in the use of words such as "militia", "British" and
    > >"savages"
    > >are found.
    > >
    > >From 1930 to 1937 a spike in the use of the word "depression" is seen.
    > >
    > Gee. I wonder why... Could it be that the Great Depression was a reality
    > within that time period?
    > >
    > >And
    > >from 1949 to 1959 "atomic" is the word with the greatest "burstiness".
    > >
    > Atomic is bursty in another way, but it's not surprising that the word
    > "atomic" would spike when the reality of Hiroshima and the
    > subsequent arms
    > race ht home.
    > >
    > >Later
    > >in the 20th century, words such as "Vietnam",
    > >
    > Well, I'd suppose that if there was a military act^H^H^H^H^war
    > going on in
    > Vietnam, that country's name might be mentioned a tad more.
    > >
    > "Soviet", "communist" and
    > >"Afghanistan" increase sharply in usage.
    > >
    > Gee, I wonder why Afghanistan might have spiked recently. It's almost
    > trivial to point it out, but if one wishes to shoehorn historical
    > data into
    > a cubbyhole of word usage trends, you'll have plenty of word usage trends
    > subsequent to historical events that bring those words into the forefront.
    > >
    > >Kleinberg presents his findings on Tuesday at the American
    > Association for
    > >the Advancement of Science's annual meeting in Denver, Colorado.
    > >
    > >
    > >
    > I'd predict that popular uttering of the names of the stars of the
    > television show "Friends" spiked sharply after that show first
    > appeared and
    > became popular. I could be wrong, but it's just a sneaking
    > suspicion I have.
    > I have no idea why...
    > In professional football (not soccer or rugby-like facsimiles), there may
    > have been word spikes over the past decade for terms like "run and shoot
    > offense" and "west coast offense".
    > All kidding aside, I suppose this information on bursting could
    > be useful,
    > to marketers especially.
    > I've subjectively noticed a very annoying tend in usage of the word
    > "absolutely". I don't know what the heck is going on with people
    > feeling the
    > need to utter this word in such emotive manners, but it needs to
    > stop. Has
    > anybody else noticed this trend? It's like the "fershure" of the 00's,
    > except I don't think Zappa or offspring had anything to do with
    > it this time
    > around.
    > _________________________________________________________________
    > Protect your PC - get VirusScan Online
    > ===============================================================
    > This was distributed via the memetics list associated with the
    > Journal of Memetics - Evolutionary Models of Information Transmission
    > For information about the journal and the list (e.g. unsubscribing)
    > see:

    =============================================================== This was distributed via the memetics list associated with the Journal of Memetics - Evolutionary Models of Information Transmission For information about the journal and the list (e.g. unsubscribing) see:

    This archive was generated by hypermail 2.1.5 : Sun 02 Mar 2003 - 12:54:00 GMT