10 days later – Twitter archive search alternatives

A nod to Paul Bradshaw for tweeting this great article in Read Write Web on the various ways Tweeters can archive their Tweets.

There’s some info in here that should concern any journalist wishing to make full use of Twitter, beyond the very basics of sharing and gathering breaking news:

At first, Twitter held onto your tweets for around a month, but as the service grew more popular, this “date limit” has dramatically shortened. According to Twitter’s search documentation, the current date limit on the search index is “around 1.5 weeks but is dynamic and subject to shrink as the number of tweets per day continues to grow…

This is something I noticed myself recently when trying to collate a list of Tweets on our recent CIJ Summer School from earlier last month – several Tweeters had used the #cij topic, but when searching for these quotes a fortnight later, I was alarmed to see many had seemingly disappeared from Twitter’s index.

In the (understandable) gold rush for currency in social (or real-time) news, its not uncommon for more staid (and less sexy) issues like provenance to get forgotten, or at the least down-graded.

But ten days is a horribly narrow time limit for any journalist trying to eke out stories, issues, potential contributors, or other newsworthy trends – indeed, by comparison even with the relatively shallow sources for mainstream news (i.e. Google News – 30 days) it is tiny.

Artists impression: the window of opportunity in Twitter search  

Artist’s impression: the window of opportunity in Twitter search

Still, while there is clearly value in the (admittedly vague) hope that most of us will diligently go about archiving their own Tweets for future use (and for the greater benefit of us all), all is not lost – a touch of advanced searching in Twitter’s main rivals will often do the trick nicely.

Personally, I’ve found the most useful Tweets I’ve ever seen (in work, and leisure) contain links – high-value social discovery. So with this in mind, it’s a good idea to include in your search some key url-shortening domains – see the following search for Tweets on the issue of micropayments in news:

micropayments news (tinyurl OR bit.ly) site:http://twitter.com/

Better still, once your results come back, hit the ‘show options’ tab at the top of the page:

Search for Tweets on micropayments in news in Google

You can then refine using the Custom Date Range option – see these result from 2007 – 2008.

But as ever in search, its no good relying solely on Google to do your research for you, so try the same search in Bing and – as sure as eggs is eggs – you’ll get a different number of results (the initial Google search yielded 143, to Bing’s whopping 234). While this discrepancy hints at content potentially being lost forever in-between these two indexes, it is at least gratifying to know that some of it is findable in future.

I’d like to think that another possible value in searching for Tweets in either Google or Bing could be that spam is less likely to be indexed, though that could well be wishful thinking on my part.

It’s a shame that Twitter has organised its folder system with the statuses folder nesting within the username – as this prevents a strict search of statuses only, but heyho – you can’t have everything.



Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

%d bloggers like this: