A custom search engine for UK political opinion

If opinions are like arseholes, then grab yourself a clothes peg – quick.

Now – say you’re after some professional opinion on a current affairs topic, any topic. How do you go about separating the chafe of ‘news’ from the wheat of opinion when you’re searching?

This question has vexed me for a while, and came back to bite again after I had a brief tinker with Silobreaker.

For those of you who haven’t dabbled, Silobreaker is a news search engine/aggregator which incorporates elements into the search mix that most other aggregators can only dream of. Like allowing you to track search term trends, and providing a means of visualising the relationships between associated words in the news (I’m not going to review Silobreaker here – but if you’re interested, there are no shortage of takers – here’s 1, 2, 3, 4, 5, 6 to start with).

Anyways, today I picked up on a piece written late last week by the CEO of Silobreaker, which extols the virtues of the search term analysis gizmos available, in the context of the US election race (and those taking part).

Surely if meaningful trends can be established on the reporting of Obama and Palin, I thought, so too should a similar comparison of UK political figures be possible, especially this week when Nick Clegg addressed his party conference as leader for the first time (today).

So I gave it a bash. But I was more than a little disappointed with the results involving the associated keywords – see here

They look like interesting terms – but if you hover over most of these keywords, you will find they are referenced from his own speeches. This is obviously not a particularly fair reflection of what journalists are choosing to pick up on, and neither does it represent what the opinion-shapers in the press are actually making a song and dance about.

Which got me back to that long-standing problem – why isn’t it possible to run a search and only have opinion pieces coming back?

And so, after about an hour of piddling about with Google Custom Search, I present to you:

The UK Political Opinion search engine

I’ve basically taken the opinion sections of most UK newspapers and made them searchable in the one place, without the unnecessary distraction of news bunging up the system.

It goes back much further than Google News (ie years), but it doesn’t include everything.

I’ve had to leave The Mail out, not because it’s opinions are so putrid their stench would knock most searchers out at a hundred paces, but because the site’s information architecture is so flakey that opinion pieces aren’t differentiated from ‘news’. Though maybe that’s a fair reflection of their editorial policy…

Anyways, no Mail – and also, no Glasgow Herald (for similar reasons). I’ve included the FT and Economist, but obviously premium content you’ll still have to pay for.

The Mirror was a bit of a faff too, it has to be said, in so far as their opinion pieces are littered amongst a range of different folders, all named according to the columnist in question – so it included adding several urls with each different columnist in the search.

But at least they are (mostly) there.

It works – check out the difference between results for Nick Clegg in Google and those results from UPOSE.

So now you can do your own personal version of What the Papers Say with your own regional/toff voices for good measure.

Any feedback (as always) is welcome…

For those of you interested in developing your own engines but who don’t know how, take a look at this post I did back in June.


Tags: , ,

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: