BTW – I may need a wee bit of help on a project I am working on. Might be in touch re: that in sept/oct.

Unfortunately I missed Mike’s talk and lab on web spidering (cij admin is a harsh mistress), but he tells me he will put the notes up on his site anon- and judging by the buzz surrounding his sessions, they would be well worth a read.

I’ve done a few more little tests and am slightly more confused, or possibly enlightened.

I have one big pipe I use every day to monitor news about journalism and journalists for the Frontline Club. I’ve just tested the exact same pipe feed in Netvibes, Google Reader, on Yahoo Pipes page itself and in NetNewsWire. All offer different result speeds. So is it an application/webtool thing and not a pipe thing?

The Yahoo Pipes page is quickest to get stuff, which would make sense. However, even when I manually refresh the other three, the new feed I can see on the Pipes does not appear in the other destinations.

I can see seven results in Netvibes that I can’t see in Google Reader – even with repeated refreshes while I write this. There are many more missing in NetNewsWire.

I’ve taken some screenshots of this and may blog it because it has me very confused. However, it does remind me of a guy I was talking to a year or so ago who ran a niche news agencies who said they couldn’t rely on RSS as it was too slow, they used something else which offered real immediacy, a bespoke system I believe.

I might have to rethink how I follow breaking news after this experiment.

I guess it would be interesting to see a comparison of raw keyword Google and Yahoo , as well as page-2-rss – all vs Yahoo Pipes.

Is it possible that Pipes does some sort of analysis on the feed before making it available, whether you’ve set filters or not?

The only other thing I can think is that for non-Yahoo RSS, that pipes has to access these indices in the same way the surfer does – according to whichever server we are directed to (and hence a possible delay), whereas Google may provide a local network for it’s RSS systems to access it’s index at source – though that could be bs to be honest.

I haven’t tried kosmix yet, but will give it a whirl this weekend – thanks for the tip.

Bytheway just had a look at the pipe you built for Thomas Wiegold – very interesting.

I took a look at the first pipe you are piping into that feed – by See-ming Lee – looks impressively complicated!

THe first thing that occurs is whether it’s possible to tell what is possible to tweak, and what isn’t. For example, is it possible to tell (using stats, and a bit of editorial judgement, over time) which of these sources tends to produce better results than others, and so prioritise them accordingly?

Great stuff though – a real eye opener.

Murray.

This is based on completely unscientific research, but Pipes seem to run in batches every few hours or so. That’s my impression at least.

In my RSS reader I have a Pipe that collects a tonne of raw keyword feeds in one feed. I also have a folder with all those raw feeds as straight RSS, no filtering, just to compare the speed of delivery and the Pipe is generally slower, but pretty accurate at pulling out only the stuff I need.

If you have any idea why this might be the case, I’d love to know…

In addition, how do you compare Pipes to search sites like Kosmix – http://www.kosmix.com/ – I find they miss too much, but are very user friendly.

Lastly, I’ve been running a smallscale experiment for some journalists by designing custom Pipes for them – http://bit.ly/vTw7S – Just hope they bother to tell me if they’re any good or not.

What I am about to tell you sounds whacko however the truth sometimes just sounds that incredible. Between 1983 and 1986, I lived on 256 Vanderbilt Ave (#4L), Brooklyn, NY. I’d just graduated from Pratt Institute with a Master’s in Communications Design. Experimental music demos I made while I lived on Vanderbilt were copied by unidentified persons. Somehow, these persons contacted Sony Music and other big labels and they got hold of my material.

I later heard, after my return in 1986 to Ghana, in hit songs from the U.S., lots of melodies I’d written–note-for-note. Guilty the most was LaFace Records and quite a few artists with Sony Music (Babyface, Boyz II Men – “End of the Road”, TLC – “Waterfalls”, Mariah Carey – “One Sweet Day”, Tony Rich, etc). I’d sung most of my material in nonsense lyrics and ad-lib; I was experimenting and didn’t worry to much about lyrical content. I even experimented with criss-cross drumbeat rhythms (from the Frafra and Dagare tribes), which found its way into and became mainstream R&B rhythmic material, courtesy of LaFace Records/Sony Music and others. There were silly, “radio drama” intros to songs that I concocted that Tony Rich used extensively as did several other guys. By sheer volume, I don’t think it is pure coincidence.

A Ghanaian (now a U.S. citizen), currently working at Brandywine Assets Management (NJ) may know how my demos got to Sony Music. It is rumoured he worked there briefly. He was at Pratt Institute with me and often remarked that my songs had the potential to be blockbuster hits. I’ve been unable to contact him for an explanation. He wouldn’t reply my e-mail.

Much later I heard other bits of my work, also note-for-note, in songs by Michael Jackson, Celine Dion, R. Kelly and Kirk Franklin’s work (”You’re Not Alone”; the van Passels lied; “My Heart Will Go On”; “I Believe I Can Fly”;”Lean On Me”, etc). I really don’t know how all those guys got hold of my stuff?

Nobody believes me when I tell them this story. It’s simply unbelievable. But I have proof. I have over 40 hours of music I composed on old TDK and Sony cassettes. Technically, the magnetic tape recordings can easily be assessed as having been made in the mid-’80s. Further, any musicologist can listen to the tracks and tell from my musical signature (a kind of compositional fingerprinting) that their compositions even with the re-arrangements are a direct rip off. (My ideas may seem eclectic but that’s where my ideas were pushing me at the time).

I’ve tried for over 15 years to get just anybody to listen to this fantastic story. I’ve hesitated pushing it too far because this all sounds a bit too kooky I guess. I’d always wanted a good investigative journalist and some brilliant lawyers to uncover the truth but couldn’t get anyone interested…and I don’t have the money either. Whatever it is, I don’t think all those ideas of mine being duplicated elsewhere is pure coincidence.

Disregard the fact that I’m an African. I grew up listening to the best music of the ’60s and ’70s. I played in several bands as keyboard player and later as a bass player/ guitarist. And though self-taught I know I was pretty creative and original.

Is anyone listening? The music industry should also focus on how to maintain the creative integrity of artists. That’s going to be difficult but it must be encouraged. Sure there’s tons of pressure and contracts and deals and all that and artists have to come up with something fantastic every now and then. But ripping off other people’s material is low, cheap, wrong and downright evil. Money drives the whole thing as I can see and that’s all right. But there’s the need for some fundamental change to how we get that money. Values may not mean much to business people but to me as an artist, hey, it’s important!

When your creative juices stop flowing, what is fair is shifting gear, moving on to new partnerships or abandoning ship. Plagiarising other people’s material cannot credibly sustain any ‘talented’ artists’ career.

The music industry giants should watch how sincere the artists they’ve signed up are and what their A&R guys are doing with all the solicited and unsolicited material. ‘And,’ as Shakespeare said, ‘there’s the rub,’

You at the BBC have your snouts buried in the trough of public funds. Poor women go to jail because they cannot pay the tax to pay you and Ross and Bland. Your fascist TV Licencing database adverts show us the contempt you hold us in. Do not be surprised when that contempt is returned with interest.