Sunday, December 12, 2010

An Interesting Twitter Experiment and What it Shows About Klout

I've had my questions about Klout and it's rankings since the first time I went there and checked out my Klout score.

This article and the authors experiment shows that my doubts about Klout's ratings may have some bases in fact. And while one experiment doesn't prove or disprove anything it certainly raises troubling questions. And Klout is sure the best metric we have but it's a trouble metric as I've studied it.

On the one had it claims to measure things that at least in my report never seem to change. How is it possible that I've still got the same 6 people listed by Klout as my influencers as I had at the end of my first week on Twitter.

No more no less and no changes. That's hard to understand as over that time I've climbed from 0 to around 2,000 followers. You would think that something about this list would change. And as troubling is the fact that while most of my metrics have shown the natural rise one would expect from an active Tweeter who tweets and isn't just a name hiding a bot I've seen sudden and unexplained drops in my Klout ratings at times.

For example in the first two days that I started on Amplified my Klout score dropped over 10 (59>43) points and only slowly came back up to almost what it was before I was using Amplify (53).

That mystifies me as while I didn't post as many Tweets on Twitter directly as before I have almost all of my Amp posts and comments also sent to Twitter. And by Twitters numbers my number of Tweets per day actually went up slightly during that time and their content changed only in their coming from Amplify.

So I'm sure that people will say well Klout looks on that kind of thing as suspect and docked you accordingly. But that can't be the reason or I wouldn't have climbed back up to almost the same Klout score while still Amping and Tweeting the same since joining Amplify.

Anyway it's an interest article and raises some good questions that Klout and Twitter should respond too. I doubt they will but it leaves me wondering if behind the scenes both Klout and Twitter are failing in their jobs and letting bots rank far too high and send what anyone can see is at best glorified spam.

Amplify’d from

Can you become influential on Twitter merely by Tweeting a lot?

A bit more than a month ago, I asked the question: Can you become influential on Twitter, and get a high Klout Score, merely by Tweeting a lot?

To test this, I set up an experiment, which involves four Twitter bots that automatically tweet the output of the Unix fortune command-line application. Fortune randomly outputs mildly humorous quotes, and was often used on Unix to produce a ‘welcome message of the day’ upon login.

The four bots Tweet once every minute, once every five minutes, once every fifteen minutes and once every thirty minutes respectively. They are completely anonymous, have no avatars or custom user profiles set, and do not follow anyone.

Now, after 80 days of running the experiment (Jules Verne style), there’s a set of pretty hot data available.

The Data (the good)

Follower accumulation over time

We can clearly see from this graph (quite surprisingly), that each bot accumulated followers linearly. Also, it seems the more they tweeted, the steeper the follower accumulation rate is, without any drop off, even for the bot that tweets every minute.

Let’s normalize the data by plotting the amount of followers against the amount of Tweets, thereby literally measuring the amount of followers per Tweet:

Followers per Tweet

Let’s test this assumption, by plotting the curves over time again, including an amplification factor equal to the amount of minutes that lapse between Tweets. That means, we assume the once every 5 minutes bot would have had 5 times more followers if it Tweeted once every minute, etc:

Normalized Amount of Followers over time

That means, the more you Tweet, the more followers you get. Period. It doesn’t matter how often you tweet, you gain an equal amount of followers for every time you Tweet.

The Followers (the bad)

Looking at the followers of these bots, many of them seem to be bots themselves (there are quite a few real people who attempt conversations with them, but they are in the minority). Most of these bots get triggered by keywords present in our bots’ Tweets, and then follow and retweet our bots’ Tweets. A good example is @BurroughsBot, which retweets Tweets that match the search term William Burroughs.

At this point I turned to Klout (which, incidentally, is the actual reason for setting up this experiment in the first place). Surely Klout should be able to make sense of this robotic mess (like Google does with link farms), shouldn’t it?

The Klout Scores (the ugly)

Klout Score: Bot 1Klout score for ‘once a minute’ bot
Klout Score: Bot 2Klout Score for ‘once every 5 minutes’ bot
Klout Score: Bot 3Klout Score for ‘once every 15 minutes’ bot
Klout Score: Bot 4Klout Score for ‘once every 30 minutes’ bot

What’s wrong with this picture? To start off with, it should not really be possible for a bot to reach a Klout Score of 50 within 80 days merely by Tweeting random (yet entertaining) rubbish every minute, should it?

Taking into account that many Twitter clients (like Hootsuite) and filter applications (like Datasift) are using Klout as a trusted way of filtering tweets, it means Klout will have to up their game on this one to stay in the game.

Or else, we might just be run by machines sooner than we think!

No comments: