The best team in the world?

The best team currently playing world cricket is South Africa, according to the ratings of the International Cricket Council (see here for the data on Wikipedia, which I'm linking to in preference to the ICC's own site because the latter doesn't indicate historical values, and so at some point will no longer confirm this assertion).  It's true, South Africa have three fine pace bowlers, Stein, Phillander and Morkel (the first two of which are arguably the best two bowlers in the world); and likewise, two of the world's very best batsmen (de Villers and Amla).  Check out the ICC Player Rankings site which does, in my opinion, a pretty good job at quantifying each player's recent efforts, to see quite how strongly this bunch have been performing for the last few years.  But all rounder Jacques Kallis and opening batsman Graeme Smith have both recently retired, and South Africa were pummelled in India and just lost the first test of their series against England.  Can it really be true that South Africa are still number one?

The first things to say are in defence of the ICC ratings.  Objective attempts to rank teams based on historical performance are, by definition, historical (and thus backwards looking): the top team at any one moment in time has had the best record (according to the chosen criteria for assessment), this does not mean it will continue to have the best record in future.  And either a system weights recent matches very heavily, in which case the ratings will prove very unstable (a great team loses a couple of games and loses its position), or it rates them less heavily (in which case a team in serious decline may stay number one for a while).  And sometimes an objective system will agree with one's subjective assessment, and sometimes it will not, and the comparison of the two is itself subjective: has one chosen the right objective criteria, in which case one's subjective assessment is revealed as false, or does the failure of the system to yield an instinctively satisfying result prove that the system is wrong? A system that suggested manifestly absurd conclusions can be rejected; in fact, looking back at the historical record, it's clear that most of the time, the best team in the world according to the ratings was also the team most followers of the game would have picked. But when no side is dominant, supporters will disagree subjectively with each other, and at least some of them will also disagree with the output of any given rating system.

But... read on a little bit more, and the Wikipedia page will also tell you something of how the ICC ratings are calculated, and it's immediately clear: it's a very complicated system.  Indeed, it contains some apparent absurdities: the quality of a team's victory is assessed according to the strength of the opposition, unless the opposition is of a very different strength to the team itself, in which case it is assessed according to the team's own strength. This just seems bizarre.  When you look at the system in more detail, you can understand how it works to produce mostly sensible outcomes.  But it's a hack, not a clean system based on first principles.

But is such a clean system possible?  In fact, a much cleaner system is possible, based on principles of the Hungarian mathematician Arpad Elo, used to rank players in chess and other games. What if we apply Elo's system to test cricket?

In fact, the possibly disappointing news is that in general the outcomes are not so different to the outcomes of the ICC's system.  The ICC system can in fact be considered a dirty hack to try and generate an Elo-like rating while taking into account certain cricket-specific factors.  But the result is a rating that is hard to interpret and subject to a large number of essentially arbitrary parameters.  What I have done is to implement a relatively pure Elo system and applied it to a database of test match results.  In these posts, I'm going to explain the system, look at the logic of the ICC system in comparison, and also to present some interesting observations about how ratings have changed over time.  This will take a while, so before I start, here are the current ratings of the sides under my system, with the scores and rankings of the teams under the ICC's system in brackets:

1. Australia    154   (109 - 3rd)
2. India    116  (110 - 2nd)
3.  Pakistan    93  (106 - 4th) 
4. South Africa    74   (114 - 1st)
5.  England    73 (99 - 6th)  
6. New Zealand    67  (99 - 5th) 
7. Sri Lanka    -4    (89 - 7th)
8. West Indies    -86  (76 - 8th) 
9. Bangladesh    -172 (47 - 9th)  
10. Zimbabwe    -315 (5 - 10th)

Firstly, don't get too excited that some scores e.g. India's are almost exactly the same under both systems.  In my system, scores can be negative; in the ICC's, zero is the lowest possible score.  The raw numbers are not strictly comparable and if they're similar, that's only by chance.  In terms of rankings, the story is more interesting: both systems agree on the bottom four, and that England and New Zealand can be found in the middle.  But the order of the top four is a point of sharp disagreement: especially as my system rates Australia as much better than their rivals, whereas the ICC puts all four of the top nations fairly close together.

But what does this mean?  For that, you'll have to wait for the next posts in this series.