Test Cricket World Ratings 26/2/2106

Coming into their recent series against New Zealand, Australia were already top of my world ratings; but only in second place in the ICC's.  And their opponents had home advantage, and their own rating (on my system) was the highest it had ever been.  It could have been a tough pair of matches for Australia: instead, they've finished the series with two comfortable wins.  Indeed, the second match was notable for the fact that Brendon McCullum scored a brilliant century for his team, New Zealand made over 700 runs in the match, and still they lost comfortably by 7 wickets.  One sign of a strong team is that they sometimes roll their opposition over, as Australia did in the first test; another sign is that even when the opposition don't do so badly, they still get crushed in the end.  This was a strong performance from Australia and now even the ICC agree with me in putting Australia in the number one spot, displacing an Indian team that held that honour briefly but didn't even get to play a match before losing it again.  Australia have had a very good run since losing the Ashes to England last summer; they could be top of the ratings for some time.  For New Zealand, however, thing now look less rosy, especially with the talented McCullum calling it a day.

So here are the latest ratings in my system: 

Australia     166  +11
India         116    
Pakistan       93    
South Africa   76    
England        71    
New Zealand    43  -11
Sri Lanka      -4    
West Indies   -75   
Bangladesh   -172    
Zimbabwe     -315

Australia's two wins have almost doubled their lead over India, which is now a decent 50 points.  Note that the series neatly indicates one way the rankings work.  Australia's first win increased their rating by 13 points (and decreased New Zealand's by the same margin): this meant a better result was predicted for the second test, so when that victory came, it was only worth +/- 11 points.

This blog might be quiet for a while as there's now a gap in the test cricket program before Sri Lanka tour England early next summer.  England followed up their win over Australia with a defeat (against Pakistan) and a victory (against South Africa) on their winter tours. Sri Lanka's highest ever ranking was 95 in 2009; a current ranking of -4 is disappointing, given that the presence in test cricket Bangladesh and Zimbabwe, two historically weak sides, reduce the quality of the average rating represented by zero. But this state of play means, of course, that England will be fancied to win.  Note, however, that England's 75 point margin is less than the 95 point gap (in the opposite direction) that now separates them from last summer's victims.

Test Cricket World Ratings 17/2/2016

When we last looked at the cricket world ratings, calculated according to my Elo-based system, Australia were in the lead, with a margin of 26 points over India, who the ICC's system currently puts in first place.  What does a lead of 26 points mean?  Well, if Australia were to play India now, the expected result is 0.54 points to Australia; a pretty small margin.  If India were to win that game, India's rating would improve by 19 points, and Australia's would fall by 19, so the two teams would swap places at the top of the rankings.  In other words, it's a fairly small margin.  As we've seen, in January 2008, Australia were also top, but were 222 points clear of their nearest rival, so on that scale, a lead of 26 is pretty small.  In other words, there's no especially dominant team right now (which also makes it less surprising that the ICC system, and mine, answer this question differently).

In fact, Australia are playing a test series at the moment, but not against India.  Instead, they're taking on the Antipodean rivals, New Zealand.  New Zealand is a small country and their cricket team has never been rated best in the world. In fact, their current rating of 67 is as high as they've ever had (and in another form of the game, they reached the world cup final last year, so in general their cricket is not in too unhealthy a state).  Within 50 points of second place, and within 100 of Australia, New Zealand are close to par with most of the other leading sides in test cricket.

Many of us remember the New Zealand team of Richard Hadlee in the 1980s, so I thought I'd look up that era's team and see how they compared (I've already covered many past Australian sides whilst reviewing the best historical teams in various posts).  So here are the ratings from November 1985:

West Indies    158    
New Zealand     28    
Pakistan        20    
England         20    
Australia      -38    
India          -69    
Sri Lanka     -119

And it's an interesting story: the West Indies were very strong, but it's notable how weak everyone else was in comparison. New Zealand were actually the pick of the rest, albeit a substantial 130 points behind.  Australian cricket, on the other hand was at close to its lowest ebb (and the team had just lost the Ashes in England).  New Zealand's rating was lower than it is now, but as we've discussed, the absolute rating is a measure of how the teams compare to their contempory rivals on average; the presence of weak Bangladeshi and Zimbabweab teams in test cricket today has increased the ratings of everyone else.

Anyway, can today's relatively strong New Zealand pull a surprise on Australia, or will Australia record another win (and maybe claim first place in the ICC's system as well, in which they had lagged India by just 1 point - the points in the two systems are not comparable, but 1 point is still a fairly small margin even under the ICC's rule)?  Well, not in the first test, which has just been won by the Australians by a convincing margin. Our updated ratings look like this:
 
Australia     155  +13   
India         116    
Pakistan       93    
South Africa   76    
England        71    
New Zealand    54  -13   
Sri Lanka      -4    
West Indies   -75    
Bangladesh   -172    
Zimbabwe     -315

So, no change in the pecking order, but Australia's win has slightly strengthened their position.  The ICC ratings won't be redone until this (short) series is over, but a win obviously can't hurt Australia's chances there either. Australia may have lost some of their better cricketers recently (with the retirements of Michael Clark and Mitchell Johnson), but for the moment, they're strengthening their claim to be considered as the world's best team anyway.  

The second test starts on Saturday. 

When Strong Meets Weak

In these posts, I’m reviewing the ICC system for rating test cricket teams, having previously outlined my own, Elo-style system.  In essence, the ICC’s system rates each team using a weighted, rolling average of recent performances; the strength of those performances is measured first by the result of individual series, but also by the strength of the opposition it has played against.  But this means that a strong team might see its rating doomed to fall no matter what margin it beats a weak team by, if that other team is sufficiently poor.  It seems wrong that a team’s rating could drop after a perfect performance. How do the ICC deal with this?

Well, if two teams in the ICC ratings have a ratings gap of over 40 points, the system changes.  Instead of assessing the teams based on their opponents’ pre-series rating, the teams get credit based on their own.  The stronger team’s series rating, in this case, equals their own rating plus ten multiplied by the number of series points that team attains, plus their own rating minus ninety multiplied by the number of points won by their opponent.  For the weaker team, the same formula applies in reverse (i.e. its own rating is increased by 90 then multiplied by its series points, and then  added to its opponents’ series points multiplied by 10 less than its own rating).

Superficially, this is bizarre.  It’s common sense to say that it’s a greater achievement to beat a strong team than a weak one; but under this system, it’s a greater achievement to simply be a strong team: a strong team will get more credit for the same result than a weaker one playing the same opponent.  Secondly, we now have two different systems for calculating a team’s series rating, and a somewhat arbitrary cut-off for moving from one system to another (it’s not accidental, however, that the cut-off is something around 40, though, because it's at ratings differences of over 50 that teams could record a perfect series record and still find themselves losing ground if the system hadn't changed.

And what’s with this strange 90-10 situation?  Basically, it means that the stronger team needs to collect at least 90% of the series points on offer in order to improve its situation (the weaker team, by contrast, needs just 10% of the available points).  So there are traces of Elo in here, but in effect, we now have with a fixed expected value of 0.9 once the ratings gap between two sides exceeds 40. This hybrid system at least ensures a team that attains a perfect record in a series will always see its rating improve.  But it’s a very contrived and counter-intuitive way of achieving this result, and the need to choose arbitrary parameters seems more pervasive than it did under Elo. To get to a comparable result, the ICCs’ system has become both Byzantine and strange.

So in conclusion, I like my Elo system.  I’m going to be maintaining it going forwards and posting periodic updates on my current estimate of the teams in world cricket.  And it will be interesting to see how it performs.

Strange formulae

We’re currently in the process of looking at how the ICC cricket team world ratings are calculated; and we’ve introduced the idea that a team’s overall rating is a rolling, weighted average of its ratings from recent series.  But how is a series rating calculated?  The basic idea is superficially very odd.  Teams score a point for each game they win in a series, a half point for a draw and a bonus point (or half-point) for the overall series result (so a 3-0 win counts 4-0 in points).  A team’s series rating is derived both this result, but adjusted to reflect the overall rating of its opponents going into that series. Beating a strong team gets you more credit: this seems sensible.   The idea gets strange, however, when one looks at exactly how the sums are done. Specifically, a team gets a series rating equal to its opponents’ previous overall rating plus 50, multiplied by the number of points that the team has scored in the series under question, to which you then add its opponents’ overall rating minus 50, multiplied by the number of points the opponents managed, all divided by twice the number of points that were at stake. So, if England played Australia and won a 3 test series 2-1, and Australia had started the series with a rating of 100, England would earn a series rating equal to ((3 x (100 + 50)) + (1 x (100 - 50))) / 4, which works out as 125.  

There are two strange things about this.  Firstly, why does the formula for England’s performance include a factor related to Australia’s score?  If Australia get more points, does England’s series rating go up?  In fact, the answer is no, and indeed, the inclusion of Australia’s score in the formula is arguably just a red herring.  The key issue here is that every point that Australia get is a point England haven’t won.  So if there are N points available in a series and England score x of them, then Austrailia have must have won N - x, and, if Australia's rating at the start of the series was y, the formula for calculating England’s series rating R can be reworked as follows:

R  = ((y + 50) x +(y – 50) (N – x)) / N

R  = (xy + 50x + Ny – 50N +yN – xy + 50x ) / N

R  = (50x + Ny – 50N +50x) / N

R  = (100x / N)  + y – 50

And just to prove that our fundamental equation has not changed, if N = 4, x = 3 and y = 100, we now have:

R = (300 / 4) + 50

R = 125

So, a team's series rating isn’t really dependent on a team’s opponents’ score at all: that was just one way of describing the formula that makes it easy to understand how the rating is calculated at the cost of introducing an unlikely additional pseudo-variable (the opposition score; though if fact, of course, that score cannot vary once N and x are set).  But thinking this through still doesn’t really help us understand why the formula takes the form it does.

In my (Elo-derived) system, an expectated result is calculated for each team using the difference in its own prior rating to that of its opponents, and each team’s rating is subsequently increased or decreased according to how its actual performance differs from expectations.  But there’s no element of calculated expectation in the ICC system.  A team’s series rating will always be +/- 50 of their opponents’ rating at the start of a series, regardless of what result was expected.  If two teams draw, each will get a series rating exactly equal to their opponent’s prior overall rating (and thus, the average rating of the stronger team will go down slightly, and of the weaker team will rise slightly).  Under most circumstances, a winning team will see its rating improve.  Note that although it’s only a side’s opponents whose prior rating goes into the formula, a team’s own prior rating determines the size of the series rating it needs in order to maintains its own average.  Thus if England start with a rating 30 points better than that of Australia, their rating will improve less drastically after a big series win than if they had started the series 30 points behind.  So the system is more Elo-like than might first appear to be the case, although lacking in the elegance of a true Elo system.

But there’s a problem.  The system works if a team starts a series with a rating broadly similar to its opponent’s.  But as we’ve seen, a perfect performance will give a team a series rating of at most 50 points more than its opponent's initial rating; which means, that if you play a team who start more than 50 points behind you, your own average rating will inevitably fall, even if every ball you bowl takes a wicket and every ball you face is hit for six.  With a true Elo system, you can’t expect more than perfection, so a perfect result always increases the rating of a winner, and intuitively, this is correct: the best a side can do is to win the matches that it gets to play. Even if a win was largely expected, the game could have been lost, and winning always adds to a team’s lustre, albeit sometimes not by much.  But in the ICC system, the use of the somewhat arbitrary number of 50 puts a limit on the range of ratings’ differences over which the system can be sensibly applied.  A strong team will be considered less strong in future simply for playing a much weaker team, no matter how comprehensively the weaker team is thrashed.

In fact, there is a mechanism to avoid this problem in the ICC’s system: that’s the subject of the next post.

ICC ratings: first principles

So, now we have to delve into the guts of the ICC system for rating test teams.  And it’s not easy, partly because it’s not clearly described anywhere.  There’s a partial description on this page from which I’ve taken my cues, but I’m going to try and explain things in a different order.

The first point is that the ICC system is not an Elo system.  Instead, it gives each team that plays in each series a series rating; and then calculates a rolling, weighted average of a team’s series ratings.  As we’ve seen, with such an approach one has to decide how recent matches have to be in order to be included, and how to discount older results with respect to new ones.  The ICC’s system starts with an oddity: there are discrete periods (and the weighting of each series depends on the period it falls in), but these are not updated continuously based on time elapsed since the match was played, but annually, on a fixed date each year.  This really is peculiar; a good run just after the intervals are updated will contribute to the average at a higher level for longer than a good run just before the reset.  I can’t see any good justification for this over a continuously rolling system save simplicity (and even then, the point is arguable: is it really more complex, say, to discount the contribution of each series proportionally to the number of days past since they were played?).

The second point is that, as we’ve already noted, the ICC ratings are advanced one series (not one match) at a time. More significantly, they score each series as if the series win was an extra match victory.  Thus, a team which wins a 3 match series 2-1 will be credited as if it had won 3-1, and so on.  Test cricket has historically been played in series between sides, and naturally, each side wants to win each series it plays.  But is a team which wins one series 2-1 and another 1-0 really better than a side that wins 3-0 then loses 0-1?  The answer is yes, if one defines the goal as winning series.  For a ratings system, the real point would be whether giving extra credit to teams that win series increases or decreases the power of the ratings to predict future outcomes. I don’t have the data to answer that.  What I can note is that I could easily adjust my ratings system so that it worked similarly.  By recalculating ratings less often, you get a bit more stability in the system (as we have already seen), but at the price of increasing the lag time it takes before recent shifts in performances are reflected.  But it doesn't seem to me to be a very important difference.

But how is a series rating calculated in the ICC system?  It’s not an Elo method, but there are some similar principles at work (as well as some surprising ones).  These will be the subject of the next post.