When Strong Meets Weak

In these posts, I’m reviewing the ICC system for rating test cricket teams, having previously outlined my own, Elo-style system.  In essence, the ICC’s system rates each team using a weighted, rolling average of recent performances; the strength of those performances is measured first by the result of individual series, but also by the strength of the opposition it has played against.  But this means that a strong team might see its rating doomed to fall no matter what margin it beats a weak team by, if that other team is sufficiently poor.  It seems wrong that a team’s rating could drop after a perfect performance. How do the ICC deal with this?

Well, if two teams in the ICC ratings have a ratings gap of over 40 points, the system changes.  Instead of assessing the teams based on their opponents’ pre-series rating, the teams get credit based on their own.  The stronger team’s series rating, in this case, equals their own rating plus ten multiplied by the number of series points that team attains, plus their own rating minus ninety multiplied by the number of points won by their opponent.  For the weaker team, the same formula applies in reverse (i.e. its own rating is increased by 90 then multiplied by its series points, and then  added to its opponents’ series points multiplied by 10 less than its own rating).

Superficially, this is bizarre.  It’s common sense to say that it’s a greater achievement to beat a strong team than a weak one; but under this system, it’s a greater achievement to simply be a strong team: a strong team will get more credit for the same result than a weaker one playing the same opponent.  Secondly, we now have two different systems for calculating a team’s series rating, and a somewhat arbitrary cut-off for moving from one system to another (it’s not accidental, however, that the cut-off is something around 40, though, because it's at ratings differences of over 50 that teams could record a perfect series record and still find themselves losing ground if the system hadn't changed.

And what’s with this strange 90-10 situation?  Basically, it means that the stronger team needs to collect at least 90% of the series points on offer in order to improve its situation (the weaker team, by contrast, needs just 10% of the available points).  So there are traces of Elo in here, but in effect, we now have with a fixed expected value of 0.9 once the ratings gap between two sides exceeds 40. This hybrid system at least ensures a team that attains a perfect record in a series will always see its rating improve.  But it’s a very contrived and counter-intuitive way of achieving this result, and the need to choose arbitrary parameters seems more pervasive than it did under Elo. To get to a comparable result, the ICCs’ system has become both Byzantine and strange.

So in conclusion, I like my Elo system.  I’m going to be maintaining it going forwards and posting periodic updates on my current estimate of the teams in world cricket.  And it will be interesting to see how it performs.