ICC ratings: first principles

So, now we have to delve into the guts of the ICC system for rating test teams.  And it’s not easy, partly because it’s not clearly described anywhere.  There’s a partial description on this page from which I’ve taken my cues, but I’m going to try and explain things in a different order.

The first point is that the ICC system is not an Elo system.  Instead, it gives each team that plays in each series a series rating; and then calculates a rolling, weighted average of a team’s series ratings.  As we’ve seen, with such an approach one has to decide how recent matches have to be in order to be included, and how to discount older results with respect to new ones.  The ICC’s system starts with an oddity: there are discrete periods (and the weighting of each series depends on the period it falls in), but these are not updated continuously based on time elapsed since the match was played, but annually, on a fixed date each year.  This really is peculiar; a good run just after the intervals are updated will contribute to the average at a higher level for longer than a good run just before the reset.  I can’t see any good justification for this over a continuously rolling system save simplicity (and even then, the point is arguable: is it really more complex, say, to discount the contribution of each series proportionally to the number of days past since they were played?).

The second point is that, as we’ve already noted, the ICC ratings are advanced one series (not one match) at a time. More significantly, they score each series as if the series win was an extra match victory.  Thus, a team which wins a 3 match series 2-1 will be credited as if it had won 3-1, and so on.  Test cricket has historically been played in series between sides, and naturally, each side wants to win each series it plays.  But is a team which wins one series 2-1 and another 1-0 really better than a side that wins 3-0 then loses 0-1?  The answer is yes, if one defines the goal as winning series.  For a ratings system, the real point would be whether giving extra credit to teams that win series increases or decreases the power of the ratings to predict future outcomes. I don’t have the data to answer that.  What I can note is that I could easily adjust my ratings system so that it worked similarly.  By recalculating ratings less often, you get a bit more stability in the system (as we have already seen), but at the price of increasing the lag time it takes before recent shifts in performances are reflected.  But it doesn't seem to me to be a very important difference.

But how is a series rating calculated in the ICC system?  It’s not an Elo method, but there are some similar principles at work (as well as some surprising ones).  These will be the subject of the next post.