Strange formulae

We’re currently in the process of looking at how the ICC cricket team world ratings are calculated; and we’ve introduced the idea that a team’s overall rating is a rolling, weighted average of its ratings from recent series.  But how is a series rating calculated?  The basic idea is superficially very odd.  Teams score a point for each game they win in a series, a half point for a draw and a bonus point (or half-point) for the overall series result (so a 3-0 win counts 4-0 in points).  A team’s series rating is derived both this result, but adjusted to reflect the overall rating of its opponents going into that series. Beating a strong team gets you more credit: this seems sensible.   The idea gets strange, however, when one looks at exactly how the sums are done. Specifically, a team gets a series rating equal to its opponents’ previous overall rating plus 50, multiplied by the number of points that the team has scored in the series under question, to which you then add its opponents’ overall rating minus 50, multiplied by the number of points the opponents managed, all divided by twice the number of points that were at stake. So, if England played Australia and won a 3 test series 2-1, and Australia had started the series with a rating of 100, England would earn a series rating equal to ((3 x (100 + 50)) + (1 x (100 - 50))) / 4, which works out as 125.  

There are two strange things about this.  Firstly, why does the formula for England’s performance include a factor related to Australia’s score?  If Australia get more points, does England’s series rating go up?  In fact, the answer is no, and indeed, the inclusion of Australia’s score in the formula is arguably just a red herring.  The key issue here is that every point that Australia get is a point England haven’t won.  So if there are N points available in a series and England score x of them, then Austrailia have must have won N - x, and, if Australia's rating at the start of the series was y, the formula for calculating England’s series rating R can be reworked as follows:

R  = ((y + 50) x +(y – 50) (N – x)) / N

R  = (xy + 50x + Ny – 50N +yN – xy + 50x ) / N

R  = (50x + Ny – 50N +50x) / N

R  = (100x / N)  + y – 50

And just to prove that our fundamental equation has not changed, if N = 4, x = 3 and y = 100, we now have:

R = (300 / 4) + 50

R = 125

So, a team's series rating isn’t really dependent on a team’s opponents’ score at all: that was just one way of describing the formula that makes it easy to understand how the rating is calculated at the cost of introducing an unlikely additional pseudo-variable (the opposition score; though if fact, of course, that score cannot vary once N and x are set).  But thinking this through still doesn’t really help us understand why the formula takes the form it does.

In my (Elo-derived) system, an expectated result is calculated for each team using the difference in its own prior rating to that of its opponents, and each team’s rating is subsequently increased or decreased according to how its actual performance differs from expectations.  But there’s no element of calculated expectation in the ICC system.  A team’s series rating will always be +/- 50 of their opponents’ rating at the start of a series, regardless of what result was expected.  If two teams draw, each will get a series rating exactly equal to their opponent’s prior overall rating (and thus, the average rating of the stronger team will go down slightly, and of the weaker team will rise slightly).  Under most circumstances, a winning team will see its rating improve.  Note that although it’s only a side’s opponents whose prior rating goes into the formula, a team’s own prior rating determines the size of the series rating it needs in order to maintains its own average.  Thus if England start with a rating 30 points better than that of Australia, their rating will improve less drastically after a big series win than if they had started the series 30 points behind.  So the system is more Elo-like than might first appear to be the case, although lacking in the elegance of a true Elo system.

But there’s a problem.  The system works if a team starts a series with a rating broadly similar to its opponent’s.  But as we’ve seen, a perfect performance will give a team a series rating of at most 50 points more than its opponent's initial rating; which means, that if you play a team who start more than 50 points behind you, your own average rating will inevitably fall, even if every ball you bowl takes a wicket and every ball you face is hit for six.  With a true Elo system, you can’t expect more than perfection, so a perfect result always increases the rating of a winner, and intuitively, this is correct: the best a side can do is to win the matches that it gets to play. Even if a win was largely expected, the game could have been lost, and winning always adds to a team’s lustre, albeit sometimes not by much.  But in the ICC system, the use of the somewhat arbitrary number of 50 puts a limit on the range of ratings’ differences over which the system can be sensibly applied.  A strong team will be considered less strong in future simply for playing a much weaker team, no matter how comprehensively the weaker team is thrashed.

In fact, there is a mechanism to avoid this problem in the ICC’s system: that’s the subject of the next post.