In the previous post, I explored how one might compensate for home advantage in the rating system. I measured actual home advantage (which was surprisingly large) and adjusted the ratings; but the new ratings turned out to be worse predictors than the old! How come? The answer turns out to be a coding bug! Measured properly, home advantage is much less than I asserted: the home team's expected yield from a game is 0.573 points, not 0.775; and that corresponds to a ratings advantage of just 42.5 points, not the massive 178.5 points I was applying yesterday. And when I use the correct factor, the predictivity of the ratings improves, with a cumulative error over all past test matches down to 1470 from 1510. So what do the adjusted ratings look like? Here's the answer:
South Africa 174
India 174
Australia 165
England 144
Sri Lanka 133
New Zealand 133
Pakistan 68
West Indies 5
Bangladesh -100
Zimbabwe -264
Ireland -301
Afghanistan -331
With this adjustment factor, Pakistan's recent draw with Australia gained Pakistan (and lost Australia) just 4 points: the prediction of a better results for the visitors (with their higher rating) partially offset by their home advantage. South Africa have been top since September; prior to that, India were top from August 2016. This almost exactly reflects the "real" ratings, wherea similar change occured, albeit one match earlier.
So how do the two ranking systems differ in the longer term? As we've discussed, the gap (in points) between any two teams has meaning, but the absolute scores less so. So one statistic we've looked at in the past is to ask, for every team that has ever held first place, what was the moment when it held that position by the greatest margin over the team in second place? Using the old rankings, our results were like this (showing also the period for which the team held the number 1 ranking):
Dec 1999 Nov 2009 Australia Dec 2007 222
Mar 1885 Jan 1898 England Aug 1890 198
Sep 1983 Dec 1991 West Indies Jul 1986 172
Aug 2016 Aug 2018 India Aug 2017 123
Nov 2012 Nov 2015 South Africa Mar 2013 94
Jan 1983 Feb 1983 Pakistan Jan 1983 22
Under the new system, the results are as follows:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 1884 Feb 1898 England Aug 1890 195
Jun 1980 Jan 1992 West Indies Nov 1984 170
Nov 2012 Nov 2015 South Africa Mar 2013 109
Aug 2016 Sep 2018 India Aug 2017 104
The timing of the peaks are nearly identical, although the periods each side's dominance lasted differ. Most notably, the West Indies are rated as having been the world's best team for over a decade in the 1980s and early 1990s, and Pakistan's brief period marginally on top of the old rankings (the only ever time they were rated number one) has disappeared. It's notable that India's recent peak is marked down, however; this reflects the fact it was based primarily on outstanding results at home.
Another question we can ask is if the new rankings more stably define teams in the number one spot. This isn't necessarily a mark of improvement - we have no absolute standard to say who really was the best team in the world at a given point in time, and how often this has changed. In fact, there's almost no difference overall in terms of length of tenure at the top: the old system has had 93 different periods of leadership, and the new system 92.
Lastly, let's look at the what's happened since the turn of the millenium. Firstly, under the old system:
Dec 1999 Nov 2009 Australia Dec 2007 222
Nov 2009 Jan 2010 India Dec 2009 31
Jan 2010 Jan 2010 Australia Jan 2010 1
Jan 2010 Feb 2010 India Feb 2010 3
Feb 2010 Feb 2010 Australia Feb 2010 16
Feb 2010 Mar 2010 India Feb 2010 1
Mar 2010 Aug 2010 Australia Jul 2010 40
Aug 2010 Jul 2011 India Oct 2010 59
Jul 2011 Feb 2012 England Dec 2011 99
Feb 2012 Nov 2012 Australia Mar 2012 33
Nov 2012 Nov 2015 South Africa Mar 2013 94
Nov 2015 Aug 2016 Australia Feb 2016 51
Aug 2016 Aug 2018 India Aug 2017 123
Aug 2018 - South Africa Sep 2018 13
And under the new one:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 2008 Jan 2009 South Africa Dec 2008 2
Jan 2009 Dec 2009 Australia Mar 2009 68
Dec 2009 Dec 2009 India Dec 2009 10
Dec 2009 Oct 2010 Australia Jul 2010 56
Oct 2010 May 2011 India Oct 2010 48
May 2011 Jun 2011 England May 2011 5
Jun 2011 Jul 2011 India Jun 2011 19
Jul 2011 Feb 2012 England Dec 2011 94
Feb 2012 Nov 2012 Australia Jun 2012 27
Nov 2012 Nov 2015 South Africa Mar 2013 109
Nov 2015 Nov 2015 Australia Nov 2015 1
Nov 2015 Nov 2015 South Africa Nov 2015 6
Nov 2015 Aug 2016 Australia Feb 2016 45
Aug 2016 Sep 2018 India Aug 2017 104
Sep 2018 - South Africa Sep 2017 5
And if we ignore brief reigns of less than 2 months, the succession is Australia; India; Australia, India; England, Australia, South Africa, Australia, India, South Africa under both systems. So although the transition periods from one dominant team to another have been handled differently, the stories are remarkably similar overall.
Finally, what's happened in the official ratings during this period? Data is available from June 2003 onwards:
Jun 2003 Jul 2009 Australia
Aug 2009 Nov 2009 South Africa
Nov 2009 Aug 2011 India
Aug 2011 Aug 2012 England
Aug 2012 May 2014 South Africa
May 2014 Jul 2014 Australia
Jul 2014 Jan 2016 South Africa
Jan 2016 Feb 2016 India
Feb 2016 Aug 2016 Australia
Aug 2016 Aug 2016 India
Aug 2016 Oct 2016 Pakistan
Oct 2016 - India
Overall, the offical ratings are more stable than mine, but also slower to react to recent results - India are still rated number one. Ignoring very short periods, both my systems have Australia on top throughout most of 2010, in between two periods of India dominance, but this was a single period of Indian ascendency according to the ICC; also note that the ICC briefly had Pakistan in the top spot, which under my old system they have last held in 1983 and under my new system, never. But overall there's a broad concordance.
But I like my new system, with it's improved predictive power, and compensation for the skew that comes with playing home or away. I think I'm going to stick with it from now on.
No comments:
Post a Comment