Two tests have been played in the last week, bith ending in home victory. For the second time in a row, India saw off the West Indies; while Pakistan, who'd failed to finish off Australia in the their first test, emphatically did so in the second. The effect on my world ratings is as follows:
India 183 +6
South Africa 175
Sri Lanka 156
England 149
New Zealand 134
Australia 129 -32
Pakistan 113 +32
West Indies 1 -6
Bangladesh -93
Zimbabwe -279
Ireland -319
Afghanistan -351
The large correction I'm now making for home advantage means India's expected victory barely moves the needle. Australia, howver, in spite of playing away, were still sufficiently favoured to be the team with higher expectations, and the increase in the k-factor means their defeat delivers them a bad blow, knocking this often mighty country down to an atypically low 6th place in the rankings.
Both series are now complete, so we can compare my rankings to the official ones:
India 116 +1
South Africa 106
England 105
New Zealand 102
Australia 102 -4
Sri Lanka 97
Pakistan 95 +8
West Indies 76 -1
Bangladesh 67
Zimbabwe 2
The ICC system doesn't yet include recent test debutantes Ireland and Afghanistan (it needs more data to be able to include them); is unsymetrical (gains of winning teams are not necessarily matched by equivalent losses of losing sides); and doesn't rate Sri Lanka nearly as highly as my system does. Nonetheless, it still has Australia in a modest 5th position. Australia were strong contenders to challenge for the number one spot, following their Ashes triumph last winter, but a cheating scandal and subsequent suspensions have for now halted their ambitions, and even England now stand ahead of them in both systems. England are next in action in Sri Lanka in November; Zimbabwe visit Bangladesh in the same month.
Saturday, 20 October 2018
Home Advantage, part 2
Recently, I've been playing around with trying to intorduce compensation for home advatage into my rating system. I'd used past results to estimate that I needed to adjust the ratings of the home team by +41 to calculate the probabilities fairly, based on the overall difference between the win/loss record of sides at home and away. But when I tested different adjustments to see which one gave made the ratings as predictive as possible, I found the answer was a relatively massive +74. I don't quite understand that one yet.
Then, with the new adjustment in place, I re-tested to find the optimum k-factor. This also went up - from 32 to 43. So, the ratings become more predictive if you let them consider home advantage, but also if you make them more sensitive to recent results.
And with this system in place, the ratings look as follows, with India already back in 1st position:
India 177
South Africa 175
Australia 161
Sri Lanka 156
England 149
New Zealand 134
Pakistan 81
West Indies 7
Bangladesh -93
Zimbabwe -279
Ireland -319
Afghanistan -351
That's enough adjusmtent for now; let's see how things change in the ratings going forward.
Then, with the new adjustment in place, I re-tested to find the optimum k-factor. This also went up - from 32 to 43. So, the ratings become more predictive if you let them consider home advantage, but also if you make them more sensitive to recent results.
And with this system in place, the ratings look as follows, with India already back in 1st position:
India 177
South Africa 175
Australia 161
Sri Lanka 156
England 149
New Zealand 134
Pakistan 81
West Indies 7
Bangladesh -93
Zimbabwe -279
Ireland -319
Afghanistan -351
That's enough adjusmtent for now; let's see how things change in the ratings going forward.
Friday, 12 October 2018
Home Advantage
In the previous post, I explored how one might compensate for home advantage in the rating system. I measured actual home advantage (which was surprisingly large) and adjusted the ratings; but the new ratings turned out to be worse predictors than the old! How come? The answer turns out to be a coding bug! Measured properly, home advantage is much less than I asserted: the home team's expected yield from a game is 0.573 points, not 0.775; and that corresponds to a ratings advantage of just 42.5 points, not the massive 178.5 points I was applying yesterday. And when I use the correct factor, the predictivity of the ratings improves, with a cumulative error over all past test matches down to 1470 from 1510. So what do the adjusted ratings look like? Here's the answer:
South Africa 174
India 174
Australia 165
England 144
Sri Lanka 133
New Zealand 133
Pakistan 68
West Indies 5
Bangladesh -100
Zimbabwe -264
Ireland -301
Afghanistan -331
With this adjustment factor, Pakistan's recent draw with Australia gained Pakistan (and lost Australia) just 4 points: the prediction of a better results for the visitors (with their higher rating) partially offset by their home advantage. South Africa have been top since September; prior to that, India were top from August 2016. This almost exactly reflects the "real" ratings, wherea similar change occured, albeit one match earlier.
So how do the two ranking systems differ in the longer term? As we've discussed, the gap (in points) between any two teams has meaning, but the absolute scores less so. So one statistic we've looked at in the past is to ask, for every team that has ever held first place, what was the moment when it held that position by the greatest margin over the team in second place? Using the old rankings, our results were like this (showing also the period for which the team held the number 1 ranking):
Dec 1999 Nov 2009 Australia Dec 2007 222
Mar 1885 Jan 1898 England Aug 1890 198
Sep 1983 Dec 1991 West Indies Jul 1986 172
Aug 2016 Aug 2018 India Aug 2017 123
Nov 2012 Nov 2015 South Africa Mar 2013 94
Jan 1983 Feb 1983 Pakistan Jan 1983 22
Under the new system, the results are as follows:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 1884 Feb 1898 England Aug 1890 195
Jun 1980 Jan 1992 West Indies Nov 1984 170
Nov 2012 Nov 2015 South Africa Mar 2013 109
Aug 2016 Sep 2018 India Aug 2017 104
The timing of the peaks are nearly identical, although the periods each side's dominance lasted differ. Most notably, the West Indies are rated as having been the world's best team for over a decade in the 1980s and early 1990s, and Pakistan's brief period marginally on top of the old rankings (the only ever time they were rated number one) has disappeared. It's notable that India's recent peak is marked down, however; this reflects the fact it was based primarily on outstanding results at home.
Another question we can ask is if the new rankings more stably define teams in the number one spot. This isn't necessarily a mark of improvement - we have no absolute standard to say who really was the best team in the world at a given point in time, and how often this has changed. In fact, there's almost no difference overall in terms of length of tenure at the top: the old system has had 93 different periods of leadership, and the new system 92.
Lastly, let's look at the what's happened since the turn of the millenium. Firstly, under the old system:
Dec 1999 Nov 2009 Australia Dec 2007 222
Nov 2009 Jan 2010 India Dec 2009 31
Jan 2010 Jan 2010 Australia Jan 2010 1
Jan 2010 Feb 2010 India Feb 2010 3
Feb 2010 Feb 2010 Australia Feb 2010 16
Feb 2010 Mar 2010 India Feb 2010 1
Mar 2010 Aug 2010 Australia Jul 2010 40
Aug 2010 Jul 2011 India Oct 2010 59
Jul 2011 Feb 2012 England Dec 2011 99
Feb 2012 Nov 2012 Australia Mar 2012 33
Nov 2012 Nov 2015 South Africa Mar 2013 94
Nov 2015 Aug 2016 Australia Feb 2016 51
Aug 2016 Aug 2018 India Aug 2017 123
Aug 2018 - South Africa Sep 2018 13
And under the new one:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 2008 Jan 2009 South Africa Dec 2008 2
Jan 2009 Dec 2009 Australia Mar 2009 68
Dec 2009 Dec 2009 India Dec 2009 10
Dec 2009 Oct 2010 Australia Jul 2010 56
Oct 2010 May 2011 India Oct 2010 48
May 2011 Jun 2011 England May 2011 5
Jun 2011 Jul 2011 India Jun 2011 19
Jul 2011 Feb 2012 England Dec 2011 94
Feb 2012 Nov 2012 Australia Jun 2012 27
Nov 2012 Nov 2015 South Africa Mar 2013 109
Nov 2015 Nov 2015 Australia Nov 2015 1
Nov 2015 Nov 2015 South Africa Nov 2015 6
Nov 2015 Aug 2016 Australia Feb 2016 45
Aug 2016 Sep 2018 India Aug 2017 104
Sep 2018 - South Africa Sep 2017 5
And if we ignore brief reigns of less than 2 months, the succession is Australia; India; Australia, India; England, Australia, South Africa, Australia, India, South Africa under both systems. So although the transition periods from one dominant team to another have been handled differently, the stories are remarkably similar overall.
Finally, what's happened in the official ratings during this period? Data is available from June 2003 onwards:
Jun 2003 Jul 2009 Australia
Aug 2009 Nov 2009 South Africa
Nov 2009 Aug 2011 India
Aug 2011 Aug 2012 England
Aug 2012 May 2014 South Africa
May 2014 Jul 2014 Australia
Jul 2014 Jan 2016 South Africa
Jan 2016 Feb 2016 India
Feb 2016 Aug 2016 Australia
Aug 2016 Aug 2016 India
Aug 2016 Oct 2016 Pakistan
Oct 2016 - India
Overall, the offical ratings are more stable than mine, but also slower to react to recent results - India are still rated number one. Ignoring very short periods, both my systems have Australia on top throughout most of 2010, in between two periods of India dominance, but this was a single period of Indian ascendency according to the ICC; also note that the ICC briefly had Pakistan in the top spot, which under my old system they have last held in 1983 and under my new system, never. But overall there's a broad concordance.
But I like my new system, with it's improved predictive power, and compensation for the skew that comes with playing home or away. I think I'm going to stick with it from now on.
South Africa 174
India 174
Australia 165
England 144
Sri Lanka 133
New Zealand 133
Pakistan 68
West Indies 5
Bangladesh -100
Zimbabwe -264
Ireland -301
Afghanistan -331
With this adjustment factor, Pakistan's recent draw with Australia gained Pakistan (and lost Australia) just 4 points: the prediction of a better results for the visitors (with their higher rating) partially offset by their home advantage. South Africa have been top since September; prior to that, India were top from August 2016. This almost exactly reflects the "real" ratings, wherea similar change occured, albeit one match earlier.
So how do the two ranking systems differ in the longer term? As we've discussed, the gap (in points) between any two teams has meaning, but the absolute scores less so. So one statistic we've looked at in the past is to ask, for every team that has ever held first place, what was the moment when it held that position by the greatest margin over the team in second place? Using the old rankings, our results were like this (showing also the period for which the team held the number 1 ranking):
Dec 1999 Nov 2009 Australia Dec 2007 222
Mar 1885 Jan 1898 England Aug 1890 198
Sep 1983 Dec 1991 West Indies Jul 1986 172
Aug 2016 Aug 2018 India Aug 2017 123
Nov 2012 Nov 2015 South Africa Mar 2013 94
Jan 1983 Feb 1983 Pakistan Jan 1983 22
Under the new system, the results are as follows:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 1884 Feb 1898 England Aug 1890 195
Jun 1980 Jan 1992 West Indies Nov 1984 170
Nov 2012 Nov 2015 South Africa Mar 2013 109
Aug 2016 Sep 2018 India Aug 2017 104
The timing of the peaks are nearly identical, although the periods each side's dominance lasted differ. Most notably, the West Indies are rated as having been the world's best team for over a decade in the 1980s and early 1990s, and Pakistan's brief period marginally on top of the old rankings (the only ever time they were rated number one) has disappeared. It's notable that India's recent peak is marked down, however; this reflects the fact it was based primarily on outstanding results at home.
Another question we can ask is if the new rankings more stably define teams in the number one spot. This isn't necessarily a mark of improvement - we have no absolute standard to say who really was the best team in the world at a given point in time, and how often this has changed. In fact, there's almost no difference overall in terms of length of tenure at the top: the old system has had 93 different periods of leadership, and the new system 92.
Lastly, let's look at the what's happened since the turn of the millenium. Firstly, under the old system:
Dec 1999 Nov 2009 Australia Dec 2007 222
Nov 2009 Jan 2010 India Dec 2009 31
Jan 2010 Jan 2010 Australia Jan 2010 1
Jan 2010 Feb 2010 India Feb 2010 3
Feb 2010 Feb 2010 Australia Feb 2010 16
Feb 2010 Mar 2010 India Feb 2010 1
Mar 2010 Aug 2010 Australia Jul 2010 40
Aug 2010 Jul 2011 India Oct 2010 59
Jul 2011 Feb 2012 England Dec 2011 99
Feb 2012 Nov 2012 Australia Mar 2012 33
Nov 2012 Nov 2015 South Africa Mar 2013 94
Nov 2015 Aug 2016 Australia Feb 2016 51
Aug 2016 Aug 2018 India Aug 2017 123
Aug 2018 - South Africa Sep 2018 13
And under the new one:
Dec 1999 Dec 2008 Australia Jan 2008 223
Dec 2008 Jan 2009 South Africa Dec 2008 2
Jan 2009 Dec 2009 Australia Mar 2009 68
Dec 2009 Dec 2009 India Dec 2009 10
Dec 2009 Oct 2010 Australia Jul 2010 56
Oct 2010 May 2011 India Oct 2010 48
May 2011 Jun 2011 England May 2011 5
Jun 2011 Jul 2011 India Jun 2011 19
Jul 2011 Feb 2012 England Dec 2011 94
Feb 2012 Nov 2012 Australia Jun 2012 27
Nov 2012 Nov 2015 South Africa Mar 2013 109
Nov 2015 Nov 2015 Australia Nov 2015 1
Nov 2015 Nov 2015 South Africa Nov 2015 6
Nov 2015 Aug 2016 Australia Feb 2016 45
Aug 2016 Sep 2018 India Aug 2017 104
Sep 2018 - South Africa Sep 2017 5
And if we ignore brief reigns of less than 2 months, the succession is Australia; India; Australia, India; England, Australia, South Africa, Australia, India, South Africa under both systems. So although the transition periods from one dominant team to another have been handled differently, the stories are remarkably similar overall.
Finally, what's happened in the official ratings during this period? Data is available from June 2003 onwards:
Jun 2003 Jul 2009 Australia
Aug 2009 Nov 2009 South Africa
Nov 2009 Aug 2011 India
Aug 2011 Aug 2012 England
Aug 2012 May 2014 South Africa
May 2014 Jul 2014 Australia
Jul 2014 Jan 2016 South Africa
Jan 2016 Feb 2016 India
Feb 2016 Aug 2016 Australia
Aug 2016 Aug 2016 India
Aug 2016 Oct 2016 Pakistan
Oct 2016 - India
Overall, the offical ratings are more stable than mine, but also slower to react to recent results - India are still rated number one. Ignoring very short periods, both my systems have Australia on top throughout most of 2010, in between two periods of India dominance, but this was a single period of Indian ascendency according to the ICC; also note that the ICC briefly had Pakistan in the top spot, which under my old system they have last held in 1983 and under my new system, never. But overall there's a broad concordance.
But I like my new system, with it's improved predictive power, and compensation for the skew that comes with playing home or away. I think I'm going to stick with it from now on.
Thursday, 11 October 2018
Cricket World Ratings 11th October 2018
Two series of test cricket have got underway in the last week. Back at home, following their defeat in England, India smashed the West Indies. Australia, meanwhile, dug themselves out of a hole to secure a draw in their match in the United Arab Emirates, which is still serving as a temporary home venue for Pakistan. The effect on my rankings is as follows:
South Africa 179
India 168 +9
Australia 159 -7
England 152
New Zealand 144
Sri Lanka 125
Pakistan 47 +7
West Indies 6 -9
Bangladesh -99
Zimbabwe -260
Ireland -295
Afghanistan -325
But one feature of my rankings is that they take no account of home advantage. Teams with a winning streak at home always move up, even if this is only to be expected. To draw in the U.A.E is arguably more of an achievement for Australia than Pakistan, but Australia have been penalised as the previously higher ranking team.
In the earliest posts on this blog, I explored how my ranking system works, with the ratings gap between the two teams used to predict a result, and then teams credited or penalised according to how much the actual results differ from the prediction. But the default prediction takes no account of home advantage. Can we change that?
Well, the first thing to do is to calculate the expected result for home sides. We've been working on an assumption it's 0.5 (scoring 1 for a win and 0.5 for a draw). In fact, it's much higher than that - over all test matches played, the home side averages 0.775 points per game, a much bigger skew than I expected. So how can we take advantage of this in the ratings system? Amazingly, if there was no home advantage, a team would need to have a ratings gap of 178.5 points in its favour to expect such an outcome.
So if we temporarily shift the ratings of each team by such a margin to calculate the expected output we should correct for home advantage. And we get a rather different set of scores:
India 235
Australia 224
South Africa 203
Sri Lanka 165
Pakistan 160
England 141
New Zealand 113
West Indies -6
Bangladesh -142
Zimbabwe -323
Ireland -366
Afghanistan -403
Most notably, India suffered in my usual ratings for losing at home to England, who gained notably; but in this revised system, England get fewer plaudits, and India suffer less punishment. And Australia actually gained a little - rather than losing - from drawing in Pakistan.
One question we can ask is, which is the better predictor, the old method or the new one? One can calculate this by summing the difference of actual results from predicted ones (the same method we used to calculate the optimal k-factor). Rather disappointingly, it makes almost no difference - the sum of the errors actually goes up a little, from 1510 to 1512.
Might there be some other weighting for home advantage that will actually minimise the error? This seems to be worth trying to calculate.Stay tuned...
South Africa 179
India 168 +9
Australia 159 -7
England 152
New Zealand 144
Sri Lanka 125
Pakistan 47 +7
West Indies 6 -9
Bangladesh -99
Zimbabwe -260
Ireland -295
Afghanistan -325
But one feature of my rankings is that they take no account of home advantage. Teams with a winning streak at home always move up, even if this is only to be expected. To draw in the U.A.E is arguably more of an achievement for Australia than Pakistan, but Australia have been penalised as the previously higher ranking team.
In the earliest posts on this blog, I explored how my ranking system works, with the ratings gap between the two teams used to predict a result, and then teams credited or penalised according to how much the actual results differ from the prediction. But the default prediction takes no account of home advantage. Can we change that?
Well, the first thing to do is to calculate the expected result for home sides. We've been working on an assumption it's 0.5 (scoring 1 for a win and 0.5 for a draw). In fact, it's much higher than that - over all test matches played, the home side averages 0.775 points per game, a much bigger skew than I expected. So how can we take advantage of this in the ratings system? Amazingly, if there was no home advantage, a team would need to have a ratings gap of 178.5 points in its favour to expect such an outcome.
So if we temporarily shift the ratings of each team by such a margin to calculate the expected output we should correct for home advantage. And we get a rather different set of scores:
India 235
Australia 224
South Africa 203
Sri Lanka 165
Pakistan 160
England 141
New Zealand 113
West Indies -6
Bangladesh -142
Zimbabwe -323
Ireland -366
Afghanistan -403
Most notably, India suffered in my usual ratings for losing at home to England, who gained notably; but in this revised system, England get fewer plaudits, and India suffer less punishment. And Australia actually gained a little - rather than losing - from drawing in Pakistan.
One question we can ask is, which is the better predictor, the old method or the new one? One can calculate this by summing the difference of actual results from predicted ones (the same method we used to calculate the optimal k-factor). Rather disappointingly, it makes almost no difference - the sum of the errors actually goes up a little, from 1510 to 1512.
Might there be some other weighting for home advantage that will actually minimise the error? This seems to be worth trying to calculate.Stay tuned...
Subscribe to:
Posts (Atom)