

Prognosticate This!

Oh, those ubiquitous experts. Every hockey telecast, every hockey magazine is rife with the opinions of hockey experts. More than anything these experts like to tell you who is going to win, and why. Of course, they are never held accountable when their predictions are wrong.
It’s time to take some experts to task. Let’s compare two groups or prognosticators: one real, and one hypothetical. The real group is the staff of the Hockey News, who publish an annual which contains predicted standings for each conference of the NHL. Presumably, these experts use all of the available information at the time, and put significant thought into predicting the standings. I calculated the coefficients of correlation between the predicted standings and the actual standings at the end of the year, for the past five years. The higher the correlation (which has a maximum value of 1), the more accurate the predictions were. Here are the results:
Year  Correl 
1996/97  0.56 
1997/98  0.42 
1998/99  0.48 
1999/00  0.66 
2000/01  0.82 
Average  0.59 
So the experts actually do quite a good job at predicting the conference standings in the NHL each year. A correlation of 0.6 is considered high, and the experts nearly reach this mark on average. It seems the experts are quite expert after all. But here's the wrinkle: the other (hypothetical) group of prognosticators that I am going to test is a group of welltrained monkeys. The only thing these monkeys are trained to do is predict that the standings this year will look exactly like the standings last year. That is, they will put no thought at all into it, and just assume that the standings will not change whatsoever from the previous year. The results:
Year 
Correl 
1996/97  0.35 
1997/98  0.44 
1998/99  0.60 
1999/00  0.64 
2000/01  0.79 
Average  0.56 
This demonstrates that last year's standings are a very good predictor of this year's standings. This also demonstrates that for all the thought and analysis that hockey experts put into their predictions, they produce results that are no better than not making any predictions at all (a difference in correlation of 0.03 is nowhere near statistically significant).
Now let’s take a slightly different approach. I will test the number of playoff teams each group correctly identifies each year (with an annual maximum of 16, of course). The results are as follows, with the half values resulting from teams being tied in the standings:
Year  Experts  Monkeys 
1996/97  12  10 
1997/98  11  12 
1998/99  11  13 
1999/00  13  13 
2000/01  13.5  14.5 
Total  60.5  62.5 
So the monkeys actually do a little better in predicting playoff teams (they got 78% correct) than the experts (who got 76%), but again, this isn’t statistically significant.
On yet another track, both groups correctly predicted exactly one of the five Stanley Cupwinning teams over this time period.
So what's the point here? The point is that predicting the future is a very difficult thing to do. Also, socalled experts are not even able to make predictions better than monkeys that don't even exist. Really, it shows that anyone, me or you, can use our knowledge of hockey to make predictions that are just as good as any expert paid to think about hockey. Far too much time is taken is trying to predict the future, rather than analyzing the past. Often more effort is expended in discussing who "should" win, rather than who actually did. So remember, the next time you’re listening to an “Insider Report” at what have you, this guy doesn’t know any better than you who’s going to win. That’s the fun of a game; if you knew who was going to win, why would you bother playing?
Please visit Puckerings for more hockey stuff by me
Copyright © 19992010 HockeyZonePlus.com  All Rights Reserved 