I wrote an article just over a year on NM that tried to explain my hesitation when it came to advanced stats (nerds say analytics) and their role in a fan’s understanding of the game.
I then went against my own train of thought and started dabbling in the world of hockey analytics (said it correctly there), and created a stat of my own that looked at the conversion of Corsi chances. (Articles are here, here and here.) I had others check my work to make sure that the numbers and theory made sense. I even sent out my work to some of the "big wigs" of analytics and asked them if my stat had any use. I didn’t get a straight answer in the replies though. One reply stated that my stat only had value if had the ability to predict future goals.
At that precise moment I had my moment of doubt about analytics come back and hit me harder than an Edler slapshot hitting the glass behind the net. Analytic models are now geared towards predicting future performance based upon past actions. The models are still not at the point that you can predict individual game performance, but instead analysts will try to predict next year’s total player and team performance. There is enough game data available to create reliable sample sizes to predict any stat.
Now I wasn’t completely surprised by this, as over the past few years there have been quite a few hires of analysts by NHL teams. The NHL re-invented the wheel by re-naming all known advanced stats on their website and posting them for to see. So the league in general has found value in these stats. And people have created a demand for their services in creating and analyzing stats for NHL teams. But I wonder how many GM’s actually believe that these models can predict future performance. But more importantly, how accurate do these analysts have to be with their predictions to keep their job? And while we have seen people hired, we have yet to see the first round of firings. Analytics is still in its honeymoon stage with teams and there is probably a long leash involved. But once one team pulls that leash tight I think there will be a lot of people in the hockey community that will once against ask questions about analytics and prediction.
I want to point that I am not against analytics, but I do wonder what the allowable percentage of error will be when it comes to predictive models. "Hey, I predicted CorsiFor% within 2%, close enough….please don’t fire me." I think once that is determined there will be a lot less people peddling their statistical wares on twitter. But who knows, there will still be a lot of people like on twitter who like analytics as a postmortem tool for players and teams. No prediction required.
There is one area in analytics that really hasn’t been explored and it is probably the most important indicator of performance, more important than team stats and more important than individual stats. I think the next big area of study will be coaching. Some will say that team stats are coaching stats and that is true to some extent. The choices coaches make lead to player’s actions and team results. But not much has been done in the field of studying coaching decisions when it comes to the offensive/defensive systems they run and the results. Nor has there been any in depth study of player situational deployment and how it affected player stats. I’m sure that any study of coaching and player results will lead to the chicken/egg discussion. Does player result lead to coaching decisions or do coaching decisions lead to player results?
I think I might take a break from stats. My head hurts. All the predictive models about the Canucks and their players and how they suck can lead one to ask, "Why play the games then, if everyone says they will lose?" Predictive stats can really suck the excitement out of an upcoming season if you choose to believe them over the reality that games have to be played to determine outcomes.
I believe there is a 100% chance of a beer after I finish writing this.