Today, Bill Meltzer wrote in his “Musings” post that advanced stats take away from the big picture. Needless to say, he doesn’t understand what he’s arguing against. That doesn’t mean he’s wrong, mind you. People are more than welcome to dismiss the “advanced statistics” in favor of what they see. But if you’re going to criticize the other side, you need to first understand the other side.
Now, I like Bill Meltzer. He’s a good writer with a long history as a good writer. He writes for the Flyers official website. He wrote a fantastic book about Pelle Lindbergh. He even writes on his blog stories most mainstream writers wouldn’t dare to touch, let alone research. He deserves respect for all of that.
And this isn’t meant to be an angry rant against Bill, so I hope it doesn’t come off like that. But his post is just incorrect. Jump for the respectful retort, blockquote style.
For example, one of the big trends has been to use the number of shots and attempted shots by the competing teams to measure the effectiveness of particular players. But these stats fail to look at bigger picture here: The team that attempts/achieves more shots on goal is not necessarily the team than wins, and the players who have the biggest positive differential for their team when they are on the ice are not necessarily the most effective players.
First of all, these stats are not perfect. This is obvious. They need context. The same is true of every stat, ever existed. Total points need context. Goals need context. Runs batted in need context. Home runs need context. Plus/Minus needs to be thrown away.
Second, people who advocate for advanced stats are huge proponents of context. Nobody should use raw Corsi scores because you would have Manny Malhotra as the 297th best player in the NHL. You have to adjust for team factors, so use CorsiRel. You have to look at where that player started his shifts. You have to look at who that player played against. These are all things people do with their eyes, except advanced statistics have objective proof of these things.
There is context. Advanced statistics requires context, just like any sort of statistics, just like any sort of personal observations. Did you watch the game with a fan of the opposing team? Did you watch the game while eating? At a bar? Were you looking for a certain player the whole time? Did you enter the contest already thinking player X is great/terrible?
Nobody is against context, except in the minds of people who hate advanced statistics. It’s simply not true.
Bruce McCurdy recently authored a tremendous research-based blog on the Edmonton Journal site that shows just how frequently in today’s NHL game that the team that outshoots its opponent is one that winds up losing the game.
Bill goes on for three paragraphs about how teams outshoot their opponents and lose. Yes, this is called score effects. Us advanced statistics proponents already know all about this, and that’s why we use CorsiTied or CorsiClose, when available. This is nothing new, and you aren’t explaining something we don’t already know.
When teams are winning, they stop focusing on offense. When teams are losing, they not only have the puck more often, and in the offensive zone more often, but they are more concerned with throwing the puck at the net and trying to force one in. This is something every person who watches hockey knows, so why is it assumed advanced statisticians don’t already know this?
We do.
To me, what some of the advanced stats fail to consider is the style that a coach wants his team to employ and the willingness of particular players to embrace their assigned roles with the system.
Coaching style and willingness to embrace their roles. First, what does that look like? If there’s a coaching style that is more offensive, wouldn’t that be similar to what the Capitals had two years ago? That showed up in the stats. The willingness of particular players to embrace their roles… how do we even know if it’s a “willingness” or an “effectiveness”? How does anybody account for that, whether they use their eyes or use stats?
Was Nik Zherdev unwilling to embrace his role with the Flyers, or was he just not good in the role that was assigned to him? If it’s a knock against stats that they don’t consider that element, I ask how anybody with just their two eyes can take that into account.
It’s not so much the quantity of shots a team generates or gives up. It’s the quality of the scoring chances that matter the most.
First, aren’t scoring chances an advanced statistic? Seeing as how they aren’t published, and only a half-dozen or so bloggers are tracking them, how is this an argument against advanced statistics?
Second, Corsi correlates very, very strongly with scoring chances. It’s been proven. But somehow scoring chances are an argument against Corsi? Bill normally does his homework, but here, he didn’t.
The shot totals were the least telling or important aspect of the game. As it also turned out, goaltending was a non-factor in the outcome.
This came after a very, very long hypothetical where one team gives up a ton of weak shots, the other team gives up very few, high-quality shots, and the team that got outshot wins. This happens in sports. In one-game sample sizes, these things happen. Just two weeks ago, the Padres took a no-hitter into the ninth, and lost the game. What’s the point? How many times does an NFL game end where the team with the most yards gained lose?
Over the course of a full season, do you think the team who consistently controls play, continues to get shots off, will continue to have such poor defensive zone coverage as in Bill’s hypothetical? I’m willing to bet that the losing team in that hypothetical is either really, really bad and outshot their opponent due to score effects, or they get yelled at by the coach, blown up by the GM, or just get better in their defensive zone where they aren’t giving up 2-on-1s and multiple rebounds.
Picking a one-game example is easy. Showing sustained counter-examples, real-life retorts to advanced statistics, is hard.
Further, if the argument is going to goalies save-percentage, since when is that an advanced statistic?
Of course, there are also nights where the shots/attempted shots and goalie stats are accurately reflective of the game play and the final score. But my point here is that no hockey statistic — no matter how complex the formula — is a panacea.
You’re fighting a straw-man here, Bill. If you meet anybody who thinks any single hockey statistic – no, any single statistic – is a panacea, you’ve met a bad statistician. Further, if we’re just arguing against the fringes, how many people think Roman Cechmanek is a terrible goalie because they saw him bad?
Arguing against the fringes is good. Arguing against the fringes, but painting the fringes as indicative of the entire community, is bad.
Always keep the big picture in mind. For teams, the bottom line is whether they win or lose. For individual players, look at the team they play on, the system employed, the ice time they get, the role they are expected to perform and their salary cap hit relative to the job they do.
Yes, the bottom line is winning or losing. But you don’t think evaluators and scouts care more about how well a team or player played? Do you think the Flyers, upon losing 1-0 to the Montreal Canadiens in April of 2010 cared more that they lost?
Peter Laviolette, after that game said:
“I’ve always tried to focus more on what we’re doing,” Laviolette said, when asked about Detroit. “Again if we can come back with the same type of game – we were really strong in the offensive zone, we pressed all pucks and lots of opportunities at the net, we just couldn’t find the back of it.
“We really limited Montreal’s chances at our net and they got one early – a quick play from the back of the net and they got a 1-0 lead. They’re goaltender played really well and that score was able to hold up.
“You look at the game and you want to come back with the same type of energy and that same type of game, and more times than not you’ll be successful. So that’s the plan.”
It’s easy to say “bottom line is, did you win”. But everybody knows this isn’t true. Why is this the retort to advanced statistics, when the better team loses the game numerous times?
Further, we’ve already agreed context is crucial. When you want to argue against advanced statistics because they don’t provide context, you’re not arguing against advanced statistics at all. You’re arguing against bad analysis, which we all agree on. Why does advanced statistics (and some not so advanced statistics) get caught up in the cross-fire?
