clock menu more-arrow no yes

Filed under:

The Curious Case of: Ryan Miller

New, comments

The Vancouver Canucks' biggest free agent acquisition, in an offseason of sweeping organizational changes, was an aging goaltender. But does newly minted GM Jim Benning know something we don't about a goalie many believe is on the proverbial "back nine?" Let's take a closer. WARNING: this article contains numbers and graphs and definitions and things. I resisted the urge to link you off to other places that handled some of it because, then you just won't read that and be even more ZZZZZZZZZ. Buckle your seatbelts. Breathe deeply. Hydrate. Take many breaks.

All by myself.
All by myself.
Dilip Vishwanat

Eddie Lack and Jacob Markstrom finished out what was a forgettable end to last season for the Vancouver Canucks. But up until July 1st, not many seemed to know exactly what GM Jim Benning had in mind for the immediate future of the club in net. Would this be a full rebuild? Would new head coach Willie Desjardins be tasked with fostering the development of the two young goaltenders while also meeting the demands of fans expecting a bounce back campaign? Or would the team indeed go after one of several established veteran goaltenders on the market?

Enter Ryan Miller.

The Canucks signed the former Sabres Blues (kind of? It never really seemed to fit) goaltender to a 3-year, $18 million ($6m average annual value) contract. But at 34...why? Was it for his past performance? And how good was that past performance leading up to the signing anyway? Or is it because he has enough good seasons left in him to stabilize the position and provide a backbone for current success?

As I see it, there are 3 narratives floating around out there regarding Miller:

Narrative 1: Ryan Miller's best years were in Buffalo, and he has been on a steady decline since his performance in the 2010 Olympics. It was never more evident than when he hit the spotlight in St. Louis. That decline will continue.

Narrative 2: Ryan Miller is a star (Hall of Fame?) goaltender who has been stuck propping up horrendous teams in Buffalo for too many years to count. He still has a lot left in the tank and can help a team take the next step (whatever that step may be).

Narrative 3: Despite everything, Miller could never really elevate the Sabres to anything better than an also-ran. And usually, he played for a basement dweller. Great goalies can win games by themselves, can't they? Overrated.

How we have quantified goaltender success to date is woefully inadequate

The prevailing way goalies are analyzed is still probably the least complex, most archaic and most vague, of any position in, like, maybe all of sports. I'm not even kidding. In part, and to be fair, that's because it's tough to do so.

Obviously, good goaltenders stop lots of pucks. We know that much, generally. From there, most analysts and media-types make value assessments on their success at that, or lack thereof, with some weighted combination of Save Percentage (SV%), Goals Against Average (GAA), and Wins. What consideration is given to technique, rebound control, how well they work within a particular defensive system (surprise! Goalies don't operate in a vacuum!), or any specific fundamental aspects of the goaltender's game are usually reserved for describing the outstanding. Quick's crab-style athleticism side to side. Lundqvist's amazing reaction speed while playing deep in his net. Luongo's ability to suck up rebounds. Rask's puck tracking abilities through traffic. Brodeur's puckhandling. And so on and so forth.

The only thing I can really think it's akin to is how baseball fans, and scouts for that matter, used to analyze (and still do, in most casual circles) pitchers in baseball with some combination of ERA, wins and, oh, I don't know, strikeouts? Heat on the radar gun? Check. Steely gazes and cool demeanor? Check. He's an ace.

Strikeouts and velocity are kind of like big glove saves for goalies. The analogy isn't perfect but you get the idea. Whatever. Shutup.

The point is, not many out there seem to know what truly makes a goalie "good," and how we might reliably predict their future performance. More importantly, goaltenders are rarely measured against something. Instead, we just kind of take those basic stats and toss them out there. And then we create narratives. Like the ones above.

We know, or think we know, that something like a .920 SV% is pretty darn good. Since league-average goals scored per team is about 2.77, give or take, year to year, I guess a GAA around 2.4 or less is cool too. I really don't even know what satisfies folks anymore. But the reality is that any of those statistics are truly arbitrary because they vary league-wide year to year and fail to adequately compare the goaltender against his contemporaries, in consideration of variables.

Let's do a short exercise. Humor me.

Here's a goalie over the past 5 seasons (reverse chronological order). Look over the stats. He's considered "good" and relatively consistent, but also, some say, in a decline.

GAA SV%
2.47 .903
2.72 .923
2.64 .918
2.81 .915
2.55 .916

He is none other than Ryan Miller, the guy who you often hear is declining with age but...doesn't appear to be really declining at all. But I'm also not quite sure he's been performing well either. At least not from those two stats. He's just kind of putting up similar-ish numbers each season. And in relation to what? We don't know.

Without the benefit of more info about what his peers are up to, we don't really know whether he's been ahead or behind the pack. Hell, you can't even tell if he is more or less likely to contribute to his team's success. And that isn't a commentary on his play. It is a fundamental failure with the statistics that are most heavily relied upon in determining who is a consistently good goalie, year over year.

It's also not merely a commentary on the media and how goalies are covered. It extends right into the highest levels of organizational decision-making, and even coaching. The fact is, goalies are either ignored or left entirely to goaltending coaches from the youngest levels of the sport. This is well established in hockey circles. Everyone thinks goalies are very important. But nobody (except other goalies) seem to know what the hell to do with them or how to actually tell how well they are doing, as compared to their peers.

It's an awful lot of "well he struggled this game so we are going to roll the other guy" and "well he had two shutouts and played great and we are going to play him vs [insert team] because I feel he gives us the best chance to win" and "I have a lot of confidence in him and I know we don't go anywhere without him." I'm paraphrasing of course. A lot of feel. A lot of intangible information. A lot of seemingly unnecessary tinkering game to game (ahem, Lu) despite talking about confidence and not needing to tinker. But the point is, there isn't a lot of talk about anything technical regarding their games. Or any statistical support to back the statements. You rarely even hear something as simple as, "out of the 22 shots he faced he put only 1 rebound back into the home plate and slot area, which indicates he was square to the shots and controlling his rebounds exceptionally well."

Further, decisions outwardly appear to be figured out on a game to game basis when, in reality, game to game SV% varies drastically and a return to expected play quality, whether better or worse than the previous game, is typically just a regression or progression to the mean performance of the goaltender. Have a look:

Game_by_game_save_per_medium

Outside of Hiller's wild outlier, game by game save% is the same rollercoaster with a different mean all season for all three.

The above generally renders the "hot hand" a myth. That's why it is best to take a long view and let things play out with the guy who has consistently proven he is capable of starting at the NHL level (anyone remember all the chatter about Lundqvist playing in Montreal during the playoffs, where he had previously had a few bad games?). Give him chunks of time to play with normal rest intervals.

Side note: I would love, just once, for a coach to state that his decision on who starts next game was based on Occam's Razor and, instead of riding the hot hand, he is going to play the goalie who has played the best over a large enough sample size to expect that he is most likely to perform well again. My head might explode into confetti. Dare to dream.

So if the coaches don't share anything concrete (or don't really know much, which is more plausible than you'd think)...what do we actually know about some of these goalies aside from taking the basics above and almost intuitively stating "that guy is playing well" and this guy isn't?"

What can we actually figure out about Ryan Miller as we head into the 2014-2015 season?

There are better and more complete resources available for evaluating goaltenders, and I am going to use them to paint a picture of Ryan Miller compared to his peers. Here goes.

The new ways to evaluate goaltending performance

Goals Saved Above Average (GSAA):

GSAA is a relatively new stat in the advanced metrics lexicon. It represents the number of goals a goalie saves for his team above (or below) league average. It's actually really simple. Basically, take the league average save percentage and apply it to the total shots faced by the particular goaltender. Out of that, you get a number of goals that the average goaltender would have given up had he faced the same number of shots as the goaltender in question.

This resulting number can be compared to the number of goals the goaltender in question actually gave up. A plus/minus is the result. If a goalie is in the positive, he is saving more than a league average goaltender might in the same situation. If the goalie is in the minus...well you get the idea.

At first it might appear like this is simply a comparison of save percentages, or akin to it. But it isn't. GSAA creates a direct comparison to league average, as opposed to a stand-alone percentage. Why is that good? Because, as mentioned above, league average SV% and GAA changes year to year. It is hard to look at either stat alone and arbitrarily determine whether it's good or not as to compared to the goaltender's peers that season. With GSAA, the assumption at a glance is that the goaltender in question is performing better or worse in comparison to the league average that season. It is a hard and fast number. And that's nice.

Of course, the statistic isn't perfect. It tends to favor goalies that play more total games. Conversely, it doesn't account for fatigue if a goaltender does play far more games than certain peers. It also doesn't take into account the quality of competition or how often teams put the goaltender in "bad" situations (penalty kills, shot quality, defensive coverage etc). But, then again, which of the basic SV%, GAA and Wins does? Of course, none of them. GSAA is as good of a "result-based" stat as is available out there for comparing goaltenders against their peers, season over season.

Let's take a quick look at the top 10 in 2013-14 GSAA for goaltenders facing at least 1000 shots (arbitrarily, about 40 games or more, unless the goalie had been getting shelled) and how Miller compared.

Rank Player Team Shots GSAA SV% GAA GP W
1. S. Varlamov COL 2013 27.45 .927 2.41 63 41
2. T. Rask BOS 1641 26.40 .930 2.04 58 36
3. C. Price MON 1828 23.51 .927 2.32 59 34
4. B. Bishop TB 1758 18.48 .924 2.23 63 37
5. S. Bobrovsky CBJ 1699 15.40 .923 2.38 58 32
6. J. Bernier TOR 1788 15.07 .922 2.70 55 26
7. A. Khudobin CAR 1076 12.72 .926 2.30 36 19
8. H. Lundqvist NYR 1810 11.96 .920 2.36 63 33
9. B. Scrivens LA/EDM 1205 9.83 .922 2.55 40 16
10. K. Lehtonen DAL 1888 9.68 .919 2.41 65 33
12. R. Miller BUF/STL 1885 8.42 .918 2.64 59 25

The first thing you might notice is that GSAA appears to have a relatively close correlation to SV%, which is good. It means it can be used in much the same way around the water cooler, at least (for better or worse). It has pretty much no correlation whatsoever with GAA or wins...which it shouldn't. Goalies can't score, so they can't win unless they pitch a shootout shutout win, which is an extremely rare occurrence. GAA can be affected by shot volume and quality, among other things, regardless of SV% performance or GSAA. That further highlights how GAA poorly indicates actual quality of performance as compared to other goaltenders.

When we look at Miller's GSAA last season, in a vacuum, it's not so bad at all. Let's take a look at his progression since 2009-2010 when compared to his peers.

2009-10 GSAA
1. Miller 36.16
2. Vokoun 27.65
3. Rask 24.34
4. Howard 23.06
5. Nabokov 22.37
6. Lundqvist 20.13
7. Bryzgalov 18.00
8. Halak 17.98
9. Kiprusoff 17.57
10. Hiller 13.04
11. Anderson 12.14
2010-11 GSAA
1. Thomas 45.77
2. Rinne 31.96
3. Luongo 26.71
4. Ward 22.90
5. Price 22.04
6. Lundqvist 19.18
7. Bryzgalov 17.12
8. Hiller 16.06
9. Vokoun 15.71
10. Niemi 11.67
14. Miller 6.10
2011-12 GSAA
1. Smith 34.63
2. Lundqvist 28.57
3. Quick 28.07
4. Elliott 26.04
5. Schneider 21.71
6. Rinne 20.15
7. Halak 14.71
8. Kiprusoff 14.38
9. Lehtonen 14.36
10. Thomas 11.44
16. Miller 4.59
2012-13 (lockout) GSAA
1. Bobrovsky 21.48
2. Anderson 19.63
3. Lundqvist 16.81
4. Rask 16.32
5. Niemi 14.45
6. Schneider 12.54
7. Howard 12.44
8. Reimer 11.64
9. Crawford 10.73
10. Dubnyk 9.70
15. Miller 3.86

As you can see, certain goalies every season jump up and simply blow it out of the water in GSAA. That's more indicative of how many quality goaltenders there are in the league than anything else. There are few, like Lundqvist, that are always in the mix at the top. Lundqvist is basically the Beetles. Maybe they didn't always win the Grammy, but everyone knows they put out the best albums year after year.

Miller went Michael Jackson's "Thriller" in 2009-2010, with his absolutely dominant Olympics performance highlighting what was otherwise a sparkling season for him. However, he dropped precipitously in 2010-11 and thereafter kind of leveled out. It could've been the heavy workload of carrying Buffalo and the U.S. Olympic team It could have been mounting injuries. This certainly didn't help anything just over a season later.

Miller merely performed admirably after his steep decline, slowly sliding down the list towards average each year and leveling out there, when compared to his peers. But in the least he has basically remained serviceable and above average in GSA while not plummeting any further.

Gsaa_contemporaries_medium

It doesn't paint the prettiest picture, but what it does tell us is that he isn't really in any sort of free-fall when compared to several of his established contemporaries in age, as was somewhat the belief after his short and mildly tumultuous stint with the Blues. It's much more likely that he fell off his unsustainable 2009-10 pace and came back down to earth. He now very consistently hovers in that range of the top 15 goaltenders or so year in and year out. Unlikely to crack the upper echelon. But also unlikely to experience another plummet from his current performance, especially in consideration of his relative improvement last season. Also unlikely to have the wildly erratic year to year play of guys like Anderson and Hiller.

The basic conclusion drawn from this information is that Vancouver got what he is and will likely get what he has been for the past few years. And what he is...is...a fairly steady top 15ish goaltender. Nothing more or less.

...

Oh you thought you were getting off easy? I just wanted to give you a primer on how GSAA works. Let's go deeper down the rabbit hole shall we?

This is where it gets good, or bad, depending on perspective

GSAA/60:

As it stands, GSAA tends to favor goalies who have a larger body of work. Because saves are cumulative and saves are a part of the formula, it makes sense that the more saves you have, the less likely a league average goalie can match your performance over the long haul. Kind of (in consideration of fatigue and the other stuff we discussed). Ok. So regardless, we can address the problem of larger sample size by simply normalizing the stat to a 60 minute game (GSAA/60). Let's take a look (1000+ shots for sample size purposes).

Rank Player Team GP SA SV GSAA GSAA/60
1. T. Rask BOS 58 1641 1526 26.40 .468
2. S. Varlamov COL 63 2013 1867 27.45 .452
3. C. Price MON 59 1828 1694 23.51 .407
4. A. Khudobin CAR 36 1076 996 12.72 .366
5. B. Bishop TB 63 1758 1625 18.48 .309
6. J. Bernier TOR 55 1788 1649 15.07 .293
7. S. Bobrovsky CBJ 58 1699 1568 15.40 .280
8. B. Scrivens LA/EDM 40 1205 1111 9.83 .267
9. H. Lundqvist NYR 63 1810 1666 11.96 .196
10. J. Halak STL/WSH 52 1385 1275 9.34 .191
14. R. Miller BUF/STL 59 1885 1731 8.42 .144

Once we normalize to a 60 minute game, our hero, Miller, drops down the charts just a tad. When compared to GSAA, you can see how GSAA/60 helps "boost" the comparative performances of guys like Khudobin and Scrivens, who did not get as many games to "accumulate" additional goals saved. Of course, there is no guarantee that had they played more games they would accumulate them anyway. They might have experienced fatigue or a natural regression over time; we just don't know without more data for them.

But at 1000 shots, I personally think it is safe enough (or at least practical, given it is around 40 games, or half a season) where we can at least begin to say "alright, this guy's play is a trend." At least within the season.

For purposes of analysis of Ryan Miller, it's not all that important. But by using GSAA/60, we see that Khudobin was performing like one of the best goalies in hockey last season for a relatively sustained period of time, far exceeding the play of most of his peers, despite Carolina's poor record and poor shot suppression statistics. An average goalie would be giving up several more goals in the same Carolina season. Interesting.

Can we dig deeper?

Yes!

EVSave%, EVGSAA, EVGSAA/60...better contemporary benchmarks

Even strength save percentage (EVSave%) further helps us distinguish how well a goalie plays "all things being equal."

In the least, EVSave% removes scenarios where a goaltender is giving up an inordinate number of goals because his team is constantly short-handed or poorly executes the PK. This general equality reveals itself in the actual EVSave% numbers, which more bunched together than standard SV%. Except for Brodeur's. His...uhm...sucked last year.

League average EVSave% last season was .920 and most goalies hovered right around there.

Note: We can take it a step further by introducing EVSave%+, which normalizes the stat across eras. But to be honest, I only care about this era and this article is far too long and cumbersome already. Another time.

Which brings me to the statistic for analyzing goaltenders that I truly wanted to share. Applying the EVSave% to GSAA and GSAA/60, we can form a stat that I do not believe has a name yet. Hooray! Let's give it a really catchy name like...EVGSAA and EVGSAA/60. That would be: Even Strength Goals Saved Above Average per 60 Minutes. Totally has a nice ring to it.

Or you can call it MERCAD and MERCAD/60 if you want. I mean, I'm not suggesting you name stats after me or anything...errrr.

Why are EVGSAA and EVGSAA/60 so important? Well, for the same reasons as are detailed in the GSAA primer above. It's comparative properties make it favorable for determining how well a goalie performed above a league average goalie in the given year. But because we have removed variables, namely special teams, and because shot quality differences across teams has been proven to be negligible, we get a much truer sense of how the goalie is performing with all things actually being as equal as possible. And because most plays, most shots, most goals, most minutes are played at even strength, we have a large enough sample size to consider the performance without fear of luck or circumstance having an overly enormous impact, as can be the case with pretty much every other goaltending statistic.

EVGSAA/60? Or MERCAD? Whatever you want. I'm not vain. I swear.

Ok ok. Let's take a look at last year's top EVGSAA/60 sort (min. 800 shots - Click table to view enlarged in separate window):

Screenshot_-_9_8_2014___6_24_07_pm_medium

Uh oh.

As you can see, when we remove variables such as special teams, Ryan Miller literally becomes, well, average. Mind you though, average at even strength isn't far from above average. The numbers, as you can see, are bunched closely enough together. His EVSave% is .920, league average. So he gets blanked on EVGSAA and EVGSAA/60.

As an aside, there's some other intriguing stuff in there. Braden Holtby's even strength play saved Washington's bacon last season. Where he doesn't show nearly as well in GSAA, when we isolate for even strength it is clear that he performed at a high level in EVSave% and therefore was saving more goals above average than folk might have assumed, considering his lack of street cred. He's just giving them up on the PK. A lot. Washington had the 16th ranked powerplay in the league. So, apparently, he was a bit of a weak link, and what prevented them from being even better than that, given their shot suppression on the powerplay. Conclusions that can be drawn are that they need to work on his puck tracking or change how they manage the shots he does see on the PK. FUN WITH NUMBERS AND FACTS AND INFORMATION!

I think more than anything else Miller's perfectly average play even strength indicates that he essentially overperforms shorthanded with a lot of opportunities to do so. He likely got pummeled often while even strength. Keep in mind, Buffalo was the worst even strength team in the league in terms of goal ratio. And nearly everything else. By like, a lot. A real lot. Two tenths of a point worse than second to last in goal ratio doesn't sound terrible until you consider that it's the Edmonton Oilers, and Buffalo wasn't even within a goal of first place. They were getting lapped by most of the league. So he was likely getting left out to dry.

And guess what? I can at least begin to prove it.

Thanks to the fine folks over at War-on-Ice.com (give them a visit, fun site currently under development), we have a great visual aid to reveal just how badly Ryan Miller was pummeled last season by shot ratio in high percentage shooting areas.

Utilizing War-on-ice's Hextally system - we can now visualize the frequency and differential of shots against from quality scoring areas in the offensive zone.

Here's a visual of the League-wide success rate (where most goals are scored):

League-wide_success_rate_medium

Here's what a guy with a really good defense sees in (1.) shot rate against even strength, and (2.) shot differential between team shots for/against even strength - Tuukka Rask (blue is better, red is worse)

Rask_shot_rate_against_medium

Rask_shot_differential_medium

Here's what a guy with a really bad defense (for most of the season - the worst defense) sees in (1.) shot rate against even strength, and (2.) shot differential between team shots for/against even strength - Ryan Miller (blue is better, red is worse)

Miller_relative_shot_rate_against_medium

Miller_shot_differential_medium

Any questions?

Yeah. Dude had been getting swarmed. It looks that bad and he played part of his season with a very competent defense in St. Louis as well. That's how bad Buffalo was. They were pretty much the definition of a quality scoring chance against machine.

What's maybe most interesting is that Miller was doing his job on the low scoring potential areas. At a clip of 97.86%, good for 7th in the NHL. But he was getting clubbed  in the high percentage areas of the ice (81.13%). Down with the likes of Fleury, Pavelec, Niemi, and ::cough:: Lack ::cough::. Not good. Because none of those guys got nearly as much pressure in the high percentage areas as Ryan Miller did.

The onslaught in the high percentage areas that he faces isn't isolated. Combining the past 5 seasons things only get worse for Miller.

Shot_rate_since_09_medium

That's a beautiful blood red, indicating how often he was peppered in the premium scoring areas of the ice. More than any other goalie over the same span of time. This explains a lot of why his EVGSAA/60 is league average despite his overall play being above average.

Conclusions

You made it! Or you skipped ahead. If you skipped ahead that's cool. Here's what you need to know. The narratives about Ryan Miller probably all get a piece of it right, but a lot of it wrong. Miller is by far and away one of the most heavily tested goaltenders in the NHL over the past 5 years. Buffalo was not kind to him.The charts above highlight this.

However, the idea that he has exceeded what other goalies might have accomplished under the same circumstances may not be entirely accurate. While he hasn't failed, his perfectly average 0 EVGSAA and EVGSAA/60, supported by a league average EVSave% indicate that he got by under the pressure. He didn't do anything extraordinary, outside of the 2009-10 season when he went absolutely bonkers.

Then again, few if any goalies have faced the sustained pressure he has, so even with the valuable comparison tool of EVGSAA/60, it's hard to precisely say how any other goalie would do under similar pressure. If you look around the league, many of the goalies that have faced the kind of constant relative shot against rate pressure (guys like Khabibulin and Dubnyk in Edmonton, Cam Ward in Carolina, James Reimer in Toronto, or even Craig Anderson in Ottawa) have by and large failed, or only had flickering success year over year.

The fact that Ryan Miller has been able to level out and play consistent, NHL top 15 starting goaltender hockey since his precipitous decline after the 2009-2010 season indicates to me that he should be able to at least put together a comparable season for the Vancouver Canucks. Especially given that the Canucks have traditionally done a far better job at shot suppression than the Sabres. Additionally, new Head Coach Willie Desjardins appears to utilize a box + 1 defensive scheme that helps clog passing lanes in the high slot, with an overloading and aggressive defense that will likely carve out some of the pressure in the low slot and corners that he has had to deal with in the past. That should pay dividends and help counteract any natural decline due to age or other circumstances.

Benning has indicated that his desire was to have stability in net first and foremost because it breeds confidence in the skaters in front of the goaltender. While Miller's performance over the past several seasons doesn't scream 3 yrs/$18m and nobody should expect a return to his 2009-10 world-beating form, he should almost certainly meet the organization's goals for a stable, presence for the majority of the contract. One that gives his team a chance to win on most nights, with league average or slightly better performance being the norm.