How Good Is That Goalie?

Evaluating a goalie's true skill requires deep statistical analysis and plenty of data.
+ More

By Chris Gorski, Inside Science News Service

(ISNS)—Dwayne Roloson of the Tampa Bay Lightning has embodied the role of the prototypical Stanley Cup playoff hot goaltender, leading his team to a 3-0 lead in the Eastern Conference semifinals against the top seeded Washington Capitals. The 41-year-old had a losing record in the NHL's regular season, but has stopped over 94 percent of shots he's faced in the playoffs—but he's probably not actually this good.

Summing up the performance of an NHL goalie with a single statistic is a difficult task. New research that makes adjustments to traditional measures attempts to do just that. This effort endeavors to remove the influence of team defense and team offense from goalie statistics, which raises questions about the ultimate meaning of sports statistics.

The Limitations Of Stats

For any sport it is difficult to develop a single formula whose solution explains everything about a player. Batting average in baseball, quarterback rating in football, points per game in basketball: each of those numbers explains something about a player, but also has limitations. For evaluating goalie performance, counting the portion of shots-on-goal that he corrals, called save percentage, is one popular statistic. It makes no distinction between goals that are primarily the netminder's fault and those due to other factors, such as poor defense or great offensive play.

Furthermore, each goalie faces a different collection of shots. Slap, snap, and wrist shots from anywhere on the ice are all equally counted. The data might show that one goalie performed better than his peers against a certain type of shot from a given area of the ice. This partially explains why save percentage provides a limited picture of goalie performance.

"Anybody who knows anything about hockey knows that not every shot has the exact same chance of going in," said Tom Awad, a writer for the website Hockey Prospectus. "So obviously if you could find some way to quantify how dangerous each of the shots is you could improve the metrics of goaltending that we already have."

Two average goalies might stop 1,800 out of 2,000 shots in a season and have identical save percentages despite facing different collections of shots. A 1 percent difference in save percentage would mean a difference of about 20 goals allowed in a season, which could mean the difference of several points in the season standings—enough to decide home-ice advantage in the playoffs.

Comparing Goalies

So how do you compare goalies to one another? Michael Schuckers, a statistician at St. Lawrence University in Canton, N.Y. tried to remove the effect of other players on a goalie's performance, a new variation of something others have attempted previously. He presented the research at a March conference at the Massachusetts Institute of Technology in Cambridge.

Schuckers constructed maps of goalie performance using data collected at each arena. The NHL tracks the type of shot, where it was taken on the ice and when either team has a power play. These maps allow Schuckers to take a goalie's actual performance and simulate his performance against any distribution of shots. 

"For last season we basically modeled what the save probability would be for every goalie for all 74,000 shots taken in the league," Schuckers said. He calls the end result a defense independent goalie rating.

Goalies that face a more difficult set of shots than the league average have higher ratings than their save percentage, while the rating drops for those who faced an easier set of shots. If the rating were a more accurate gauge of performance, it could demonstrate differences over a long season, even though the gap between one season's save percentage and the rating is usually less than 1 percent.

"The places we see big differences are in teams with either very, very good defenses or teams with bad defenses," Schuckers said. "If a goalie is on a team with a [poor] defense, they're more likely to face shots that are in close."