For starters, my main deviation from the original Marcels hockey system was that I wanted to project 5v5 Save % as opposed to regular Save %, with the thinking that 5v5 was a slightly more reliable indicator of goaltender quality. Speaking of sample size, the considerably small amount of High Danger Zone (DZ) Saves that Lehner, Martin Jones, Cam Talbot and Eddie Lack have had to make thus far in their careers made it far from the best tool to project going forward, despite it being the most correlated with overall goaltending success.
Some may also take up issue with the use of weighting the most recent season more heavily than the seasons before, something that affects Lehner much more negatively than the other goalies in this sample due to his tumultuous 2014-15 season. But even when I messed around with increasing the importance of saves from more than one season ago, the effect it had on the goaltenders in this sample was pretty minimal. In addition to past performance, the regression to the mean methodology (which adds 1,525 shots saved at the league average rate to each goaltenders prior four years) has a huge impact on projected performance in that a goalie that's below the league average has his projection pulled up - i.e., Lehner. Some may disagree with regressing a below average goalie to the mean, but if we are to account for exorbitant amount of good luck that Jones and Talbot experienced in their stops, then we should also adjust for bad luck Lehner likely experienced in Ottawa.
Hockey Marcels is also based on a Bayesian approach developed by Brian MacDonald, for which has a niftier way to account for the small sample size issue. The lesser number of shots a goaltender has faced in his career, the higher the variance is for a save percentage that a goaltender is expected to have with 95% confidence. As a result, the Bayesian approach would forecast goaltenders averages in the ballpark of the following values, highlighting that it is impossible to deem any one of the five goaltenders as being a significantly superior option based on forecasted performance alone:
But what I like better about Hockey Marcels is a) it applies a weighting methodology that places higher value on saves made in more recent seasons and b) it accounts for aging. While I am interested in further exploring the aging methodology, I will save that for another time; what's important to me about any projection system is that younger goalies' future performance should be regressed relatively less than older goalies. As a result, a soon-to-be 32-year old, .924 5v5 Career Save % goalie like Antti Niemi is projected to have a lower save percentage over the next three seasons than soon-to-be 24-year old, .918 5v5 Career Save % goalie like Robin Lehner.
As also highlighted in the other article though, there's many downfalls to simply using Save % to evaluating goaltender quality - hence ongoing research into and development of alternate measures of goaltending quality such as Adjusted Save % and Stephen Valiquette's Royal Road project. In addition, luck can play a huge role in early career save percentages - hence why an undrafted goaltender like Martin Jones can fetch a 1st round pick for Boston despite having an unsustainable High DZ Save % in 36 career NHL games.
While I have analysis on the Lehner trade to finish up, the direction I would like to toy with tacking Marcels is a weighting methodology by DZ, and then apply aging to that Adjusted Save %. Long term, all goaltenders should have Small DZ Save %s in the 97%-98% - of the 49 NHL goaltenders with 3,000 shots against from 2007-15, only Tim Thomas (98.12%) and Chris Mason (96.98%) fall outside of that range - while the weighting regression methodology currently being applied to overall Save % should be applied to High DZ Save %s. Medium DZ may be more of a hybrid between Low and High DZ, and is where much of the experimentation will come in.
No comments:
Post a Comment