Tag: bias

Takeaways and giveaways are two turnover stats officially recorded by the NHL at every game. However much like other “real-time stats” like hits and blocked shots, there are issues to discuss before we can consider lending any credibility to the numbers.

Defining Takeaways and Giveaways

Perhaps the most concerning thing about takeaways and giveaways is that there is no definition for them from the NHL. The stats page does nothing to describe what either of these stats actually are. Worse yet there is no glossary or list of definitions on the entire NHL.com site that describes exactly what a takeaway or giveaway is. I asked Eric Hornick, the statistician for Islanders home broadcasts since 1982, if league had ever defined what giveaways and takeaways are:

@Chris_Beardy I’ve never seen [definitions] in writing anywhere. Same with any of the real time stats (except ice time). Lot of subjectivity.

— Eric Hornick (@ehornick) February 12, 2016

After extensively using my google-fu, I was able to turn up a lone 2013 article from Dave Mishkin, radio broadcaster for the Lightning, that offered a definition. However, this text does not appear elsewhere on NHL.com and seems to originate from a Columbus Dispatch article in 2011. Nonetheless, Mishkin states:

Giveaways/Takeaways: Here are the league definitions: A giveaway occurs when a player’s own actions and decision making results in the loss of team possession of the puck.

A takeaway occurs when a defensive player causes a turnover and takes possession of the puck or when a defensive player makes a definitive effort to intercept a pass attempt and takes possession of the puck.

I would argue that this definition does not help us at all. It’s probably what most of us intuited before reading it. And that leads us to core problem of these two stats: They are highly subjective. While there are certainly going to be giveaways that are clearly giveaways (such as passing the puck right onto an opponent’s stick) and takeaways that are obviously takeaways (like lifting someone’s stick to take the puck), there are some times where it is less clear if a turnover was a bad offensive play or a good defensive play.

While I may sound like I am splitting hairs here, the subjective nature of these stats have led to unreliable record keeping. There have been many articles over the years describing “home rink bias” for turnovers, hits, blocked shots, and even more well-defined stats like shots on goal and assists. Any use of home stats for any player is going to introduce home rink bias into the statistics and make them rather suspicious. Below is a chart of home vs. away turnover stats for Colorado during the 2013-14 and 2014-15 seasons. The stats are for all situations and only for players with 80+ total games.

While the giveaway stats look fairly reasonable, it is exceedingly clear that Colorado’s home rink statistician is tracking takeaways in a way that deviates from the rest of the league’s statisticians. For this reason, I would advise the use of away stats only or the use of adjustment coefficients for home data.

(Note: While it is possible that the Avalanche’s statistician is recording takeaways in a way that is biased for the home team, we cannot say that for sure from this data alone. If his record of takeaways by Colorado’s opponents is similar, then the issue is instead a deviation of how this statistician defines takeaways relative to the rest of the league. That investigation would be another post for another day.)

Situational Bias

The turnover stats reported by the NHL are for all situations – even strength, power play, and shorthanded. In addition, these stats are reported as base counts, completely ignoring the influence of varying time on ice across players. I was actually inspired to look into this topic yesterday while reading the NHL Arbitrators blog yesterday on finding comparable players to Kucherov for estimating his next contract value. It was a great read and I really like the blog concept and approach. However, the author made use of turnover base counts from the NHL when making one of his comparisons:

For instance, over the last two [full seasons], O’Reilly had a takeaway-giveaway ratio of 181-59 while Stepan’s was 96-69.

In the course of my research, I found that this difference largely disappeared when using away stats and switching to rate-based statistics:

Player	Home GV / 60	Home TK / 60	Away GV / 60	Away TK / 60
Ryan O’Reilly	1.22	4.70	1.00	2.07
Derek Stepan	1.99	2.17	1.06	2.08

The two players are almost exactly identical when removing the home rink bias from each sample. The switch to rate-based statistics adjusts for the fact that O’Reilly had a higher average time on ice per game as well as the fact that Stepan was injured at the start of the 2014-15 season, causing him to play less games than O’Reilly. Below is a further breakdown of the away stats by situation for the two players:

Player	5v5 GV / 60	5v5 TK / 60	PK GV / 60	PK TK / 60	PP GV / 60	PP TK / 60
Derek Stepan	1.12	2.25	0.00	2.75	0.94	0.31
Ryan O’Reilly	1.03	2.19	0.55	3.84	1.51	1.21

Do note that the PK and PP samples are rather small (100 -200 mins) so they may not necessarily be close to what they would be over a longer stretch of time.

Ultimately, I think it was still fair for NHL Arbitrators to say that O’Reilly had more favorable turnover stats than Stepan, but I would argue that they were much closer than the base counts implied.

An Incomplete Picture

Giveaways and takeaways make implicit descriptions about possession during a game. A giveaway signals that the player had the puck before the event occurred while a takeaway signals that the player did not. From there we can ask if a team’s ability to possess the puck is having an impact on the player’s stats.

Returning to the O’Reilly vs. Stepan comparison from before we can see that there might be something to this. Despite being one of Colorado’s best possession players, O’Reilly’s away 5v5 score-adjusted Corsi for was 45.42% over the last two full seasons. Stepan on the other hand had a 50.52% Corsi for in the same situations. This 5.10 percentage point difference suggests that the Rangers had possession of the puck more often than the Avalanche did. This makes it more likely that Stepan was in situations where he could commit a giveaway than O’Reilly, while O’Reilly would have had more opportunities to commit a takeaway than Stepan.

Now, it should be noted that Corsi is a proxy for possession because all it actually measures are certain types of shot attempts. It does not quantify time of possession, zone entries and exits, pass attempts, etc. And this also does not quantify O’Reilly’s or Stepan’s actually roles and successes on the ice. It tells nothing of how much each skates with the puck on their stick, how their positioning is, etc. All of that could be another topic for another day (depending on how much data is out there for these sorts of things).

This bridges over into a criticism that Mishkin had back in his 2013 article:

You would figure a player with a high giveaway total is prone to making bad decisions while one with a lofty takeaway total is adept at reading plays. But the player’s “giveaway” total only highlights potential bad decisions. There’s no corresponding number for how many good decisions he makes with the puck. (At least in football, the quarterback offsets his interceptions with other figures, such as completion percentage, yardage and touchdowns).

Mishkin is right that we need to know a lot more to make use of these turnover statistics when comparing players. It just further enforces the need for more data to properly make use of these statistics. Perhaps the biggest indication that we need to re-evaluate the use of these statistics is the fact that 2015 Norris trophy winner Erik Karlsson led the league in giveaways in the last two full seasons combined. Behind him were PK Subban and Joe Thornton. While these three players “did the most bad things with the puck” over those two seasons, I highly suspect that some record of the “good things they did with the puck” would vindicate them. Unfortunately, no such data is tracked by the NHL.

In Conclusion

To wrap it up, it’s important to do the following when using turnover stats in player evaluation:

1. Either stick to away stats or find/calculate an adjustment coefficient for home statistics.
2. Use rate-based statistics, preferably ones based on time-on-ice.
3. Consider the situational usage of the players you’re using.Either use 5v5 stats or make a note of how PP and PK stats influence the players’ composite stats.
4. Think about how a player’s stats can be influenced by their style of play as well as that of the team around them.
5. Add a disclaimer about the shortfalls of turnover stats and use other metrics in your comparisons.
6. Reconsider if using turnover stats is a good idea.