Towards a real-time possession value framework in ice hockey

. Measuring the individual performance of players is an important task in sports analytics. Traditional statistics-based approaches for evaluating hockey players fail to account for context and long-term impact. Recent advances in data gathering have enabled valuing posses-sions and actions directly to address these issues. This talk describes the implementation of the first real-time possession value framework for ice hockey.


Introduction
Being able to objectively quantify performance in ice hockey has important implications for player evaluation and acquisition. Traditional goal-and shot-based statistics are problematic in this regard as they ignore the impact of preceding plays and do not differentiate between situational contexts. Advanced metrics derived from expected goals address the latter issue by incorporating information on how dangerous the shots a team takes or faces, but are unable to directly measure the value of non-shot actions and fail to consider situations where no shot is taken.
With the increasing availability of high-granularity event data streams and tracking data in recent years, various approaches in different team sports have been proposed for directly modeling the value of individual actions or possessions [1,2,4]. Related work in ice hockey has been scarce [5,6], however, owing to a lack of the aforementioned data at least in the public domain. Our work intends to bridge this gap by proposing and implementing the first hockey-specific framework for deriving the total value of any given possession in real time, as well as valuing the risk and reward of individual actions separately.

Methodology
Estimating the value of possessions can be framed as a Markov decision process (MDP) where the possible actions a player in possession of the puck can take is represented by the discrete set A for all possible match states S. The actions we consider to be part of this action space are shots, passes, moving with the puck, dump-ins and dump-outs. Each action can be further separated depending on their outcome: whether a shot results in a goal, a pass reaches its intended target, a player maintains possession while moving with the puck, a dump-out is recovered by a teammate or a dump-out successfully exits the zone.
Players can be assumed to perform actions that intend to increase the probability of scoring for their team and decrease the probability of conceding a goal. Some actions that are valuable offensively, however, are inherently more risky despite their greater reward. To represent this trade-off we model the return of an action separately for both outcomes with success states yielding the probability of a team in possession scoring and failure states the probability of a team in possession conceding. We assume that only successful actions have positive returns and unsuccessful actions have negative returns, as even though some failed actions may lead to a positive outcome (e.g. a missed pass is received by another teammate in a relatively good position), from a modeling standpoint we want to consider only the intended target.
A common approach for representing the immediate and future impact of an action has been to use either a time window or a fixed number of future actions to assign positive labels for actions that end up affecting the score and negative labels for actions that do not [3,4]. We experiment with different approaches and observe in our data that using a fixed window of eight seconds from the start of an action provides the best balance between short-and long-term return. When modeling reward actions we label actions that lead to the team in possession scoring a goal in the next eight seconds as positive, and when modeling risk we label actions that lead to conceding a goal in the next eight seconds as positive.
To formalize this we follow the definition of Fernández et al., where the value of a possession P t is taken to be the total expectation of all actions in a given state [4]. The probability to take action a and its expected value are learned from X t , which is the feature vector representation of state s derived from a tracking data snapshot at time t.
As the outcome of successful and unsuccessful actions is modeled separately for all actions except shots (which we assign a fixed value of zero risk due to lack of a true failure condition), the expectation of an action can be generally decomposed as the difference between its expected reward and risk and how likely it is to succeed or fail. Because we assume a single end location for moves, the expected value of a move action follows this formulation exactly.
Because any teammate excluding goalies can be considered as the possible receiver of a pass, we take the expected value of a pass action to be the total expectation of all possible passes. We define an additional transition probability

Towards a real-time possession value framework in ice hockey
Linköping Hockey Analytics Conference 2023 P(R t |X t ) for all receivers R t to represent how likely player r becomes the receiver of a pass.
For dump-ins we consider all players except the goalie of the team in possession to be able to recover the puck. As dump-ins do not always have an intended receiver, but a general location instead, we model their success probability as P(A, R t |X t ) to represent how likely player r is to recover the dumped-in puck first. We take the expected value of a dump-in action to be the difference between the total expectation of all teammates T and all opponents O that can recover a dump-in.
For dump-outs we consider all opponents except the goalie to be able to intercept the puck in the offensive zone. We model P(A, I t |X t ) to represent how likely opponent i is to intercept the dump-out and use the total interception probability to determine the success probability. The expected value of a dumpout action is taken to be the difference between expected reward from successfully exiting the zone and the total expectation of risk for each possible interceptor.

Data & Modeling
We use event and tracking data from the Liiga regular season matches 2020-21 and 2021-22 provided by Wisehockey. The player and puck tracking data is gathered using an indoor positioning system and sampled at a frequency of 20 Hz. The tracking snapshots are automatically synchronized by the system to align with the start timestamps of the events. We split 75% of the 872 matches in our data to train our models and use the remaining 25% as the test set. Events where the team whose scoring probability is being modeled faces an empty net are omitted due to the dynamics of scoring changing significantly in such situations. An overview of the events is presented in Table 1.

Towards a real-time possession value framework in ice hockey
Linköping Hockey Analytics Conference 2023 Because the system provides situations where players move with the puck as continuous sequences, we split the puck controls into discrete one-second length actions. We assume that moves have one possible end location and set this as the player's position one second into the future based on their velocity at the time of the event. For unsuccessful passes we determine the intended receiver using a nearest-neighbor approach. We take the direction and velocity of the known trajectory of the pass and project it forward from the starting position of the pass at different time steps. The intended receiver is then chosen as the player who is closest to the projected end positions most frequently.
Each model uses positional features engineered from tracking data like location, velocity, direction, distance to goal, angle to goal, distance to puck and angle to puck for the player in possession of the puck and the target of the action (if applicable). To represent the local context of an action these features are calculated for the closest teammate and opponent of the player and the target. We also adapt the pressure model of Andrienko et al. to a hockey context and to consider the velocity of the players [7]. For the global context of an action we derive features using hierarchical clustering with two clusters on the players' locations to represent the tactical structure of both teams.
For the pass and shot expectation models we calculate features relating to the positioning of the goalie and how much of the net the goalie has to cover to make a save against the shooter or pass target. Pass expectation models also include information about the crowdedness and width of the pass lane between the passer and the target. The dump-in models have additional features based on the area where the puck is estimated to be played in for a given target. We use a simple convolutional neural network that takes raw tracking data snapshots as its input to estimate this end location. Finally, we include game and score state features in the action probability model to represent how teams adjust their playing style depending on the overall match situation.
We train our models using XGBoost [8] and optimize hyperparameters for tree depth and various regularization parameters using Bayesian optimization with cross-validation. To prevent overfitting into particular feature combinations we use moderate to high regularization parameter bounds for each model. As many of the learning tasks for our models consist of imbalanced data, we use a low max delta step parameter to ensure well-calibrated posterior probabilities. Based on domain knowledge we enforce monotonicity constraints on some features like

Towards a real-time possession value framework in ice hockey
Linköping Hockey Analytics Conference 2023 distance to goal as this improves predictive performance and helps in dealing with outliers like goals scored from the neutral zone. Model metrics on the test set are presented in Table 2. We summarize the calibration of the models using expected calibration error (ECE), which takes the weighted average of the difference between binned output probabilities and data points.

Results
As we can determine the total value of any given possession P t , we can use this to evaluate the impact of each action. The value of an action is then defined by taking the difference between total possession value at the start of the action P Start and the total possession value at end of the action P End for all actions except shots, for which the total value is defined as-is. We define actions that have a positive impact as progressive and divide actions that have a negative impact into three categories: regressive, lost and conceded. Regressive refers to how much value is lost through successful actions that decrease the probability of the team scoring, lost refers to how much value is lost through unsuccessful actions, and conceded refers to the value gained by the opponent through unsuccessful actions. To account for any possible error in the detection of the moment of reception for passes, dump-ins and dump-outs, we take a one-second window after the end timestamp of the action to determine its end value. The value of move actions is calculated continuously during matches, so we take the cumulative sum of a puck control sequence as the total value of a move

Towards a real-time possession value framework in ice hockey
Linköping Hockey Analytics Conference 2023 action. As the value of move actions is the most sensitive to how other players are positioned, and we do not want to punish the puck carrier for factors outside their control, we limit the amount of regressive and lost value between two consecutive puck controls by the average value of a move action in our data (approximately one goal scored per 100 moves) and divide this limit by the sampling rate used to generate tracking data snapshots.
Because there are no existing ground-truth labels for evaluating the quality of a player's actions, we compare the total estimated possession value of common in-game situations and how often they have led to a goal being scored in the Liiga playoffs 2022-23. A comparison of the best forwards and defencemen that shows how their actions have added and lost value is also presented. We use playoff matches as our out-of-sample prediction because there is a recognized difference in play styles between the regular season and the playoffs, where players are allowed by the referees to play a more physical and disruptive game. This is illustrated by the fact that in our training data the average number of goals scored per match is 5.3, while in the 2022-23 playoffs it was 4.6 goals. We surmise that the features engineered from tracking data provide enough context to help overcome this domain shift. Table 3 displays the danger level between controlled zone entry types at evenstrength. We take the danger of a zone entry to be average of the total possession value in a one-second window following the moment that the offensive blueline has been crossed. We denote a zone entry to have resulted in a goal using the same eight-second window as with our models. Our framework correctly identifies that breakaways and odd-man rushes are likely to result in more dangerous situations than entries where the team in possession has an equal or lower number of players involved compared to the defending team. The estimated danger level generally correlates well with how often the different entry types result in goals, though some types are affected by the natural variance of goal-scoring in relatively small sample sizes. Based on observing individual entries, it is also fairly common for entry types to change a couple seconds after the entry is performed in the favor of the defending team as their forwards hustle to join the backcheck.  Table 4 displays the danger level of successful even-strength passes by type. We take the danger level of a pass to be its progressive impact as defined above. Passes into the slot, which end in the area between the goal and the first hash marks of the faceoff circle, are intuitively the most dangerous with lateral (eastwest) passes and behind-the-net passes inside the offensive zone following suit. We define entry passes as passes that facilitate a zone entry and outlet passes as passes that lead to a zone exit. Low-to-high and high-to-low passes are passes inside the offensive zone that do not cross the center line. It can be seen that the value added by these pass types closely follows the true scoring probabilities.  Tables 5 and 6 represent the best-performing forwards and defencemen by total value added and lost at even-strength play. Passes, dump-ins and dump-outs have been consolidated together into a single value. The positive impact of an action indicates the progressive value added and the negative impact indicates the sum of regressive, lost and conceded value as defined before. Forwards generate more progressive value on average, but tend to lose more as well through attempting more ambitious and difficult actions. A greater proportion of the negative impact by defencemen is through conceded value, however, as their unsuccessful actions tend to occur closer to their defensive zone.

Conclusions
In this talk we have shown that a real-time possession value framework can be implemented in an ice hockey context. The out-of-sample performance of the framework in typical in-game scenarios and events as well as in differentiating the value created by players in different positions matches both domain knowledge and the true underlying scoring probabilities. In the future it would be interesting to extend the framework to model defensive actions and the possibility of shots being indirect passes. Similarly, considering banked and rimmed passes separately would likely improve the performance of pass-related models [9]. Another promising avenue of research would be to use graph-convolutional neural networks with tracking data snapshots, which has been shown to improve model performance over tree-based models and remove the need for advanced feature engineering [10].