FIFA - FIFA QUALITY PROGRAMME FOR EPTS

FIFA QUALITY PROGRAMME FOR EPTS TEST MANUAL FOR BROADCAST EPTS 2024 EDITION VERSION: NOVEMBER2 CONTENTS 1. INTRODUCTION 3 2. TEST PROTOCOL 3 2.1 Testing scenario 3 2.2 Test area 3 2.3 Download files content 3 2.4 Submission time 3 2.5 Testing scenario overview 3 2.6 Available camera footage for testing 5 2.7 Reference system 6 2.7.1 Assessment of the accuracy of the Reference system 6 2.7.2 Set-up of the Reference system 8

3. DATA ANALYSIS 9 3.1 Data analysis processes 9 3.2 Reference system data preparation 9 3.3 Supplier data preparation 9 3.4 Reference system and supplier’s data synchronisation 9 3.5 Position alignment 9 3.6 Statistical analysis 101. INTRODUCTION In light of the growing demand for more cost-efficient match analysis technologies, FIFA has launched a new standard within the FIFA Quality Programme for Electronic Performance and Technology Standard (EPTS) that aims to contribute to the democratisation of football technologies. This new standard, referred to as Broadcast EPTS, assesses supplier systems capable of generating player performance data using existing infrastructure only, including broadcast footage and team-filmed footage, without the need for additional personnel on site. The test protocol for this evaluation is explained in this document. 2. TEST PROTOCOL 2.1 Testing scenario FIFA will provide footage from an official professional match played in a football stadium with pitch dimensions according to official FIFA regulations. The pitch dimensions will be professionally surveyed using a total station and provided to the participating supplier. 2.2 Test area The test area will consist of the full size of the pitch in which the players’ movements during the official professional match are captured with a reference system, and a professional broadcast system. 2.3 Download files content A secure link to download broadcast footage will be provided to each supplier, which will choose from a selection of camera views (see Table 1 below). Suppliers will also receive the pitch dimensions and the team sheets. Given the file size of the footage, a strong internet connection will be required for the download. 2.4 Submission time The submission time will be calculated using the download timestamp of the video and the upload timestamp of the data. Suppliers will have up to 24 hours to upload data, and the official report on supplier performance will document the time to the nearest minutes taken to supply the results. Please note that if any changes are made, or any additional data files are uploaded, the earlier download and the latter upload timestamps will be used. 2.5 Testing scenario overview Players will be tracked in all match activities within the pitch dimensions.2.6 Available camera footage for testing WORLD FEED WORLD FEED

Type of footage Derived metrics Limitations Video feed as broadcast on TV, multiple camera angles, replays, dynamic cameras with varying frame rate - Instantaneous position - Instantaneous velocity As the feed does not include all players at all times, it can only be used to contextualise event data and understand the game at specific moments in time when camera 1 is being used CAMERA 1 FEED

Type of footage Derived metrics Limitations This camera angle typically makes up 65% of the broadcast feed that can be seen on TV - Instantaneous position - Instantaneous velocity As the feed does not include all players at all times, it can only be used to contextualise event data and understand the game at specific moments in time when camera 1 is being used DYNAMIC TACTICAL FEED

Type of footage Derived metrics Limitations Operator aims to keep all outfield players in shot at all and at least one goalkeeper at all times - Instantaneous position - Total distance covered - Instantaneous velocity - Maximum velocity - Distance covered in speed bands - Potential for human error in filming - Incomplete GK data

Table 1: Overview of available camera angles, types of player coverage, examples of possible derived metrics, limitations and sample shot of footage.2.7 Reference system 2.7.1 Assessment of the accuracy of the reference system Please refer to the research paper here. To determine the accuracy of the reference system, the level of agreement with a three-dimensional motion capture system (VICON) was established. Participant position (XY) and movement (velocity) were determined by a large-scale three-dimensional motion capture system. To track participants, four dual-measure (active and passive) markers were placed on each hip and shoulder joint, with a fifth placed on the device (where applicable) or mid-back section of each participant. This allowed for an approximation of the centre of mass. The playing volume was reconstructed into three-dimensional space from 36 VICON Vantage cameras with a sampling frequency of 100 Hz. The cameras were positioned around the 30 x 30 m test area used for both the circuit and small-sided games. The test of reference system accuracy was conducted in two separate stadiums. The first test venue was a stadium currently used for national-level football competition. It had a regulation football pitch and grandstands with seating for approximately 15,000 people. The second test venue was a stadium currently used for both national and international football matches. This stadium had a regulation football pitch and grandstands with seating for approximately 100,000 people. VICON and reference system data were time-synchronised using cross correlation of position data (Buck, Daniel and Singer, 2002). Once synchronised, data was trimmed for time on the pitch, combined and extracted into individual data files. To determine the level of agreement between the two sources, the root-mean square difference (RMSD) for velocity was obtained between VICON and reference system data. For position, the mean absolute error between VICON and reference system was calculated.There was excellent agreement between VICON and reference system. The mean absolute error for speed 0.04 m·s-1 and the RMSD for position was 0.15 m. The distribution of these differences are shown in Figure 1 below. VICON and reference system data were time-synchronised using cross correlation of position data (Buck, Daniel and Singer, 2002). Once synchronised, data was trimmed for time on the pitch, combined and extracted into individual data files. To determine

position was 0.15 m. The distribution of these differences are shown in Figure 1 below. VICON and reference system data were time-synchronised using cross correlation of position data (Buck, Daniel and Singer, 2002). Once synchronised, data was trimmed for time on the pitch, combined and extracted into individual data files. To determine the level of agreement between the two sources, the root-mean square difference (RMSD) for velocity was obtained between VICON and reference system data. For position, the mean absolute error between VICON and reference system was calculated. There was excellent agreement between VICON and reference system. The mean absolute error for speed 0.04 m·s-1 and the RMSD for position was 0.15 m. The distribution of these differences are shown in Figure 1 below. 40000 - 30000 - 20000 - 10000 - 0 - -1.0 -0.5 0.0 0.5 1.0 Velocity Error (m/s)

15000 - 10000 - 5000 - 0 - 0.00 0.25 0.50 0.75 1.00 xy Error (m)

Figure 1: Histograms of speed and position differences between the reference system and Vicon.2.7.2 Set-up of the reference system Camera set-up specifications: • Sixteen cameras running at 25 fps and 1936 x 1216 pixels used to collect the video for reference system. • Cameras positioned as high as possible around the test area (e.g. on the stadium grandstands). • Cameras synchronized so the images are captured at exactly the same time and will all have the same frame count. • Cameras set up in six clusters as shown on Figure 2: two clusters with three cameras on either side, and a cluster with two cameras behind each goal. Each of the six clusters provided complete coverage of the pitch, when combining the cameras within the cluster. Figure 2: Position of cameras.3. DATA ANALYSIS 3.1 Data analysis processes All of the suppliers shall be assessed at 25 Hz. Where providers do not provide data at 25 Hz they will be either upsampled or downsampled to match the 25 Hz reference data. 3.2 Reference system data preparation Reference system data shall be provided as X and Y coordinates at a sample rate of 25 Hz. The file shall be sorted by player and frame number, with files saved for each individual player. Displacement shall then be established from the X and Y coordinates using the three point centrum method, subsequently allowing the calculation of speed. Speed shall then be smoothed using a second-order low-pass Butterworth filter with a 1 Hz cut off. 3.3 Supplier data preparation Supplier data MUST be provided in .csv format and shall be firstly split into individual player. The file MUST be named using the following naming convention: “systemname_playerid_DDMMYYYY_time”.csv where systemname is the name of the actual supplier system being tested, playerid is the unique jersey number (i.e., black19), date of test and start time of the test session. All files MUST include the following data columns: 1. Frame 2. Time (if applicable) 3. Team name 4. Player name and jersey number 3. X coordinate 4. Y coordinate 5. Speed 6. Visible/Non-visible status

(i.e., black19), date of test and start time of the test session. All files MUST include the following data columns: 1. Frame 2. Time (if applicable) 3. Team name 4. Player name and jersey number 3. X coordinate 4. Y coordinate 5. Speed 6. Visible/Non-visible status (1 = Visible, 0 = Not Visible) Supplier data shall be provided using the FIFA EPTS Standard Data Format. Supplier files provided at 25 Hz shall be checked for consistency of sample rate (i.e. dropped data points, inconsistent sample rate). Further to the raw data provided by suppliers, a second-order low-pass Butterworth filter with a 1 Hz cut-off will be applied to all suppliers’ velocity data. In order to ensure the best agreement between your data and the reference data the supplier must use the kick-off as the starting point for data submission and the end of the half as the end of the data collection. Suppliers must provide documentation explaining the coordinate system in relationship to the camera selected.3.4 Reference system and supplier’s data synchronisation The individual player files (reference system data) shall be imported and synchronised with individual supplier data. Synchronisation shall then be established between the supplier speed data using cross correlation and the reference system velocity data. The resulting data shall be trimmed and combined. The final file for statistical analysis shall contain the reference system data and the supplier data. 3.5 Position alignment To examine the difference in position data, X and Y coordinates should be rotated to match the X and Y co-ordinates from reference system data. This rotation of data will be achieved by rotating the suppliers’ data by 1 degree through 360 degrees until the lowest error in position data is achieved. Once the closest 1 degree rotation is known, the supplier X and Y coordinates will be further adjusted to the closest 1/100 by 1 degree either side of the best alignment. Once aligned, data will then be shifted left or right, up or down to ensure X and Y position data are synchronised. The difference between the supplier X and Y coordinates and motion capture X and Y coordinates will be quantified as the straight line difference between the coordinates. 3.6 Statistical analysis Where possible, the differences between the supplier’s and the reference system data for both position (in m) and velocity (m.s-1 and km.h-1 if desired) will

The difference between the supplier X and Y coordinates and motion capture X and Y coordinates will be quantified as the straight line difference between the coordinates. 3.6 Statistical analysis Where possible, the differences between the supplier’s and the reference system data for both position (in m) and velocity (m.s-1 and km.h-1 if desired) will be assessed using both absolute and relative measures. For absolute measures, the mean difference between the two data sources will be obtained to provide an indication of any systematic differences between the two data sources. Accompanying 95% confidence intervals will also be included to determine the range by which 95% of these differences lie. To determine the level of agreement between the two sources, for speed the RMSD will be obtained. This represents the sample standard deviation of the differences between the two sources. For position, mean absolute error will be used for the same purpose. During the test event, tolerance thresholds for position and speed data will be defined to establish a pass or fail requirement. These thresholds will be determined based on the pooled data points collected during the test events to reflect the contemporary accuracy capabilities of the Broadcast EPTS industry. Based on these thresholds, suppliers will be graded as pass or fail. For the submitted data, analysis and comparisons will be conducted separately for a) visible players, b) non-visible players and c) pooled players (visible and non-visible). However, pass or fail criteria will apply solely to the tracking data for visible players. Results relating to non-visible players, if submitted, will be noted in the report for informational purposes only.As the camera angle used by suppliers can dictate when players are in view, a visualisation of the timeline when players were in view and data supplied will also be shown, as per the example in Figure 3 below.

Figure 3. Example timeline of data submission against time which the tracked players were in field of view.