Edd Webster CFG Junior Data Scientist Data Challenge Submission
This repository is the code and analysis submitted by Edd Webster for the CFG Junior Data Scientist Data Challenge submission, including a list of publicly available resources published by the football analytics community that are related to the project. For a summary pack in Google Slides, please see the following [link], or the PowerPoint version [link] or PDF version [link] saved locally in this repository.
👋 About this Repository and Author
For more information about this repository, I am available through all the following channels:
- eddwebster.com;
- edd.j.webster@gmail.com;
- @eddwebster;
- linkedin.com/in/eddwebster;
- github/eddwebster; and
- public.tableau.com/profile/edd.webster.
📋 Contents:
📔 Notebooks
For code used to produce the Chance Quality Models and engineered dataset of game two of the Metrica Sports sample data, see the notebooks subfolder, in which the workflow is divided into the following:
- Metrica Sports - analysis of the major chances that each team created during game 2 with an exported dataset of shots for predictions to be made by the following Chance Quality Models developed from the shots data (static version).
- Chance Quality Models from Shots Data to calculate the probability of a shot resulting in a goal. Two models have been created (so far), the first, a Logistic Regression model and the second using the Gradient Boosting algorithm XGBoost:
- Logistic Regression model - initial model created as part of the submitted task (static version); and
- Secondary model created using XGBoost - an improved model created after the initial submisssion on 21/03/2021, that explores trying to create a model with improved performance using Gradient Boosted algorithms (static version).
💾 Data
ℹ️ Data Sources
Due to the 100mb file size limitation in GitHub, all datasets used in this analysis, including the Tracking data, have been exported and made publicly available to view and download in Google Drive at the following [link]. However, all the important files are below 100mb, including the shots data provided, and the exported Metrica Sports shot data with calculated features including Interfering and Intervening players, both raw and engineered, are available in the data folder of this repository.
- The provided sample of shots data; and
- Metrica Sports Sample Tracking and corresponding Event data. For code to work with this data, see the
LaurieOnTracking
GitHub repo by Laurie Shaw and the corresponding Friends of Tracking tutorials;
📄 Documentation
All documentation saved locally in the documentation subfolder, including:
- Provided notes and definitions for the a sample of shots [link]; and
- Metrica Sports event definitions.
🏛️ Libaries
The Python libraries used in this notebook include:
- NumPy;
- pandas;
- matplotlib;
- Plotly;
- scikit-learn;
- SciPy;
- XGBoost; and
- SHAP.
📑 Resources and Further Reading
Please see my football_analytics
repository for my attempt to create as concise a list of possible of publicly available resources published by the football analytics community.
The follow resources are those that were specifically used to inform and create my submission for the CFG Junior Data Scientist Data Challenge, specifically focusing on Expected Goals and Tracking data. I have also included links to other topics related to the role such as the application of Reinforcement Learning in football. Credits to all those cited below.
⚽ Football Analytics
👨🎓 Tutorials
- Friends of Tracking YouTube channel [link] and Mathematical Modelling of Football course by Uppsala University [link]. The GitHub repo with all code featured can be found at the following [link]. Lectures of note include:
- Laurie Shaw's Metrica Sports Tracking data series for #FoT - Introduction, Measuring Physical Performance, Pitch Control modelling, and Valuing Actions. See the following for code [link];
- David Sumpter's Expected Goals webinars for #FoT - How to Build An Expected Goals Model 1: Data and Model, How to Build An Expected Goals Model 2: Statistical fitting, and The Ultimate Guide to Expected Goals. See the following for code 3xGModel, 4LinearRegression, 5xGModelFit.py, and 6MeasuresOfFit;
- Peter McKeever's 'Good practice in data visualisation' webinar for Friends of Tracking. See the following for code [link];
- Soccer Analytics Handbook by Devin Pleuler. See tutorial notebooks (also available in Google Colab) that notably include: 3. Logistic Regression, and 7. Data Visualization:
- FC Python tutorials [link];
- DataViz, Python, and matplotlib tutorials by Peter McKeever [link] - I think his website is currently in redevelopment, with many of the old tutorials not currently available (28/02/2021). Check out his revamped How to Draw a Football Pitch tutorial;
- McKay Johns YouTube channel;
- Tech how-to: build your own Expected Goals model by Jan Van Haaren and SciSports.
- Fitting your own football xG model by Dato Fútbol (Ismael Gómez Schmidt). See GitHub repo [link];
- Python for Fantasy Football series by Fantasy Futopia (Thomas Whelan). See the following posts:
- Building an Expected Goals Model in Python by Peter McKeever (using WayBackMachine);
- An xG Model for Everyone in 20 minutes (ish) by Football Fact Man (Paul Riley).
🏛️ Libaries and GitHub Repos
Friends-of-Tracking-Data-FoTD
;SoccermaticsForPython
- repo by David Sumpter dedicated for people getting started with Python using the concepts derived from the book Soccermatics;LaurieOnTracking
by Laurie Shaw - Python code for working with Metrica tracking data; andExpected Goals Thesis
by Andrew Rowlinson. See both his thesis [link] and the following notebooks:expected_goals_deep_dive
by Andrew Puopolo. See the following notebooks:soccer_analytics
by Kraus Clemens. See the following notebooks:xg-model
] by Dato Fútbol (Ismael Gómez Schmidt)soccer-xg
by Jesse Davis and Pieter Robberechts - a Python package for training and analyzing expected goals (xG) models in soccer (not used this this assignment but referenced here); andGoogle Research Football
. See the Kaggle Competition alongside Manchester City [link] (ended October 2020).
✒️ Written Pieces
For a full list of Expected Goals literature, see the following [link].
Papers
The following Shiny App from Lars Maurath is a great tool for looking up publications [link].
- Routine Inspection: A Playbook for Corner Kicks (2020) by Laurie Shaw and Sudarshan 'Suds' Gopaladesikan. Accompanying talk - 2020 Harvard Sports Analytics Lab];
- Dynamic Analysis of Team Strategy in Professional Football (2019) by Laurie Shaw and Mark Glickman. Accompanying talks - NESSIS 2019, 2020 Google Sports Analytics Meetup;
- Football Shot Quality: Visualising the Quality of Soccer/Football Shots by Andrew Rowlinson. See his GitHub repo for code [link]; and
- Game Plan: What AI can do for Football, and What Football can do for AI (2020) by Karl Tuyls, Shayegan Omidshafiei, Paul Muller, Zhe Wang, Jerome Connor, Daniel Hennes, Ian Graham, Will Spearman, Tim Waskett, and Dafydd Steele, Pauline Luc, Adria Recasens, Alexandre Galashov, Gregory Thornton, Romuald Elie, Pablo Sprechmann, Pol Moreno, Kris Cao, Marta Garnelo, Praneet Dutta, Michal Valko, Nicolas Heess, Alex Bridgland, Julien Perolat, Bart De Vylder, Ali Eslami, Mark Rowland, Andrew Jaegle, Remi Munos, Trevor Back, Razia Ahamed, Simon Bouton, Nathalie Beauguerlange, Jackson Broshear, Thore Graepel, and Demis Hassabis;
- Google Research Football: A Novel Reinforcement Learning Environment (2020) by Karol Kurach, Anton Raichuk, Piotr Stańczyk, Michał Zając, Olivier Bachem, Lasse Espeholt, Carlos Riquelme, Damien Vincent, Marcin Michalski, Olivier Bousquet, Sylvain Gelly. See the GitHub repo [link];
- A Framework for the Fine-Grained Evaluation of the Instantaneous Expected Value of Soccer Possessions (2020) by Javier Fernández, Luke Bornn and Daniel Cervone;
- Decomposing the Immeasurable Sport: A deep learning expected possession value framework for soccer (2019) by Javier Fernández, Bornn, and Dan Cervone. Accompanying talks - SSAC19, StatsBomb conference;
- Ready Player Run: Off-ball run identification and classification (2020) by Sam Gregory;
- Beyond Expected Goals (2018) by Will Spearman;
- Wide Open Spaces: A statistical technique for measuring space creation in professional soccer (2018) by Javier Fernandez and Luke Bornn;
- “The Leicester City Fairytale?”: Utilizing New Soccer Analytics Tools to Compare Performance in the 15/16 & 16/17 EPL Seasons (2017) by Hector Ruiz, Paul Power, Xinyu Wei, and Patrick Lucey;
- Not all passes are created equal: objectively measuring the risk and reward of passes in soccer from tracking data (2017) by Paul Power, Hector Ruiz, Xinyu Wei, and Patrick Lucey. See Paul Power's talk [link] (downloadable MP4), and the webpage [link];
- “Quality vs Quantity”: Improved Shot Prediction in Soccer using Strategic Features from Spatiotemporal Data (2015) by Patrick Lucey, Alina Bialkowski, Mathew Monfort, Peter Carr, and Iain Matthews; and
- A Framework for Tactical Analysis and Individual Offensive Production Assessment in Soccer Using Markov Chains (2011) by Sarah Rudd. Accompanying NESSIS talk on Metacafe [link].
Blogs
- Sam Green's xG model;
- Michael Caley's xG model;
- Using Data to Analyse Team Formations by Laurie Shaw;
- Structure in football: putting formations into context by Laurie Shaw;
- xG explained by FBref;
- What are expected Goals? by American Soccer Analysis;
- David Sumpter's Expected Goals pieces:
- Michael Caley's Expected Goals pieces:
- Jesse Davis and Pieter Robberechts' Expected Goals pieces for KU Leuven;
- Will Gürpinar-Morgan's Expected Goals pieces:
- Martin Eastwoood (Pena.lt/y)'s Expected Goals pieces [link];
- Expected Goals For All.
- Actual Goals Versus Expected Goals;
- Expected Goals Updated;
- Expected Goals: The Y Axis;
- Expected Goals And Exponential Decay;
- Expected Goals: Foot Shots Versus Headers;
- Expected Goals And Support Vector Machines;
- Expected Goals and Uncertainty; and
- Sharing xG Using Multi-touch Attribution Modelling.
- Garry Gelade's Expected Goals pieces:
- Expected Goals and Unexpected Goals (using WayBackMachine);
- Assessing Expected Goals Models. Part 1: Shots (using WayBackMachine);
- Assessing Expected Goals Models. Part 2: Anatomy of a Big Chance (using WayBackMachine);
- Introducing xGChain and xGBuildup by Thom Lawrence;
- Quantifying finishing skill by Marek Kwiatkowski;
- The Dual Life of Expected Goals (Part 1) by Mike L. Goodman;
- A close look at my new Expected Goals Model by by 11tegen (Sander IJtsma] (using WayBackMachine);
- An analysis of different expected goals models by Benjamin Cronin;
- Expected Goals 3.0 Methodology by Matthias Kullowatz;
- Explaining and Training Shot Quality by Ted Knutson;
- A simple Expected Goals model by Cricket Savant;
- How we calculate Expected Goals (xG) by Fantasy Football Fix; and
- Una mirada al Soccer Analytics usando R — Parte III by Dato Fútbol (Ismael Gómez Schmidt).
News Articles
- Liverpool sign up for StatsBomb 360: Ted Knutson explains why this stats revolution will change the game (18/03/2021) by Adam Bate for Sky Sports News;
- Man City’s Big Winter Signing Is a Former Hedge Fund Brain (31/01/2021) by David Dellier and Adam Blenford for Bloomberg;
- Man City land big signing in quest to be the best in data science (17/01/2021) by Simon Austin for Training Ground Guru;
- Man City launch AI football competition with Google (13/10/2020) by Simon Austin for Training Ground Guru;
- Manchester City hire Huddersfield Town recruitment co-ordinator (05/08/2020) by Simon Austin for Training Ground Guru;
- Manchester City appoint Sisman to new role of Performance Physicist (20/07/2020) by Training Ground Guru;
- Prestidge promoted to top data science job at Man City (02/01/2020) by Simon Austin for Training Ground Guru;
- Man City Head of Data Insights leaves after six years (10/11/2019) by Simon Austin for Training Ground Guru; and
- Manchester City create new first team data science role (20/06/2019) by Simon Austin for Training Ground Guru.
Books
- The Numbers Game by Chris Anderson and David Sally;
- Football Hackers by Christoph Biermann;
- Soccermatics by David Sumpter;
- Soccernomics by Simon Kuper and Stefan Szymanski;
- Money and Football: A Soccernomics Guide by Simon Kuper and Stefan Szymanski; and
- Data Analytics in Football by Daniel Memmert and Dominik Raabe.
📼 Videos
For a YouTube playlist of videos collated around the topics of Expected Goals, see [link]. For a Tracking data in Football specific playlist, see [link].
Webinars and Lectures
- Laurie Shaw's Metrica Sports Tracking data series for Friends of Tracking (see the following for code [link]):
- Demystifying Tracking data Sportlogiq webinar by Sam Gregory and Devin Pleuler;
- Will Spearman's masterclass in Pitch Control for Friends of Tracking;
- How Tracking Data is Used in Football and What are the Future Challenges with Javier Fernández, Sudarshan 'Suds' Gopaladesikan, Laurie Shaw, Will Spearman and David Sumpter for Friends of Tracking.
- Introduction to tracking data in football by David Sumpter for Friends of Tracking;
- Learning to Watch Football: Self-Supervised Representations for Tracking Data by Karun Singh. See accompanying blog post [link];
- David Sumpter's Expected Goals webinars for Friends of Tracking (see the following for code 3xGModel, 4LinearRegression, 5xGModelFit.py, and 6MeasuresOfFit):
- 'Good practice in data visualisation' webinar by Peter McKeever for Friends Of Tracking. See the following for code [link];
- The Ultimate Guide to Expected Goals David Sumpter for Friends of Tracking;
- How to explain Expected Goals to a football player by David Sumpter;
- What is xG? by Tifo Football;
- Opta Expected Goals by The Analyst (formally Opta);
- What are Expected Goals? by David Sumpter and Axel Pershagen;
- Anatomy of a Goal by Numberphile Brady Haran);
- Sam Green OptaPro Interview;
- How Did These Goals Go In? - We Explain How Goal Probability Works by the Bundesliga;
- Soccer Analytics: Expected Goals by Dan Altman; and
- Anatomy of an Expected Goal by 11tegen (Sander IJtsma);
- Anatomy of a Goal (with Sam Green) by Numberphile:
- "Is Our Model Learning What We Think It Is?" Estimating the xG Impact of Actions in Football by Tom Decroos from the 2019 StatsBomb Innovation in Football Conference;
- Statsbomb Data Launch - Beyond Naive xG by Ted Knutson;
- Karol Kurach - Google Research Football;
- Karol Kurach (Google Brain) "Google Research Football: Learning to Play Football with Deep RL;
- Google Research Football by Piotr Stanczyk;
- Google's AI Plays Football…For Science! by Two Minute Papers;
- Changing the soccer transfer market with big data by Giels Brouwer;
- Soccermatics: how maths explains football by David Sumpter;
- Data Robot Opening Remarks & Keynote: Making Better Decisions, Faster with Brian Prestidge;
Miscellaneous
YouTube Channels
- Friends of Tracking with David Sumpter, Javier Fernández, Laurie Shaw, Sudarshan 'Suds' Gopaladesikan, Pascal Bauer, and Fran Peralta;
- McKay Johns;
- Mark Glickman – for NESSIS talks, uploaded to his personal channel. Old talks are available on his Metacafe channel. See the official website [link];
- 42 Analytics – for SSAC conferences;
- StatsBomb;
- Opta - including Opta Pro Forum talks; and
- Tifo Football.
🔊 Podcasts
List of notable episodes:
- All Stats Aren't We:
- Analytics FC Podcast:
- The Conor J Show:
- Expected Value
- The Football Collective Podcast:
- The Football Fanalytics Podcast
- The Football Pod:
- Football Today
- Pinnacle Podcast:
- The Scouted Football Podcast
- #56: Dominic Calvert-Lewin & Explaining Expected Goals - Spotify and YouTube by The Scouted Football Podcast.
- Squawka Talker Football Podcast:
- Tifo Podcast:
- Trademate Sports:
- UCN/USF Sport Management - Sports Business Podcast:
🐦 Tweets
- The benefits of including fake data in an Expected Goals model by David Sumpter [link].
🧪 Data Science
Mathematics
- Find if a point lies inside a Circle by Utkarsh Trivedi
- How to know if a point is inside a circle?
- Check whether a given point lies inside a triangle or not
Classification Metrics
General
Overview
ROC AUC
- Receiver operating characteristic Wiki
- Understanding AUC-ROC
- How to Use ROC Curves and Precision-Recall Curves for Classification in Python by Jason Brownlee
- scikit-learn ROC AUC score
- Intuition behind ROC-AUC score
Log Loss
- Log Loss
- Accuracy, Recall, Precision, F-Score & Specificity, which to optimise on?
- Understanding binary cross-entropy / log loss: a visual explanation
- Intuition behind Log-loss score
- Log Loss or Cross-Entropy Cost Function in Logistic Regression
- [Lecture 6.4 — Logistic Regression | Cost Function]](https://www.youtube.com/watch?v=HIQlmHxI6-0) by Andrew Ng
- Binary Cross Entropy/Log Loss for Binary Classification
- A Gentle Introduction to Cross-Entropy for Machine Learning by Jason Brownlee
- What is Log Loss? by Dan Becker
Modeling
Logistic Regression
- scikit-learn Logistic Regression official docs
- Logistic Regression wiki
- Logistic Regression by StatQuest (Josh Starmer)
- Logistic Regression for Machine Learning by Jason Brownlee
- Logistic Regression Tutorial for Machine Learning by Jason Brownlee
XGBoost
- Greedy Function Approximation: A Gradient Boosting Machine by Jerome H. Friedman
- XGBoost: A Scalable Tree Boosting System by Tianqi Chen and Carlos Guestrin (the authors of XGBoost);
- XGBoost GitHub repo;
- Awesome XGBoost repo;
- XGBoost official documentation. See the tutorials [link]:
- Gradient Boosting wiki;
- A Gentle Introduction to XGBoost for Applied Machine Learning by Jason Brownlee
- How to Develop Your First XGBoost Model in Python by Jason Brownlee
- How to Configure XGBoost for Imbalanced Classification by Jason Brownlee
- How to Visualize Gradient Boosting Decision Trees With XGBoost in Python by Jason Brownlee
- Story and Lessons Behind the Evolution of XGBoost - brief history and backstory to the creation of XGBoost by Tianqi Chen;
- XGBoost A Scalable Tree Boosting System - talk by Tianqi Chen at the LA Machine Learning Meetup Group on 02/06/2016;
- Kaggle Winning Solution Xgboost Algorithm - talk by Tong He (author of the R XGBoost package) at the NYC Data Science Academy. Mo
- Gradient Boosting Machine Learning - talk by Professor Trevor Hastie
- XGBoost lessson, part of the Intermediate Machine Learning course by Kaggle
- Hyperparameter Optimization for Xgboost by Krish Naik
- XGBoost in Python from Start to Finish by StatQuest (Josh Starmer)
- XGBoost + k-fold CV + Feature Importance by Prashant Banerjee
- A Guide on XGBoost hyperparameters tuning by Prashant Banerjee
- Hyperparameter Grid Search with XGBoost
- Hyperopt the Xgboost model by Yassine Alouini
- Using XGBoost in Python by Manish Pathak
- Getting started with XGBoost by Kevin Lemagnen
- A Beginner’s guide to XGBoost by George Seif
- Boosting your Machine Learning Models Using XGBoost by Derrick Mwiti
- XGBoost Algorithm: Long May She Reign! by Vishal Morde
- Gradient Boosting and XGBoost by Gabriel Tseng
- Gradient Boosting from scratch by Prince Grover
- HyperParameter Tuning — Hyperopt Bayesian Optimization for (Xgboost and Neural network) by Tinu Rohith D
- Complete Guide to Parameter Tuning in XGBoost with codes in Python by Aarshay Jain
Feature Interpretation
SHAP
- Official GitHub repo
- Official documentation
- A Unified Approach to Interpreting Model Predictions by Scott M. Lundberg and Su-In Lee
- True to the Model or True to the Data? by Hugh Chen, Joseph D. Janizek, Scott Lundberg, and Su-In Lee
- Interpretable Machine Learning with XGBoost by Scott Lundberg (creator the of SHAP library)
- Explain Your Model with the SHAP Values by Dr. Dataman
- SHAP Values Explained Exactly How You Wished Someone Explained to You