The only slightly more advanced than shots/possession statistics thread

[Brain Traysers]

Intro post
I’ll aim to periodically update this top post with new and relevant data sources, blogs and twitter accounts - please suggest any that you have seen and like. If you don’t like this sort of stuff – it’s probably a thread to avoid, but hopefully enough people are interested and we can stimulate an interesting debate, share ideas and how bits seen elsewhere apply to Reading.

Blurb
The application of analytics to football remains in its infancy, and naturally the EFL get a lot less coverage than the PL and major foreign leagues. The environment is a little challenging right now, a company called StrataBet used to provide a load of high quality (event level) data to aspiring analysts for free, but they have closed that branch of their business following a change of ownership, and nobody has (yet) filled the void. This means the majority of readily available free data are already aggregated, limiting what you can do with it. Despite that, there are some people doing some really good work:

Blogs/twitter accounts:
Experimental 361 – great data visualisations covering the PL down to League 2
Blades Analytics twitter (naturally a Sheffield U focus, but covers the whole FL) and their new blog EFL Analytics

Data sources (covering EFL):
Fivethirtyeight predictions (including xG and nsxG for Championship games since 2016/17)
infogol (I find it easier to access via the app, as the website is a bit clunky)
Squawka
Whoscored

Others (non EFL focus)
Statsbomb (great blog, plus free data for women’s football and 2018 World Cup)
Understat (xG and more complex measures for top leagues)
Twitter: AshwinRaman_ (frequently tweets good data visualisations at team and player level)
Twitter: FC_rstats (someone learning R via football analytics, often using the Statsbomb WC data)
Twitter: Tacticsplatform (another active tweeter of good visulisations)

Glossary (just some basics, since the more advanced measures rarely reach EFL teams)
xG: expected goals – a simple (and imperfect) way of taking the quality of shots into account. Detailed explanation under ‘Methodology'. Interactive map here. High xG over a game = high quality chances made (or loads of low quality ones)
nsxG: non-shot expected goals – an attempt to add value to build up play beyond key passes and assists – how did action x (e.g. cross) increase the chance of that possession ending in a goal, even if it ultimately did not end in a shot? Explanation here. High nsxG over a game = lots of attacks in dangerous areas, without converting those possessions into chances/goals.
xA: expected assists – applying the xG value of a shot to the passer

[Brain Traysers]

A starter for 10 - our strikers in focus

Having in form strikers has been a bit of surprise development this season – the emergence of Meite, and Bod’s headline figures have meant the two summer signings have either had to often play a wider role (Baldock) or limited opportunities (McNulty).

Digging a little deeper, Experimental361 recently updated their ‘attacking breakdown threat’ charts

1. Bodvarsson is hugely outperforming his stats, with his goals almost double his xG. Despite the overperformance (above the shaded area puts him in the top 10% of performers), he is our striker with the best xG per 90. Given nothing in his career suggests he is an exceptional finisher (which would be required to consistently outperform xG), I think we should reasonably expect him to mean revert.
2. Meite is performing exactly as well as his xG suggests he should – he is having a genuinely good breakout season.
3. Bacuna is the only other player outperforming his xG, albeit with a much fewer xG per 90.
4. Aluko is the only player underperforming his xG. Unfortunately this is in line with every chart I have seen of this nature, including his Fulham days – he has always underperformed his xG so he is either extremely unlucky, or a bad finisher.

Blades Analytics plotted the league's top 40 strikers, with xG per 90 on the x-axis and shots per 90 on the y-axis.

1. After Maupay, who is essentially in his own league, Bod is in the second tier of players with 0.6-0.7 xG per 90 (this is a different data source/model to the previous chart, hence the different numbers – without knowing how they make their models I can’t suggest which is a better measure). But he is only taking 2.2 shots per game, with Brighton’s Tomer Hemed top of that batch of strikers, getting 50% more chances per 90.
2. Meite sits nicely within the next group, among some elite names at this level – James Rodriguez and Tammy Abraham. Interestingly Meite is taking an extra shot per game than Bod – is he better at finding space/ generating opportunities? Or alternatively does the choice of Meite vs Bod change the set up of the team?
3. Baldock is our third entry on the top 40 list, with 0.32 xG per 90, off 1.6 shots per game. It’s hard to read much into this as some will be up front, and some will be from out wide.

[Hendo]

Just an FYI to anyone who’s interested. The above data is from Ben Mayhew’s website https://experimental361.com/

As I feel like this should be credited somewhere.

[Vision]

Just an FYI to anyone who’s interested. The above data is from Ben Mayhew’s website https://experimental361.com/

As I feel like this should be credited somewhere.

'greed. Credit should also be given for the bloke at West Brom who persuaded James Rodriguez to play in the championship.

Meite sits nicely within the next group, among some elite names at this level – James Rodriguez and Tammy Abraham.

[NewCorkSeth]

'greed. Credit should also be given for the bloke at West Brom who persuaded James Rodriguez to play in the championship.

+1.

Also good thread. I hope you keep it up.

[Brain Traysers]

Just an FYI to anyone who’s interested. The above data is from Ben Mayhew’s website https://experimental361.com/

As I feel like this should be credited somewhere.

'greed. Credit should also be given for the bloke at West Brom who persuaded James Rodriguez to play in the championship.

Meite sits nicely within the next group, among some elite names at this level – James Rodriguez and Tammy Abraham.

I had linked to the blog, but yes I agree - full credit should be given in future posts

And no... not actually James Rodriguez... it is of course Jay.

[Brain Traysers]

The BBC have a nice article on how Huddersfield have launched a partnership with a company offering data and videos of young players across Europe. Seems like an interesting service which has had success elsewhere.

https://www.bbc.com/sport/football/46280781

Interesting that both they and Brentford (both EPPP cat 2) have essentially shut down their youth academies - it would appear they believe the money spent developing their own players would be better spent elsewhere, and their u18s and u23s will be built from players released from other clubs on the assumption they can find hidden gems.

We have taken a very different approach with a big investment in our academy, which has paid off in recent years in player sales.

[Brain Traysers]

There has been a lot of debate over our central midfielders this season, and strong criticism of Kelly, Swift and Bacuna at various points. Helpfully, EFL Analytics first post covering the Championship looks at how centre mids across the league rank in terms of passing, using a couple of simple metrics (I've been a little slow to post this one, so the dataset is missing the last two games)

EFL Championship – Who are the Pass Masters?

Ex royal Oliver Norwood is performing incredibly strongly across all measures looked at – he seems to have really clicked at Sheffield and is a big part of their form this season.

In terms of current players, we perform surprisingly strongly here:
• Liam Kelly ranks 10th best for progressive passes in the league (forward passes + passes to final 3rd), at 28 per 90 minutes. Whoscored data sells me he is the only RFC centre mid averaging >50 passes per 90, and plays more long balls per game than the other CMs (8.3 o/w 3.8 are ‘accurate’)
• Swift ranks 5th overall for most ‘dangerous passes’ (through balls + passes to penalty area + deep completions) per 90 mins, at 6.9 per game. I think this solidifies how important Swift is for generating chances – he is one of the best in the league, and if we can keep him consistently fit we will make more chances. Squawka data tells me that for every backward pass Swift makes, he makes 1.1 forward passes – Bacuna and Kelly are at 0.6 - 0.7 forward passes per backward pass.
• Leandro Bacuna ranked 11th best for ‘Chance creation quality’ (xA per key pass), and Swift in 24th. Bacuna’s stats (and others') are almost certainly skewed by set pieces (1.8 ‘accurate corners’ out of 4.6 corners per 90 according to whoscored, also logged as 0.8 key passes from corners per 90). Squawka data indicates he has created 28 chances this season, and while only half have been from from open play, that is still enough to be leading Swift (13 albeit in 30% fewer mins) and Kelly (12).

Overall, some indication that offensively at least we are actually pretty competent at passing and creating chances from CM – and a few bits in there to challenge the ‘Kelly is a bad passer’ narrative. Defensive weakness from these positions still stands, and is a likely area for a future look.

[John Smith]

There has been a lot of debate over our central midfielders this season, and strong criticism of Kelly, Swift and Bacuna at various points. Helpfully, EFL Analytics first post covering the Championship looks at how centre mids across the league rank in terms of passing, using a couple of simple metrics (I've been a little slow to post this one, so the dataset is missing the last two games)

EFL Championship – Who are the Pass Masters?

Ex royal Oliver Norwood is performing incredibly strongly across all measures looked at – he seems to have really clicked at Sheffield and is a big part of their form this season.

In terms of current players, we perform surprisingly strongly here:
• Liam Kelly ranks 10th best for progressive passes in the league (forward passes + passes to final 3rd), at 28 per 90 minutes. Whoscored data sells me he is the only RFC centre mid averaging >50 passes per 90, and plays more long balls per game than the other CMs (8.3 o/w 3.8 are ‘accurate’)
• Swift ranks 5th overall for most ‘dangerous passes’ (through balls + passes to penalty area + deep completions) per 90 mins, at 6.9 per game. I think this solidifies how important Swift is for generating chances – he is one of the best in the league, and if we can keep him consistently fit we will make more chances. Squawka data tells me that for every backward pass Swift makes, he makes 1.1 forward passes – Bacuna and Kelly are at 0.6 - 0.7 forward passes per backward pass.
• Leandro Bacuna ranked 11th best for ‘Chance creation quality’ (xA per key pass), and Swift in 24th. Bacuna’s stats (and others') are almost certainly skewed by set pieces (1.8 ‘accurate corners’ out of 4.6 corners per 90 according to whoscored, also logged as 0.8 key passes from corners per 90). Squawka data indicates he has created 28 chances this season, and while only half have been from from open play, that is still enough to be leading Swift (13 albeit in 30% fewer mins) and Kelly (12).

Overall, some indication that offensively at least we are actually pretty competent at passing and creating chances from CM – and a few bits in there to challenge the ‘Kelly is a bad passer’ narrative. Defensive weakness from these positions still stands, and is a likely area for a future look.

And all of the above a totally irrelevant waste of time when we can't beat the team bottom of the league at home

[Brain Traysers]

What can xG data tell us about some of the prospective new RFC managers?

Background – RFC
A chart I have posted a new times over the last year or so provides the current situation – it shows the rolling 6 game average expected goals (xG) for and against over time. The dashed line shows the break between different seasons, and any managerial changes are shown by a red line. All data behind the calculations are publicly available on the fivethirtyeight website.

The last time I posted this was after the Swansea game, with Clement surviving a further 5 games before being sacked a couple of weeks ago. While the general consensus appeared to be that we were improving (one win, three draws and one loss in those 5 games), the xG story tells the opposite story. Assigning expected points (xP) to the respective xG (where a difference of +0.5xG ‘earns’ 3xP for deserving a win, while an absolute xG difference of less than 0.5 ‘earns’ 1xP), suggests we actually only deserved to earn 1 point (a draw against Ipswich) over that period. In fact, this measure suggests we only deserved to win one (H – Hull) and draw one (H- Ipswich) of Clements final 12 games. Put another way, our ability to consistently convert what limited chances we make at a much higher rate than is statistically likely continued to generate us points.

The chart also shows the gap between xG for and against continued to widen up to his sacking, with the 6 games summarized in the final data points of Clements reign 1.7 xG apart. The dataset of 9000 games suggest teams win around 5% of games with a -1.7 xG difference, and lose 80% of the time. I believe the results were not sustainable, and would have caught up with us sooner rather than later – from this angle, a managerial change looks quite justified, and while it’s too early to make a conclusion (as the final datapoint contains 4 Clement games) there are tentative signs the gap is closing under Scott Marshall.

Potential candidates

Gomes
A similar chart generated for Rio Ave shows José Gomes’ impact at the club. He was only appointed only at the start of the season, and with Rio Ave finishing 5th-7th in the last three seasons (a league of 18 teams), it suggests they are an upper mid table team who occasionally challenge for a Europa League place. While the level of xG against is significantly lower than for Reading for the vast majority of the chart, there is a fairly worrying trend in xG against over the last few games, almost doubling from 1.01 to 1.97 (consistent with our latest datapoint of 2.03 xG A), while ‘xG for’ remains flat. The current gap of -1.23 xG over the last 6 games is associated with a 10% win rate and 66% loss rate. Considering Rio Ave are a ‘better’ team in their respective league than us, these numbers are far from impressive, and don’t show potential for a recovery based on shoring up a defense.


Parky
Another name with short odds, and my preferred pick in all honesty, is ex club captain Phil Parkinson. Like Reading, Bolton’s 'xG for' numbers have rarely exceeded their 'xG against' figures over the last season and a half. However, I would suggest we have a much stronger squad with Bolton only getting promoted in 2016/17, and serious financial issues limiting their scope for signings. Indeed, fivethirtyeight's model assigned them a 28% probability of relegation before the start of last season, compared to our 19%.

I note two things from the chart:
- The gap between xG for and against has been pretty consistent this season, at around half a goal. Performances with these numbers win 23% of games, and lose 46% of the time. Extrapolated this is equivalent to a 46 point season (2 more than we got last season with a much better squad, and 3 more than they achieved) – while this is likely to be in and around relegation every season (enough to stay up in 4 of last 6 seasons), again it is with a very limited squad.
- Apart from a spell in Feb 2018, there is quite a strong correlation between the two lines – I interpret this as being tactically astute, as it shows the theoretical sacrifice of being more defensively solid at the expense of reduced attacking threat, and vice versa. Of course the best possible trend is 'xG for' rising and 'xG against' falling (getting better at generating chances, without conceding more), and the most worrying is the lines moving further apart (conceding more xG without generating more). In my experience of looking at this data, correlation is a surprisingly rare find, and indeed may not be indicative of anything deeper.
Unfortunately, my xG dataset doesn’t extend down to League 1/2 yet so I can’t produce the same chart for Ainsworth.

Luis Castro
While it seems this one is dead in the water, I thought I would produce a chart for Guimarães anyway, who operate in a similar mini league to Rio Ave, and are on a run of 10 consecutive finishes between 4th and 10th place. Again he has only been in place since the start of the season, but their results appear promising, with that magic combo of ‘xG for’ rising and ‘xG against’ falling. I think its too early to draw any firm conclusions here, but he looks to be positively effecting Guimarães at both ends of the pitch.



Make of it what you will, but I'm hardly overwhelmed by Gomes, and would personally much rather it was given to Parky.

[Snowflake Royal]

I think you've just demonstrated why xG is a flavour of the month overrated crock of shit.

Any statistical model that says reality is wrong is pretty worthless.

[Brain Traysers]

I think you've just demonstrated why xG is a flavour of the month overrated crock of shit.

Any statistical model that says reality is wrong is pretty worthless.

Imperfect, yes, but I don't agree its useless at all. I assume you are referring to us picking up a lot more points over Clements final 5 than the most simple possible model (my crude transformation of xG into xP, not accounting for things like scoreline etc)? I think any fan would be hard pushed to admit we deserved to beat Bristol - the game was entertaining, but they let us off with really poor finishing while we took the few chances we had.

The dataset shows 36 games in total with a goal difference between -2.78 and -2.68 (plus/minus 0.05 around that game's -2.73), only 2 resulted in wins (us plus Girona beating Valencia 1-0 in March 2018), with 2 draws and 32 losses. I stand by the assessment that we were very lucky [that Bristol were so poor on the day]. "Reality", as you put it, says we were the better team because we won- while that is ultimately all that matters - I disagree, and so does an empirical analysis. 'Reality' actually has a whole load of variance (often random), which models cannot fully account for. Yet, we care about a load of models that people would say go against reality - e.g. what would people in ex-industrial heartlands say about models showing record-high employment in the UK?

Nevertheless, the point stands that there isn't anything exciting about a potential appointment of Gomes from their performance this season.

[Brain Traysers]

I haven't seen a whole of RFC related stats stuff so dug into the numbers to have a look at where we stand now we are a decent chunk into Gomes' reign (all these data are available from here)


The most obvious impact is at the back, where the 6 game rolling average of expected-goals against (xGA) continued its downward trend having peaked in early Jan - we conceded 7.7xGA over (a) Wigan, (a) Leeds and (h) Stoke. Its settled around 1.4 xGA per game, although has risen slightly since the Sheff U game (2.5 xGA). This is quite a bit different to the narrative of attacking at the expense of defending around Gomes' appointment, which I had contributed to.

Rolling expected goals for (xGF) has remained remarkably flat at just under 1 goal per game, the level it has persisted at since the end of Clements reign. We appear to remain blunt up front, but haven't been as clinical/lucky. I mentioned earlier in the season how we were massively outperforming xG early into the season (Bod and Meite especially). Under Clement we scored 26 goals this season from 19.4 xGF (we also outperformed our 36 xGA, by conceding 32), while under Gomes we have scored 8 from 8.8 xGF and conceded 14 from 16.6 xGA.


Digging into home and away form, under Gomes we have become less threatening at home and much more defensively solid away. But we still are conceding chances worth around half a goal more than we generate each game - this xG difference very roughly translates as around 50% probability of a loss, and only 25% of a win.

But how do our average xG for and against compare to the rest of the league? Well I probably didn't need to highlight us in the below chart...


We rank dead last for average xGA per game (1.7), over 0.2 worse than Ipswich and Rotherham, and 0.4 worse than Bolton. We only fare marginally better on average xGF; our 0.9 xGF is third worse, ahead of only Ipswich (0.64) and Bolton (0.78). That ranks us 23rd overall for average xG difference (0.78 goals per game) for the season.

I'm given comfort by this type of analysis by checking the other end of the spectrum - the good teams are where you expect them to be. The top five by xG difference are the current top 5 in the league.

But what impact has Gomes had here? From the first chart we see we were very open at the end of Clements reign, so I split the plot into two charts - one before Gomes (all 20 of Clement's games + Marshall's 3) and one since he was appointed.



Sure enough, over Gomes 11 games we are now only 4th worst at conceding chances, overtaking Ipswich, Bolton and Forest - hooray! However, we remain bad at creating chances compared to the rest of the league, remaining the third worst. Ipswich remain the worst, while Derby have fallen below us since Mount has been injured. Bolton have overtaken us, and Rotherham have improved from 1.06 to 1.28, the 12th best in the league since Gomes was appointed! Maybe a draw wasn't so bad after all...

Overall that means from an average xG difference view, under Gomes we have only been the 22nd worst team in the league, having been the worst over the 23 games before he was appointed.

Overall there isn't a lot of comfort in these numbers - I have been somewhat optimistic that the loan signings would help out at both ends of the pitch- while xGA has come down, its hard to determine how much of that can be attributed to the signings, and xGF has remained flat despite now having a spearhead in Oliveria. Hopefully the numbers begin to turn, otherwise we remain in a very tight race with Rotherham, who appear to have been very unlucky in recent weeks. That said, a huge part of this sport is luck, and we have had our fair share of it over the last two seasons (xG says we shouldn't have been anywhere near the playoffs in 16/17, and should have been dead and buried last season).

[SCIAG]

I disagree with putting the difference between xG and actual G down to "luck". It can also reflect things like deciding to shoot or pass, and particularly the quality of your strikers and goalkeeper.

[Brain Traysers]

I disagree with putting the difference between xG and actual G down to "luck". It can also reflect things like deciding to shoot or pass, and particularly the quality of your strikers and goalkeeper.

(this is another long post, not so much an attack of SCIAG, but sharing my thoughts and links around his fair challenges... SCIAG, thanks for joining the conversation )

To some degree there is more than just 'luck' to it, as since xG is constructed based on the average conversion of shots, it therefore reflects an 'average' finisher - Aguero's shots from 10 yards are averaged out with Bentekes... However I am yet to be convinced I that finishing 'skill' allows players/teams to outperform xG in the long run (usually over a season, but certainly over multiple seasons).

We can verify this by looking at the 'elite' finishers in the PL (I picked the top 4 scores this season) and in other leagues:



All figures from understat.com. Red = underperforming xG by 10% or more, Green = outperforming by 10% or more.

Here we only see evidence that Salah, Kane and Messi have meaningfully outperformed xG over the last 3 seasons - the latter two are penalty takers which will skew the numbers a bit. There therefore is some evidence here that suggests these are elite finishers. Regardless, we see players we would expect to be elite by eye (Aguero, Aubameyang, Suarez, Ronaldo[not an elite finisher imo, just elite at taking lots of shots], and Lewandowski) all in line with their numbers over the long term. But there is variation within that - Aguero marginally underformed in 16/17, but marginally overperfomed by a similar level the season after - did something change? Similarly Suarez overperformed in 16/17, and is underperforming this season - as is Lewandowski who xG suggests should have 6.8 more goals than his tally of 13. People might argue both of these are getting older, but really that shouldn't impact their ability to shoot as much as their physical ability to get into the same good positions that their younger fresher legs could.

Indeed some players go through electric spells - the ball happens to fall through to them in the right place at the right time (think Michu that one season at Swansea) or in extreme cases it deflects off them and in (Kane last season), but fans and pundits also talk about confidence which I subscribe to a degree. I think a good example is Salah last season, I remember one flicked header that looped into the top corner that he had no right to score, and probably wouldn't even attempt this season now that his goals have reverted in line with xG (I dont believe for a second he is having a bad season, he just isn't at the sustainability high levels of last season). I think this applies to Bod/Meite - both started the season on fire, but have regressed since (albeit with injuries). I find it more likely that they were getting lucky than they magically unlocked the secret to being elite finishers - especially given their previous low goalscoring records.

SCIAG is right to talk about tactics, since current (publicly available) xG measures don't account for number or position of defenders, or position of goalkeeper. This has led to some tentative-yet-plausible evidence that counter attacking players are likely to outperform xG - this post by EightyFivePoints suggests that as the reason Hazard, Salah and Son all outperformed xG (+- 1.s.d) over 16/17 and 17/18. Work on goalkeepers is still at an early stage, although it does seem to suggestJoe Hart sadly needs to be put to pasture.

Indeed some teams/managers have managed to outperform these measures for an extended period - us included. Burnley are another great example, who have performed xG magic... up until this season (possibly due to Hart as linked above...) Man U last season were another example, outperforming partly due to De Gea magic, but I assert partly luck - that luck ran out (and De Gea stopped performing miracles) and Jose got the boot.

On passing vs shooting, xG is only registered when a shot is taken, but non-shot xG is also calculated (the xG value of dangerous situations that didn't result in a shot - e.g. an intercepted cross or pas into box). We don't fare much better here - 6th worst overall for generating nsxG (2nd worst for conceding it), and haven't really improved under Gomes (1.08 vs 1.06 before him). For those reasons I'm not convinced our decision making or ability to get a shot off from an attack has changed under Gomes, and I'd prefer to stick to luck/randomess as the explanation.

Overall, it would be great to believe we have unlocked the secret to beating these stats, but I really find that hard to believe. I think we are slightly better under Gomes this season, but have been less fortunate.

[Snowflake Royal]

Thanks.

[RG30]

Fivethirtyeight had Rotherham 41% chance relegation and us at 39% chance prior to last night's fixtures. Now gone 54% Rotherham and 22% ourselves. Still got a SPI of a bottom 3 side

[Snowball]

Ipswich no odds


1/500 Bolton

2/5 Rotherham

5/1 Reading

8/1 Wigan

12/1 Millwall


Last game of the season maybe both Wigan and Millwall need a point and play a tippy-happy 0-0 draw

But it could very easily be winner-stays-up and losers is relegated.

Wigan are almost certain to need something there, probably a win

So I think they (both) should be much lower odds

[Zip]

Ipswich no odds


1/500 Bolton

2/5 Rotherham

5/1 Reading

8/1 Wigan

12/1 Millwall


Last game of the season maybe both Wigan and Millwall need a point and play a tippy-happy 0-0 draw

But it could very easily be winner-stays-up and losers is relegated.

Wigan are almost certain to need something there, probably a win

So I think they (both) should be much lower odds

Yep baffling why Wigan have been consistently fancied to stay up in recent weeks. Their next two games could well be losses. Let’s see what their odds are then if they do lose them.

[Hound]

someone must know something about Wigan that we don't. 8-1??

1 win in 11 and that was against an awful and defeated Bolton side. Next 4 games are Leeds, Norwich, local rivals Preston and Birmingham (a)

2 decent players - 1 that James chap, and the other Powell, who I assume got taken off injured last night

Surely they are favourites to go?