We received a submission from a reader, and Professor, regarding the Crist vs Rees debate. While we typically don’t post submissions, Jack did a lot of really great work on this, so I wanted to share a part of it. (Jack also wrote a freaking thesis on it, which he will share more comprehensively in the Sideline Forum – so I took some liberty in writing a summary here, but all analysis is Jack’s)
Ultimately, Jack came down to something similar to mine from the spring, just a bit more sophisticated. (To be fair to me, Jack teaches stats at a top tier university. I use them in my job, and to win internet arguments…)
He attempts to take into account factors that might skew the stats such that we are comparing 1) actual efforts after attempting to standardize playing conditions and 2) examine those efforts relative to Ws/Ls only AFTER considering the other potential factors that could have led to the outcomes. Essentially he’s using standardized measures to ‘un-skew’ any skews in the data.
To do the analysis, Jack created these relative metrics across a host of stats (Points Scored, Points Scored Against, Total Yards/Against, Number of Plays, Total Yards Passing/Against, etc) for ND and for the team’s opponents, taking into account THEIR opponents. So we get to see how ND performed relative to all the other teams a given opponent played. See the genius there? And that’s where Jack went the extra mile, and therefore the post. To wit: He examines how ND performed against a certain team on a stat. He then compared that performance against how that team’s opponent’s performed against them on the same stat. Then he calculated a ratio. If ND was >1, then they outperformed the opponent’s opponents (on average). If it was < 1, ND underperfomed. Pretty straightforward yes? Okay, not really, but it's cool. So Jack ran all these numbers and came up with a relative scoring index for each game. Green means good, pink means bad, and you'll see the scale in between. He also scored major points with me for crediting Tulsa to Rees. Crist threw 2 passes in that game, that was Rees' game. Enough preamble, yeah? The Results!
Sorry this is tough to read – it’s a big chart in a small space. Jack will post a more detailed set of visuals in Sideline. Or, see the image at: link
At the highest level, the data shows that the two QB’s reigns on the field were very, very different. In fact, across all 28 stats that Jack analyzed, only one remained fairly consistent across the two ‘halves’ of the season: ND Scoring. Everything else was very different.
The Irish defense was a key difference. In Crist’s games, the Irish D held ND opponents to 86% of their average scoring performance. (To clarify on this first stat so we can all be on the same page: This means that ND performed 14% better than the average of our opponent’s opponents. So, if the average team on Purdue’s schedule scored 100 points against Purdue, ND would’ve only given up 86 points – and then this stat is calculated across all opponents’ opponents for all the games Crist played, and then all against Rees, and compared. Got it? Great.) So, 86%. Good job, ND Defense, you outperformed the average with Dayne Crist at the helm. BUT, the D helped out Rees more. Towards the end of the season, the D held its opponents to 42% of their average scoring. Less than half, very impressive.
We can then look across all the defensive stats (Jack calls them ‘markers’) and see how the team looked overall. And what we see is a whole lot of green for good/outstanding for Rees, and a whole lot of pale yellows or ‘okay’ for Crist.
The Rushing game for Crist was much worse than Rees. Crist’s offense achieved only 80% of the average rushing yards against our opponents, while Ree’s offense achieved 107%. And that includes Crist’s own rushing performances which were much better than Rees’.
In essence, every single one of the stats that the QB does not have direct control over is much better for Rees compared to Crist. So, Rees got a ton of help that Crist did not. (Of course, one could argue that the QB does actually have some impact on these stats as well, but teasing out ‘how much’ is very difficult). Rees had an 18% to 62% advantage across these stat markers. He got a lot of help (and thank the Lord he did!).
In terms of passing, Crist performs at 104% of our opponents’ opponents average in Passing Efficiency. Rees performed at 86%. In effectiveness, the gap is even bigger with Crist at 128% of the average, and Rees at 88%. At the end of the day, Crist performed better than other team’s QBs against the same competition, and Rees performed slightly below average. And when you compare within team stats, it’s interesting to note that Crist’s relative performance was one of the lone highlights in the first 8 games. Most of the team’s stats markers were underperforming, but Crist wasn’t. He overperformed while those around him were average to slightly below average. At the end of the season, it was the opposite for Rees. His team was overperforming on metrics, while he was average to slightly underperforming. Net-net, Crist pulled the team up while Rees kept it right about average.
Personally, I like Tommy Rees. He did a great job, especially when you consider he was a true frosh at the time. But, Crist was superior last year, and I expect him to be the starter this Fall. Once Crist moves on, Rees will have a shot to compete with Hendrix and Golson to be the future starter at ND. And for a true sophomore that got the experience he got last year, that shouldn’t be considered a bad thing.
*The latin essentially means: correlation does not equal causation