Thursday, January 15, 2009

Causation

Cold, Hard Football Facts does an excellent job finding a correlation between particular statistics and team wins. What CHFF does not always do, however, is show a causation between particular stats and team wins. For example, CHFF has clearly shown there is a correlation between throwing multiple interceptions and losing playoff games (see here, here, and here). What I do not see, however, is clear evidence that not throwing interceptions causes a team to win playoff games. The data CHFF provides could be explained by a combination of two other factors:

1. Teams that fall behind tend to call more pass plays and more high-risk pass plays.
2. Inferior teams tend to throw more interceptions against superior teams.

These factors combine to make losing teams throw a lot of interceptions: superior teams jump out to leads against inferior teams, then the inferior team has to throw the ball more, but because the opponent is superior, it is able to stop the inferior team, sometimes by intercepting the ball. Furthermore, if superior teams are more likely to win the game in general, and if inferior teams are more likely to struggle against a superior team in general, a team with more interceptions may end up with losses more often in general, without the interceptions necessarily being the cause of the loss.

In other words, falling behind early causes interceptions, and being an inferior team causes interceptions. This is just a theory--I'm not really proving anything. I respect CHFF for finding and frequently citing the correlation between interceptions and playoff losses, but I'd like to see CHFF prove the causation between not throwing interceptions and winning playoff games. For example, in this 2006 article, CHFF cites Dan Marino's interceptions in playoff losses. But in Marino's ten playoff losses, seven times his Dolphins were playing against a team with a better record, and in eight of the games, his Dolphins were down by more than 10 points at some point during the game. I'd want to look closer at detailed box scores, but it's possible that my theory above about reverse causation of interceptions would hold for Marino. Incidentally, in Marino's playoff losses, the Dolphins allowed an average of 34.5 points per game--I think it quite reasonable to claim Miami's poor defense was a major cause of Marino's playoff losses (I might ask questions about the other examples cited. Is it incidental that Tom Brady started throwing playoff interceptions when his 10-6 team had to play on the road against a 13-3 team? And how about team context--did playing with a bunch of Hall of Famers help Bart Starr win a lot of playoff games--and possibly avoid interceptions?).

There is another recent situation in which Cold, Hard Football Facts finds a correlation without attempting to prove causation. Here, Kerry Byrne looks at how home field advantage has disappeared and the playoffs have been difficult to predict since 2002's realignment. But, the reason road teams may be performing better since realignment is actually brought up in Byrne's article--sometimes the Wild Card road team is superior to the Division Champ home team (regardless of record--a weaker team might have a better record because of a poor division, and a Wild Card team might have a lesser record because of tougher division competition). Byrne may be right that realignment devalues the regular season (I don't think so--it remains true that only 12 of 32 teams make the playoffs, meaning 62.5% of teams don't make the playoffs. You still must perform well in the regular season just to get to the wide-open playoff). But home field advantage may have disappeared because the home team is simply not so superior to the road team.

And does realignment account for all the silliness of 2008? For laughs, let's try put the 2008 AFC season into pre-2002 division alignment. It is hard to do--we can't simply transfer the 2008 teams' records to 2001 divisions because their schedules--and thus likely records--would be different. But we'll do our best. San Diego would still win the AFC West (and they'd probably have 9-10 wins, since they'd get to add two games against Seattle to their schedule). Indianapolis moves to the AFC East, and probably wins it, though it would be a tight battle between the Colts, Dolphins, and Patriots (possibly the Jets). The AFC Central would be wild, with Tennessee, Baltimore, and Pittsburgh beating up on each other twice a year (this season Tennesse crushed Pittsburgh, Pittsburgh swept Baltimore, and including playoffs, Baltimore split with Tennessee). But in such a conference, the Patriots might still miss the playoffs (though they might make the playoffs because Tennessee, Baltimore, or Pittsburgh may have had a lesser record because of the difficult schedule). And such alignment did, after all, lead to an 8-8 team making the playoffs and an 11-5 team missing the playoffs in 1985 (thanks Bismuth).

Here, Byrne further argues that realignment has created a mess of the playoffs, writing

"Consider the chaos of the past four years, and the unlikely champions it’s yielded"

I would argue, however, that chaos hasn't exactly had such a long reign. In 2005, the Pittsburgh Steelers were a #6 seed, but an 11-5 team (the same record as the division winning Bengals, but with better point differential). This is not quite so chaotic: 11-5 teams won the Super Bowl in 1980 and 2001, before realignment (an 11-4 team won in 1987, a 10-6 team in 1988). In 2006, the Indianapolis Colts were a #4 seed, but 12-4 teams have won seven Super Bowls. the surprising 2007 Giants could be a mark of chaos, but if 11-5 Baltimore or 12-4 Pittsburgh wins this year, it will be hard to call 2008 chaos.

Byrne then shows that while home-field has been an historical strong advantage, that advantage hasn't shown in three of the past four seasons. Byrne is right to note that realignment is rewarding some lesser teams with homefield advantage. And Byrne may be onto something by finding a correlation between four-team divisions and the disappearance of home-field advantage. What I don't see Byrne showing, however, is that realignment caused home-field advantage to disappear.*

Byrne is correct to dismiss the myth of parity as an explanation (I've doubted and challenged the myth of parity for a while). But Byrne shifts blame for playoff chaos to realignment of four four-team divisions, and makes an historical comparison to 1967. Byrne concludes:

"In the expansion and realignment of 2002, the NFL spit up the lessons in moral hazard it should have digested in the late 1960s. So what we have today is postseason chaos on an even larger scale: a system in which 8-8 teams host playoff games against 12-4 teams, 9-7 teams host not one but two playoff games, 11-5 teams sit at home, and a pair of nine-win teams battle for the right to go to the so-called Super Bowl.

"It's not a pretty picture. And with no rival league and no merger on the horizon, the NFL needs to find another way to recapture the importance of its bone-crushing regular season and rescue the dignity of its once-proud postseason."

If the argument is that the system unfairly rewards lesser teams with home games and leaves good teams out of the playoffs, then I agree with Byrne. I'm even willing to entertain arguments for a restructuring of the playoff system.

However, Byrne shows no evidence that four division realignment is the cause of playoff upsets or the reason for the diminishing home-field advantage. If it is Byrne's goal to show four team realignment causes these two effects, he has shown absolutely no evidence that it does so (and if that is not his goal, then why did he bring up the 2005-2008 anomoly teams or the diminishing records of home teams?). I am not saying it is not a cause--I'm saying, show me the evidence. How do four divisions account for 11-5 Pittsburgh winning games at 14-2 Indianapolis and 13-3 Denver? How do four divisions account for 10-6 New York winning games at 13-3 Dallas and 13-3 Green Bay, then winning on a neutral field against 16-0 New England? How do four divisions account for 9-7 Arizona winning at 12-4 Carolina, 11-5 Baltimore winning at 13-3 Tennessee, or 9-6-1 Philadelphia winning at 12-4 New York? How do these events specifically "[call] into question the wisdom of four-team divisions and the realignment of 2002"?

In this case, again, CHFF has shown a correlation--after the 2002 realignment, there have been some surprising champions, and home-field advantage appears to be diminished. But why? How does realignment to four four-team divisions cause upsets, or diminish home-field advantage for superior teams?

As Stephen's Guide to Logical Fallacies points out, "the relationship between cause and effect is a complex one" (and follow the link to see how the cause-effect relationship sometimes gets misconstrued. For relevance to this post, see specifical the Post Hoc fallacy). Cold, Hard Football Facts sometimes notices correlations that others don't, and I commend the writers for that. CHFF provides historical insights and relevant statistical facts that are extremely useful for football fans. But sometimes CHFF presents correlations and implies causation, without fully exploring the cause-effect relationship.

*It may be implied that home-field advantage is disappearing because inferior teams are being granted home games. If that is the case, however, then the superior team is still winning the games, and that is an explanation (in which case the only complaint is that those superior teams should have had home games, not that they won games they shouldn't have). But this explanation is not used, would not support Byrne's earlier finding of "chaos," and it would not necessarily explain how teams like the 2005 Steelers, 2007 Giants, or any of 2008's surprise teams won several road games against teams with better records.

7 comments:

  1. Byrne is not happy his Patriots didn't make the playoffs.

    ReplyDelete
  2. I realized a deeper problem. Not only does Byrne provide no evidence that four-division realignment causes playoff upsets/diminished home-field advantage, Byrne doesn't even offer a THEORY on HOW four-division realignment created playoff upsets/diminished home-field advantage.

    I don't even require evidence here; I'd just like a theory on how having four divisions causes playoff upsets/diminished home-field advantage. No such theory is provided. It seems to be a pretty blatant Post Hoc fallacy (B follows A, therefore, A caused B).

    ReplyDelete
  3. I'll go ahead and further express my doubts about those "anomalous" champions, as well as a general point.

    --I think you could probably look at a lot of individual champs and call them anomalous, maybe the majority. The 80 Raiders were the first Wild Card Super Bowl champ. The 92 Cowboys were the first Super Bowl champ with a rushing leader. The 93 Cowboys were the first team to start 0-2 and win the Super Bowl. A lot of Super Bowl champs are anomalous in some meaningful way.

    --The 06 Colts are held to be anomalous in part because of their poor run defense. But CHFF themselves repeatedly point out that run defense doesn't correlate to winning (and at any rate, if they were poor at run defense, they were obviously excellent enough in other facets to go 12-4).

    --Obviously, the more teams that make the playoffs, the greater the chance that the best regular season team won't win the Super Bowl. And obviously, the more teams you allow into the playoffs, the more teams have a chance to win the Super Bowl (including teams that were weaker in the regular season, that wouldn't have made the playoffs at all decades ago--when they are let into the "playoff," they have a chance). So when Byrne calls these teams "an anomaly by historical standards," that standard really goes back to 1990, when the NFL expanded playoffs to 12 teams, not back to '66 when the Super Bowl started. That's not quite so impressive an anomaly.

    I think that a lot of four year stretches lead people to think there's some massive change going on. From 99-01, out-of-nowhere teams won the Super Bowl, and so a lot of people cried "parity!" In reality, this was a period of transition as some of the 90s powers diminished (another time of transition was 80-82, when some of the 70s powers diminished and three relatively upstart teams won the Super Bowl). A wider perspective, I think, won't make the last four seasons that anomalous.

    ReplyDelete
  4. I think you are exactly right on the interceptions correlation v causation issue.

    [] Logically, teams that are behind to begin with throw more often to catch up (especially as it gets late in the game), throw against Ds that are expecting it, thus throw "uphill" against aggressive pass rushes and DBs sitting on the pass ... and so throw more picks at a higher rate than teams that are winning.

    [] Empirically one can see this in QB situational "splits", as NFL stats show QBs have significantly higher passing ratings when playing ahead than behind.

    E.g. from my clip file for 2006...

    Tom Brady's passing rating when:
    Ahead by 9-16 .... 136
    Ahead ............. 99
    Behind............. 67
    Behind by 9-16 .... 57

    Drew Brees, MVP runner-up:

    Ahead .............. 106
    Behind .............. 87

    Tony Romo:

    Ahead .............. 121
    Behind .............. 87, etc.

    So the quality of a team's defense can materially affect the quality of its QB's play and his rating, another example of the QB getting both excessive credit and blame in a team game ... but I digress.

    [] Anecdotally, I have the misfortune to be a long-time Jets fan, and Pennington 2007-2008 is a great example of all of the above.

    2007: The Jets had a dreadful awful D during the first half of the season while CP was starting. Time and again he had the lead for most of the game but after the D collapsed in the second half, he spent the last minutes chucking uphill to try to come from behind. Time and again the game ended a loss after he threw a bad-looking pick.

    Jets fans' reaction: "Killer pick after killer pick! Popgun with his noodle arm can never carry this team to a big win, especially from behind when you really need it".

    So the guy was benched and run out of town and replaced with Favre, though career-wise CP had one the lowest pick rates in NFL history ... and replacing him, or anyone with Favre to reduce bad picks ... hey! ... but I digress again.

    2008: Penny returns to near league-best pick rate with 7 in 16 games.

    Then, in the playoffs against a much superior Ravens team, playing almost all the way from behind, he throws 4 in one game.

    The Jets fans who had wanted to get rid of him say, "See, we were right! Popgun and his noodle arm can never beat a good team -- he lost that game by throwing 4 picks! How could they win when he did that?"

    CHFF would probably agree. But I'd suspect the causation was probably the other way around. And that not recognizing the distinction between correlation and causation can lead teams to make bonehead personnel decisions such as may leave them without a decent QB for years to come ... but, oh, enough digressing.

    ReplyDelete
  5. It occurs to me that the 2001 Patriots were a far bigger fluke than the 2005 Steelers, 2006 Colts, or 2007 Giants. That team was average or below average in pretty much every statistical category as I recall. At least the last three champs were known to be exceptional in at least area, and the Giants showed big improvement over the playoffs that carried into this season. As Football Outsiders pointed out, if you discount Weeks 13-16, when the division was already in hand, the Cardinals rank as the 7th best team (in DVOA of course). So even if they win the Super Bowl, I don't know how much of a fluke it'd be.

    And I'm not convinced that the supposed diminishment of home-field advantage is anything but random variation. HFA is significantly smaller against division rivals, and maybe more division rivals are meeting in the first two rounds now.

    ReplyDelete
  6. Derek, the 01 Patriots ranked 6th in points scored and 6th in points allowed, but the other numbers are pretty average (19th in offensive yards, 24th in defensive yards, 24th in rush yards per attempt and 21st in rush yards per attempt allowed, 15th in net yards passing per attempt and 20th in net yards passing per attempt allowed. It's interesting comparing them to the 07 Giants, who were 14th in points scored and 17th in points allowed, but were actually a top-10 rush offense and top-10 rush defense.

    One thing I think we may be seeing (with the 06 Colts and 08 Cardinals)--if you take one outstanding unit to the playoffs (in this case offense), the other unit can get hot or lucky and perform well enough for a deep playoff run. In the Colts' case, they had an elite offense, but in the first two rounds of the playoffs the defense was outstanding in leading them to wins.

    ReplyDelete
  7. Anonymous8:48 PM

    Cold Hard Football Facts is a crap website and Kerry J. Byrne is a pile of garbage that should be committed!

    Good job Pacifist Viking, you did a good job ripping apart Byrne's BS. I personally like having 8 divisions. Sure it has it's flaws but the same can be said about any type of realignment and the fact that the 8 division set-up allows a team to play home and away against all (remaining) 31 teams within an 8 year span is good enough to outweigh any negatives that the set-up has.

    If they went back to six divisions, it would be a chaotic scheduling nightmare as you would have 4 five team divisions and 2 six team divisions which would horrible. All in all I'm proud of the 8 division alignment. Good work with your rant too Pacifist Viking.

    Oh and don't worry, I can guarantee you 100% that the Minnesota Vikings are NEVER gonna move. You have my full support.

    ReplyDelete