Causation

Cold, Hard Football Facts does an excellent job finding a correlation between particular statistics and team wins. What CHFF does not always do, however, is show a causation between particular stats and team wins. For example, CHFF has clearly shown there is a correlation between throwing multiple interceptions and losing playoff games (see here, here, and here). What I do not see, however, is clear evidence that not throwing interceptions causes a team to win playoff games. The data CHFF provides could be explained by a combination of two other factors:

1. Teams that fall behind tend to call more pass plays and more high-risk pass plays.
2. Inferior teams tend to throw more interceptions against superior teams.

These factors combine to make losing teams throw a lot of interceptions: superior teams jump out to leads against inferior teams, then the inferior team has to throw the ball more, but because the opponent is superior, it is able to stop the inferior team, sometimes by intercepting the ball. Furthermore, if superior teams are more likely to win the game in general, and if inferior teams are more likely to struggle against a superior team in general, a team with more interceptions may end up with losses more often in general, without the interceptions necessarily being the cause of the loss.

In other words, falling behind early causes interceptions, and being an inferior team causes interceptions. This is just a theory--I'm not really proving anything. I respect CHFF for finding and frequently citing the correlation between interceptions and playoff losses, but I'd like to see CHFF prove the causation between not throwing interceptions and winning playoff games. For example, in this 2006 article, CHFF cites Dan Marino's interceptions in playoff losses. But in Marino's ten playoff losses, seven times his Dolphins were playing against a team with a better record, and in eight of the games, his Dolphins were down by more than 10 points at some point during the game. I'd want to look closer at detailed box scores, but it's possible that my theory above about reverse causation of interceptions would hold for Marino. Incidentally, in Marino's playoff losses, the Dolphins allowed an average of 34.5 points per game--I think it quite reasonable to claim Miami's poor defense was a major cause of Marino's playoff losses (I might ask questions about the other examples cited. Is it incidental that Tom Brady started throwing playoff interceptions when his 10-6 team had to play on the road against a 13-3 team? And how about team context--did playing with a bunch of Hall of Famers help Bart Starr win a lot of playoff games--and possibly avoid interceptions?).

There is another recent situation in which Cold, Hard Football Facts finds a correlation without attempting to prove causation. Here, Kerry Byrne looks at how home field advantage has disappeared and the playoffs have been difficult to predict since 2002's realignment. But, the reason road teams may be performing better since realignment is actually brought up in Byrne's article--sometimes the Wild Card road team is superior to the Division Champ home team (regardless of record--a weaker team might have a better record because of a poor division, and a Wild Card team might have a lesser record because of tougher division competition). Byrne may be right that realignment devalues the regular season (I don't think so--it remains true that only 12 of 32 teams make the playoffs, meaning 62.5% of teams don't make the playoffs. You still must perform well in the regular season just to get to the wide-open playoff). But home field advantage may have disappeared because the home team is simply not so superior to the road team.

And does realignment account for all the silliness of 2008? For laughs, let's try put the 2008 AFC season into pre-2002 division alignment. It is hard to do--we can't simply transfer the 2008 teams' records to 2001 divisions because their schedules--and thus likely records--would be different. But we'll do our best. San Diego would still win the AFC West (and they'd probably have 9-10 wins, since they'd get to add two games against Seattle to their schedule). Indianapolis moves to the AFC East, and probably wins it, though it would be a tight battle between the Colts, Dolphins, and Patriots (possibly the Jets). The AFC Central would be wild, with Tennessee, Baltimore, and Pittsburgh beating up on each other twice a year (this season Tennesse crushed Pittsburgh, Pittsburgh swept Baltimore, and including playoffs, Baltimore split with Tennessee). But in such a conference, the Patriots might still miss the playoffs (though they might make the playoffs because Tennessee, Baltimore, or Pittsburgh may have had a lesser record because of the difficult schedule). And such alignment did, after all, lead to an 8-8 team making the playoffs and an 11-5 team missing the playoffs in 1985 (thanks Bismuth).

Here, Byrne further argues that realignment has created a mess of the playoffs, writing

"Consider the chaos of the past four years, and the unlikely champions it’s yielded"

I would argue, however, that chaos hasn't exactly had such a long reign. In 2005, the Pittsburgh Steelers were a #6 seed, but an 11-5 team (the same record as the division winning Bengals, but with better point differential). This is not quite so chaotic: 11-5 teams won the Super Bowl in 1980 and 2001, before realignment (an 11-4 team won in 1987, a 10-6 team in 1988). In 2006, the Indianapolis Colts were a #4 seed, but 12-4 teams have won seven Super Bowls. the surprising 2007 Giants could be a mark of chaos, but if 11-5 Baltimore or 12-4 Pittsburgh wins this year, it will be hard to call 2008 chaos.

Byrne then shows that while home-field has been an historical strong advantage, that advantage hasn't shown in three of the past four seasons. Byrne is right to note that realignment is rewarding some lesser teams with homefield advantage. And Byrne may be onto something by finding a correlation between four-team divisions and the disappearance of home-field advantage. What I don't see Byrne showing, however, is that realignment caused home-field advantage to disappear.*

Byrne is correct to dismiss the myth of parity as an explanation (I've doubted and challenged the myth of parity for a while). But Byrne shifts blame for playoff chaos to realignment of four four-team divisions, and makes an historical comparison to 1967. Byrne concludes:

"In the expansion and realignment of 2002, the NFL spit up the lessons in moral hazard it should have digested in the late 1960s. So what we have today is postseason chaos on an even larger scale: a system in which 8-8 teams host playoff games against 12-4 teams, 9-7 teams host not one but two playoff games, 11-5 teams sit at home, and a pair of nine-win teams battle for the right to go to the so-called Super Bowl.

"It's not a pretty picture. And with no rival league and no merger on the horizon, the NFL needs to find another way to recapture the importance of its bone-crushing regular season and rescue the dignity of its once-proud postseason."

If the argument is that the system unfairly rewards lesser teams with home games and leaves good teams out of the playoffs, then I agree with Byrne. I'm even willing to entertain arguments for a restructuring of the playoff system.

However, Byrne shows no evidence that four division realignment is the cause of playoff upsets or the reason for the diminishing home-field advantage. If it is Byrne's goal to show four team realignment causes these two effects, he has shown absolutely no evidence that it does so (and if that is not his goal, then why did he bring up the 2005-2008 anomoly teams or the diminishing records of home teams?). I am not saying it is not a cause--I'm saying, show me the evidence. How do four divisions account for 11-5 Pittsburgh winning games at 14-2 Indianapolis and 13-3 Denver? How do four divisions account for 10-6 New York winning games at 13-3 Dallas and 13-3 Green Bay, then winning on a neutral field against 16-0 New England? How do four divisions account for 9-7 Arizona winning at 12-4 Carolina, 11-5 Baltimore winning at 13-3 Tennessee, or 9-6-1 Philadelphia winning at 12-4 New York? How do these events specifically "[call] into question the wisdom of four-team divisions and the realignment of 2002"?

In this case, again, CHFF has shown a correlation--after the 2002 realignment, there have been some surprising champions, and home-field advantage appears to be diminished. But why? How does realignment to four four-team divisions cause upsets, or diminish home-field advantage for superior teams?

As Stephen's Guide to Logical Fallacies points out, "the relationship between cause and effect is a complex one" (and follow the link to see how the cause-effect relationship sometimes gets misconstrued. For relevance to this post, see specifical the Post Hoc fallacy). Cold, Hard Football Facts sometimes notices correlations that others don't, and I commend the writers for that. CHFF provides historical insights and relevant statistical facts that are extremely useful for football fans. But sometimes CHFF presents correlations and implies causation, without fully exploring the cause-effect relationship.

*It may be implied that home-field advantage is disappearing because inferior teams are being granted home games. If that is the case, however, then the superior team is still winning the games, and that is an explanation (in which case the only complaint is that those superior teams should have had home games, not that they won games they shouldn't have). But this explanation is not used, would not support Byrne's earlier finding of "chaos," and it would not necessarily explain how teams like the 2005 Steelers, 2007 Giants, or any of 2008's surprise teams won several road games against teams with better records.