Offseason strength of schedule is not a good prediction metric

Over the last few days, I’ve started seeing articles like this one pop up on the Internet. It’s a yearly exercise that many NFL analysts perform: using records from the previous season to predict the strength of schedule a team is going to face in the upcoming season.

Is it relevant? Is the difficulty of a team’s schedule before the season starts a useful metric in figuring out how the team is going to perform in the season itself?

Not really, no.

I took a look at how the whole thing played out in practice last season. I compared each team’s preseason strength of schedule to how things actually turned out, and the results were somewhat telling:

If you can’t zoom in on the chart, allow me to explain. The teams are listed from left to right in order of their preseason strength of schedule; the hardest schedules are on the left, and the easiest are on the right. The bars indicate the difference between the projected strength of schedule and the actual outcome; the blue bars indicate those teams had an easier schedule than the preseason rank had indicated, while the white bars show teams where the opposite was true.

You might notice that there’s a clear delineation after the eleventh team. I didn’t construct the chart that way...there was a natural break in the data at that point. The teams with the 11-most difficult preseason strength of schedule ratings (a group that includes the Bills, at 10) all ended up with easier schedules than the projections gave them. Most of the other 21 teams all ended up facing harder schedules than were indicated before the season (the Titans are an exception, as their preseason SOS and actual SOS were both .473).

Of course, prognosticators don’t look at strength of schedule to predict what a team’s eventual strength of schedule is going to be in the next season. They use it to help predict what team’s record is going to be.

So, does preseason strength of schedule do a good job of foretelling a team’s eventual record in the upcoming season? Not even a little bit.

Here’s a simple scatter plot, comparing each team’s preseason strength of schedule to their actual winning percentage:

Even without being able to zoom in, you can probably notice that there’s almost no correlation between the two; the points are all over the place. For example, there were two teams tied for the most difficult preseason strength of schedule heading into 2016: one of them (the Falcons) played in the Super Bowl, and the other (the 49ers) is picking second overall in the draft.

At the the opposite end of the spectrum, there were two teams tied for second-easiest schedule at the start of the year. One of them, the Giants, won 11 games and made the playoffs. The other, the Bears, won three games and are picking third overall in the draft.

There’s a measurement in statistics called correlation coefficient. Basically, it’s a measurement on a scale of -1 to +1 that measures the linear relationship between two sets of data; as the correlation coefficient moves closer to either -1 or +1, it’s a better indicator that one set of data is closely related to the other set. If it’s close to zero, that’s an indicator that one set of numbers doesn’t have much to do with the second.

The correlation coefficient between a team’s preseason strength of schedule and their eventual winning percentage is -0.0861. If you want mathematical proof that the slate for a team going into a season has almost nothing to do with how the season will turn out, well, there you go.

You already knew that, though. I already knew that, and everybody who writes one of these preseason strength of schedule articles already knows that. There’s so much that goes into a team’s record that’s entirely independent of their schedule that using the preseason slate to try and project the final outcome is an entirely fruitless endeavor.

Keep that in mind when you read about the Bills having the fifth-hardest schedule for 2017. It might look that way now, but it’s almost certainly not going to end there, and even if it does it’s not going to matter as much to their record as their actual performance will.