FanPost

Improving Penalty Stats: Assessing Penalty Harm

Alright, so Dan isn't the only one looking to get better stats. I already invented a new penalty stat to calculate the Predicted Penalty Disparity between a team and their opponents to help understand differences from year to year. I have some validity testing to do on that still, but it's my hope that maybe it could be used in a couple other ways.

Anyway, this is not really about that. The single biggest question I always seem to see with any penalty discussion is some variant of "How bad did ___ hurt the team?" It's always also followed up with something like "Do the Bills shoot themselves in the foot more often and/or worse than other teams?"

Well I think I have a stat to answer those questions! And I could use a little help in refining it before putting it into action. I hope to have it mostly settled by the regular season so I can go live with it for the Bills (and perhaps their opponents). I for SURE won't have enough time to do the whole league, but it would be nice to calculate "Buffalo vs. Opponents."

What I'm proposing is a simple, straightforward way to give a "grade" to each penalty. Now, despite being "simple" it will take a lot of effort to grade each one individually which is why I don't think a league wide look by a single person is feasible.

Ultimately I'd like to judge the harm (or benefit) caused by a penalty by measuring based on three criteria:

  1. Points called back due to penalty. This would also encompass "likely" points. For instance, a team on 4th down in FG range has a penalty that takes them out of range
  2. Distance lost. Field position matters, so the yardage given up needs to be accounted for. This would account for the direct penalty yardage AND any yardage called back,etc.
  3. Opportunities given. Offside on defense on 3rd and long is not as big a deal as 3rd and short. Giving that first down up hurts a lot. Same idea with takeaways. If a teammate recovered a fumble but the play was called back because of penalty, that's a big deal. More so than the yards it lost.
The problems arise in calculating and comparing the different harm types. Two issues specifically. Which aspect hurts the most? Should all the harm be cumulative, or should there be a hierarchy?

For the first question, I would prioritize points as the worst. Probably opportunities given up as second most harmful, and finally yardage. You'll see my proposal below.

For the second question, I'm leaning toward cumulative. What that means (I'll have scenarios below to illustrate) is that you'd look at all three aspects and simply add them up. Working on a hierarchy on the other hand would first look at say...points...and assign the rating strictly on points and forget the other two considerations. If no points were altered by the penalty, then assess based on opportunity while forgetting yardage, etc.

Here's what I have for a scoring system so far (feel free to comment):
  • 0.1 units for every yard given up. This creates a maximum 9.9 points for yardage (a 99 yard DPI penalty for instance)
  • 1 unit for every point affected. A chop block that negates a TD for instance would get 7 points. While this does create a situation where yardage theoretically could be a bigger harm than points (9.9 over 7), if a cumulative assessment was done, this hypothetical chop block would be 7 + yardage lost which would likely put it close to the 9.9 (8+ in most situations)
  • Field Goals get tricky, so I think a simple route would be to carve the field into thirds. A defensive penalty that allows the opponents to get to your 33 or so, it opens up a FG in the mid 40s (highly likely to be made) and that's decently clean of a break without making a FG opportunity too dependent on other statistics. This is brought up mainly because...
  • Putting an opponent into a scoring opportunity is something else I'm leaning toward adding. If you place the other team into FG range and the kicker misses I don't know if the penalty gets a pass based on the kicker not doing his job
  • Opportunities given would probably be a 1 and/or 2 unit affair. I tend to see a negated turnover as worse than an extra first down so I'm proposing 1 unit for any penalty giving the other team a first down and 2 units for any penalty that negates a turnover. This would make a first down penalty equal to a 10 yard penalty.
One other consideration...
I'm contemplating just using actual results to assess the damage based on points. What I mean is that a TD that gets called back isn't 7 points right away. You get to replay the down usually and if the team still scores a TD then the damage was really the yardage. If it turns into a FG, then it'd be assessed as 4 units of harm instead of 7, etc. For FG, giving an opponent a realistic opportunity (33 or inside) they wouldn't have otherwise had would be counted. So for instance, on a 3rd and 10 the opponent is at the 40 and gains 3 yards. The play itself keeps them out of the range I selected if a clean stop. However, a 5 yarder puts them at 3rd and 5 at the 35 and they gain that same 3 yards. Well now they're 4th and 2 at the 32 instead of 4th and 7 at the 37 due to the penalty. That instance would be assessed 3 units.

This would be fundamentally the same for offense. If a play resulted in a first down that was called back, it'd be the same 1 unit plus actual yardage lost.

This could result in assessment of "good penalties." A DPI breaks up a TD it would be assessed -7 units (-4 if the other team ultimately got a FG) which could outweigh the yardage lost, etc. Or at least come out as a wash

Finally, some examples from the Bills vs. Giants last year. Play by play summary is from nflpenalties.com
  • E. Manning throws incomplete pass from NYG 38. Gilmore assessed DPI for 8 yards. 8 yards + 1st down = 1.8 Units of Harm
  • E. Manning incomplete pass at NYG 6. L. Donnell false start assessed for 2 yards. Down replayed so yardage assessment only = 0.2 Units of Harm
  • Manning pass to D. Harris for 13 yard gain on 1st and 10 (NYG 44 to BUF 43). S. Vereen called for OPI for 10 yards. 10 yards + 13 yards + lost 1st down = 3.3 Units of Harm
  • Here's a fun one... J. Brown kicks 44 yard FG which is good on 4th and 2 from BUF 25. K. Williams called for illegal formation for 5 yards. Drive continues and results in TD. 5 yards + 1st down + change of 4 pts. in game = 5.5 Units of Harm
  • T. Taylor from NYG 31 runs for TD. K. Urbik called for holding and assessed for 10 yards. Drive resulted in realistic opportunity for FG (Carpenter missed but from NYG 17 which he should have made using my rule of thirds). So then 31 yards + 10 yards + 4 points difference = 8.1 Units of Harm
So far I really like it. It jumps out that Urbik's holding was far and away a bigger problem for the Bills than Gilmore's DPI call earlier in the game. Illegal formation isn't generally a big deal, but Kyle's under context really did have a big impact on the game. These two plays directly account for 11 points of swing in a 14 point loss and their "Harm Rating" accurately reflects some pretty big damage done. Woof. Vereen's OPI isn't assessed as particularly backbreaking using these scores, but reflects a decent hole he put the team in. It turns out they converted eventually, but at one point they were 3rd and 27. In most situations that'd be essentially a lost drive,which the 3.3 does a decent job of reflecting.

Let me know your thoughts!

Just another great fan opinion shared on the pages of BuffaloRumblings.com.