The dog days of the offseason are upon us, with several months until the July 28th full start of training camp but after the draft and schedule release. Sure there will be a few moves, particularly post June 1st this offseason, but the news cycle for the NFL is slowing to a crawl compared to most times of the year. As such, this is the time for most football content creators and media to create rankings, tier lists, a few thought-provoking discussions, and much hotter takes to keep audiences engaged. While no media company is above some of these practices, only one does so while claiming to be an analytical firm listened to by all 32 teams…
I’m, of course, referring to Pro Football Focus.
Before I go any further, I will say this: I have a love-hate relationship with PFF. They do an excellent job in their research department on counting statistics, both traditional and new. They try to do what few can truly hope to: quantify the game of football in all of its aspects. It’s a bold and worthy endeavor, and I applaud their efforts to do so with statistical data. They are very useful in quantifying what was previously unquantifiable. I respect their work especially in areas of the game that get glossed over by traditional stats like Offensive Line play. On the data analytics side, I really like what PFF produces content-wise.
However, PFF, like many other things in life, is subject to human error. PFF has its flaws, often when things are more subjective than empirical, most notably in their player rating system. I’ll allow PFF’s own site to describe it for you to hear their perspective.
Via PFF
Are there parts I agree with? Yes, of course. They value consistency and production, adjust for expectations, and don’t use an average; they use a scale for season-long grades. In theory, this is a powerful methodology. However, it is subjective in nature and prone to mistakes. PFF can misdiagnose what a player’s responsibility is and then value it poorly. For example, take Darius Leonard in Zone coverage. Leonard allows a high completion % because of the Colts Zone D, and sometimes gets categorized as a “poor coverage LB” by PFF. However, he is focused on limiting Yards after the catch, making good tackles, and trying to force turnovers in this scheme. Because of this, he isn’t focused on more consistent and better graded plays like Pass deflections or forcing incompletions and can be miscategorized as having a bad play by allowing the catch despite doing exactly what he is being asked to do in the scheme. Is he an elite Coverage LB? In 2019 he had a case to be made as one of the best, and while he regressed in 2020 there, he was still a solid coverage LB.
The grading scale is based on an analyst’s opinion on the play, and how they define it on the 9-grade scale can tilt the scales. Because it is independent of the cold hard data, it can also be subject to the natural human biases that we are subject to. There is nothing wrong with admitting that PFF can improve and try to shift to more data-driven analysis. Still, the player grading scale being subjective to human opinion is an additional caveat that means they shouldn’t inherently be more valued than other respected analysts.
If the grades weren’t enough, there are also the opinion pieces of PFF that can be driven by these grades. Most notably is the recent PFF player positional rankings for 2021. These are projection articles that inherently are opinion-based, but they predominantly rely on the PFF grades to justify these rankings. Colts fans are most familiar with their infamous takes this past week about Jonathan Taylor being the 16th best RB of 2021 and Darius Leonard being the 6th best LB of 2021, both by PFF’s Sam Monson. While I disagree with their opinions on both, as PFF underrates them, projections are predictions for the next year and are subject to debate. But it seems a bit disrespectful to underrate these players to such a degree. The Taylor part was particularly heinous, as it put him behind other backs like Austin Ekeler, Kareem Hunt, Chris Carson, Joe Mixon, and fellow 2020 rookie Antonio Gibson who many would not project as better backs than Taylor on a per touch and/or cumulative basis.
Do I hate PFF? Not really. They are a useful tool in the research and statistical research aspects, but fans and other media outlets have seemed to hold PFF grades up as all-knowing and universal truths of football over the last few years. The pedestal they’ve been built upon is rickety and uneven, with several holes in it. There is always room for improvement, and PFF should welcome such constructive criticism as a challenge to rise to their hype and live up to it by continuing to strive to be better. To the dozens of people with “PFF” in their Twitter handles and their supporters who will be potentially upset with me for this article, I say to their analysis:
