Draft assessment has been my line of focus this draft season, and now that the draft is completed, we’re all supposed to give grades, right? Grades are fun, but the basis for them should be a large part of the consideration. If the grades you’re reading are pretending to tell you who will be the best player or have the best career, it’s time to move on. There are too many wildly impactful variables like coaching, scheme fit, scheme change over time, player desire, and medical information that any of us on the outside are kidding ourselves to say we can predict.

Positional Surplus Value

There are newer ways of evaluating draft success, and some folks give out grades for those too, but they have their own flaws. If you follow me on Twitter (@lowbuffa), you’ve heard me talking about the work Kevin Cole is doing on his site Unexpected Points. He and PFF’s Timo Riske have been working on value-based draft evaluation. The foundation of the method moves away from player performance, which can only be evaluated over time on the field, and shifts toward contractual value as the main factor in ascertaining how well a GM performed.

Simply put, each drafted player has monetary value based on the difference between his rookie contract and the cost it would take to replace that player with a similarly skilled not on a rookie deal. That difference is the surplus value. So, there is a higher amount of surplus value in each pick of a player from positions the league has demonstrated it values more.

In terms of average cap hit for a position group, EDGE is the most valued position in the league. It overtakes QB because most teams carry more EDGE players than they do QBs, and because so many teams pursue rookie deal QBs because the cost so high otherwise. This chart is fed from data by Over The Cap and Spotrac, and there is overlap in which players are considered EDGE and LB, which appears to drive up the price for LB. Another note, the cap numbers in this article are from 2022, because the 2023 numbers between the two sites don’t line up or are generally wonky still as free agency and the draft settle out.

Because they skew the data, Cole removes QB from the equation and treats them as their own entity. To put his theory into practice, Cole simplifies down into tiers, so you have Non-QB Tier 1 positions, which are T, IDL, and EDGE. Tier 2 positions are CB and WR. Tier 3 positions are IOL, LB, RB, TE, and S. Yes, we’re tossing out specialists. You’ll notice the tiers don’t exactly line up with positional value as cap percentage. Cole says he looks at the average top 5 or 10 salaries, but that doesn’t align either.

Please don’t get me wrong. I’m not trying to undercut Cole’s work. I believe he has a methodology for determining positional group value. It’s just that whichever source he is pulling data from does not match what we have to work with to explain why his work and what you’ll find herein don’t perfectly line up.

The main point is to give you an idea of some of the other work being done in draft evaluation. There is real value in reexamining how we critique drafting, and positional surplus value has worth and can give the immediate results we’re all looking for. Cole’s positional surplus value assessment of the Bills’ draft does not favor a team that took TE, IOL, and LB – three Tier 3 positions – with its first three picks.

From Unexpected Points

If you’re interested in the rest of Cole’s categories, you should check out his AFC Analytical Draft Grades. As a forewarning, it is paywalled. Quickly though, trading up is significantly frowned upon, so going up even two spots in Rd 1 gets a negative score that other later-round trades don’t make up for. Discipline refers to how closely actual draft slots came to consensus pre-draft rankings (which we’ll look at below). So, in terms of value in this methodology, the Bills didn’t do so hot. Here’s where we get to the main problem with this methodology though, did you catch that line earlier where we said it shifts away from player performance? Be sure to note that this entire method, while fascinating and offering real time evaluation, misses the crux of drafting in the first place – acquiring players who will play well.

Beane and Surplus Value

I thought that the Bills might score better in positional surplus value if we assessed the draft against how the Bills have paid out for positions instead of the comparing it to the entire NFL. Each team has its own valuations based on scheme, existing players, and existing contracts. It seemed only fair to solve for surplus value based on how the Bills value positions.

Again, these are 2022 numbers, and Josh Allen’s contract will continue to push higher (even though his 2023 contract still only has a cap  hit of $18.6M). You can see the Bills are relatively similar to the NFL with a couple of exceptions. They value CB a great deal more at close to twice the cap percentage. They also have a lower cap percentage to DEs than typical. That could very well be impacted by all the draft capital spent there. Having so many rookie contracts filling the position keeps the price low.

Before I went further and tried to reverse calculate Cole’s formulas for positional value, it seemed wise to figure out if the Bills even remotely follow this thinking in how they have drafted in the McBeane Era. If they do, there would be a general downward “trendline” if we chart their picks by round and position against their positional spending.

Here, each X is a player drafted at that position by round from 2017 through 2023. A red X is a 2023 pick. If the Bills were following a positional value approach, we’d see a cluster of X’s moving from the top left down to the bottom right.

It’s not there.

Not even a little bit really. And it doesn’t magically appear when we look by NFL positional values or Cole’s Tiers.

Going by Cole’s Tiers actually appears to come closest, but we’ve already established those tiers don’t mesh well with the way Beane and Company invest the Pegula Bucks. I even looked to see if using the Bills’ average positional RAS scores correlated. Nope. We can’t say the Bills are drafting with positional value as a leading factor.

So, what can we say?

Best Player Available or Need?

The traditional schools of thought on how GMs approach a draft boil down to finding value or filling holes. Beane has often said they are looking for the best players.

“We’re going to take the best player,” Beane shared. “We really are. There’s going to be a good player there.”

From The Buffalo Bills

From outside the Bills’ war room, BPA is generally formed through pre-draft rankings. The draft community’s work on consensus pre-draft rankings is bringing the rest of us closer to how the draft actually falls. We’ll always be far from 100%, but folks like Arif Hasan at PFN, Mock Draft Database, and Ben Robinson at Grinding the Mocks are building consensus tools that offer a “wisdom of the crowd.” If the Bills are consistently taking BPA, there might be a relationship between the players they select and consensus ranks.

Using 4 different consensus boards to create an average expected ranking and comparing that rank to where the Bills actually selected a player gives us an idea of value. We’ll never know the one board that truly matters, but even though all the work available to us is from the outside doesn’t mean we can’t learn anything, as long as we’re willing to keep those caveats in back of mind.

The Bills’ best value was Nick Broeker, as a steal of 60 spots. However, that value – and the reach on Justin Shorter – are lessened by the fact that the reach/steal scale is reduced deeper into the rankings. Overall, the best value the Bills got was O’Cyrus Torrence, a talent ranked toward the end of the 1st round that they drafted at the back end of the 2nd without needing to trade up. Plus, he filled a position of need.

Mock Draft Database maintains a record of each teams’ needs heading into the draft year by year. Most every Bills’ fan could name the needs from this last year. Again, this is a consensus list of a variety of sources, so we eliminate the possibility of being trapped into a single point of view. The classifications of Urgent, Necessary, and Should have been added.

The needs chart lets us look at the Bills’ picks versus that year’s needs. The chart below is slightly different from the previous versions above. A red X is no longer for a 2023 pick, but rather an Urgent Need. A blue X is Necessary, and it follows that a green X is Should. A black X is considered Unlisted.

There is red throughout the chart, suggesting that Beane looks to address Urgent needs regardless of round, but pay attention to the degree of red in the first two columns, the first and second rounds. From 12 1st/2nd Rd picks, 8 have been on Urgent needs. Eleven of 12 have been on Urgent and/or Necessary needs. Here I’ll argue that it’s truly 12/12 because – positional purists be damned – Dalton Kincaid is more pass catcher than anything else, and thereby satisfies the 2023 Urgent need of receiver.

The totals give the game away.

If you accept the Kincaid renegotiation, then every Day 1 or 2 pick has been used to meet a need, and all told, 36 of 42 picks have gone to need. The sweet spot is when those needs meet value in players, but Brandon Beane drafts for need. That might sound simplistic, but before this project I genuinely believed Beane was using FA to allow for a modified BPA approach. Now that the results are quantified, that belief has to adapt. Also, Beane hates the 4th rd, but we knew that.

Conclusion

The McBeane Era Bills use the draft to meet needs. It appears they use free agency not to permit going a pure version of BPA, but more to ensure roster needs are given a floor if the draft refuses to fall their way. It’s not the “gimme athletic freaks early” mantra that it seemed like Beane was espousing for the last couple of years. This approach feels safer, more managed.

The results were not what I expected to find, but I hope you still find value in this piece because I think we did learn something about the how the Bills attack the draft.