PicoBlog

A few months ago I had an idea to combine the scouting community's knowledge with analytics to predict the outcomes of draft prospects more accurately. Today I am sharing how the model works and its Top 100 prospects, and will be publishing position breakdowns in the coming days during the lead-up to the draft.

There are three primary components to my model:

And I’m doing all of this using data and information that can be accessed publicly.

I believe this method can add to the accuracy of predictions at each position and also better order the positions by on-field value. From a position-agnostic standpoint, Brock Bowers is one of the best players in this draft. That doesn’t necessarily mean he should be drafted ahead of the 3rd-5th players at some positions simply because he’s the best Tight End. I’ll explain why my big board will have him lower than the consensus further into the article.

I’m predicting the career outcomes for every prospect in the draft using Pro Football Reference’s Approximate Value (AV) metric as the dependent variable. Each position has its own model (random forests) instead of using a one-size-fits-all approach.

AV has some faults, with many in the analytics community taking issue with how it divides credit between players at some positions. And there are more predictive single-number metrics available, such as PFF’s Wins Above Replacement.

However, I chose to model AV instead of other metrics for a few reasons.

PFF’s write-up of their WAR metric showed that while PFF WAR is more stable year-to-year, AV demonstrates predictive value.

It’s also cumulative, which removes the need to account for differences in playing time that would probably be necessary if I were predicting EPA/play or PFF grade. This is especially important because of how I am ordering prospects on the overall big board (more on that later).

And finally, I’m choosing to predict the career outcomes of prospects rather than the rookie contract window. I think the two are different questions that should be considered, balancing early impact with long-term returns. And given that some premium positions take longer to develop (OT and DL), predicting the rookie contract window may result in overvaluing WRs and CBs, positions where it’s easier to have an immediate impact. In the future, I’d like to explore a model predicting the outcomes of a player’s rookie contract window but that is out of the scope of this project.

Now that we established what we’re predicting, let’s get into how the model works.

Many draft models I’ve seen on Twitter seem to only focus on production and athletic testing and don’t include scouting data to generate their projections. By doing so, they are leaving out one of the most predictive features in determining prospect success.

The Consensus Big Board is a collection of draft rankings that are compiled to produce the consensus opinion and is living proof of the power of the wisdom of the crowds. As Brad DeWees and Julia A. Minson write, it is when one asks “many people for their opinions and suggestions, and then combines them to form the best overall decision. Evidence suggests that the combination of multiple, independent judgments is often more accurate than even an expert’s individual judgment.”

Previous research by PFF’s Timo Riske and Arif Hasan (who every year compiles one of the most widely-used consensus big boards) into the topic has demonstrated how the effect applies to the NFL Draft.

The consensus ranking was shown to be predictive of NFL success for every position and is one of the most important features in each positional model.

The second step in my process is adding more color to each prospect’s profile by looking at other predictive data points.

For production metrics (all courtesy of PFF) I first determined which career metrics were sustainable from a college-to-pro standpoint at each position and then filtered down to those that were also predictive of career AV.

A similar process was used for the six athletic tests conducted at the NFL Combine and how they predicted NFL outcomes at each position.

The last two data points I considered were the level of competition and age. The degree to which they impacted a player’s outcome at the NFL level depended on the specific position.

After generating predicted AV values for each prospect, I had to figure out a way to best order the prospects in a way that balanced the positional value and quality of the player. The process I settled on leverages the NFL marketplace to tell us how we should value players at each position.

The goal of every draft pick (unless you’re drafting a kicker in the Top 100) should be to find the player with the best chance of becoming an impact player. By looking at contract data, we can see how the NFL views elite talent at each position and devise a system to translate our projected AV values into a dollar value to order the prospects at every position.

I looked at the Top 10 veteran players last year, as determined by AV, and the AAV of their contract. I then calculated the price per AV for each position and multiplied that by each prospect’s predicted AV value. This translates the position-specific AV prediction to an overall score that we can use to determine the big board order, accounting for positional value as well as the quality of each prospect.

NOTE: Some adjustments were made to the quarterback position to account for how much larger their contracts are relative to other positions to reduce the confidence level to which they should be valued over other players.

The final product is an analytically-driven big board, rooted in the insights of the collective (public) scouting community, that attempts to properly weigh each position's value.

Brock Bowers is the top tight end on my board, but I have him as the 24th overall prospect instead of his consensus big board ranking of 7th because he plays a less valuable position.

The other positions in my top 32 are QB, WR, OT, EDGE, DT, and CB, the premium positions of modern football.

One big difference I noticed from the consensus big board is how cornerbacks are valued. Some in the analytics community are beginning to believe cornerback is more of a premium-lite position given pass defense is a weak-link system and cornerback salaries have stagnated a bit. And that is borne out in my draft rankings, with the top cornerback (Quinyon Mitchell) being ranked behind the top 5 quarterbacks, top 4 offensive tackles, top 5 wide receivers, top 2 defensive ends, and the top defensive tackle.

In terms of predictiveness, the consensus big board is still more aligned with how the draft will play out. However, when looking at the early returns of recent draft classes my model shows a slight improvement in projecting career outcomes. The two exceptions are tight end and linebacker, positions the NFL has also had a hard time determining the best players.

One thing to note is currently my model only includes prospects who were invited to the NFL Scouting Combine. Next year I’m hoping to include prospects who weren’t invited and only have Pro Day testing numbers available. I just haven’t had the time to set up a system to adjust for the inflation typically seen in Pro Day numbers relative to the combine (this is a hobby after all).

I’ll have more to share in the lead-up to and during the draft including some quick positional previews. But for now, here is a preview of the player cards I will be tweeting during the draft and the model’s Top 100 players (subject to change with the final consensus big board rankings).

ncG1vNJzZmirmJa4prDOsKWtoJWjwq6uxKuqZ6ull8C1rcKkZZynnWS9cLXNramonKWYtq%2BzjKawZqaWoXqlvsCfq2aln5myrQ%3D%3D