Prospect Valuations for Fantasy

Intro

You may have noticed over the years that RotoGraphs has not typically published dynasty rankings. This is likely for any number of reasons. There is no lack of dynasty content or rankings on other sites and, absent a ranking model that adds particular value relative to the rest of the industry, there isn’t much incentive to, as my editor put it, “vibe out a list of rankings.”

I wanted to change that, though. The full rankings model is not ready to hit the press yet, but I wanted to walk you through some of the steps and methodology that are going into my rankings because I both value transparency and think that there are useful insights to glean along the way. First up is the key differentiator in any dynasty ranking: prospects.

Overview

In a real-life context, valuing prospects has been covered extensively both here and elsewhere, including by Ben Clemens in a recent major update to our green neighbor’s valuation methodology.

These studies typically aim to assign a WAR value (and subsequently, dollars-per-WAR) to prospects over their team control years to create dollar values. This can then be totaled to determine the value of a particular caliber of prospect, or summed based on MLB team to ascertain who has the best farm system. This is less strictly applicable in a fantasy context however, because WAR doesn’t typically win our leagues.

The framework, though – assigning dollar value to prospects based on rank – is extremely useful and can be easily adapted because we already have a natural method of assigning dollar values to fantasy performance: auction values. From what I can tell, this is the first study explicitly designed to assess prospect value in terms of fantasy performance, which is a lens that comes with its own benefits and drawbacks.

We avoid having to build additional assumptions into the value framework by relying on an existing and established valuation methodology. Using a dollars-per-WAR framework is sensitive to both your WAR flavor of choice and the WAR-per-dollar methodology you use. For our purposes, dollar values are dollar values. There might be slight differences based on methodology, but by and large, we will have a fairly straightforward translation of performance to value built in.

The main drawback, and it’s not a small one, is that prospect lists consider defense into their ranks for hitters. This gets factored into WAR and the downstream value calculations. Meanwhile, for our purposes, defense is not materially a factor. This presents issues when aggregating based on prospect ranks because a player who might be a 60 FV based on their bat alone might get elevated to a 65 FV or 70 FV based on their defense. This will cause them to climb the prospect lists in a way that is not relevant to our use case. Likewise, a 50 FV player might have a 40 bat but be an elite defender and thus accrue useful real life value with terrible fantasy stats. Across the board, adjustments like this are going to suppress the aggregated values for a given prospect rank. I devised an effective (if blunt) workaround which I will discuss in greater detail later.

Data

For my data, I derived historical auction values based on the same methodology that I used in previous articles. These are based on a large export (1985-2025) of individual season data from the FanGraphs Leaderboard leveraging a multi-year range with the split-season feature. The league settings consider typical 5×5 roto scoring with 12 teams and 23-man rosters.

I garnered prospect data from The Baseball Cube, which houses the ordinal ranks of major prospect lists for each year dating back to Baseball America’s first list in 1990. While I generally prefer the FanGraphs lists philosophically, they are only easily available back to 2017. To this end, I used Baseball America’s lists exclusively. They had the longest history and allowed me to keep a relatively apples-to-apples record of performance relative to their ranks.

Methodology

I quickly realized this exercise was more complex than simply tallying average career value based on prospect rank. First off, I had to create a reasonable cutoff to avoid biasing my results from truncation (the idea that players on more recent lists have simply had fewer years to accrue value). For the analysis below, I chose 2013; this allowed for an approximate minimum of 10 MLB seasons to accrue value, which would consider a 2016 debut. Likewise, following Craig Edwards and the Clemens study linked above, I include all rankings for all years even if players are ranked multiple times.

In addition, as some of my forebears in this style of analysis note, pitchers and hitters of the same rank are not always created equal. This has leveled out somewhat in recent lists as prospect prognosticators have become hipper to pitcher bust rates, but it still stands to potentially bias results across the board.

With this in mind, my first step was to take the average career auction value, per rank, based on a hitter/pitcher split. The results, needless to say, were surprising and confusing:

Hitters were quite negative throughout the entire ranking population and significantly lower overall than pitchers. This aggressively failed the smell test.

Addressing bias

I started poking around in the data and realized that the bias I mentioned before, where defense’s factor in prospect ranks will lower the overall value of hitters, was manifesting. Hitters were being penalized in a number of ways.

For one, across the sample, pitchers more frequently missed the majors altogether than hitters. So I had a situation where a marginal bat might crack the majors, stick around for a couple of years, accrue a decent amount of negative fantasy value either through bad hitting, low playing time, or both. Meanwhile, a pitcher of similar overall talent might blow out in AA, never see the majors, and thus never count against their pitcher rank slot.

For another, to the extent that top-ranked hitters are truly elite players having long careers, most are playing past a point which they are fantasy viable and therefore accruing a large amount of negative value and biasing their overall rank downward. I suspect this is less the case for pitchers (an interesting note for future analysis); usually they are good enough to pitch or they aren’t. Miguel Cabrera got five years of run as a sub-replacement player; Adam Wainwright got one.

Last and perhaps most significantly, there are any number of glove-first hitters over the years who played frequently and for a long time. Consider the career of Alex Gonzalez, a top-10 prospect in both 1994 and 1995.

Contrast the above with 5.9 fWAR in his team control years and 11.3 for his entire career. Other prospect valuation studies would probably consider his career to be somewhat of a disappointment, below what you might expect for a top prospect but certainly not a disaster. Meanwhile for fantasy he was unrosterable for nearly his entire career.

I grappled with a few options for how to address this. For point one above, I thought about simply nerfing pitchers. For point two, I thought about assigning some type of negative value to players who never made the majors to more properly account for higher overall pitcher bust rates.

But these both would involve a significant finger on the scale of the calculation by adding new assumptions and decisions which would themselves require their own justification. Further, neither really addressed point three above, and wouldn’t really fix the notion that most of the values across the rank spectrum for either position group were negative. This would seriously hurt prospects when plugged into the downstream ranking methodology and would be counterintuitive to the entire exercise.

Where I landed was flooring season auction values at zero. To this end, I believe that what I’m measuring reasonably captures what we’re looking for – the amount of fantasy value a given player accrues throughout his career. The difference between a -$40 and -$5 season for our purposes is not especially valuable; most likely, a player would not be rostered in either instance. This floor also allowed players who played a long time but were not fantasy viable (like Gonzalez above) to not be penalized more relative to a player who never played at all. With a zero-dollar floor, Gonzalez’s career is just a hair’s breadth more valuable than that of Kiki Jones, which matches the reality of comparing the players in a fantasy context.

To me, this solution is both simple and analytically consistent with what we’re trying to measure. In other words – a good fit. I then recalculated the average career value per ranking slot accounting for this fix:

Booya! Hitter and pitcher value are now positive across the board (though this is somewhat trivial, because mechanically they can’t fall below zero). Hitters are also valued across the board higher than pitchers based on the fit lines.

Handling noise

I was now one step closer but not quite done; I still had to devise a way to deal with the noise inherent in these data. One potential way was simply to fit a trend line (like the one above) for each hitter/pitcher group and use it for each place along the ranks. This is certainly defensible, but I didn’t prefer it because it requires assuming that ranks in sequence are monotonically less valuable as the rank increases. I’m simply not convinced that this is the case.

Prospect lists since time immemorial have preached their tier-stratified nature and that individual ranks within tiers should not be construed as hard and fast, but instead more reflective of that evaluator’s preference.

Is 20-odd data points really enough time to tell whether a ranking of five is actually, genuinely better than a ranking of 10, or even whether a ranking of 51 is better than 100? As it turns out, the data say no:

Think of the tests (link for a more detailed description) above to be like t-tests for cases when the data used for them are not normally distributed (which my data emphatically are not). Note the p-values; in all cases – pitcher and hitter splits as well as combined – the mean career value of a 5th-ranked player is not statistically different (p < 0.05) than a 10th-ranked player, nor is it even for rank 51 versus 100.

To me, this is intuitive and is not something I consider to be a material flaw in the data; instead, it is just something we have to work around. If you’ve read any of my work for the past two months, you’ll know I love grouping players into buckets, so that’s what I did. After some experimenting, I settled on the following:

• Hitters: Top 2, 3 through 10, 11 through 25, 26 through 50, and 51+
• Pitchers: Top 15, 16 through 40, and 41+

I arrived at these groups by testing for statistical significance using the Kruskal-Wallis test, which is similar to the Mann-Whitney U test described above but that allows for multiple groups to be compared and identifies that there is at least one difference between the groupings. The subsequent Dunn tests compare each group’s medians as pairs to verify that they are all indeed statistically different.

Notably, these are not relative ranks within the groups. These represent buckets for the actual numerical rank of a prospect for a given year. So yes, this means that a pitcher with a number one overall ranking is treated the same as a pitcher with a number 15 ranking. I thought it was crazy too, but consider:

• Grouping top 15 pitchers in buckets of five does not produce a statistically significant difference in mean career value
• Comparing each individual rank of 1 through 5 does not produce a statistically significant difference
• Neither individual rank of one, two, or three is statistically different from rank 15

This surprised me somewhat, but ultimately I think it’s a key finding of this analysis, proving that TINSTAAPP is as evergreen as, well, ever.

Results and Discussion

Having determined some reasonable rank groupings, I looked at the mean and median career values for each:

Notably, median values are much lower than the means. This is because my distributions tend to look like this:

These are for the top two buckets in each position group; move further down the rankings and the effect will be even more pronounced.

And while they might seem somewhat low, $90 for a top hitting prospect translates to about four or five seasons as an excellent (~$15-20) fantasy performer. This represents essentially the Dansby Swanson of number-one ranked prospects; not a superstar, not a bust, but a success by any reasonable measure.

I then sought to more closely understand the types of careers existing within these buckets by looking for players whose career and peak values most closely matched the respective percentiles within the buckets. This, in my opinion, is the most fun part of the analysis – who doesn’t love to remember some guys? In each of these tables below, the Target Career and Peak Value columns represent the value of that percentile for the rank bucket. Compare it with the Player Career and Peak Values for the selected comps. I’ve removed percentiles which returned negative target peaks. Note also that a player might not necessarily be showing for their highest eventual ranking.

To me, this really tracks. Whatever your personal feelings about him may be, it’s not arguable that A-Rod is the elite of the elite when it comes to top prospects. Meanwhile, a guy like Mark Teixeira was consistently great but probably never really blew your socks off, and Jason Heyward was on net disappointing but not altogether terrible.

As we’d expect, the typical career for each bucket gets just a little bit worse as we go down the line.

By the fourth bucket, you’d have to get pretty lucky to land someone meaningfully valuable to your dynasty team:

Meanwhile, it will generally not be worth rostering hitters on the back half of the list. Nothing against any of these guys, but when Juan Rivera and Orlando Hudson are 70^th percentile outcomes, you are generally treading in rough straits.

For pitchers, unsurprisingly, the situation becomes dire much more quickly:

This is admittedly a blunt way to look at it, but consider that the hitter lists had Hall-of-Famers and Hall-of-Very-Gooders dotting the upper percentiles in the first few ranking buckets; that distinction is much sparser among the pitcher ranks.

In the final bucket, outside of Max Scherzer, there isn’t too much to see.

In addition to being the most fun, these tables are also the most useful, because they contain the values which I will plug into aging curves. I will detail this more in my next article, but aging curves seek to establish a peak value and the rate at which a player ascends to and descends from their apex. Using the results from above, I can build out a projection of a top prospect’s career based on a typical player’s aging pattern. This allows me to create an empirically rigorous forecast of a typical prospect’s full career without needing to build a full projection system with millions of simulations tailored to individual traits or players.

Conclusion

But aside from that, these results are useful on their own. I have established rank groupings (annoying but important caveat: based on data through 2013) and career values that you can use as a standalone resource. This is where attaching the player comps is particularly valuable – conjuring the image of Bryce Harper is a lot easier than imagining $250 of cumulative non-negative fantasy seasons when trying to contextualize an 80^thpercentile outcome for a top hitting prospect. Likewise, knowing that Dice-K represents the typical career of a top pitching prospect might make you think twice before spending resources on one. For every Paul Skenes, there are far more Darren Dreiforts or Chad Billingsleys. So as you go into your league’s trade season, keep these pearls in mind, and be on the lookout for more in the coming weeks.