Adjusting Projection Analysis to League Run Scoring Environment

The following analysis is beyond nerdy.

I’ll try to keep it simple as possible, but no guarantees. This past week, I examined the results of several hitter projection systems. In the comments of the first article, Mays Copeland and Skin Blues brought up a near 15-year-old thread on the InsideTheBook blog between Tom Tango and Nate Silver.

Tom stated that to evaluate projection accuracy, the values need to be adjusted to the league average. Then he spells out how to do it.

Nate Silver agrees that for rate stats, it’s best to adjust but comes back with a stipulation boxed in red.

The ever-changing baseball over the last few seasons definitely falls under the “or something like that”. But that’s not all. When fantasy managers go into the season, some of us use a system to convert projections into an overall 5×5 roto value. Most of those systems are based on the previous season or a few seasons of standing results. So, does it matter if a projection system correctly guesses the run-scoring environment? Even when I explained the concept to my co-author, Tanner Bell, I got the, “I’ll have to think about that” (i.e. hopefully Jeff’s just drunk and will forget about this conversation).

Here is an example of why I think it matters. I understand I could be wrong, but for the two people still reading, please let me know.

Assume the following player would perform differently depending on how juiced is the ball. This hitter is in the sweet spot where his production can vary quite a bit. Some sluggers will just clear the fence with any ball. Some rabbits on the other hand won’t have the strength to hit any ball out.

I did was increased or decreased his Runs, RBI, and Home Runs by 20%. Now assume a fantasy manager projected the middle stat to use in a 15-team NFBC league. In the 2021 Edition of The Process, Tanner Bell and I worked through setting up a Standings Gained formula for 2021. We ended up with:

SGP = R/13.8+HR/5.6+RBI/14.2+SB/4.3+((1733+H)/(6665+AB)-.2595)/.012

The following table applies the formula the above hitter

Depending on the type of ball used, where his profile provides value changes. If the ball softens up, his manager will be disappointed with the number of Runs, RBI, and Home Runs accumulated.

What this last exercise shows that guesstimating the correct run environment to pair with how the projections generate values could be key to extracting any value.

I went back to the home run projection and found how the projected home run environment compared to the actual one.

A near 10% range of production existed with a huge jump from the BatX to Mastersball. This quick analysis leads to two more questions. How does the RSME value compare if the stats are adjusted to the league? Also, how does the difference compare across different projections and stats?

First, here are the five stats with their original RSME and being adjusted to the league scoring environment, with the Average always coming out on top with the Bats, ATC, and Depth Charts each having a nice showing.

Overall, the projection league adjustment changes the projections just a bit with the Average now dominating all categories.

Here is a look at how the projections estimated the run-scoring environment. First, here are the raw values for reference with the percentages following.

Of the four hitting categories, only PECOTA’s batting average didn’t over project. This is not a surprise with the league run scoring down the same amount. ZiPS was the only projection that had a standard deviation over 3% for the differences in the four offensive categories. The Bat’s and the Average posted small standard deviations on the four and when paired with their previous results, they stand out.

With stolen bases, Pods and PECOTA were the only two to under projected, but as a whole everyone was way too optimistic.  When it comes to stolen bases, the two Bat’s dominate with the Average holding their own.

So getting back to the original question, of needing to adjust the projection for the league average, I think performing the analysis is the “right” answer looking back even though the result might not change. There was a major change in the projected run-scoring environment, but each of the four offensive categories missed by generally the same amount. I’m not sure if this will always be the case. Also, the results were clean since stolen bases were, on a whole, also over projected. Everything was headed in the same direction this past season with the Average of all five categories over projected by 5% with a range of 3.2% to 6.0%.

One step I’ll likely add in the preseason is to compare the projections coming into the season and point out any discrepancies. Otherwise, I’ll just add a few steps to my future analysis.