Projection Accuracy: Late March Hitter Rate Stats

I’ve been slowly working my way through the hitter projections and that journey comes to an end today as I examine how each projected hitter rate stats stand up. Besides batting average, I turn each of the counting stats into a rate by dividing by plate appearances. Finally, I adjust each value to the actual league rates. Again, any combination of projections stick out along with the BAT.

For reference, here are the projections used.

  • Steamer (FanGraphs)
  • ZIPS
  • DepthCharts (FanGraphs)
  • The Bat
  • The Bat X
  • Davenport
  • ATC (FanGraphs)
  • Pod (Mike Podhorzer)
  • Masterball (Todd Zola)
  • PECOTA (Baseball Prospectus)
  • RotoWire
  • Razzball (Steamer)
  • ZEILE (Fantasy Pros)*
  • Paywall #1
  • Average of the above projections

To create a list of players to compare for accuracy, I took the NFBC Main Event ADP (players in demand at that time) and selected the hitters in the top-450 drafted players (30-man roster, 15 teams in the Main Event). To determine accuracy, I calculated the Root Mean Square Error (RMSE) for two different sets of values. RMSE is a “measure of how far from the regression line data points are” and the smaller a value the better. Additionally, I included the actual and league average rates for reference.

 

Late Hitter Projection Accuracy: Batting Average
Projection Avg Value Projection RMSE Projection Lg Adj RMSE
PECOTA .249 Bat 0.0334 Average 0.0300
Actual .250 PECOTA 0.0336 ZEILE 0.0329
Razzball .253 DepthCharts 0.0337 DepthCharts 0.0330
Bat .254 BatX 0.0337 Bat 0.0332
BatX .254 Razzball 0.0340 ATC 0.0332
Steamer .256 Average 0.0340 BatX 0.0334
Zips .257 ATC 0.0341 Pods 0.0335
DepthCharts .257 Steamer 0.0343 PECOTA 0.0336
ATC .257 ZEILE 0.0345 Steamer 0.0337
Average .257 Pods 0.0348 Razzball 0.0338
Davenport .259 Davenport 0.0357 Paywall #1 0.0340
Pods .259 Zips 0.0360 Mastersball 0.0342
ZEILE .260 Paywall #1 0.0362 Davenport 0.0343
Mastersball .262 Mastersball 0.0366 Zips 0.0352
Rotowire .262 Rotowire 0.0385 Rotowire 0.0359
Paywall #1 .262

 

Late Hitter Projection Accuracy: Runs
Projection Avg Value Projection RMSE Projection Lg Adj RMSE
Bat 12.48% BatX 0.0217 Average 0.0205
BatX 12.48% Bat 0.0218 BatX 0.0217
Actual 12.49% DepthCharts 0.0223 Bat 0.0218
Pods 12.60% Average 0.0225 DepthCharts 0.0222
Steamer 12.61% ATC 0.0226 ATC 0.0222
DepthCharts 12.69% Pods 0.0226 ZEILE 0.0222
Zips 12.77% Steamer 0.0229 Pods 0.0225
Razzball 12.87% ZEILE 0.0230 Steamer 0.0228
ATC 12.92% Zips 0.0235 Zips 0.0233
Average 12.92% Razzball 0.0243 Mastersball 0.0235
PECOTA 13.03% Mastersball 0.0249 Razzball 0.0239
ZEILE 13.05% PECOTA 0.0250 Paywall #1 0.0240
Davenport 13.19% Paywall #1 0.0258 PECOTA 0.0242
Mastersball 13.25% Davenport 0.0280 Rotowire 0.0254
Paywall #1 13.34% Rotowire 0.0280 Davenport 0.0265
Rotowire 13.56%

 

Late Hitter Projection Accuracy: Home Runs
Projection Avg Value Projection RMSE Projection Lg Adj RMSE
Actual 3.5% BatX 0.0121 Average 0.0120
Bat 3.5% Bat 0.0126 BatX 0.0121
BatX 3.5% ATC 0.0127 ATC 0.0124
PECOTA 3.6% DepthCharts 0.0128 Razzball 0.0124
Zips 3.7% Razzball 0.0128 Steamer 0.0125
DepthCharts 3.7% Steamer 0.0129 DepthCharts 0.0125
ATC 3.7% ZEILE 0.0130 Bat 0.0126
Mastersball 3.7% Average 0.0130 ZEILE 0.0126
Average 3.7% PECOTA 0.0131 Zips 0.0129
Steamer 3.8% Zips 0.0132 PECOTA 0.0130
Pods 3.8% Pods 0.0133 Pods 0.0130
Rotowire 3.8% Mastersball 0.0135 Mastersball 0.0131
Razzball 3.8% Rotowire 0.0138 Rotowire 0.0132
Paywall #1 3.8% Davenport 0.0142 Davenport 0.0135
ZEILE 3.8% Paywall #1 0.0144 Paywall #1 0.0137
Davenport 3.9%

 

Late Hitter Projection Accuracy: RBI
Projection Avg Value Projection RMSE Projection Lg Adj RMSE
Bat 12.5% Bat 0.0218 Average 0.0205
BatX 12.5% BatX 0.0218 Bat 0.0218
Actual 12.5% DepthCharts 0.0223 BatX 0.0218
Pods 12.6% Average 0.0225 DepthCharts 0.0222
Steamer 12.6% Pods 0.0225 ATC 0.0222
DepthCharts 12.7% ATC 0.0226 ZEILE 0.0222
Zips 12.8% Steamer 0.0228 Pods 0.0225
Razzball 12.9% ZEILE 0.0230 Steamer 0.0228
ATC 12.9% Zips 0.0235 Zips 0.0233
Average 12.9% Razzball 0.0242 Mastersball 0.0235
PECOTA 13.0% Mastersball 0.0248 Razzball 0.0238
ZEILE 13.1% PECOTA 0.0250 Paywall #1 0.0239
Davenport 13.2% Paywall #1 0.0256 PECOTA 0.0242
Mastersball 13.2% Davenport 0.0280 Rotowire 0.0255
Paywall #1 13.3% Rotowire 0.0280 Davenport 0.0265
Rotowire 13.5%

 

Late Hitter Projection Accuracy: Stolen Bases
Projection Avg Value Projection RMSE Projection Lg Adj RMSE
PECOTA 1.14% Bat 0.0088 Average 0.0083
Pods 1.19% BatX 0.0088 Bat 0.0088
Actual 1.32% Average 0.0089 Zips 0.0089
Davenport 1.32% DepthCharts 0.0090 DepthCharts 0.0089
Bat 1.35% Steamer 0.0091 BatX 0.0089
BatX 1.35% ATC 0.0091 ATC 0.0089
Average 1.41% Pods 0.0091 ZEILE 0.0089
Rotowire 1.42% ZEILE 0.0091 Pods 0.0091
Steamer 1.44% Razzball 0.0092 Rotowire 0.0091
ATC 1.44% Zips 0.0094 Steamer 0.0092
Razzball 1.45% Rotowire 0.0094 Razzball 0.0092
ZEILE 1.48% Davenport 0.0095 Paywall #1 0.0094
DepthCharts 1.49% Paywall #1 0.0100 Davenport 0.0095
Paywall #1 1.53% Mastersball 0.0109 Mastersball 0.0097
Zips 1.54% PECOTA 0.0110 PECOTA 0.0113
Mastersball 1.63%

 

Final thoughts on hitters

  • While a few projections sneak into the top-six RMSE values (e.g. Razzball with HR, Pods with Runs), the BATs and any of the “averages” (Average, Depth Charts, ATC, ZEILE) reign supreme. The six end up occupying 90% of those top-six spots. For me, I’m thinking of using some select-crowd strategy (i.e “The select-crowd strategy is thus accurate, robust, and appealing as a mechanism for helping individuals tap collective wisdom.“). Trying to go with the better performing, non-correlated sources (e.g. Pods, Razzball, and the Bat X). Some dude has a spreadsheet that helps to average several different projections.
  • The BAT’s distinguished themselves as stand-alone projections, but they need a reliable playing time estimate to go with them (e.g. ATC).
  • One interesting point is with stolen bases. Rotowire does their projections by hand and they moved up in the pack. There might be something to adding a personal touch to the stolen base estimates.

After the first couple of articles, there was a trend that just continued with each successive analysis, use the Wisdom of the Crowd. Not every projection can catch every nuance, but a combination stands out. I guess the same results will materialize for pitchers, but that analysis will have to wait. I am going to take a break in order to catch up on Mining the News.

 

* ATC, Clay Davenport, Depth Charts, Harper Wallbanger, Luke Gloeckner, Mike Podhorzer, CBS Sports, ESPN, RotoChamp, Razzball, numberFire, Steamer, THE BAT, and ZiPS.





Jeff, one of the authors of the fantasy baseball guide,The Process, writes for RotoGraphs, The Hardball Times, Rotowire, Baseball America, and BaseballHQ. He has been nominated for two SABR Analytics Research Award for Contemporary Analysis and won it in 2013 in tandem with Bill Petti. He has won four FSWA Awards including on for his Mining the News series. He's won Tout Wars three times, LABR once, and got his first NFBC Main Event win in 2021. Follow him on Twitter @jeffwzimmerman.

1 Comment
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Rotoholicmember
9 months ago

It would be interesting to look at the collinearity between the different projection sets, as you mentioned. Try to get an idea of which ones to use and how to weight them. I’d think that a mix of Pods/BATx/Steamer would all be different enough to get pretty close to an average of the whole group. And just a lot easier to manage.