Projection Accuracy: Late March Hitter Rate Stats by Jeff Zimmerman November 5, 2021 I’ve been slowly working my way through the hitter projections and that journey comes to an end today as I examine how each projected hitter rate stats stand up. Besides batting average, I turn each of the counting stats into a rate by dividing by plate appearances. Finally, I adjust each value to the actual league rates. Again, any combination of projections stick out along with the BAT. For reference, here are the projections used. Steamer (FanGraphs) ZIPS DepthCharts (FanGraphs) The Bat The Bat X Davenport ATC (FanGraphs) Pod (Mike Podhorzer) Masterball (Todd Zola) PECOTA (Baseball Prospectus) RotoWire Razzball (Steamer) ZEILE (Fantasy Pros)* Paywall #1 Average of the above projections To create a list of players to compare for accuracy, I took the NFBC Main Event ADP (players in demand at that time) and selected the hitters in the top-450 drafted players (30-man roster, 15 teams in the Main Event). To determine accuracy, I calculated the Root Mean Square Error (RMSE) for two different sets of values. RMSE is a “measure of how far from the regression line data points are” and the smaller a value the better. Additionally, I included the actual and league average rates for reference. Late Hitter Projection Accuracy: Batting Average Projection Avg Value Projection RMSE Projection Lg Adj RMSE PECOTA .249 Bat 0.0334 Average 0.0300 Actual .250 PECOTA 0.0336 ZEILE 0.0329 Razzball .253 DepthCharts 0.0337 DepthCharts 0.0330 Bat .254 BatX 0.0337 Bat 0.0332 BatX .254 Razzball 0.0340 ATC 0.0332 Steamer .256 Average 0.0340 BatX 0.0334 Zips .257 ATC 0.0341 Pods 0.0335 DepthCharts .257 Steamer 0.0343 PECOTA 0.0336 ATC .257 ZEILE 0.0345 Steamer 0.0337 Average .257 Pods 0.0348 Razzball 0.0338 Davenport .259 Davenport 0.0357 Paywall #1 0.0340 Pods .259 Zips 0.0360 Mastersball 0.0342 ZEILE .260 Paywall #1 0.0362 Davenport 0.0343 Mastersball .262 Mastersball 0.0366 Zips 0.0352 Rotowire .262 Rotowire 0.0385 Rotowire 0.0359 Paywall #1 .262 Late Hitter Projection Accuracy: Runs Projection Avg Value Projection RMSE Projection Lg Adj RMSE Bat 12.48% BatX 0.0217 Average 0.0205 BatX 12.48% Bat 0.0218 BatX 0.0217 Actual 12.49% DepthCharts 0.0223 Bat 0.0218 Pods 12.60% Average 0.0225 DepthCharts 0.0222 Steamer 12.61% ATC 0.0226 ATC 0.0222 DepthCharts 12.69% Pods 0.0226 ZEILE 0.0222 Zips 12.77% Steamer 0.0229 Pods 0.0225 Razzball 12.87% ZEILE 0.0230 Steamer 0.0228 ATC 12.92% Zips 0.0235 Zips 0.0233 Average 12.92% Razzball 0.0243 Mastersball 0.0235 PECOTA 13.03% Mastersball 0.0249 Razzball 0.0239 ZEILE 13.05% PECOTA 0.0250 Paywall #1 0.0240 Davenport 13.19% Paywall #1 0.0258 PECOTA 0.0242 Mastersball 13.25% Davenport 0.0280 Rotowire 0.0254 Paywall #1 13.34% Rotowire 0.0280 Davenport 0.0265 Rotowire 13.56% Late Hitter Projection Accuracy: Home Runs Projection Avg Value Projection RMSE Projection Lg Adj RMSE Actual 3.5% BatX 0.0121 Average 0.0120 Bat 3.5% Bat 0.0126 BatX 0.0121 BatX 3.5% ATC 0.0127 ATC 0.0124 PECOTA 3.6% DepthCharts 0.0128 Razzball 0.0124 Zips 3.7% Razzball 0.0128 Steamer 0.0125 DepthCharts 3.7% Steamer 0.0129 DepthCharts 0.0125 ATC 3.7% ZEILE 0.0130 Bat 0.0126 Mastersball 3.7% Average 0.0130 ZEILE 0.0126 Average 3.7% PECOTA 0.0131 Zips 0.0129 Steamer 3.8% Zips 0.0132 PECOTA 0.0130 Pods 3.8% Pods 0.0133 Pods 0.0130 Rotowire 3.8% Mastersball 0.0135 Mastersball 0.0131 Razzball 3.8% Rotowire 0.0138 Rotowire 0.0132 Paywall #1 3.8% Davenport 0.0142 Davenport 0.0135 ZEILE 3.8% Paywall #1 0.0144 Paywall #1 0.0137 Davenport 3.9% Late Hitter Projection Accuracy: RBI Projection Avg Value Projection RMSE Projection Lg Adj RMSE Bat 12.5% Bat 0.0218 Average 0.0205 BatX 12.5% BatX 0.0218 Bat 0.0218 Actual 12.5% DepthCharts 0.0223 BatX 0.0218 Pods 12.6% Average 0.0225 DepthCharts 0.0222 Steamer 12.6% Pods 0.0225 ATC 0.0222 DepthCharts 12.7% ATC 0.0226 ZEILE 0.0222 Zips 12.8% Steamer 0.0228 Pods 0.0225 Razzball 12.9% ZEILE 0.0230 Steamer 0.0228 ATC 12.9% Zips 0.0235 Zips 0.0233 Average 12.9% Razzball 0.0242 Mastersball 0.0235 PECOTA 13.0% Mastersball 0.0248 Razzball 0.0238 ZEILE 13.1% PECOTA 0.0250 Paywall #1 0.0239 Davenport 13.2% Paywall #1 0.0256 PECOTA 0.0242 Mastersball 13.2% Davenport 0.0280 Rotowire 0.0255 Paywall #1 13.3% Rotowire 0.0280 Davenport 0.0265 Rotowire 13.5% Late Hitter Projection Accuracy: Stolen Bases Projection Avg Value Projection RMSE Projection Lg Adj RMSE PECOTA 1.14% Bat 0.0088 Average 0.0083 Pods 1.19% BatX 0.0088 Bat 0.0088 Actual 1.32% Average 0.0089 Zips 0.0089 Davenport 1.32% DepthCharts 0.0090 DepthCharts 0.0089 Bat 1.35% Steamer 0.0091 BatX 0.0089 BatX 1.35% ATC 0.0091 ATC 0.0089 Average 1.41% Pods 0.0091 ZEILE 0.0089 Rotowire 1.42% ZEILE 0.0091 Pods 0.0091 Steamer 1.44% Razzball 0.0092 Rotowire 0.0091 ATC 1.44% Zips 0.0094 Steamer 0.0092 Razzball 1.45% Rotowire 0.0094 Razzball 0.0092 ZEILE 1.48% Davenport 0.0095 Paywall #1 0.0094 DepthCharts 1.49% Paywall #1 0.0100 Davenport 0.0095 Paywall #1 1.53% Mastersball 0.0109 Mastersball 0.0097 Zips 1.54% PECOTA 0.0110 PECOTA 0.0113 Mastersball 1.63% Final thoughts on hitters While a few projections sneak into the top-six RMSE values (e.g. Razzball with HR, Pods with Runs), the BATs and any of the “averages” (Average, Depth Charts, ATC, ZEILE) reign supreme. The six end up occupying 90% of those top-six spots. For me, I’m thinking of using some select-crowd strategy (i.e “The select-crowd strategy is thus accurate, robust, and appealing as a mechanism for helping individuals tap collective wisdom.“). Trying to go with the better performing, non-correlated sources (e.g. Pods, Razzball, and the Bat X). Some dude has a spreadsheet that helps to average several different projections. The BAT’s distinguished themselves as stand-alone projections, but they need a reliable playing time estimate to go with them (e.g. ATC). One interesting point is with stolen bases. Rotowire does their projections by hand and they moved up in the pack. There might be something to adding a personal touch to the stolen base estimates. After the first couple of articles, there was a trend that just continued with each successive analysis, use the Wisdom of the Crowd. Not every projection can catch every nuance, but a combination stands out. I guess the same results will materialize for pitchers, but that analysis will have to wait. I am going to take a break in order to catch up on Mining the News. * ATC, Clay Davenport, Depth Charts, Harper Wallbanger, Luke Gloeckner, Mike Podhorzer, CBS Sports, ESPN, RotoChamp, Razzball, numberFire, Steamer, THE BAT, and ZiPS.