Early Projections: Hitting Counting & Rate Stats
In my last article, I focused on the early-season projection accuracy of hitter playing time. Today, the rest of the standard 5×5 Roto hitting stats finally take center stage. Besides the counting stats, I turn each of them into rate stats to help determine projection accuracy. After completing the analysis, three options stick out.
As a reminder, here are the projections I used and some background on the analysis.
Projections
- Steamer (FanGraphs)
- ZIPS
- DepthCharts (FanGraphs)
- The Bat
- The Bat X
- Davenport
- ATC (FanGraphs)
- Pod (Mike Podhorzer)
- Masterball (Todd Zola)
- PECOTA (Baseball Prospectus)
- RotoWire
- CBS
- Razzball (Steamer)
- Paywall #1
To create a list of players to compare for accuracy, I took the TGFBI ADP (players in demand at that time) and selected the hitter in the top-450 drafted players (30-man roster, 15 teams in TGFBI). Generally, all the players were projected with the following exceptions. Rotowire and Pods didn’t have a Josh Rojas projection while Pods also didn’t have projections for Jazz Chisholm Jr. and Mike Brosseau. Additionally, CBS was missing several projections. I am blamed for part of it because I forgot to pull designated hitters and they didn’t project as many outfielders. Finally, I just removed the projection for Yasiel Puig.
To determine accuracy, I calculated the Root Mean Square Error (RMSE) for three different sets of values. RMSE is a “measure of how far from the regression line data points are” and the smaller a value the better.
- All hitters with the three Pod and Rotowire missed.
- Removed the three along with several who missed quite a bit of time (Lewis, Mondesi, Trout, Ozuna, Hicks, Rendon).
- Every hitter that I pulled from CBS.
Finally, I created two additional projections. One is the average of the above projections. For the second one, I averaged the results from the best two projections. With all the theatrics out of the way, here are the results for the four counting stats (CBS didn’t include stolen bases).
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE | CBS List | RMSE |
---|---|---|---|---|---|
Average | 26.2 | Average | 26.4 | Average | 24.6 |
Razzball & ATC | 26.5 | ATC & Steamer | 26.7 | ATC & Steamer | 24.9 |
ATC | 26.7 | ZiPS | 26.8 | ATC | 25.1 |
Razzball | 26.7 | ATC | 26.9 | Steamer | 25.2 |
Steamer | 26.8 | Steamer | 27.0 | Pods | 25.4 |
Pods | 26.9 | Razzball | 27.1 | Razzball | 25.4 |
ZiPS | 26.9 | Pods | 27.1 | ZiPS | 25.6 |
DepthCharts | 27.4 | DepthCharts | 27.6 | DepthCharts | 25.8 |
Davenport | 28.0 | PECOTA | 28.3 | PECOTA | 26.4 |
PECOTA | 28.2 | Davenport | 28.4 | Mastersball | 26.7 |
Mastersball | 28.5 | Mastersball | 28.7 | Davenport | 27.0 |
Bat | 28.6 | BatX | 28.9 | Bat | 27.0 |
BatX | 28.6 | Bat | 28.9 | BatX | 27.1 |
Rotowire | 29.0 | Rotowire | 29.0 | Rotowire | 27.3 |
Paywall #1 | 31.2 | Paywall #1 | 31.9 | CBS` | 29.5 |
Paywall #1 | 30.8 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE | CBS List | RMSE |
---|---|---|---|---|---|
ATC & PECOTA | 9.65 | ATC & PECOTA | 9.70 | ATC & PECOTA | 9.17 |
PECOTA | 9.74 | PECOTA | 9.78 | ATC | 9.27 |
ATC | 9.78 | ATC | 9.85 | PECOTA | 9.31 |
Average | 9.80 | Average | 9.91 | Average | 9.35 |
Razzball | 9.91 | Davenport | 9.98 | Razzball | 9.52 |
Davenport | 9.93 | Razzball | 9.99 | Davenport | 9.55 |
ZiPS | 10.01 | ZiPS | 10.00 | ZiPS | 9.59 |
Steamer | 10.02 | Steamer | 10.14 | BatX | 9.61 |
BatX | 10.08 | Pods | 10.19 | Steamer | 9.62 |
Pods | 10.10 | BatX | 10.23 | Pods | 9.64 |
Mastersball | 10.12 | Mastersball | 10.25 | Mastersball | 9.68 |
DepthCharts | 10.25 | DepthCharts | 10.37 | Bat | 9.76 |
Bat | 10.26 | Bat | 10.43 | DepthCharts | 9.82 |
Paywall #1 | 10.39 | Paywall #1 | 10.46 | Paywall #1 | 9.82 |
Rotowire | 10.53 | Rotowire | 10.55 | Rotowire | 9.97 |
CBS | 11.61 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE | CBS List | RMSE |
---|---|---|---|---|---|
ATC | 25.9 | ATC | 26.3 | ATC | 24.4 |
ATC & Pods | 26.2 | Pods | 26.7 | ATC & Pods | 24.6 |
Average | 26.3 | ATC & Pods | 26.7 | Average | 24.8 |
Pods | 26.4 | Average | 26.7 | Pods | 24.8 |
Razzball | 26.5 | Razzball | 26.8 | Razzball | 25.1 |
Paywall #1 | 26.6 | Paywall #1 | 27.1 | Paywall #1 | 25.3 |
Steamer | 27.0 | PECOTA | 27.2 | PECOTA | 25.6 |
PECOTA | 27.0 | Steamer | 27.4 | Steamer | 25.7 |
Mastersball | 27.0 | Mastersball | 27.6 | Mastersball | 25.7 |
BatX | 27.4 | Rotowire | 27.8 | Davenport | 25.9 |
Bat | 27.4 | BatX | 27.9 | Rotowire | 26.0 |
Davenport | 27.7 | Bat | 27.9 | Bat | 26.2 |
Rotowire | 27.7 | Davenport | 28.1 | BatX | 26.2 |
DepthCharts | 29.2 | DepthCharts | 29.6 | DepthCharts | 27.6 |
ZiPS | 29.6 | ZiPS | 29.7 | ZiPS | 28.1 |
CBS | 29.7 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE |
---|---|---|---|
Razzball & Bat | 5.57 | BatX & Razzball | 5.12 |
Razzball | 5.58 | BatX | 5.14 |
Average | 5.65 | Bat | 5.16 |
Bat | 5.70 | Average | 5.16 |
BatX | 5.70 | Razzball | 5.23 |
Pods | 5.75 | Steamer | 5.24 |
ATC | 5.81 | Pods | 5.29 |
Steamer | 5.81 | ATC | 5.34 |
PECOTA | 5.85 | DepthCharts | 5.46 |
Davenport | 6.00 | Davenport | 5.55 |
DepthCharts | 6.03 | PECOTA | 5.58 |
Paywall #1 | 6.12 | Paywall #1 | 5.65 |
Rotowire | 6.23 | Rotowire | 5.73 |
ZiPS | 6.26 | ZiPS | 5.80 |
Mastersball | 6.72 | Mastersball | 6.18 |
The quick takeaway from here is that the systems that dominated the playing time analysis (ATC, Razzball, and the Average), are the top finishers. This finding should not be a surprise with playing time, along with its close cousin, injury projection, being the holy grails of fantasy analysis.
I have a few hot takes, but first, here is how the systems did at projection batting average and the counting stats as rate stats. I included CBS with the AVG but they didn’t have plate appearances in their projections. With their previous bad showings, I might just move past them.
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE | CBS List | RMSE |
---|---|---|---|---|---|
Bat & PECOTA | 0.0334 | Bat & PECOTA | 0.0336 | Bat & PECOTA | 0.0332 |
Bat | 0.0337 | PECOTA | 0.0340 | PECOTA | 0.0336 |
BatX | 0.0340 | Bat | 0.0341 | Bat | 0.0338 |
PECOTA | 0.0341 | DepthCharts | 0.0344 | DepthCharts | 0.0340 |
DepthCharts | 0.0343 | BatX | 0.0344 | Razzball | 0.0340 |
Razzball | 0.0343 | Razzball | 0.0344 | BatX | 0.0341 |
Steamer | 0.0346 | Steamer | 0.0346 | Steamer | 0.0343 |
Average | 0.0346 | Average | 0.0347 | Average | 0.0343 |
ZiPS | 0.0348 | ZiPS | 0.0348 | ZiPS | 0.0344 |
ATC | 0.0349 | ATC | 0.0350 | ATC | 0.0346 |
Pods | 0.0358 | Pods | 0.0359 | Pods | 0.0355 |
Davenport | 0.0370 | Davenport | 0.0369 | Davenport | 0.0367 |
Mastersball | 0.0371 | Mastersball | 0.0374 | Mastersball | 0.0370 |
Paywall #1 | 0.0374 | Paywall #1 | 0.0374 | Paywall #1 | 0.0371 |
Rotowire | 0.0395 | Rotowire | 0.0390 | Rotowire | 0.0386 |
CBS | 0.0388 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE |
---|---|---|---|
BatX & DC | 0.0218 | BatX & DC | 0.0216 |
BatX | 0.0219 | DepthCharts | 0.0219 |
Bat | 0.0220 | BatX | 0.0219 |
DepthCharts | 0.0222 | Steamer | 0.0221 |
Steamer | 0.0225 | Bat | 0.0221 |
Pods | 0.0226 | Average | 0.0224 |
Average | 0.0226 | Pods | 0.0225 |
ATC | 0.0229 | ATC | 0.0228 |
ZiPS | 0.0232 | ZiPS | 0.0229 |
Razzball | 0.0239 | Razzball | 0.0235 |
PECOTA | 0.0249 | PECOTA | 0.0246 |
Mastersball | 0.0250 | Mastersball | 0.0249 |
Paywall #1 | 0.0264 | Paywall #1 | 0.0260 |
Davenport | 0.0271 | Davenport | 0.0273 |
Rotowire | 0.0285 | Rotowire | 0.0279 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE |
---|---|---|---|
BatX | 0.0124 | BatX | 0.0123 |
BatX & ATC | 0.0125 | BatX & ATC | 0.0124 |
Bat | 0.0128 | Bat | 0.0128 |
Average | 0.0129 | Average | 0.0128 |
ATC | 0.0129 | ATC | 0.0128 |
Steamer | 0.0130 | Steamer | 0.0129 |
DepthCharts | 0.0131 | DepthCharts | 0.0130 |
Razzball | 0.0131 | PECOTA | 0.0130 |
PECOTA | 0.0132 | Razzball | 0.0130 |
ZiPS | 0.0134 | ZiPS | 0.0133 |
Mastersball | 0.0136 | Mastersball | 0.0135 |
Pods | 0.0136 | Pods | 0.0135 |
Davenport | 0.0138 | Davenport | 0.0136 |
Rotowire | 0.0140 | Rotowire | 0.0139 |
Paywall #1 | 0.0145 | Paywall #1 | 0.0144 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE |
---|---|---|---|
BatX | 0.025 | BatX | 0.025 |
BatX & ATC | 0.026 | BatX & ATC | 0.025 |
Bat | 0.026 | Bat | 0.025 |
ATC | 0.026 | ATC | 0.026 |
Average | 0.027 | Average | 0.026 |
PECOTA | 0.027 | Pods | 0.027 |
Pods | 0.027 | Steamer | 0.027 |
Steamer | 0.027 | PECOTA | 0.027 |
Razzball | 0.029 | Razzball | 0.028 |
Mastersball | 0.029 | Paywall #1 | 0.028 |
DepthCharts | 0.029 | DepthCharts | 0.028 |
Paywall #1 | 0.029 | Mastersball | 0.028 |
Rotowire | 0.031 | Rotowire | 0.030 |
Davenport | 0.032 | Davenport | 0.032 |
ZiPS | 0.033 | ZiPS | 0.033 |
Minus missing 3 | RMSE | Minus missing 3 & Injuries | RMSE |
---|---|---|---|
BatX & ATC | 0.0083 | BatX | 0.0082 |
BatX | 0.0085 | BatX & DC | 0.0082 |
Bat | 0.0085 | Bat | 0.0083 |
Average | 0.0086 | Average | 0.0084 |
ATC | 0.0087 | DepthCharts | 0.0086 |
DepthCharts | 0.0088 | ATC | 0.0087 |
Steamer | 0.0089 | Steamer | 0.0087 |
Razzball | 0.0089 | Razzball | 0.0087 |
Pods | 0.0090 | Pods | 0.0088 |
ZiPS | 0.0092 | ZiPS | 0.0090 |
Rotowire | 0.0092 | Rotowire | 0.0090 |
Paywall #1 | 0.0096 | Davenport | 0.0091 |
Davenport | 0.0097 | Paywall #1 | 0.0093 |
Mastersball | 0.0105 | Mastersball | 0.0099 |
PECOTA | 0.0107 | PECOTA | 0.0099 |
While I plan on analyzing pitchers and hitter projections from projections pulled right before the season starts, I have three possible recommendations with these results.
- Use ATC for all analysis except batting average component (bottom half finish) and stolen bases. Since it combines and weights the playing time from other projections, its counting stats are consistently near the top.
- Use the rates from The BatX with the average or ATC playing times projections. The BatX just dominates the rate stats and just needs help with playing time.
- Use the average of several of the “better” projections. It’s tough to beat the Wisdom of the Crowds.
An important point is that these projections are from early March. I pulled the same projections right before the season started and I will see if there are any changes to the analysis with them after I analyze early pitcher projections.
Jeff, one of the authors of the fantasy baseball guide,The Process, writes for RotoGraphs, The Hardball Times, Rotowire, Baseball America, and BaseballHQ. He has been nominated for two SABR Analytics Research Award for Contemporary Analysis and won it in 2013 in tandem with Bill Petti. He has won four FSWA Awards including on for his Mining the News series. He's won Tout Wars three times, LABR twice, and got his first NFBC Main Event win in 2021. Follow him on Twitter @jeffwzimmerman.
This is all based on last season only, right? Not to be a Debbie Downer, but I would have a hard time making any definitive decisions based on one season.
Also, I’d be interested to see not just RMSE for the players but a global analysis as well for each system. A lot of offensive categories were down last year as compared to the last full season of 2019, so if one system consistently undershot one or more categories for every player, it’s not necessarily a bad thing if they’re all being missed by roughly the same amount, since it’s the environment that’s being underestimated, rather than particular players being over/underestimated.
I would think you’d want to not just do a raw RMSE, but maybe do some adjustment based on population total. For example, I just pulled the top 261 hitters (approximately the # of hitters in the top 450 according to the ADP in the projections), and ZiPS overestimated RBI/PA by 0.009 for the league as a whole, which is on the order of 5.6 RBI/600 PA. I don’t know how you’d necessarily go about this, but I would think you’d want to reward the ones that are best relative to their assumed statistical environment, at least from a fantasy standpoint.
Oh, I know …. https://twitter.com/jeffwzimmerman/status/1450869177485598729?s=20
Ha, yes, it is a thankless task to be sure, Jeff.
To use your example for ZiPS and RBI/PA:
The hitters in this sample averaged 0.120 RBI/PA in 2021. (That doesn’t weight by PA, which it should, but it’s a start.)
ZiPS projected an average of 0.132 RBI/PA for those same players.
To correct it, you need to switch from RBI/PA to RBI/PA above the average. So Jose Abreu got 0.178 RBI/PA, which is 0.58 above the average (0.120). ZiPS projected him for 0.190 RBI/PA, which is also 0.58 above its average. You base your RMSE off of that 0.58 and 0.58.
You should double-check my math, but I think that’s the basic idea of it.