Projection Accuracy: Early March Pitcher Counting Stats

Now that the analysis hitter projection comparisons (part 1, 2, 3, 4, and 5) are done, it is time for the pitchers to take center stage. For the first article, I’ll measure the accuracy of counting stats from early March. Razzball had a near clean sweep as it only missed on Saves.

First, here are the projections analyzed.

• Steamer (FanGraphs)
• ZIPS
• DepthCharts (FanGraphs)
• The Bat
• Davenport
• ATC (FanGraphs)
• Pod (Mike Podhorzer)
• Masterball (Todd Zola)
• PECOTA (Baseball Prospectus)
• RotoWire
• Razzball (Steamer)
• Paywall #1
• Average of the above projections

To create a list of players to compare for accuracy, I took the TGFBI ADP (players in demand at that time) and selected all the pitchers in the top-450 drafted players (30-man roster, 15 teams in the Main Event) in ten or more leagues. Then I removed all the pitchers who never threw an inning.

To determine accuracy, I calculated the Root Mean Square Error (RMSE) for two different sets of values. RMSE is a “measure of how far from the regression line data points are” and the smaller a value the better. Additionally, I included the actual and league average rates for reference.

I ran into a bit of an issue as 16 pitchers* were missing from one or more of three different projections.

Projection: Players missing
Pods: 7
The Bat: 6
Rotowire: 5

For this reason, I created two comparisons. One with all the projection systems but fewer pitchers and one with all the pitchers but fewer projection systems.

As for predicting Saves, two projections (Zips and the Bat) just don’t project Saves so I removed them. Also, I limited the player pool down to any pitcher from the initial subset who accumulated one Save or was projected for one Save.

Finally, here are the results:

 

Early March Innings Pitched Projection Accuracy: All Projections
Projection RMSE Projection Avg
Razzball 41.9 Actual 102.7
Pods 45.2 Zips 110.9
PECOTA 45.4 PECOTA 111.8
Average 45.7 Razzball 113.8
Zips 45.8 Pods 118.6
ATC 46.6 Average 120.8
Mastersball 47.8 Mastersball 121.8
Bat 48.0 ATC 122.6
DepthCharts 48.1 Steamer 123.7
Steamer 48.1 Bat 124.4
Rotowire 49.6 DepthCharts 124.4
Paywall #1 50.0 Davenport 124.9
Davenport 50.6 Paywall #1 125.9
Rotowire 127.1

 

Early March Innings Pitched Projection Accuracy: All Pitchers
Projection RMSE Projection Avg
Razzball 42.1 Actual 99.0
PECOTA 44.8 PECOTA 108.0
Average 45.1 Zips 108.0
Zips 45.6 Razzball 109.0
ATC 46.1 Average 116.0
Mastersball 47.3 Mastersball 117.0
Steamer 47.5 ATC 118.0
DepthCharts 47.6 Steamer 119.0
Paywall #1 49.0 DepthCharts 120.0
Davenport 49.6 Paywall #1 121.0
Davenport 121.0

 

Early March Strikeout Projection Accuracy: All Projections
Projection RMSE Projection Avg
Razzball 47.9 Actual 108.1
Pods 51.6 Zips 117.5
ATC 51.7 Razzball 117.9
Average 51.7 PECOTA 119.5
Bat 52.2 Pods 123.7
Zips 52.2 Bat 124.5
PECOTA 52.5 Davenport 124.7
Steamer 53.7 Average 125.6
DepthCharts 54.2 ATC 126.2
Davenport 55.7 Steamer 128.0
Mastersball 56.0 Mastersball 128.9
Paywall #1 56.3 DepthCharts 130.1
Rotowire 56.8 Paywall #1 131.7
Rotowire 134.5

 

Early March Strikeout Projection Accuracy: All Pitchers
Projection RMSE Projection Avg
Razzball 47.7 Actual 104.3
Average 50.8 Razzball 113.1
ATC 50.9 Zips 114.0
PECOTA 51.5 PECOTA 114.7
Zips 51.6 Davenport 120.1
Steamer 52.9 Average 120.9
DepthCharts 53.4 ATC 121.1
Davenport 54.4 Steamer 123.6
Mastersball 54.6 Mastersball 123.6
Paywall #1 54.9 DepthCharts 125.6
Paywall #1 126.7

 

Early March Wins Projection Accuracy: All Projections
Projection RMSE Projection Avg
Razzball 3.9 Actual 6.5
ATC 3.9 PECOTA 7.1
Average 3.9 Razzball 7.4
PECOTA 3.9 Bat 7.5
Steamer 4.0 Mastersball 7.5
Pods 4.0 Zips 7.5
Bat 4.0 ATC 7.5
Zips 4.0 Steamer 7.5
DepthCharts 4.0 Average 7.6
Paywall #1 4.1 Davenport 7.7
Rotowire 4.2 Pods 7.8
Davenport 4.3 Paywall #1 7.9
Mastersball 4.3 DepthCharts 8.0
Rotowire 8.1

 

Early March Wins Projection Accuracy: All Players
Projection RMSE Projection Avg
Razzball 3.8 Actual 6.2
ATC 3.8 PECOTA 6.7
Average 3.8 Razzball 7.1
PECOTA 3.8 Mastersball 7.1
Steamer 3.9 ATC 7.2
DepthCharts 4.0 Steamer 7.2
Zips 4.0 Zips 7.3
Paywall #1 4.1 Average 7.3
Davenport 4.2 Davenport 7.4
Mastersball 4.2 Paywall #1 7.6
DepthCharts 7.7

 

Early March Saves Projection Accuracy: All Projections
Projection RMSE Projection Avg
Paywall #1 9.2 Razzball 9.2
Rotowire 9.3 Pods 12.2
Average 9.3 Actual 12.9
Razzball 9.4 Steamer 12.9
ATC 9.5 DepthCharts 12.9
Davenport 10.1 Paywall #1 13.3
PECOTA 10.6 Average 13.3
Steamer 10.7 ATC 13.6
DepthCharts 10.7 Rotowire 14.1
Mastersball 10.7 PECOTA 14.1
Pods 11.5 Davenport 15.0
Mastersball 16.1

 

Early March Saves Projection Accuracy: All Players
Projection RMSE Projection Avg
Paywall #1 9.1 Razzball 8.2
Average 9.3 Actual 11.5
Razzball 9.4 Paywall #1 11.8
ATC 9.4 Steamer 11.8
Davenport 9.9 DepthCharts 11.8
Steamer 10.6 Average 12.0
DepthCharts 10.6 ATC 12.2
PECOTA 10.7 PECOTA 13.0
Mastersball 10.7 Davenport 13.3
Mastersball 14.6

 

Thoughts

• The boys over at Razzball crushed the rankings by being close to the innings thrown. And it wasn’t by just projecting fewer innings thrown. Zips and PECOTA were projected for fewer innings, but their per player accuracy was not as good as Razzball. Like with hitters, nailing the initial playing time projections led to good standings for the other counting stats.

• Two of the aggrigator projections (ATC and the average) consistently placed in the top half.

• The Saves rankings are in a comletely different order than any other set of rankings. If there was any one value I’d take an average of, it would be Saves. It’s the stat where doing more leg work can pay off and fantasy managers might as well leverage the Wisdom of the Crowds for the leg work.

• In early deep drafts, I might back away from the three projections that were missing players. The projections aren’t taking into account players being drafted in the top-450, yet alone in 15-team, 50-round draft-and-holds (750 players).

So that is all I have on pitcher counting stat projection accuracty for the early offseason. Getting close to the playing time as possible was key and in the next installment, the rate stats get compared.

 

* Brusdar Graterol, Kyle Wright, Chad Green, Garrett Crochet, Mike Foltynewicz, Hunter Harvey, Nick Wittgren, Logan Gilbert, Bryan Garcia, Trevor May, Sean Doolittle, Stefan Crichton, Rich Hill, Gregory Soto, Emmanuel Clase, Daniel Ponce de Leon





Jeff, one of the authors of the fantasy baseball guide,The Process, writes for RotoGraphs, The Hardball Times, Rotowire, Baseball America, and BaseballHQ. He has been nominated for two SABR Analytics Research Award for Contemporary Analysis and won it in 2013 in tandem with Bill Petti. He has won four FSWA Awards including on for his Mining the News series. He's won Tout Wars three times, LABR once, and got his first NFBC Main Event win in 2021. Follow him on Twitter @jeffwzimmerman.

Comments are closed.