2019 Projection Systems Comparison – Hitting vs. Pitching

In my previous article, I compared a number of baseball projection systems for the 2019 season using a game theory approach. We looked at the profitability of each projection system in the context of simulating what would transpire at a fantasy baseball auction. We measured each projection’s successes and failures.

Several readers had approached me to further split out the resulting analysis into the hitter and pitcher components. By popular demand, I have decided to do exactly that. Today’s article will detail the analysis by its offensive and defensive elements.

For a refresher on the process and methodology, or for reference, please refer to the original post which can be found here.

Overall Results:

First, let’s quickly remind ourselves of the results of overall total profitability by projection system in 2019.

As we previously saw, ATC and Steamer were the two best overall systems according to this analysis in 2019.

Below are the two-year profitability totals for all projection systems:

ATC and THE BAT were the two best systems over the 2018-2019 period. They were the only two sets of projections to turn a profit on the entire player pool in the aggregate.

What strikes me as interesting, is that from 2018 to 2019, for all projection systems (and for the market as well) – profitability decreased rather uniformly up and down the player pool.

The above chart is the “All Players” perspective, which shows that the average player purchased in 2019 lost almost $1 more profit (to fantasy owners) than they did in 2018. The game is becoming a contest of managing losses, rather than of turning profits.

Hitting Results:

Onto looking at profitability for the hitters. Let’s start with the number of players purchased by each system, along with their hit rates:

In terms of drafting successful players, THE BAT and ZiPS were the superior systems in 2019 for the hitters. What is interesting here, is that THE BAT is the system that purchases the greatest number of hitters, whereas ZiPS purchases the fewest – yet both showed superior hit rates. Compared to the other systems, Pod performed worse than the others, especially high up in the draft. ATC on the other hand, did well for high priced players.

The other item to note is that the success rates for each projection system beat the “All Players” selections at every level of the player pool. No matter which projection system you might have used, you were ahead of the game.

Let’s now look at the total profitability, factoring in the magnitude of all gains and losses:

Steamer is the clear winner for 2019 for the hitters for overall profitability. Razzball and ZiPS do well in the top 150 players, and ATC does quite well especially after player 250. Steamer becomes profitable in the aggregate by player 250 due to their excellent mid-draft selections. Other than Razzball, all other systems eventually also become profitable.

Below is the two-year profitability analysis for hitters:

ATC wins the 2018-2019 two-year analysis handedly here, being profitable in the top 50, as well as after player 100. No system comes close. ZiPS overall is the worst system on the 2-year basis. Razzball is the 2^nd worst system according to this chart, but keep in mind that their figures are 2019 only (1 year) – which isn’t directly compatible to the others.

Pitching Results:

Onto the profitability for pitchers. First, below are the number of pitchers selected by each system along with their respective success rates:

What comes across first is the dearth of pitcher selections by THE BAT. One reason for that comes from the Hitter/Pitcher split employed by THE BAT with the straight Z-Score method calculation. The implied split is more heavily tilted towards hitting as compared to the other systems. Most other systems generate splits somewhat close to the actual observed NFBC AAVs.

Even with few players selected, for the players that THE BAT did draft, 0% of the players were profitable on the pitching side. Meaning, those who followed THE BAT’s top pitching projection bargains – did not do well.

ATC clearly dominated the pitcher hit rates of 2019, with Razzball coming in 2^nd place.

Now factoring in the magnitude of gains vs. losses in the above, we can see that ATC clearly dominated pitching this year. Although Pod did not have the best of success rates in pitching, it made up for it with its magnitudes (the hits were better than others, and the busts were not as dramatic). THE BAT, as we saw with a 0% hit rate throughout, came in last.

As opposed to hitting, no projection system came out profitable in the aggregate for pitching. ATC, the best of the 2019 projections still lost $3 on a per pitcher basis. It is also worth noting that ATC was the only system that beat the “All Players” line in all parts of the curve. ATC excelled in selecting profitable $1-2 endgame pitching darts.

Below are the two-year pitching figures:

In general, 2019 was not a particularly great year for pitching projections. This can be witnessed by viewing the wide gap between the profitability of 2019 and the two-year results. In 2018, only one system (Steamer) did NOT outperform the market in the aggregate. In 2019, only one system (ATC) outperformed the market profitability.

ATC wins the two-year pitching race. For the overall figures, THE BAT comes in 2^nd place, which meant that THE BAT dominated the lower part of the draft in 2018. Up higher, it is Razzball that takes 2^nd place behind ATC in the top 100 and 150 players.

Assorted Notes:

• In both components (hitting and pitching), ATC had the best two-year profitability.
• Steamer had the best 2019 offensive profitability, while ATC was the best defensively in 2019.
• Both hitter success rates as well as overall profitability metrics outperformed their respective pitching figures. This was true both in 2018 as well as in 2019.
• As mentioned in my prior article, ZiPS differs from the other systems in that it makes selections more heavily at top of the draft. The shape of their curve is not currently compatible to market tendencies.
• As mentioned above, when valuating THE BAT using a Z-Score method, prices generated are tilted more towards hitting than the prices generated by other projection systems. To use THE BAT more appropriately (to mimic market tendencies), one may need to slightly adjust the hitter vs. pitcher split. Doing so would provide THE BAT with more pitcher selections, although it may not necessarily generate better pitcher profitability. Of course, altering the hitter vs. pitcher split would in turn take away some of its hitter selections, and possibly harm some of its profitability.
• Further to the above bullet – whereas most systems come close to mimicking the NFBC hitter/pitcher split, THE BAT comes the closest to generating the actual 2019 calculated hitter/pitcher split. I am not sure if this was an intended consequence.
• As mentioned in my previous post – ATC has been the most stable projection system (least variation in how it performed) over the past two years. One intention of how ATC is designed and created is to smooth out results.