THE BAT X – 2020 Projections Review via Game Theory

Many thanks to Derek Carty of RotoGrinders for his assistance on this article, and for his player notes on a few 2020 player projections.

In my previous article, 2020 Projection Systems Comparison – A Game Theory Approach, I compared several excellent projection systems in terms of fantasy baseball profitability for 2020. It was not the typical statistical comparison, rather – I used a game theory approach. This was the third such annual article that I had put forth in evaluating projection systems.

Earlier this year, Derek Carty unveiled a new version of his already excellent THE BAT projection system. The new system is called, THE BAT X. The major innovation of THE BAT X is that it incorporates Statcast data into the fold. You can read more about THE BAT X works in Carty’s introductory article found here on the pages of FanGraphs.

I have typically evaluated THE BAT within my 2020 Projections comparison. With this season (despite the short duration) as the inaugural run of THE BAT X – Derek asked me to take a deeper look into how his new projection system had performed. To do this, I went back and revisited the same game theory methodology applied to THE BAT X. The initial results look very promising for the young system.

In this article, I will go through what had changed between THE BAT and THE BAT X as far as the game theory simulations. For a few of the largest and most impactful player performance differences, I will also include some analysis from Derek Carty himself as to why THE BAT X made those adjustments.

Methodology & Prior Results Recap

As a quick reminder, here is a brief overview on how the game theory comparison methodology works:

• Start with the raw pre-season projections.
• Produce a starting auction value for each player. I use a Z-Score method to calculate values.
• Create a price point for each player. For higher valued players, this might be $1-3 above the calculated auction values. For lower valued players, this might be $1-3 below the auction values.
• Obtain an auction cost. I am using actual NFBC AAV data during the month of July for this year’s analysis.
• Determine the players that are “purchased” by each projection system [Price Point >= Auction Cost].
• Compute the end of season rotisserie values for each player.
• Track and analyze results of profitability by projection system.

For 2020, I evaluated six projection systems. Above are the results by cumulative auction cost tiers, on a profit per player basis.

Especially this past year, you will notice that most figures are negative. This is unsurprising. A player injured for the majority of the season (yet selected in an NFBC auction) would produce a negative value. A top-50 valued player who does not earn top-50 player value, would produce a negative value, etc. Especially at the top of the draft, negative values for this type of analysis are normal. These days, drafting is more about avoiding losses than anything else.

THE BAT was already one of the best performing projection systems in 2020 by this methodology. Let’s now dig into what changes when moving to THE BAT X.

Top 50 Players

As we have previously seen, due to the definition of the pricing curve of this algorithm, projection systems tend not to outright purchase many players within the top 50. The typical system usually values 2-6 elite players above the market (AAV), accounting for 4-12% of the available players. ZiPS, which is geared towards a stars and scrubs approach – has at times purchased upwards of 18 elite players (36%) in a season.

In 2020, projection systems were on the low side – with both THE BAT and THE BAT X drafting only 3 elite players apiece. Both systems would have selected both Max Scherzer and Jose Altuve, producing a $34 and $24 loss, respectively.

THE BAT would have additionally purchased Rockies’ star Nolan Arenado, while THE BAT X would have vied for the World Series Champion Mookie Betts.

In the below, GREEN denotes a system ‘buy’ and RED denotes a system ‘pass.’

Max Scherzer has the same valuation under both THE BAT and THE BAT X. This is not an accident, as THE BAT X is only an intended improvement upon the offensive. Pitcher statistics are the same under both versions of the model. For the duration of this article, I will no longer refer to pitching.

Jose Altuve was similarly valued by both systems – at or above the market price. But for both Arenado and Betts, we see a tremendous difference in valuation.

This is what Derek Carty had to say about the sources of the model differences for the duo:

Mookie Betts – THE BAT X saw Mookie as equal or better to the players everyone was drafting at the top of the first round (Yelich, Acuna, Bellinger), even though he had a worse 2019. The drop in wOBA from his MVP 2018 to 2019 was 70 points, but the drop in his THE BATcast xwOBA was half of that. He was a bit lucky in 2018, a bit unlucky in 2019, and people (and traditional projection systems) overreacted to that. His 2018 xwOBA was at the 100^th percentile, and was still 95^th in 2019. Even though a lot of his barrels and exit velocity numbers dipped in 2019, THE BAT X saw some things (secret sauce type stuff that I can’t go too deep into) to expect a rebound.

Nolan Arenado – Many of Arenado’s Statcast metrics had been gradually falling off for years, but 2019 was a big drop for many of them. While his 2019 average launch angle was a strong 81^st percentile – an improvement over his 2018 LA – his underlying LA metrics weren’t so rosy. Arenado’s launch angle on his hardest hit balls had stayed in the 70-85^th percentile range most of his career, but cratered to 53^rd in 2019. His percentage of balls hit between 23 and 34 degrees (a range where many home runs are hit) hovered between the 80^th and 90^th [percentiles] in 2017 and 2018, but was in the low 70s in 2019. He was also not hitting the ball as hard in ‘19. Nolan’s percentage of air balls over 100 mph was only at the 43^rd percentile. His exit velocity on fly balls was 48^th. His max EV was 38^th. His average EV had usually been at the 70^th percentile, but was barely 50^th in 2019. Having a Barrel% that used to be in the top quartile, he was now barely above average in 2019. Some of this may be due to the humidor reducing EV a bit, but he has played in Coors each year, and so the pattern of decline should not have been impacted by it.

These were two excellent cases where the incorporation of Statcast metrics affected THE BAT X’s buying decision. In both cases – THE BAT X made the right alternative purchasing choice, in hindsight.

Although Mookie Betts earned a -$1 profit, and technically was “unprofitable” according to the game theory methodology – Betts was a tremendous buy for 2020, amassing $42 while sporting a $43 auction cost. He returned 98% of his investment, which is amazing for a first-round caliber player.

* In the main game theory projections comparison article, I noted that a future modification to the classification of “profitable” should consider a success curve so that high-valued player would be considered profitable even if they realize a small seasonal loss. i.e., a value return of 98% for Mookie Betts should be classified as a “success.”

Where THE BAT X distinguished itself from THE BAT in this tier – was its pass signal on Nolan Arenado. While THE BAT X simply chose not to buy this distressed asset, THE BAT X ponied up $35 on a semi-strong buy ($3 over value signal) and unfortunately walked away with a $30 hit to team profits.

In other words, if you would have exclusively used THE BAT as your projection pricing model, you would have had a decent chance of both selecting, and subsequently busting on Nolan Arenado.

THE BAT X still trailed almost all other projection systems in the first 50 players, but shedding Nolan Arenado was quite impactful.

Players 50-100

Rather than looking at every player purchased by either THE BAT or THE BAT X, for subsequent player tiers, I will focus solely on the players that were purchased by one system or the other (i.e., the players for which THE BAT X produced a buying decision change).Jose Abreu, the 2020 American League’s Most Valuable Player, was a tremendous fantasy purchase this season. As the 4^th most profitable fantasy hitter of 2020, it obviously does wonders to THE BAT X’s profitability (over THE BAT) by shifting his name into the ‘purchase’ column.

Carty had to say this about THE BAT X’s love of the White Sox slugger:

Jose Abreu – Abreu drastically underperformed his Statcast metrics in both 2019 and 2018 (and slightly in 2017). THE BAT X was simply willing to bet on him being better than the surface numbers indicated, in terms of both BABIP and power. His Barrel% jumped to the 88^th percentile in 2019, and his exit velocity metrics were 90^th percentile or better across the board (average, max, top 5%, on flyballs, etc.). Abreu’s surface numbers were only a bit better than average in 2018 and 2019, but his Statcast metrics said he was a very-good-bordering-on-great hitter who just wasn’t performing that way.

Tim Anderson was a player that THE BAT did get right this year, for which THE BAT X did not. Anderson would have earned its users $6 of fantasy profit, while THE BAT X users would have disregarded him.

Aside from Abreu, the strength of THE BAT X over THE BAT in this tier was its ability to limit a few extra poor purchases, most prominently of Anthony Rizzo and Kris Bryant. Bryant ended up earning below replacement level value for the 2020 season.

Here is some of what Derek Carty had to say about the biggest busts of this player tier:

Anthony Rizzo – Rizzo’s surface and Statcast numbers took a dive in 2018, but the surface numbers rebounded in 2019. The Statcast metrics did not. His average launch angle and the launch angle on the top 1/3 of his hardest hit balls dropped for the fourth straight year, falling into below-average territory. Not only was the lift missing, but he wasn’t hitting the ball as hard. The percentage of his air balls that were hit 100 mph or harder was 75^th percentile in 2017, but by 2019 it had fallen to the 49^th percentile. His Barrel% was 42^nd percentile in 2019, below average for the second straight year. Rizzo had gotten worse; the surface numbers just were not showing it yet.

Kris Bryant – A similar story for Bryant. Despite a very good .379 wOBA in 2019, his Statcast metrics were declining, particular his ability to hit the ball hard. Basically, all of the EV metrics had been declining, but simple average EV tells the story simplest: 72^nd percentile in 2015, 59^th in 2016, 32^nd in 2017, 11^th in 2018, 22^nd in 2019.  Bryant had three straight years of bottom-third exit velocity. Even with decent surface numbers, THE BAT X just wasn’t going to buy him at the level most were going to.

Players 100-200

For the player range 100-200, THE BAT X was not as helpful. THE BAT X did identify the purchase of Christian Walker, but it also more strongly recommended Scott Kingery and Andrew Benintendi. As the -$5 represents a capped value, Kingery and Benintendi were in reality even worse than what the figures indicate.

Since the game theory methodology currently institutes a binary purchasing mechanism, the profit per player ended up being roughly the same between the systems for this player range. However, upon inspection – THE BAT X showed stronger buys on more of the value drainers. For $5-$15 players, I would have preferred THE BAT in 2020.

Below are Derek Carty’s notes on Yuli Gurriel and Andrew Benintendi. THE BAT X was far lower on Gurriel than THE BAT (and rightfully so), but unfortunately it got Andrew Benintendi wrong.

Yuli Gurriel – I think that most systems saw Gurriel’s .364 wOBA / 31 HR 2019 season as fraudulent, but THE BAT X really thought so. There was just nothing to like in his Statcast numbers whatsoever, especially once you dug past the most obvious. His average launch angle spiked to the 63^rd percentile (pretty good!), but many of those were pop-ups, as his percentage of balls that hit 38 degrees or higher (very little HR chance that high) was at the 80^th percentile. His launch angle on his hardest hit balls was just 6^th percentile … and he was not hitting balls hard. A 54^th percentile average exit velocity belied a 19^th percentile on flies and a 21^st percentile in terms of air balls hit 100+ mph. His Barrel% was 15^th percentile.  In other words, he was hitting balls higher and harder, but rarely at the same time and rarely in actual positive ways.

Andrew Benintendi – Despite a down surface year in 2019, many of Benintendi’s Statcast numbers had been trending up for the past three seasons.  Launch angle on hardest hit balls percentiles from 2017 to 2019: 40^th, 75^th, 80^th. 23-to-34-degree percentage: 46^th, 66^th, 93^rd. Barrels: 40^th, 48^th, 52^nd. Some of his exit velocity numbers did fall off in 2019, but THE BAT X expected a bit of a rebound given his fairly stable levels in 2017 and 2018.

Players 200-300

In the 200-300 player range ($2-4 AAV), THE BAT X correctly identified a number of profitable players such as Travis d’Arnaud, Brandon Lowe, Gio Urshela and Kevin Kiermaier. Of the group, THE BAT X indicated that Brandon Lowe was the strongest buy. With a $9 price point – THE BAT X users had a particularly good chance of owning the Ray. Here is a bit of narrative (from Carty) on THE BAT X’s Lowe love:

Brandon Lowe – Lowe had good surface stats in 2019, but his Statcast numbers were incredible. 97^th percentile barrels. 99^th percentile launch angle on his hardest hit balls. 92^nd percentile exit velo on flyballs. Nearly across the board, Statcast showed simply stellar ‘stuff,’ indicating big time power potential. 

THE BAT X did lose out on moderately strong pushes for Yandy Diaz and Nomar Mazara, but at this stage in the draft – due to the relatively low investment cost – the upside heavily outweighs the downside. While THE BAT lost $2 for its owners for these 10 players, THE BAT X made an average profit of $6.

Players 300+

For players ranked 300 or higher by AAV, we are getting into the $1-2 range of players. Rather than listing every player that switched between THE BAT & THE BAT X – here are a few key players with wider valuation differences:

There were no strong buys for either system after player 300, other than Ender Inciarte. THE BAT users might have identified Inciarte as a strong late round target, due to his speed potential.

THE BAT X showed a lower batting average, less power and worse run production. It was enough to knock Ender below replacement level for the system.

The strength of THE BAT X for the lowest valued players was the further avoidance of unprofitable players. For the 8 players above with the widest value differences between the systems – THE BAT X was correct 6 times, with THE BAT the wiser only twice. Brandon Nimmo and Asdrubal Cabrera turned out to be profitable players acquired by THE BAT, but all others were unprofitable.

For THE BAT X – Mike Yastrzemski was an enormous low value selection, which provided $24 of rotisserie profit. Here is why THE BAT X was high on the SF Giant:

Mike Yastrzemski – Similar to Lowe, Yastrzemski displayed really strong Statcast numbers in a lot of categories in 2019. 82^nd percentile barrels. 90^th percentile launch angle on his hardest hit balls. 92^nd percentile in percentage of balls hit between 23 and 34 degrees (a strong predictive range for home runs). His average exit velocity was only at the 61^st percentile, but his EV on flies was 75^th and the percentage of his air balls that were hit over 100 mph was at the 81^st percentile.

Conclusion

Looking at the incremental values from three of the better projection systems of 2020, here are a few notable observations:

• THE BAT X’s performance in the 50-100 player range was enormous. While all other systems produced negative value (which is typical) – THE BAT X was accretive by almost $3 a player.
• With the respect to the 50-100 range, the gains by THE BAT X, and the avoidance of bust players (as seen above) cannot be understated.
• From player 150 and on, THE BAT X performed very similarly to ATC.
• In almost every value range, THE BAT X performed as good or far better than THE BAT.
• Based on the game theory methodology, THE BAT X’s incorporation of Statcast data greatly helped the overall performance of the model, as intended.

In terms of cumulative profit per player purchased (for the entire player pool), here is how THE BAT X stacked up amongst the other 2020 projections that were previously evaluated.

Yes, 2020 is just one season – but it was a good one for Derek Carty’s first season of THE BAT X. With its improvements using Statcast data over THE BAT, it was able to leapfrog ATC for the top spot in projection system profitability for the year.

To note, for the 2020 season – THE BAT X had not yet been incorporated into ATC, aside from homeruns. After running a quick back-test of 2019 data, I had concluded that THE BAT X was superior to THE BAT in its HR rates, and I judgmentally swapped out THE BAT’s HR rates with those from THE BAT X. I did not have enough historical data to credibly modify any of the other offensive category weights to incorporate THE BAT X at the time. However, this analysis provides me with a reason to accelerate THE BAT X’s incorporation into ATC heading into 2021.

Hopefully, this analysis has given you a deeper look into how the game theory projections comparison work. At the same time, it appears that the incorporation of Statcast into player valuation is beneficial. It is also worth taking a look at Carty’s THE BAT X system for consideration into your 2021 fantasy baseball preparatory work.