Backtesting “The Perfect (New) Recipe” for Drafting Pitchers

Thanks to reader and LOTR enthusiast “Gandalfsstaff” for the comment they made on my article last week:

What if you hopped in the DeLorean and used the new formula pretending it was last year? Would it have predicted better picks in hindsight?

In that article, I wrote about the recipe I concocted, on the shoulders of giants, to target pitchers. I used the skills components in Ron Shandler’s LIMA plan and metrics from Eno Sarris’ pitching models to try and identify great pitchers in the upcoming 2024 season. The recipe included 2023 end-of-season stats and 2024 projected stats:

ATC 2024 Projections

• LIMA: K%>=25%, BB<10%, HR/9<1.3

2023 End-of-Season Actuals

• Stuff+ Fastball (FA, SI, FC) >=100
• Stuff+ Secondary (SL, CH, KC, CU, FS) >=100
• A called strike rate (CStr%, SIS) >12%
• Pitched at least 50 innings in 2023

Names like Zac Gallen, Gerrit Cole, and Corbin Burnes (new Oriole, no big deal) floated to the top of the bubbling, steaming pot. Ok, enough with the cooking metaphor. There’s a problem, though, with back-testing this recipe on last year’s data as Gandalfsstaff suggested. The K%, BB%, and HR/9 used in the recipe were projected by ATC. I don’t have data on last season’s projections. Next season’s Stuff+ metrics, also, aren’t specifically projected. Stuff+ is used to make projections, but we don’t see a fastball Stuff+ projection for Gerrit Cole in 2024. However, as you will see, Cole’s fastball will likely be very similar in 2024 to how it was in 2023. As complicated as that all sounds, back-testing to see which pitchers met all the requirements of the recipe by year’s end is not. Let’s go back in time. Here are the pitchers who accomplished all the bullet points above by the end of the 2022 season:

–

Now that looks pretty darn good. Anyone of those seven starters could have anchored a fantasy rotation. The true secret ingredient here is a dominant fastball. My attempt at creating a recipe pre-season 2023 did not turn out well because I wasn’t using the best metrics. pVals are not predictive and they didn’t belong in my preseason analysis. Stuff+, however, is predictive:

You can read all about the predictive power of Stuff+ and other pitching models, but the chart above explains a lot. Fastball Stuff+ is generally repeatable year-to-year. This is nothing new. Once a pitcher has a dominant fastball, they can work their secondaries with more success. Hitters, bless their hearts, have enough to deal with when a fastball’s Stuff+ rating gets above 100. Furthermore, the ability of a pitcher to earn called strikes is important because he needs something that brings the bat off the hitter’s shoulder when the hitter has the advantage. Sure a pitcher has a good fastball and secondaries, but if those pitches rarely get a “Strike!” from the umpire, hitters can just become observers. Finally, the LIMA plan skill components from days of old were just as good in 2021 and 2022. You can’t fake striking someone out. Let’s take a look at who followed this recipe to the flour-stained, bottom of the pages in 2021:

–

–

Once again, the recipe yields positive results. Don’t get too hopeful about this recipe, it’s difficult to predict which pitchers will end 2024 having met all of the very challenging criteria above. It’s even more difficult for even the top-most gifted pitchers in the world to go out and do it! If I were stuck with only one statistic to predict with confidence for the upcoming season, I would choose innings pitched every single time. Nestor Cortes was awesome in 2022 but dealt with injury all 2023 long, limiting his innings pitched and therefore, a repeat great season. If only I were Biff Howard Tannen. Unfortunately, we can’t predict anything with that much confidence, so relying on repeatable skills and moving forward with fingers crossed seems to be the best way to go.