Referencing Pitch Quality Models to More Traditional Stats

by
May 5, 2023

WARNING: If you are reading this article, some or most the exact values are out of date. The pitch quality models seem to go through at least a yearly adjustment so I can’t verify if all the numbers will hold up. With that caveat, it’s useful to have an overall idea of what each one means.

Last week, I was looking into Joey Lucchesi and I created this convoluted mess of a table.

Joey Lucchesi’s Pitch Modeling Stats
Model SI CU/CH FF/FC Stuff Overall
Bot 52 46 37 43 49
Stuff+ 89 91 72 86 98
pERA (AAA) 5.54 -0.44 4.79 2.74 3.14
pERA (AAA comps) 4.72 2.89 4.23 – –

To start off with, having three different metrics using three different scales is confusing. Not as obvious was that I didn’t know exact what the two “Stuff” metric were exactly measuring. I had some idea listening to their creators and others using them. I decided to take a step back and put some perspective on the two pitch quality models so others and myself could correctly reference them and know what other metrics they corelate to.

Note: When I mention stuff metrics, I’m just referring to the Stuff values for Stuff+ and Pitching Bot. I know it can be confusing, especially with one system having the name Stuff+.

Two start out with, this article won’t answer two questions. First, I’m not looking into the predictiveness of the stats. While I have done some work on it, I feel that should be its own article. Second, I’m just looking at the combined values, not the individual pitches. Again, a separate article for another day.

Here at FanGraphs, we introduced the pitch modeling metrics over a month ago introducing PitchingBot and Stuff+ with separate writeups.

Here is a short description of each from the original articles.

PitchingBot

In short, PitchingBot takes inputs such as pitcher handedness, batter handedness, strike zone height, count, velocity, spin rate, movement, release point, extension, and location to determine the quality of a pitch, as well as its possible outcomes. Those outcomes are then aggregated and normalized on a 20-80 scouting scale, which is what is displayed on the leaderboards.

Stuff+

Stuff+ only looks at the physical characteristics of a pitch, including but not limited to: release point, velocity, vertical and horizontal movement, and spin rate.
…
Stuff+, Location+, and Pitching+ are all on the familiar “+” scale (like wRC+), with 100 being average.

While both supply a reason behind their values, it sucks that they each have their own scale. Personally, I have my pERA values and similar pitches on an ERA scale so there is readily recognizable reference.

The first item of business was to put put both of the metrics on an ERA scale. By lining up the values from 2021 and 2022 (min 40 IP) with the pitcher’s actual ERA, the following two formulas were created.

    • PitchingBot values to an equivalent ERA (r-squared of .992): 22.697*e^(-.035*Bot Metric)
    • Stuff+ values to an equivalent ERA (r-squared of .996): 49.19*e^(-.025*Stuff+ Metric)

With the two formulas, here is a quick reference table for stuff values and the ERA equivalent.

Conversion Table for ERA to “Stuff” Equivalents
ERA Equivalent BotPlus Stuff+
1.50 78 135
2.00 69 124
2.50 63 115
3.00 58 108
3.50 53 101
4.00 50 96
4.50 46 92
5.00 43 87
5.50 41 83
6.00 40 80

For an example, say a pitcher has a Stuff+ of exactly 100. We would expect the hitter to have an BotStuff around 52 and an ERA around 3.60.

The next step I did was bucket the three metrics for each PitchingBot (stuff, command, and overall) and Stuff+ (Stuff+, Location+, and Pitching+) and then compare them to other pitching metrics. To start with, here is a limited comparison (limited table size) with all the values in this Google Doc.

Pitching Bot

Comparison of PitchingBot’s Stuff to Other Metrics
Range botOvr botStf botCmd Pitching+ Stuff+ Location+ ERA K/9 BB/9 WHIP HR/9
>70 62 72 49 106 125 98 3.05 11.9 3.9 1.13 0.8
65-70 62 67 53 106 120 100 3.01 11.1 3.3 1.12 0.8
60-65 58 62 52 104 113 99 3.30 10.5 3.5 1.17 0.9
55-60 56 57 53 102 107 100 3.62 9.8 3.3 1.20 1.0
50-55 54 52 53 101 102 100 3.81 9.0 3.2 1.25 1.1
45-50 52 47 54 99 97 101 4.16 8.4 3.1 1.28 1.2
40-45 49 42 53 97 91 100 4.63 7.7 3.1 1.35 1.4
<40 46 36 54 96 86 101 4.60 6.7 2.9 1.36 1.3

 

Comparison of PitchingBot’s Command to Other Metrics
Range botOvr botStf botCmd Pitching+ Stuff+ Location+ ERA K/9 BB/9 WHIP HR/9
>70 62 44 70 101 88 108 4.06 9.4 1.4 1.11 0.9
65-70 64 50 66 105 104 107 3.75 8.8 2.0 1.16 1.1
60-65 59 50 62 104 101 104 3.59 9.0 2.3 1.16 1.2
55-60 56 50 57 102 101 102 3.85 8.7 2.7 1.21 1.1
50-55 53 52 52 101 103 100 3.83 9.1 3.3 1.26 1.1
45-50 49 51 47 98 99 97 4.37 8.7 3.9 1.36 1.1
40-45 46 53 42 97 102 95 4.28 9.3 4.5 1.35 1.1
<40 43 56 35 96 104 92 4.11 9.6 4.6 1.37 1.0

 

Comparison of PitchingBot’s Overall to Other Metrics
Range botOvr botStf botCmd Pitching+ Stuff+ Location+ ERA K/9 BB/9 WHIP HR/9
>70 72 69 61 112 128 105 2.39 11.9 2.2 0.95 0.8
65-70 67 62 61 109 118 104 3.07 10.7 2.4 1.05 1.0
60-65 61 58 58 105 110 103 3.41 10.2 2.7 1.13 1.0
55-60 57 54 56 102 104 101 3.59 9.3 3.0 1.21 1.0
50-55 52 49 53 100 99 100 4.05 8.5 3.2 1.29 1.2
45-50 47 45 50 97 93 99 4.40 8.1 3.6 1.34 1.2
40-45 42 44 45 95 92 97 4.87 7.8 4.0 1.43 1.3
<40 37 47 38 93 94 93 4.84 8.6 4.8 1.43 1.2

Stuff+

Comparison of Stuff+’s Stuff+ to Other Metrics
Range Pitching+ Stuff+ Location+ botOvr botStf botCmd ERA K/9 BB/9 WHIP HR/9
>130 111 137 102 65 68 54 2.38 12.5 2.6 0.92 0.9
125-130 107 127 100 61 65 52 2.97 12.0 3.4 1.12 1.0
120-125 106 121 99 61 65 52 3.02 10.2 3.3 1.12 0.7
115-120 106 117 101 60 62 54 3.24 10.8 3.3 1.14 0.9
110-115 104 112 100 58 57 54 3.25 10.0 3.2 1.16 1.0
105-110 102 107 100 54 54 52 3.63 9.5 3.2 1.20 1.0
100-105 100 102 100 52 51 52 3.87 9.1 3.4 1.25 1.1
95-100 99 97 100 52 49 53 4.17 8.4 3.2 1.30 1.2
90-95 98 92 100 49 45 53 4.49 8.0 3.2 1.33 1.3
85-90 96 87 100 49 44 53 4.78 7.4 3.3 1.41 1.3
80-85 95 82 100 48 41 53 4.64 6.9 2.9 1.38 1.4
75-80 94 78 101 46 38 54 4.64 6.8 3.0 1.39 1.3
<75 92 70 101 46 37 55 5.97 6.0 2.9 1.54 1.6

 

Comparison of Stuff+’s Command+ to Other Metrics
Range Pitching+ Stuff+ Location+ botOvr botStf botCmd ERA K/9 BB/9 WHIP HR/9
105-110 105 104 106 61 50 63 3.49 8.9 2.0 1.13 1.1
100-105 102 101 102 55 50 56 3.83 8.9 2.8 1.22 1.1
95-100 98 100 97 50 52 48 4.14 8.8 3.8 1.32 1.1
90-95 96 104 93 45 57 41 4.44 9.9 5.1 1.44 1.0
85-90 95 106 87 41 63 33 4.32 11.3 5.3 1.26 0.8

 

Comparison of Stuff+’s Pitching+ to Other Metrics
Range Pitching+ Stuff+ Location+ botOvr botStf botCmd ERA K/9 BB/9 WHIP HR/9
>115 116 140 105 72 69 60 2.35 12.3 1.7 0.83 0.9
110-115 111 125 104 67 64 59 2.84 11.3 2.4 1.00 0.9
105-110 107 114 102 61 58 57 3.16 10.2 2.7 1.10 1.0
100-105 102 104 101 55 53 54 3.65 9.2 3.1 1.22 1.0
95-100 97 94 99 49 47 51 4.30 8.2 3.5 1.33 1.2
<95 93 87 96 43 44 46 5.38 7.5 4.1 1.51 1.4

 

Looking over the information, both of the “Stuff” values seems to generally catch what each is trying to describe. The stuff values corelate to strikeouts and the command/location grades point walk rate.

After reading through the definitions of how the batted ball data is collected, I expected the Bot values to have a larger variance in the StatCast values (linked spreadsheet). That concept wasn’t the case and there ended up being almost not correlation to any of the measures to actually limiting hard contact. With hard contact not being predictive, I was surprised when I got to WHIP.

How much WHIP changed in the two “Stuff” models was almost too good to be true. In both cases, the walk rate increased as a pitcher’s stuff got better, but the hit suppression was so large that the WHIP declined.

The ability to detect hit suppression is on another scale than has ever been measured. It’s almost too good to be true.

Overall, I see two major issues with the stuff metrics:

  • The formulas behind the values is a black box so there is no way to back check the results. Also, the calculations are constantly changing so it’s tough to know which formula is being used. It’ll be impossible to incorporate the information if it keeps changing
  • The pitch model metrics are trained off of just the 2021 and 2022 data. Of course the data is going to almost lineup perfectly for now. It’ll be interesting to see how they hold up this season and three to four seasons down the road.

The next step for me will be to dive into the small sample of 2023 data. Does the near perfect accountability of all batted ball outcomes continue based on just pitch metrics or were the metrics correlated too close to the actual results I’m examining.

I did get a few questions answered but working through these Pitch Quality Models but I generated a ton more. As I get time, I’ll keep diving into the subject to see what is usable going forward.





 

Jeff, one of the authors of the fantasy baseball guide,The Process, writes for RotoGraphs, The Hardball Times, Rotowire, Baseball America, and BaseballHQ. He has been nominated for two SABR Analytics Research Award for Contemporary Analysis and won it in 2013 in tandem with Bill Petti. He has won four FSWA Awards including on for his Mining the News series. He's won Tout Wars three times, LABR twice, and got his first NFBC Main Event win in 2021. Follow him on Twitter @jeffwzimmerman.

7 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
baseBRAWLmember
11 months ago

Jeff, super interesting “stuff”. Thank you.

3