Referencing Pitch Quality Models to More Traditional Stats

May 5, 2023

WARNING: If you are reading this article, some or most the exact values are out of date. The pitch quality models seem to go through at least a yearly adjustment so I can’t verify if all the numbers will hold up. With that caveat, it’s useful to have an overall idea of what each one means.

Last week, I was looking into Joey Lucchesi and I created this convoluted mess of a table.

Joey Lucchesi’s Pitch Modeling Stats

Model	SI	CU/CH	FF/FC	Stuff	Overall
Bot	52	46	37	43	49
Stuff+	89	91	72	86	98
pERA (AAA)	5.54	-0.44	4.79	2.74	3.14
pERA (AAA comps)	4.72	2.89	4.23	–	–

To start off with, having three different metrics using three different scales is confusing. Not as obvious was that I didn’t know exact what the two “Stuff” metric were exactly measuring. I had some idea listening to their creators and others using them. I decided to take a step back and put some perspective on the two pitch quality models so others and myself could correctly reference them and know what other metrics they corelate to.

Note: When I mention stuff metrics, I’m just referring to the Stuff values for Stuff+ and Pitching Bot. I know it can be confusing, especially with one system having the name Stuff+.

Two start out with, this article won’t answer two questions. First, I’m not looking into the predictiveness of the stats. While I have done some work on it, I feel that should be its own article. Second, I’m just looking at the combined values, not the individual pitches. Again, a separate article for another day.

Here at FanGraphs, we introduced the pitch modeling metrics over a month ago introducing PitchingBot and Stuff+ with separate writeups.

Here is a short description of each from the original articles.

PitchingBot

In short, PitchingBot takes inputs such as pitcher handedness, batter handedness, strike zone height, count, velocity, spin rate, movement, release point, extension, and location to determine the quality of a pitch, as well as its possible outcomes. Those outcomes are then aggregated and normalized on a 20-80 scouting scale, which is what is displayed on the leaderboards.

Stuff+

Stuff+ only looks at the physical characteristics of a pitch, including but not limited to: release point, velocity, vertical and horizontal movement, and spin rate.
…
Stuff+, Location+, and Pitching+ are all on the familiar “+” scale (like wRC+), with 100 being average.

While both supply a reason behind their values, it sucks that they each have their own scale. Personally, I have my pERA values and similar pitches on an ERA scale so there is readily recognizable reference.

The first item of business was to put put both of the metrics on an ERA scale. By lining up the values from 2021 and 2022 (min 40 IP) with the pitcher’s actual ERA, the following two formulas were created.

- PitchingBot values to an equivalent ERA (r-squared of .992): 22.697*e^(-.035*Bot Metric)
- Stuff+ values to an equivalent ERA (r-squared of .996): 49.19*e^(-.025*Stuff+ Metric)

With the two formulas, here is a quick reference table for stuff values and the ERA equivalent.

Conversion Table for ERA to “Stuff” Equivalents

ERA Equivalent	BotPlus	Stuff+
1.50	78	135
2.00	69	124
2.50	63	115
3.00	58	108
3.50	53	101
4.00	50	96
4.50	46	92
5.00	43	87
5.50	41	83
6.00	40	80

For an example, say a pitcher has a Stuff+ of exactly 100. We would expect the hitter to have an BotStuff around 52 and an ERA around 3.60.

The next step I did was bucket the three metrics for each PitchingBot (stuff, command, and overall) and Stuff+ (Stuff+, Location+, and Pitching+) and then compare them to other pitching metrics. To start with, here is a limited comparison (limited table size) with all the values in this Google Doc.

Pitching Bot

Comparison of PitchingBot’s Stuff to Other Metrics

Range	botOvr	botStf	botCmd	Pitching+	Stuff+	Location+	ERA	K/9	BB/9	WHIP	HR/9
>70	62	72	49	106	125	98	3.05	11.9	3.9	1.13	0.8
65-70	62	67	53	106	120	100	3.01	11.1	3.3	1.12	0.8
60-65	58	62	52	104	113	99	3.30	10.5	3.5	1.17	0.9
55-60	56	57	53	102	107	100	3.62	9.8	3.3	1.20	1.0
50-55	54	52	53	101	102	100	3.81	9.0	3.2	1.25	1.1
45-50	52	47	54	99	97	101	4.16	8.4	3.1	1.28	1.2
40-45	49	42	53	97	91	100	4.63	7.7	3.1	1.35	1.4
<40	46	36	54	96	86	101	4.60	6.7	2.9	1.36	1.3

Comparison of PitchingBot’s Command to Other Metrics

Range	botOvr	botStf	botCmd	Pitching+	Stuff+	Location+	ERA	K/9	BB/9	WHIP	HR/9
>70	62	44	70	101	88	108	4.06	9.4	1.4	1.11	0.9
65-70	64	50	66	105	104	107	3.75	8.8	2.0	1.16	1.1
60-65	59	50	62	104	101	104	3.59	9.0	2.3	1.16	1.2
55-60	56	50	57	102	101	102	3.85	8.7	2.7	1.21	1.1
50-55	53	52	52	101	103	100	3.83	9.1	3.3	1.26	1.1
45-50	49	51	47	98	99	97	4.37	8.7	3.9	1.36	1.1
40-45	46	53	42	97	102	95	4.28	9.3	4.5	1.35	1.1
<40	43	56	35	96	104	92	4.11	9.6	4.6	1.37	1.0

Comparison of PitchingBot’s Overall to Other Metrics

Range	botOvr	botStf	botCmd	Pitching+	Stuff+	Location+	ERA	K/9	BB/9	WHIP	HR/9
>70	72	69	61	112	128	105	2.39	11.9	2.2	0.95	0.8
65-70	67	62	61	109	118	104	3.07	10.7	2.4	1.05	1.0
60-65	61	58	58	105	110	103	3.41	10.2	2.7	1.13	1.0
55-60	57	54	56	102	104	101	3.59	9.3	3.0	1.21	1.0
50-55	52	49	53	100	99	100	4.05	8.5	3.2	1.29	1.2
45-50	47	45	50	97	93	99	4.40	8.1	3.6	1.34	1.2
40-45	42	44	45	95	92	97	4.87	7.8	4.0	1.43	1.3
<40	37	47	38	93	94	93	4.84	8.6	4.8	1.43	1.2

Stuff+

Comparison of Stuff+’s Stuff+ to Other Metrics

Range	Pitching+	Stuff+	Location+	botOvr	botStf	botCmd	ERA	K/9	BB/9	WHIP	HR/9
>130	111	137	102	65	68	54	2.38	12.5	2.6	0.92	0.9
125-130	107	127	100	61	65	52	2.97	12.0	3.4	1.12	1.0
120-125	106	121	99	61	65	52	3.02	10.2	3.3	1.12	0.7
115-120	106	117	101	60	62	54	3.24	10.8	3.3	1.14	0.9
110-115	104	112	100	58	57	54	3.25	10.0	3.2	1.16	1.0
105-110	102	107	100	54	54	52	3.63	9.5	3.2	1.20	1.0
100-105	100	102	100	52	51	52	3.87	9.1	3.4	1.25	1.1
95-100	99	97	100	52	49	53	4.17	8.4	3.2	1.30	1.2
90-95	98	92	100	49	45	53	4.49	8.0	3.2	1.33	1.3
85-90	96	87	100	49	44	53	4.78	7.4	3.3	1.41	1.3
80-85	95	82	100	48	41	53	4.64	6.9	2.9	1.38	1.4
75-80	94	78	101	46	38	54	4.64	6.8	3.0	1.39	1.3
<75	92	70	101	46	37	55	5.97	6.0	2.9	1.54	1.6

Comparison of Stuff+’s Command+ to Other Metrics

Range	Pitching+	Stuff+	Location+	botOvr	botStf	botCmd	ERA	K/9	BB/9	WHIP	HR/9
105-110	105	104	106	61	50	63	3.49	8.9	2.0	1.13	1.1
100-105	102	101	102	55	50	56	3.83	8.9	2.8	1.22	1.1
95-100	98	100	97	50	52	48	4.14	8.8	3.8	1.32	1.1
90-95	96	104	93	45	57	41	4.44	9.9	5.1	1.44	1.0
85-90	95	106	87	41	63	33	4.32	11.3	5.3	1.26	0.8

Comparison of Stuff+’s Pitching+ to Other Metrics

Range	Pitching+	Stuff+	Location+	botOvr	botStf	botCmd	ERA	K/9	BB/9	WHIP	HR/9
>115	116	140	105	72	69	60	2.35	12.3	1.7	0.83	0.9
110-115	111	125	104	67	64	59	2.84	11.3	2.4	1.00	0.9
105-110	107	114	102	61	58	57	3.16	10.2	2.7	1.10	1.0
100-105	102	104	101	55	53	54	3.65	9.2	3.1	1.22	1.0
95-100	97	94	99	49	47	51	4.30	8.2	3.5	1.33	1.2
<95	93	87	96	43	44	46	5.38	7.5	4.1	1.51	1.4

Looking over the information, both of the “Stuff” values seems to generally catch what each is trying to describe. The stuff values corelate to strikeouts and the command/location grades point walk rate.

After reading through the definitions of how the batted ball data is collected, I expected the Bot values to have a larger variance in the StatCast values (linked spreadsheet). That concept wasn’t the case and there ended up being almost not correlation to any of the measures to actually limiting hard contact. With hard contact not being predictive, I was surprised when I got to WHIP.

How much WHIP changed in the two “Stuff” models was almost too good to be true. In both cases, the walk rate increased as a pitcher’s stuff got better, but the hit suppression was so large that the WHIP declined.

The ability to detect hit suppression is on another scale than has ever been measured. It’s almost too good to be true.

Overall, I see two major issues with the stuff metrics:

The formulas behind the values is a black box so there is no way to back check the results. Also, the calculations are constantly changing so it’s tough to know which formula is being used. It’ll be impossible to incorporate the information if it keeps changing
The pitch model metrics are trained off of just the 2021 and 2022 data. Of course the data is going to almost lineup perfectly for now. It’ll be interesting to see how they hold up this season and three to four seasons down the road.

The next step for me will be to dive into the small sample of 2023 data. Does the near perfect accountability of all batted ball outcomes continue based on just pitch metrics or were the metrics correlated too close to the actual results I’m examining.

I did get a few questions answered but working through these Pitch Quality Models but I generated a ton more. As I get time, I’ll keep diving into the subject to see what is usable going forward.

BAL	CHW	ATH
BOS	CLE	HOU
NYY	DET	LAA
TBR	KCR	SEA
TOR	MIN	TEX

ATL	CHC	ARI
MIA	CIN	COL
NYM	MIL	LAD
PHI	PIT	SDP
WSN	STL	SFG