2016 Weighted Arsenal Scores
Around this time last year (edit: actually, it was more like sometime in 2014), Eno Sarris introduced the Arsenal Score. It was, and still is, a novel concept: for every pitcher, evaluate each of his pitches based strictly on their strikeout- and ground ball-inducing tendencies. Each pitch would be evaluated relative to its contemporaries — in other words, Corey Kluber’s slider would be compared to all other sliders in the league.
I’ll speak for Eno when I say the original Arsenal Scores weren’t meant to be especially rigorous. They received some flak for being mathematically inaccurate — to which I say, it doesn’t really matter. Originally, Eno calculated separate Z-scores for the ground ball rate (GB%) and swinging strike rate (SwStr%) — called “Z-BIP” and “Z-Whiff,” respectively, in the results to follow — of each pitch for every pitcher. The aggregate Z-scores — two Z-scores times X number of pitches — comprise the full Arsenal Score.
This time around, I propose a few tweaks:
- GB% + PU%: Ground balls are excellent, but pop-ups are the theoretically optimal batted ball outcome — the ball-in-play equivalent to a strikeout. Thus, I added pop-ups (PU%) to ground balls to capture critical outcomes, albeit simplistically.
- Pitch weights: Eno had equally weighted every pitch in a pitcher’s repertoire. This isn’t heavy criticism; I simply saw this as a flaw in which, for example, the game’s low-key best pitch might be overrated if it wasn’t actually thrown very often. Instead, I chose to weight each pitch by the actual frequency at which it was thrown.
- Z-score weights: This is more experimental than anything, and less of a critical update than the first two bullet points: Eno weighted the SwStr% Z-score twice as heavily as the GB% Z-score based on each statistic’s correlation with various ERA and FIP metrics (see original post). I’ll provide three separate scores: (1) equal weights, (2) Eno’s “2/3” weights, and FIP weights by which strikeout (or, in this case, strikeout inputs) are weighted 6.5 times more heavily than ground balls.
It’s true — this isn’t a mathematically legitimate way to do things. Frankly, I don’t care. Again, this isn’t (and wasn’t) mean to be anything more than a crude but interesting way to evaluate pitchers, especially in terms of identifying guys who may have breakout potential. A mathematically valid approach would likely involve building a regression equation that ultimately results in something closely reflecting FIP. It wouldn’t necessarily be a redundant exercise — a piecemeal FIP could be really fascinating — but it’s much more labor-intensive than this was perhaps ever meant to be.
So, let’s dig in. But first, the nitty gritty: I excluded all pitches thrown fewer than 50 times last year. Accordingly, a pitcher’s repertoire might not incorporate all his pitches. One can argue the pitches a pitcher throws least frequently are his least-effective. Their omissions, therefore, likely inflate the scores you see below by some undetermined amount for each pitcher. The “Freq” column indicates, from 0 to 100 percent, how much of the pitcher’s repertoire is accounted for in his scores. A lower “Freq” should be interpreted as the scores being more volatile and also perhaps inflated; a score of 100% means you have nothing to worry about. Lastly, I notice some names, such as my boy David Phelps, who don’t show up as starting pitchers here. It all depends on how Baseball Prospectus’ PITCHf/x leaderboard (P.S. thank you!) classifies a pitcher; I toggled by starting pitchers only.
Also: the nominal magnitude of the scores themselves mean nothing. Use them strictly for ranking purposes. Everything is relative.
With that said, here are your top-100 Weighted Arsenal Scores for 2016 starting pitchers , sorted by their FIP-weighted scores. (Note: Sort any column by clicking on its header!)
| Rk | Name | Z-Whiff | Z-BIP | Equal Weights | 2/3 Weights | FIP Weights | Freq | Pitches |
|---|---|---|---|---|---|---|---|---|
| 1 | Max Scherzer | 1.98 | 0.26 | 2.24 | 4.23 | 13.16 | 100% | 5 |
| 2 | Clayton Kershaw | 1.85 | 0.97 | 2.82 | 4.67 | 13.02 | 100% | 3 |
| 3 | Noah Syndergaard | 1.70 | 0.92 | 2.62 | 4.33 | 12.00 | 100% | 5 |
| 4 | Michael Pineda | 1.69 | 0.53 | 2.22 | 3.91 | 11.51 | 100% | 3 |
| 5 | Jose Fernandez | 1.67 | 0.40 | 2.07 | 3.74 | 11.26 | 100% | 4 |
| 6 | Mike Montgomery | 1.56 | 1.11 | 2.67 | 4.22 | 11.24 | 93% | 4 |
| 7 | Lance McCullers | 1.39 | 1.27 | 2.66 | 4.06 | 10.33 | 98% | 3 |
| 8 | Matt Shoemaker | 1.40 | 0.41 | 1.81 | 3.20 | 9.50 | 99% | 4 |
| 9 | Corey Kluber | 1.36 | 0.55 | 1.91 | 3.27 | 9.38 | 100% | 5 |
| 10 | Yu Darvish | 1.41 | 0.11 | 1.53 | 2.94 | 9.30 | 98% | 5 |
| 11 | Chris Archer | 1.30 | 0.76 | 2.06 | 3.37 | 9.23 | 100% | 3 |
| 12 | Carlos Carrasco | 1.30 | 0.75 | 2.05 | 3.34 | 9.17 | 100% | 5 |
| 13 | Jharel Cotton | 1.16 | 1.25 | 2.41 | 3.57 | 8.79 | 84% | 3 |
| 14 | Cole Hamels | 1.22 | 0.74 | 1.96 | 3.18 | 8.67 | 100% | 5 |
| 15 | Justin Verlander | 1.28 | 0.19 | 1.47 | 2.75 | 8.50 | 99% | 4 |
| 16 | Jon Gray | 1.20 | 0.63 | 1.83 | 3.03 | 8.45 | 100% | 5 |
| 17 | Sean Manaea | 1.18 | 0.71 | 1.90 | 3.08 | 8.41 | 100% | 3 |
| 18 | James Paxton | 1.17 | 0.75 | 1.93 | 3.10 | 8.38 | 100% | 5 |
| 19 | Danny Duffy | 1.26 | 0.08 | 1.35 | 2.61 | 8.30 | 99% | 4 |
| 20 | Dylan Bundy | 1.17 | 0.64 | 1.81 | 2.98 | 8.24 | 97% | 3 |
| 21 | Alex Reyes | 1.07 | 1.15 | 2.22 | 3.29 | 8.10 | 87% | 3 |
| 22 | David Price | 1.14 | 0.49 | 1.63 | 2.77 | 7.92 | 100% | 5 |
| 23 | Kenta Maeda | 1.13 | 0.55 | 1.68 | 2.82 | 7.92 | 100% | 5 |
| 24 | Nick Tropeano | 1.17 | 0.21 | 1.38 | 2.56 | 7.84 | 100% | 4 |
| 25 | Kevin Gausman | 1.06 | 0.86 | 1.92 | 2.98 | 7.77 | 100% | 5 |
| 26 | Robbie Ray | 1.10 | 0.63 | 1.73 | 2.82 | 7.75 | 100% | 5 |
| 27 | Vince Velasquez | 1.15 | 0.01 | 1.16 | 2.32 | 7.51 | 100% | 5 |
| 28 | Madison Bumgarner | 1.09 | 0.41 | 1.50 | 2.59 | 7.50 | 100% | 4 |
| 29 | Francisco Liriano | 1.01 | 0.88 | 1.88 | 2.89 | 7.42 | 100% | 4 |
| 30 | Tyler Glasnow | 0.98 | 0.99 | 1.97 | 2.95 | 7.39 | 97% | 2 |
| 31 | Jacob deGrom | 1.02 | 0.65 | 1.67 | 2.69 | 7.30 | 100% | 5 |
| 32 | John Lackey | 1.07 | 0.31 | 1.37 | 2.44 | 7.24 | 100% | 5 |
| 33 | Stephen Strasburg | 1.05 | 0.37 | 1.42 | 2.47 | 7.21 | 100% | 4 |
| 34 | Danny Salazar | 1.00 | 0.66 | 1.66 | 2.65 | 7.15 | 100% | 5 |
| 35 | Chad Green | 1.09 | 0.01 | 1.11 | 2.20 | 7.12 | 97% | 4 |
| 36 | Chris Sale | 1.03 | 0.39 | 1.43 | 2.46 | 7.12 | 100% | 4 |
| 37 | Rich Hill | 0.98 | 0.71 | 1.69 | 2.67 | 7.09 | 98% | 3 |
| 38 | Cody Anderson | 1.02 | 0.25 | 1.28 | 2.30 | 6.90 | 94% | 3 |
| 39 | Marco Estrada | 1.01 | 0.34 | 1.34 | 2.35 | 6.87 | 100% | 4 |
| 40 | Drew Pomeranz | 0.94 | 0.74 | 1.68 | 2.61 | 6.82 | 99% | 5 |
| 41 | Jose De Leon | 0.93 | 0.80 | 1.72 | 2.65 | 6.82 | 87% | 2 |
| 42 | Masahiro Tanaka | 0.93 | 0.75 | 1.68 | 2.62 | 6.81 | 100% | 6 |
| 43 | Michael Fulmer | 0.89 | 1.04 | 1.92 | 2.81 | 6.80 | 100% | 4 |
| 44 | Junior Guerra | 0.93 | 0.77 | 1.70 | 2.62 | 6.79 | 100% | 4 |
| 45 | Zack Godley | 0.91 | 0.79 | 1.70 | 2.61 | 6.71 | 100% | 4 |
| 46 | Jake Arrieta | 0.86 | 1.06 | 1.92 | 2.79 | 6.68 | 100% | 5 |
| 47 | Tyler Anderson | 0.88 | 0.94 | 1.82 | 2.70 | 6.68 | 99% | 4 |
| 48 | German Marquez | 0.96 | 0.31 | 1.27 | 2.23 | 6.54 | 88% | 2 |
| 49 | Jeremy Hellickson | 0.91 | 0.58 | 1.49 | 2.40 | 6.48 | 100% | 5 |
| 50 | Joe Ross | 0.92 | 0.48 | 1.40 | 2.33 | 6.48 | 99% | 4 |
| 51 | John Gant | 0.89 | 0.63 | 1.52 | 2.41 | 6.41 | 100% | 3 |
| 52 | Blake Snell | 0.96 | 0.15 | 1.11 | 2.07 | 6.41 | 100% | 4 |
| 53 | Drew Smyly | 0.97 | 0.09 | 1.06 | 2.03 | 6.38 | 100% | 4 |
| 54 | Julio Teheran | 0.94 | 0.25 | 1.19 | 2.13 | 6.37 | 100% | 5 |
| 55 | Garrett Richards | 0.87 | 0.63 | 1.50 | 2.37 | 6.28 | 96% | 4 |
| 56 | Adam Morgan | 0.93 | 0.13 | 1.07 | 2.00 | 6.19 | 100% | 5 |
| 57 | Luis Cessa | 0.84 | 0.70 | 1.55 | 2.39 | 6.17 | 100% | 4 |
| 58 | Eduardo Rodriguez | 0.93 | 0.05 | 0.98 | 1.91 | 6.11 | 100% | 5 |
| 59 | Alex Wood | 0.76 | 1.15 | 1.92 | 2.68 | 6.11 | 100% | 3 |
| 60 | Rubby de la Rosa | 0.78 | 1.06 | 1.84 | 2.62 | 6.11 | 100% | 4 |
| 61 | Daniel Norris | 0.90 | 0.24 | 1.14 | 2.04 | 6.09 | 100% | 5 |
| 62 | Jon Lester | 0.84 | 0.64 | 1.48 | 2.31 | 6.09 | 100% | 5 |
| 63 | Collin McHugh | 0.86 | 0.48 | 1.34 | 2.19 | 6.06 | 100% | 5 |
| 64 | Matt Andriese | 0.86 | 0.46 | 1.32 | 2.17 | 6.04 | 99% | 4 |
| 65 | Zack Greinke | 0.83 | 0.64 | 1.47 | 2.30 | 6.03 | 100% | 5 |
| 66 | Carlos Martinez | 0.74 | 1.18 | 1.93 | 2.67 | 6.02 | 99% | 4 |
| 67 | Bud Norris | 0.79 | 0.88 | 1.67 | 2.46 | 6.01 | 100% | 5 |
| 68 | Tim Lincecum | 0.88 | 0.27 | 1.15 | 2.03 | 5.99 | 100% | 5 |
| 69 | Julio Urias | 0.84 | 0.47 | 1.31 | 2.15 | 5.94 | 99% | 4 |
| 70 | Matt Moore | 0.85 | 0.38 | 1.22 | 2.07 | 5.88 | 100% | 5 |
| 71 | CC Sabathia | 0.70 | 1.07 | 1.77 | 2.47 | 5.60 | 99% | 5 |
| 72 | Jorge de la Rosa | 0.75 | 0.73 | 1.48 | 2.23 | 5.59 | 100% | 5 |
| 73 | Nate Karns | 0.80 | 0.32 | 1.12 | 1.92 | 5.53 | 100% | 4 |
| 74 | Matt Harvey | 0.78 | 0.46 | 1.24 | 2.01 | 5.51 | 98% | 4 |
| 75 | Taijuan Walker | 0.75 | 0.61 | 1.36 | 2.11 | 5.49 | 98% | 4 |
| 76 | Henry Owens | 0.87 | -0.17 | 0.70 | 1.57 | 5.48 | 86% | 3 |
| 77 | Johnny Cueto | 0.67 | 1.06 | 1.74 | 2.41 | 5.44 | 100% | 6 |
| 78 | Kyle Hendricks | 0.70 | 0.92 | 1.61 | 2.31 | 5.44 | 100% | 4 |
| 79 | Steven Matz | 0.67 | 1.04 | 1.71 | 2.38 | 5.40 | 100% | 5 |
| 80 | Aaron Nola | 0.66 | 1.08 | 1.74 | 2.41 | 5.39 | 100% | 4 |
| 81 | Carlos Rodon | 0.77 | 0.38 | 1.15 | 1.92 | 5.38 | 100% | 4 |
| 82 | Adam Conley | 0.77 | 0.34 | 1.11 | 1.88 | 5.32 | 100% | 3 |
| 83 | Jason Hammel | 0.78 | 0.27 | 1.05 | 1.82 | 5.32 | 100% | 5 |
| 84 | Nathan Eovaldi | 0.68 | 0.86 | 1.55 | 2.23 | 5.31 | 100% | 5 |
| 85 | Ian Kennedy | 0.81 | 0.03 | 0.84 | 1.65 | 5.28 | 100% | 4 |
| 86 | Ervin Santana | 0.72 | 0.55 | 1.27 | 1.99 | 5.21 | 99% | 4 |
| 87 | Robert Gsellman | 0.63 | 1.10 | 1.73 | 2.36 | 5.21 | 91% | 4 |
| 88 | Reynaldo Lopez | 0.74 | 0.36 | 1.11 | 1.85 | 5.20 | 100% | 3 |
| 89 | Marcus Stroman | 0.58 | 1.39 | 1.97 | 2.54 | 5.13 | 100% | 6 |
| 90 | Chris Young | 0.88 | -0.61 | 0.27 | 1.15 | 5.11 | 99% | 2 |
| 91 | Tim Adleman | 0.77 | 0.07 | 0.85 | 1.62 | 5.11 | 100% | 4 |
| 92 | Cody Reed | 0.66 | 0.81 | 1.47 | 2.13 | 5.09 | 100% | 4 |
| 93 | Felix Hernandez | 0.66 | 0.78 | 1.44 | 2.10 | 5.07 | 99% | 5 |
| 94 | Mike Foltynewicz | 0.73 | 0.34 | 1.07 | 1.79 | 5.06 | 100% | 5 |
| 95 | Clayton Richard | 0.52 | 1.63 | 2.15 | 2.66 | 4.99 | 99% | 4 |
| 96 | Joe Kelly | 0.65 | 0.77 | 1.42 | 2.07 | 4.98 | 91% | 4 |
| 97 | Steven Brault | 0.74 | 0.17 | 0.91 | 1.65 | 4.97 | 100% | 4 |
| 98 | Aaron Blair | 0.67 | 0.61 | 1.28 | 1.95 | 4.97 | 100% | 5 |
| 99 | Kyle Gibson | 0.65 | 0.77 | 1.42 | 2.06 | 4.97 | 100% | 5 |
| 100 | Dallas Keuchel | 0.57 | 1.25 | 1.82 | 2.39 | 4.96 | 99% | 5 |
The results seem to validate themselves. Would you hope or expect to see anyone else other than Kershaw, Scherzer and Syndergaard near the top? I won’t break down this list to exhaustion. Instead, I will point out some of the most obviously interesting names (sorted by FIP weights) and include the rank of their scores in the following order: FIP weights / Eno’s weights / equal weights.
Michael Pineda (4th / 6th / 8th): What to do with this guy? Filthy yet simultaneously hittable stuff. Enigmatic. If the home run rate regresses heavily, he could return a hefty profit, but this guy screams of red flags after underperforming his peripherals for two years.
Mike Montgomery (6th / 4th / 2nd): I won’t dive too deep into Montgomery; I featured him exclusively in this September 2016 post. The 10-second version: he threw five pitches that induced ground balls more than 50 percent of the time, and two of those pitches induced spectacular whiff rates. Until further notice, he’s the Cubs’ No.-5. His current NFBC ADP: SP83 (he doesn’t even show up on the starting pitcher page), 315th overall. Absolute bargain.
Lance McCullers (7th / 5th / 3rd): Ah, yes — Weighted Arsenal Scores are immune to walks. McCullers has dominant stuff; if he can chip (er, maybe chunk?) away at his walk rate, he could become a monster.
Matt Shoemaker (8th / 13th / 30th): The FIP weights punish Shoemaker less harshly for his fly ball propensities. Yet 30th among starting pitchers is still pretty dang good, and his swinging strike rate ranked fifth among all pitchers who threw at least 150 innings. At SP66, he’s also a low-risk, high-reward option.
Jharel Cotton (13th / 8th / 5th): Cotton! He has been a Carson Cistulli Fringe Fiver for some time now, and for good reason. Perpetually overshadowed by Julio Urias, Cotton has escaped to Oakland to pursue a full-time gig. His cutter is bonkers, and while his other pitches lag, he has shown real promise even in barely 30 Major League innings. A note, though: his low Freq indicates his score is a bit volatile and likely inflated. Bump him down a few pegs, but at least make sure he’s on your radar if he wasn’t already. Current NFBC ADP: SP71.
Other highly ranked names: Manaea, Paxton, Duffy, Bundy, Reyes, Tropeano (huh!), Gausman, Ray, Velasquez… all in the top-30 FIP-weighted Arsenal Scores.
* * *
Acknowledging the shortcomings of the Arsenal Score approach, I (and I’m sure Eno, too) would love to hear your feedback.
It’s important to remember that this is nowhere near the end-all, be-all of pitcher evaluation. Not even close. Sale, Strasburg, Arrieta, Cueto, Aaron Sanchez, and many more don’t rank particularly well here. There’s obviously a lot that this approach misses. But if you’re a fan of strikeouts, ground balls and pop-ups as shorthand analytical tools, then I hope this exercise points you in the direction of some pitchers you may not have given fair consideration prior to this.
“The results seem to validate themselves.” If you stop at about 20. This seems to fall apart in the middle of the list with some pretty crazy rankings (Cessa>Lester; Green>Sale and so on). Any ideas why this might fall apart beyond the very top?
(in before Luis Cessa and Chad Green are 2017 breakout fantasy MVPs)
I was alarmed by Cessa and Green, too; I’d chalk that up to very small sample sizes. They threw some of their pitches more than 50 times, but I don’t realistically think any of us would bet the house on a dude with 30 MLB innings. So, I guess an additional caveat is: be cognizant of the actual player sample sizes. I think the results become much more interesting (and meaningful) for pitchers with more robust 2016 samples under their belts. (An innings-pitched column would have been helpful.)
yeah i love this post but lincecum over urias is a little hair-raising. still, if you want to make an omelet you gotta break some eggs
Arsenal scores might be my favorite thing ever.
On the other hand, you’re sort of spoiling all my SP sleeper picks (Paxton, Montgomery, Cotton, I even had my eye on Cessa)
Fun fact: Only Clayton Kershaw and Luis Cessa had a higher SwStr% and lower BB% than Cotton
I ran some analysis for each of the three weights and their correlations to FIP, xFIP, and SIERA. Eno’s 2/3rds weight showed the strongest correlation for all three ERA estimators. If you can’t decide which weighted leaderboard to use, I recommend the 2/3rd weight. This showed a 0.40 R-squared correlation to both xFIP and SIERA. Including BB% as a variable upped the R-squared to 0.62 (to be expected).
If anyone is interested in producing their own Arsenal-based xSIERA here is the formula:
(2/3rd Weight * -0.451) + (BB% * 10.65) + 4.3
Cool! Thanks for doing the legwork. It makes some sense; Eno picked that weighting method because of their correlations with those metrics in the first place. Glad to see they hold up under duress. (Also, my insistence on sorting by FIP-weighted implicitly asserted that they were superior. Inadvertent, but obviously a mistake on my part!)
some prominent / decent MLB SP that didn’t make the top 100:
Jose Quintana
Gio Gonzalez
Yordano Ventura
Tanner Roark
Rick Porcello
Not a knock on this metric but maybe an interesting place to start on where stuff diverges from results
Except Porcello and Roark pitched over their heads, and Ventura has never lived up to how good his stuff appears to be. I’d make an argument for Quintana and maybe Gio… but I know what you mean. There’s definitely something missing here. On a rainy day, I’d love to turn arsenal scores into something more than just GBs, PUs, and whiffs.
Porcello had a career year, but it’s not like it was that crazy lucky. Same with Roark. He has a soft contact god all year.
I won’t dispute either claim; I even wrote about Roark’s contact management skills in the summer. But both guys benefited from extremely generous strand rates. Regardless, their skill sets — which rely not on whiffs nor contact type but contact quality — won’t be duly rewarded by Arsenal Scores.
Good stuff, love the Arsenal Score! Question – For guys who both started and relieved, I assume all pitches are included just the same in this calc, right? Perhaps that helps explain why guys like Montgomery, Cody Anderson, and Cessa rank higher than expected. The pitches they threw in relief with a little extra juice are being compared to a true SP’s 80th-90th-100th pitch thrown. Would be cool if you could filter for only pitches thrown as SP (not sure if possible).
Yeah, I don’t know. I don’t think the leaderboard makes that distinction. As for Montgomery specifically, he only made seven starts but produced an xFIP almost identical to his xFIP as a reliever with more strikeouts. In terms of whiffs, his cutter and change-up (arguably his two best pitches) played up even better during his starts, and they both still induced tons of ground balls. That’s why I’m excited about him as a possible #5; there’s a lot of potential there! Potentially. Potential potential.
Poor Cotton. Being good at inducing ground balls is probably a bad thing when you’re on a team where the infielders s__k (fill in the blank).
Spatterdock?
Lmao! Now I have to shift from FanGraphs to dictionary.com!
One disconnect between Arsenal Scores and results is the ability to throw strikes. A high walk rate can ruin a pitcher.
Theres a couple of obvious flaws in this approach. First of all, PU% isnt a stable or sustainable metric, so you are not really capturing their arsenal by including this, you are capturing what their arsenal produced in the past and might not ever do again. For instance, say a guy produced 10 PUs with a pitch last year but none this year or 2 yrs ago. Same exact pitch, frequency, movement, but a random change in PU%. It shouldn’t be included in a pure arsenal analysis. Same could be said for GB%, which also has little YTY correlation. SwStrk does have a good YTY correlation.
As for Pineda, its strange you dont mention this research: http://www.fangraphs.com/fantasy/pitcher-stuggles-explained-with-breaking-ball-zone-rate/ Do you not buy this explanation or what? Maybe his Fastball gets a good gb% and pu% which drives his AS upwards. But his zone% being so low overall (as the article mentions) limits his ability to get ahead and his fastball gets crushed.
Uhhhhhh…
First: PU% is a sustainable metric. Its year-to-year correlation is .61 (r-squared = .38). A pitcher’s PU% has to come somewhere. It’s likely that each pitch experiences fluctuations in PU%, but it’s unlikely that they induce *random* PU%s that all just *happen* to result in an overall PU% that correlates strongly with the previous year’s PU%.
Second: I did not come to Pineda’s defense. I said exactly what everyone else says about him: his peripherals look good, but he is extremely hittable. I don’t know why you’re up in arms. As for Zimmerman’s research, it’s nice, but the connection is purely anecdotal — there’s no rigorous demonstration of a correlation between this metric and BABIP (hittability) or HR/FB or ISO or OPS (power).
Very good, interesting read. I’m curious about the regression method– how do you envision going about that?