FIP Challenge 2010
Last year at the All-Star break I did a piece that collected all the pitchers who had a difference of at least 0.50 between their FIP and xFIP. At the end of the season I collected the 2nd half ERA for each pitcher in the survey to see if FIP or xFIP did a better job of predicting the results. The raw results favored xFIP, as that metric did a better job predicting 20 of the 34 pitchers. However, FIP did a better job of predicting how the best pitchers in the group would fare.
The results were definitely interesting, but it was hard to make any proclamations off one year of data. So, now is the time to assemble this year’s chart of pitchers with a discrepancy of at least half a run between their FIP and xFIP. Like last year, this list was crafted by hand, so please let me know if you see any omissions. I am looking for pitchers who had 70 or more innings pitched at the All-Star break.
| Name | HR/FB | ERA | FIP | xFIP |
|---|---|---|---|---|
| Paulino | 1.9 | 4.40 | 3.25 | 4.60 |
| Liriano | 2.5 | 3.86 | 2.18 | 2.97 |
| A. Sanchez | 3.4 | 3.66 | 3.46 | 4.52 |
| Buchholz | 3.6 | 2.45 | 3.45 | 4.26 |
| J. Johnson | 3.8 | 1.70 | 2.31 | 3.06 |
| Matsuzaka | 4.1 | 4.56 | 3.83 | 4.98 |
| J. Santana | 4.5 | 2.98 | 3.62 | 4.69 |
| Vargas | 4.7 | 3.09 | 3.62 | 4.84 |
| Verlander | 5.3 | 3.82 | 3.11 | 3.89 |
| Zito | 5.3 | 3.76 | 3.91 | 4.79 |
| Jimenez | 5.4 | 2.20 | 3.13 | 3.71 |
| Gorzelanny | 5.4 | 3.16 | 3.26 | 3.92 |
| Danks | 5.6 | 3.29 | 3.41 | 4.13 |
| Hanson | 5.6 | 4.13 | 3.26 | 4.02 |
| Cain | 5.7 | 3.34 | 3.82 | 4.72 |
| Kershaw | 5.7 | 2.96 | 3.11 | 3.79 |
| C. Lee | 5.8 | 2.64 | 2.58 | 3.34 |
| Wilson | 6.1 | 3.35 | 4.14 | 4.71 |
| L. Hernandez | 6.2 | 3.37 | 4.02 | 4.71 |
| Fister | 6.3 | 3.09 | 3.75 | 4.38 |
| Carmona | 6.3 | 3.64 | 4.08 | 4.61 |
| Floyd | 6.5 | 4.20 | 3.28 | 3.78 |
| Buehrle | 6.6 | 4.24 | 4.16 | 4.85 |
| Morrow | 6.7 | 4.86 | 3.42 | 3.93 |
| Cueto | 6.9 | 3.42 | 3.91 | 4.45 |
| Lackey | 6.9 | 4.78 | 4.39 | 4.98 |
| Correia | 15.7 | 5.26 | 4.82 | 4.22 |
| Hamels | 15.2 | 3.78 | 4.53 | 3.85 |
| Blackburn | 14.8 | 6.40 | 5.89 | 5.14 |
| Millwood | 14.8 | 5.77 | 5.03 | 4.32 |
| Karstens | 14.5 | 5.42 | 4.88 | 5.50 |
| Duke | 14.5 | 5.49 | 4.89 | 4.36 |
| Shields | 14.3 | 4.87 | 4.11 | 3.55 |
| Bannister | 14.0 | 5.56 | 5.26 | 4.69 |
| Wolf | 13.9 | 4.56 | 5.81 | 5.24 |
| Davis | 13.7 | 4.69 | 5.69 | 5.10 |
| Nolasco | 13.7 | 4.55 | 4.39 | 3.84 |
| Kennedy | 13.7 | 4.12 | 4.83 | 4.31 |
This year there are 38 pitchers in our survey. Five pitchers are repeats from a season ago – Kershaw, Lee, Blackburn, Verlander and Bannister. However both Blackburn and Bannister had HR/FB rates below average last year at the break while they are both above average this year. Kershaw, Lee and Verlander are the only ones who “beat” the average HR/FB rates in both seasons. As far as predicting 2nd half ERA goes, Kershaw and Lee were wins for FIP while xFIP did a better job of predicting Verlander.
To determine which metric is better at forecasting 2nd half ERA, I am going to take the midpoint between their FIP and xFIP and compare it to their real life ERA in the second half of the season.
Using Kennedy as an example, 4.57 is the midpoint between his FIP and xFIP. So, if Kennedy’s ERA in the second half is 4.44, I will count that as a “win” for xFIP. On the flip side, if Kennedy’s second half ERA is 4.66, I will count that as a “win” for FIP.
Like last year, I will check in on this list after the end of the regular season.
Love this analysis. Really allows me to use either or to stick as a trading point in the future.
Maybe a stupid questions, but did you include first half ERA as the third ‘predictor’ of second half ERA?
I did not but you can see the original list here
http://www.fangraphs.com/fantasy/index.php/should-fantasy-owners-use-fip/
Typo in the Cain column, I know he gets lucky but I doubt his FIP is really 43.82.
Fixed – thanks for the heads up.
Typo on Cain’s FIP
Sorry fixed already 🙂
I don’t see Dan Haren
Last year at the All-Star break I did a piece that collected all the pitchers who had a difference of at least 0.50 between their FIP and xFIP.
Haren’s only .3.
I’m not an expert in statistics but wouldn’t it be better to take all pitchers’ 1st half FIP and xFIP and compare it to the ERA of the 2nd half. Colin Wyers already did a comparison of the root mean square error of ERA, FIP, xFIP, and tRA by comparing even to odd numbered days over a five year period. http://www.hardballtimes.com/main/article/how-well-can-we-predict-era/
Even then the RMSEs where quite large in that sample. A half season’s innings isn’t going to tell you much anyway regardless of what system you’re using.
Also by limiting it to pitchers with some discrepancy between FIP and xFIP aren’t you likely introducing bias as well artificially limiting sample size?
BPro did a study when they were working on the SIERA model and they found xFIP to be the best year to year correlation for pitchers.
What I think you’ll find is that xFIP doesn’t work for the very best pitchers or for pitchers in extreme parks or with extremely good or bad defenses behind them. All of those types of situations tend to break the standard model.
I think FIP is a pretty useless stat for a single season personally, why correct for some of the luck in a pitchers stat line and not the rest of it and while yeah pitchers control their HR/FB some they control their own LOB and BABIP some too so correct some and not all of it. Just makes no sense to me.
Nice work Brian, I appreciate the effort. Good stuff to look at, just have to consider underlying factors that could have lead to second half inconsistencies to go along with these #s at the end of the season.
xFIP = (3*BB-2*K + 13 * (FB * HR/FB)/IP, but is HR/FB park adjusted in xFIP or does xFIP just use league average? Because if it doesn’t use park adjustment, I would go with FIP since it’s more likely to give a picture of the pitch in which he throws. But if it’s park adjusted, I would use xFIP.
By the way, you pretty much have to throw Cliff Lee out of the study if he gets traded to/from a hitters park to a pitcher’s park, like both of the last seasons. There’s just no way that you can perform a valid study on Lee going from Safeco to Arlington.
Every discussion on xFIP I’ve read either states or implies that the league-average HR/FB rate is inserted into that equation. This is done I guess because xFIP was conceived as a playing-field-leveler, i.e. a tool for deciding how well the pitcher pitched with luck removed. But since most people use xFIP for fantasy purposes, to know how to expect the guy to do going forward, I don’t know why someone doesn’t put out a version that inserts the player’s home park HR/FB rate into that equation (or mix it 50-50 with the league average, since half his games will be on the road).
Why nobody in the sabermetrics world bothers to do this – a stat which would be enormously more popular and useful – is beyond me. Of course, you’d want to use a big enough sample size to get that HR/FB figure, but for most ballparks we have multiple years to aggregate. How about it, guys?
I did this article last year in response to a claim by THT’s Derek Carty that he considered FIP useless for fantasy purposes. The other studies and suggestions mentioned above are all worthwhile – they just answer a different question.
Even with the move to a hitterr’s park last year, Lee’s ERA was better predicted by FIP. Even if it was not, I would be hesitant to throw out anyone from the sample, assuming they pitched enough innings in the second half.
I can see why you would be reluctant to throw it out, but you can’t really consider the data point to be indicative of much either. It’s not data tampering when, while running a medical study, you throw out one subject because they were diagnosed diabetic in the middle of the study.
But doesn’t the fact that he moved to a hitter’s park last year and still exceeded his xFIP projection act as a huge reason to keep him? At the break last year he had an xFIP of 4.13 but he had an actual second half ERA of 2.92
He may do the exact opposite of that this year but I would rather include all pitchers who threw a bunch of innings in both halves to get the largest sample.
You might want to be careful about including pitchers like Cliff Lee who get traded, as the ballpark you pitch in weighs heavily on your HR/FB (xFIP) ratio.
Look forward to the results.
vr, Xeifrank
i gotta question…
How do you account for the difference in (CURRENT ERA v. FIP) [VS] (Zips Projected FIP) for projecting future value?