Is Year-to-Year Hitter Consistency Consistent?
Whether I like it or not, I’ve opened Pandora’s box on year-to-year consistency. The concept states that if a hitter’s overall year-to-year production is consistent, the consistent production will continue. Therefore, good, consistent hitters should be valued more highly since owners know what they’ll be getting on draft day. The problem is that consistent overall production doesn’t lead to future consistency.
The discussion started last week when I wrote that Eric Hosmer and Edwin Encarnacion had similar fantasy values but Hosmer’s NFBC ADP (average draft position) was quite a bit lower. Reader’s stated in the comments they devalued Hosmer because of his year-to-year inconsistency.
Now, some readers disagreed with the pair’s overall value because of different stances on their projections. Some valid arguments exist on the projections but it is a discussion for another day. The focus now is on year-to-year consistency being predictive of future consistency.
The discussion continued Friday on Twitter with several industry experts weighing in. Just expand the replies and read away on the hundred or so comments.
Consistency suckkkks! I know this guy agrees with me 👉 @RyanBHQ#FPAZ17 https://t.co/gwoYzDhABL
— Derek Carty (@DerekCarty) January 26, 2018
Two main camps emerged. Those experts who find consistency at the beginning of draft important and those that don’t care about consistency at all.
In the above Twitter thread, some in-season consistency was discussed. If readers want to read up and understand about in-season consistency, start with this article (and the ones linked from it) by Bill Petti at the Hardball Times. He sums up his findings as:
Hitters that tend to hit the ball in the air for power tend to produce in a more volatile fashion, while groundball hitters with higher on-base skills appear to produce more closely to their average on a daily basis.
He’s posted his daily volatility (consistency) values going back to 1974 for those interested. Enough on in-season numbers and on to the year-to-year discussion.
It’s time to determine if consistent hitters remains consistent. Here are the parameters I used.
To measure player talent, I’m going with OPS. While some other measures (e.g. wOBA or wRC+) are probably a better measure of true talent, OPS is easy to find and calculate. For the consistency metric, I took the standard deviation of the OPS values in question. The higher the deviation, therefore the less consistent the player. For the player’s talent level, I’m using OPS weighted by his plate appearances from X previous seasons (wOPS).
Note: Please let me know if you want to see other stats, benchmarks, seasons instead of the following ones used. Depending on the ease of manipulating the data, I can get back with an immediate response or I may take several responses and group them into a future article.
Math for those who care. Findings in summary for those smart enough to skip this section.
To start off the analysis, I took three seasons of data with minimum 100 PA in each and compared them to the next season. I cut-and-diced the data several ways to see if I could find any patterns of consistency. I found almost nothing.
First, I started out with the regulars who averaged 400 PA in the previous three seasons and compared their three-season deviation to the absolute difference between wOPS and actual value. The r-squared was .0005. Not good. Using this same set of players, I took the 50 least and most consistent hitters and compared their wOPS and the actual 4th-season values. The most consistent hitters averaged an absolute difference of .07232. For the least consistent hitters, the value was .072333. Nothing so far.
To see if better hitters or worse hitters were more consistent, I divided the group into two by wOPS and ran the same analysis. The r-squared jumped to a still pathetic .0035 for the more talented group and was .00074 for the less talented players.
The talented group’s average absolute difference is .0589 for the least consistent hitters and .0784 from the volatile. Now, a 20-point difference is decent. More on this in a bit but first, the weaker hitters. The least consistent group’s absolute average difference from actual OPS and wOPS is .0912. For the most inconsistent bad players, the average is .0775 for a ~14-point difference but in the opposite way compared to the good hitters. None of the data is consistent. Good, consistent players remain consistent but bad, consistent become inconsistent. I don’t buy it. I’m going to keep the results as reference and move on to find any possible pattern.
Even though I took the players with an average number of plate appearances over 400, those with more plate appearances were more consistent. The more data available, the better the outcome. The top-50 consistent hitters averaged 554 PA while the bottom-50 consistency players were only at 496 PA. By comparing the standard deviation in PA to OPS, an r-square of .05 emerges. While not great, it’s the best correlation produced so far. It’s not surprising since the more plate appearances a hitter gets, the more likely they are to be near their true talent level each season. With fewer at-bats, the more likely variation will occur. This factor is a major consideration going forward.
First, here are the r-squared values between different datasets (all available here to analyze).
| Group | R-sqaured |
|---|---|
| 3-years, 100 PA min | .0023 |
| 3-year, >= 250 Avg PA | .0006 |
| 3-year, >= 400 Avg PA | .0005 |
| 3-year, >= 500 Avg PA | .0013 |
| 3-year, >= 600 Avg PA | .0004 |
| 4-year, 100 PA Min | .0059 |
| 4-year, >= 250 Avg PA | .0098 |
| 4-year, >= 400 Avg PA | .0041 |
| 4-year, >= 500 Avg PA | .0072 |
| 4-year, >= 600 Avg PA | .0075 |
No values or trends of interest in that table. I’m throwing in the towel for today. No more math.
Summary
For those who skipped the math, congrats. Here’s the only points.
- There is little-to-no year-to-year correlation with consistency when just using OPS as the input for consistency.
- If just using OPS, playing time does matter. Hitters with the most plate appearances are the more consistent because they’ve had more chances to reach their true talent level.
Again, let me know if any other parameters could be changed to consistency predictable.
Does this mean year-to-year consistency isn’t predictable? No, not close. As I was working on the analysis, I kept going back to Bill Petti’s day-to-day correlation studies. It’s not the combined values but the overall talent inputs. Plate discipline and contact stabilizes quickly while BABIP and ISO don’t. While I have it run the numbers yet, I would not be surprised if player types (e.g. high K% sluggers) need to be the focus for consistency. It almost makes too much sense now. Stay tuned to find out if the idea is true in a day or two.
Jeff, one of the authors of the fantasy baseball guide,The Process, writes for RotoGraphs, The Hardball Times, Rotowire, Baseball America, and BaseballHQ. He has been nominated for two SABR Analytics Research Award for Contemporary Analysis and won it in 2013 in tandem with Bill Petti. He has won four FSWA Awards including on for his Mining the News series. He's won Tout Wars three times, LABR twice, and got his first NFBC Main Event win in 2021. Follow him on Twitter @jeffwzimmerman.
In your follow-up study, I and others might be interested in weekly consistency. A lot of fantasy leagues use weekly decisions, and intra-season consistency over one or two years might plausibly be predictive of reliability the next year (say, performance is similar to projections) Perhaps the consistent-week-to-week hitter is less sensitive to matchups or avoids long slumps.
BaseballHQ already does this to a large extent with their QC ratings. I’ve found them to be really helpful in generating a consistent team, personally. The Mayberry ratings seem to be helpful as well, particularly the reliability ratings.
I wonder if using OPS as your key indicator may be to narrow to accurately quantify consistency. Maybe something like points or standings gain points (either at an overall or category level) over a 3 year period would be more instructive, and fantasy relevant to understand consistency.
OPS is a holistic measure of hitting in a particular season, but for the purposes of your analysis perhaps a more holistic fantasy measure to understand the full value of a player will lend some insight.
Agree with the idea. With SGP being different for each league, I wanted to stay away from league specific settings.
After digging some more, the biggest factor for remaining consistent is PA.
What about survivorship bias? If a player is bad for 400PAs two years in a row, it doesn’t seem likely he’ll get 400PAs again the third year. He’d have to have posted a good season somewhere in that 3 year range, which would make him inconsistent by your measure. Well, unless he’s Alcides Escobar.
Then how does the player get or not get a consistency label?
I did drop the PA to 100 for several examples.
I think the way you analyzed it, consistency doesn’t matter but I think paying a premium in a draft for a “safe” player is worth it in the early rounds. They’re two different things but I think they get mixed up.
My definition of safe is a player no older than 31, who’s had a few seasons of 600+ PA and who’s had more than just one good season. Last year it meant staying away from guys like Miguel Cabrera, Trea Turner, Charlie Blackmon, Trevor Story and Jonathan Villar.
It doesn’t always work out but I think it gives a better percentage of success in the early rounds.
Agree and a few 600 PA seasons means he’s consistent.
I don’t think it is just raw consistency, but consistency above some certain level – that is what people value. Who cares if a player had back-to-back MVP caliber seasons that were dissimilar. Consistent high-level performance is what we should value.
I understand why you chose three seasons as your time frame as that creates a more manageable and larger database of players from which to do your analysis. For a similar reason, I also understand why you set a relatively low baseline of at-bats (100).
That said, I have questions about the time frame and the low baseline of at-bats. This discussion began with your comparison between Hosmer and Encarnacion. So while I do think there is great value in your efforts to ask and answer broader questions about volatility, efforts I will address, I do want to start with the relative standing of Hosmer and Encarnacion among fantasy players. Hosmer has been a more volatile hitter for the entirety of his seven years while Encarnacion has been a much less volatile hitter the past six years. Here are their respective OPS for each of the past six years, with Encarnacion listed first and Hosmer in parentheses: .941 (.663) .904 (.801) .901 (716) .929 (.822) .886 (761) .881 (.883). I wonder, then, if Encarnacion and Hosmer are outliers whose volatility might be missed when you place players in groups of 50 and for three or four years. I would be curious, then, for the past six seasons, how those two would rank in volatility among players with at least 2,500 at bats (that’s a group of 135 batters) and at least 3,000 at-bats (87 players).
In raising these questions, I should note that this sort of multi-season approach has been taken by the author you linked, Bill Petti. Perhaps his conclusion have changed since, but this is what he wrote in 2011 (and the link to the article: https://www.beyondtheboxscore.com/2011/9/7/2406195/hitter-volatility-part-vii) :
“There are many statistics that do not correlate strongly, year-to-year. That being said, even for those weaker correlated statistics we tend to see variation by players over the course of numerous seasons into higher and lower performers relative to average. For example, while Batting Average on Balls in Play has a relatively weak correlation year-to-year (.36), there are clearly hitters that over time display that their baseline is consistently higher or lower than the average hitter. So while Volatility in year 1 may not be a good predictor of volatility in year 2, there certainly may be hitters that are more or less volatile over the course of their careers than others. And that is something we can measure and take into account when evaluating hitters . . . So while Volatility may bot be a reliable metric that we can use to evaluate player skill on a yearly basis it appears that if we look at multiple seasons individual consistency does emerge and is likely an attribute that varies by player.”
Here’s what Petti wrote in 2012 for Fangraphs:
“The one bit of inferential analysis I’ve completed was a look at the year to year correlation of VOL. Turns out, this new formulation has a higher correlation year to year than my previous one (.39 vs. .23). Overall, it’s still low — basically, it’s as reliable year to year as batting average — but there is a decent relationship and we do see evidence in the data that, like BABIP, over the course of a career players will sort by generally higher or lower VOL. For example, the correlation between a hitter’s average VOL for years one and two and a hitter’s volatility in year 3 is .42. “
I should mention that it appears Petti revised his formula after the 2014 article that you linked, and that the more current is shown here: https://www.fangraphs.com/tht/corrvol-updating-how-we-can-measure-hitter-volatility/ This article limits itself to in-season volatility, but perhaps he has adapted his math too for season-to-season volatility.
In any case, you raised and researched a meaningful question, one that is relevant to real and fantasy baseball owners alike. I truly appreciate that you tackled this head-on and look forward to your future work in this area.
With Petti’s work, it’s out of context here. Petti’s VOL metric measures daily volatility for a season, not a full season to full season consistency.