The Great Valuation System Test: The Divisive Players

The aftermath of The Great Valuation System Test continues! Before reading further, be sure you know what was being tested and our process, along with the results:

The Process
The Results

At the end of the results article, I mentioned that although we now have a good idea of which valuation system proved best at converting a player’s statistical line into a dollar value, that doesn’t necessarily mean that the system does the best job for every individual player or player type. So today I’ll look into different player groups and individual players and see if we could find any patterns.

I will be excluding ESPN because their rating number is on a different scale and only using the Razzball – 0% Pos Adjustment & 100% versions. I also don’t have room to fit all the systems in so had to cut down in some manner.

Catchers

Easily the most divisive position is catcher. This is especially true in two catcher leagues, because depending on the league size and format, players qualifying at the position could get a significant boost in dollar value solely because they are catchers. Many fantasy owners are uncomfortable with this, preferring to punt the position and be satisfied drafting the bottom tier catchers.

I’m obviously a proponent of the positional boost, if warranted, because when generating dollar values, enough players at each position to fill every active roster slot must be worth at least a buck. Without some sort of positional adjustment, it’s very likely your last few catchers will be worth in the negative (though that won’t always be the case, at which point no positional adjustment is necessary). You’re certainly not going to be given extra auction dollars to add to your budget for taking these guys, so they must be boosted to a buck, which removes money from other players.

So let’s begin by taking a look at the top 12 catchers, using the names from Zach’s z-scores simply because that’s the position designations I have on my file. These aren’t necessarily the same top 12 from the other systems, but it doesn’t matter.

Note that Carlos Santana was listed at 1B in Zach’s values, which I assume means he was valued there, rather than at catcher, so if Zach had him at catcher, he would likely be worth more. I also assume all other systems valued him at catcher.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms
Buster Posey	22.26	22.60	25.00	17.60	15.30	23.04	21.60	22.03
Jonathan Lucroy	16.15	15.25	16.00	11.30	17.10	15.24	14.98	14.94
Devin Mesoraco	13.95	12.96	14.00	11.70	10.10	13.95	13.39	13.55
Yan Gomes	13.24	12.01	13.00	10.30	7.40	12.00	11.99	12.63
Carlos Santana	10.89	13.39	12.00	12.90	19.50	11.85	13.58	14.28
Salvador Perez	8.56	6.15	9.00	6.00	4.90	5.41	6.58	7.33
Brian McCann	8.32	6.19	7.00	7.00	3.60	4.76	6.43	7.73
Russell Martin	7.35	4.92	7.00	4.30	8.70	6.61	6.25	5.52
Dioner Navarro	5.60	2.67	5.00	3.00	5.20	4.21	4.19	3.66
Evan Gattis	4.78	1.88	4.00	3.20	-0.70	3.62	3.51	3.49
Wilin Rosario	3.38	0.25	3.00	1.00	-0.40	0.88	1.44	1.61
Derek Norris	2.99	-0.22	3.00	0.40	1.00	0.38	1.00	1.01
Total	117.47	98.06	118.00	88.70	91.70	101.96	104.95	107.78

Surprise, surprise, Razzball’s 0% Pos Adjustment valued the catchers the least amount. What is surprising, however, is that the 100% adjustment valued this group only $3 higher and was still far below the rest of the systems! I’m rather shocked that the REP system fell in the middle. In every auction and draft I’ve ever been in, catchers either go cheaper than my values or get drafted after I have them ranked. Perhaps fantasy owners know their inherent values, but still refuse to pay full price given the perception of added risk?

Jason told me that he was surprised by the fact that when running his z-scores, there was no position adjustment necessary. So his values are not adjusted, because all starting positions made it into the positive dollar pool. So it’s no surprise his z-scores value this group third lowest. But, it’s still $9 more than the Razzball system, also without any position adjustment. Zach’s z-scores and LPP are easily the highest and have nearly identical totals. It’s odd that Razzball’s 100% Pos Adjustment values Santana significantly higher than every other system, and he’s their top catcher, whereas Buster Posey is tops for everyone else. A breakdown of category dollars earned would have been helpful.

Speed

With my pride riding on Billy Hamilton this season, stolen bases is a category in which its primary contributors are viewed very differently. Many fantasy owners will shy away from drafting one-category speedsters given the potential hits they will take in homers and runs batted in. On the other hand, some owners realize the value of contributing such a high percentage of a category. Let’s see how the valuation systems handle this. I’ll choose a majority of the top base stealers from last year to compare values for.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms
Dee Gordon	25.71	26.06	26.00	20.70	14.70	33.71	32.45	22.11
Denard Span	19.58	19.69	18.00	14.70	12.40	21.68	21.47	17.84
Ben Revere	18.24	17.73	17.00	13.90	10.40	25.18	23.45	15.04
Billy Hamilton	17.72	17.33	16.00	15.50	11.70	23.26	23.63	14.97
Alcides Escobar	15.57	13.86	18.00	11.00	9.00	17.01	16.50	12.57
Rajai Davis	15.47	14.80	14.00	12.70	10.20	19.58	18.77	13.35
Leonys Martin	11.96	10.59	10.00	9.20	6.80	13.78	13.93	9.86
Jarrod Dyson	1.32	-2.00	-1.00	-0.10	-3.20	4.97	4.12	-2.24
Total w/o Altuve	125.57	118.07	118.00	97.60	72.00	159.17	154.33	103.51
Jose Altuve	35.86	37.89	37.00	28.90	27.80	48.06	42.95	32.67
Total w Altuve	161.43	155.95	155.00	126.50	99.80	207.24	197.28	136.18

I separated Jose Altuve from the group because of his huge batting average contribution as I wanted to try isolating just the stolen base component.

WOW! And there it is. The REP system values this group highest, with the winning SGP system close behind. After that is an enormous gap where the two sets of z-scores and LPP sit. Check out the low values for Razzball’s 100% Pos Adjustment system! Even the 0% Pos Adjustment is relatively low. I’m curious what Rudy has to say about why stolen bases are seemingly devalued so significantly.

You’ll notice that REP values Altuve highest by a wide margin. This is likely for two reasons — obviously for the stolen bases, of which we see the system values more than all the other systems from the non-Altuve group, and because of his league leading batting average. In my experience with the system, it seemingly calculates a larger range for batting average contributions than other systems. So, Altuve will get more positive value for his average than another system will dole out, while a low batting average guy will get a greater negative value.

On an island by their lonesome, is REP and SGP correct with their valuation of stolen bases or are they being overvalued? Well, when I run the REP system, I allocated the same dollars to each of the five hitting categories. So in a 12-team league with a $260 budget, that means that each category is allocated $430.56, assuming a 69/31 hitting/pitching split. For that very reason, an argument that the system overvalues stolen bases cannot be made. It values steals exactly the same as the other four categories! You may choose to strategically devalue steals when entering your draft, but that decision doesn’t actually change these hitters’ inherent values.

Power

If I believe that stolen bases are undervalued by the majority of valuation systems, then it follows that power must be overvalued. In order to try isolating just the power stats of homers and runs batted in, I will try to put together a group of hitters who were strong in those two categories, but relatively neutral in the other three.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms
Nelson Cruz	30.57	33.12	34.00	28.70	29.50	32.01	31.00	32.79
Adrian Gonzalez	23.10	27.90	26.00	23.40	18.20	26.32	25.03	27.59
Anthony Rizzo	22.11	26.89	25.00	22.60	21.50	26.85	25.22	26.68
Edwin Encarnacion	20.46	24.94	23.00	22.00	17.40	24.90	23.24	25.15
David Ortiz	17.62	21.25	19.00	19.80	18.70	22.13	20.15	21.96
Marcell Ozuna	17.29	17.27	16.00	15.20	15.40	15.90	15.96	17.79
Marlon Byrd	17.18	17.08	16.00	15.40	15.60	15.74	15.92	17.77
Lucas Duda	16.42	20.03	18.00	18.20	18.60	19.20	18.58	20.63
Adam LaRoche	15.50	18.97	16.00	16.90	14.00	18.15	17.45	19.45
Chris Carter	14.83	18.11	16.00	18.10	18.30	17.83	17.58	19.25
Brandon Moss	12.51	11.77	13.00	11.70	11.90	9.79	10.40	13.17
Total	207.59	237.33	222.00	212.00	199.10	228.81	220.54	242.24

Well this was unexpected! The REP system actually values this group third highest! Five of the eight systems value this group between $220 and $242. Another headscratcher is those Razzball systems again. They were lowest or near lowest for the catchers, again for speed and now again for power! Where are those dollars allocated to?!

Let’s take a look at where each system ranked in dollar values allocated for power and speed. The speed ranks exclude Altuve.

	Speed Value	Power Value	Total Value	Speed Rank	Power Rank
Todd Zola REP	159.17	228.81	387.99	1	3
SGP – Winning Fantasy Baseball Denoms	154.33	220.54	374.87	2	5
Jason’s Z-scores	118.07	237.33	355.40	4	2
SGP – Tony Fox Denoms	103.51	242.24	345.75	6	1
Last Player Picked	118.00	222.00	340.00	5	4
Zach’s Z-scores	125.57	207.59	333.16	3	7
Razzball – 0% Pos Adjustment	97.60	212.00	309.60	7	6
Razzball – 100% Pos Adjustment	72.00	199.10	271.10	8	8

One might expect to find the systems to be relatively close in total value between speed and power. You would assume that if a system is valuing one highly, it would value the other less so to balance things out, as after all, every system is working with the same $260 per team budget to allocate those dollars. But this is certainly not what we see here. Maybe it’s just a matter of assuming different hitting/pitching splits. That could very well be at least part of the explanation, as my valuations were based on a 69/31 split.

Batting Average

As mentioned in the Speed section when discussing Altuve, I believe the REP system weights batting average more heavily than other systems. It might be argued that it is doing things correctly, whereas the other systems aren’t. And since my batting average dollars are the same as the other categories, it’s difficult to argue the math is wrong. Let’s check out a group of high average hitters.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms
Melky Cabrera	19.42	19.77	19.00	15.60	15.40	19.10	18.39	19.42
Adrian Beltre	17.91	22.92	22.00	17.70	18.80	23.68	20.95	22.42
Justin Morneau	14.40	17.34	17.00	13.90	9.40	18.86	16.10	17.37
Jon Jay	3.90	1.21	5.00	0.90	0.00	2.04	1.84	2.10
Justin Turner	1.00	2.85	3.00	2.40	3.50	6.15	4.07	3.69
Juan Uribe	-1.00	0.24	1.00	0.90	0.70	3.07	1.00	0.90
Adam Lind	-3.19	-3.45	-2.00	-2.30	-5.20	-1.00	-2.57	-1.76
Total	52.44	60.88	65.00	49.10	42.60	71.91	59.78	64.14

BOOM. I was correct. The REP method values these players by $7 more than the next highest system, LPP. Bringing up the rear are the Razzballers once again, which makes me think Rudy must have allocated significantly fewer dollars to hitters than the other systems.

Now let’s check the low batting average guys. I would expect REP to value the group the lowest. Let’s find out.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms
Chris Carter	14.83	18.11	16.00	18.10	18.30	17.83	17.58	19.25
Brandon Moss	12.51	11.77	13.00	11.70	11.90	9.79	10.40	13.17
Khris Davis	11.44	10.38	10.00	10.30	10.00	8.38	9.62	11.62
Ryan Howard	7.83	9.76	9.00	10.60	4.90	6.94	8.28	11.50
Dayan Viciedo	5.61	3.52	3.00	5.10	4.90	0.69	2.79	5.68
Chris Davis	4.25	5.64	5.00	7.70	8.30	3.19	4.82	7.72
Oswaldo Arcia	3.01	0.41	1.00	2.80	2.50	0.08	0.74	2.46
Luis Valbuena	1.68	3.74	5.00	4.30	6.30	2.09	2.94	5.47
Mark Teixeira	1.33	2.08	2.00	4.40	0.40	0.88	1.81	4.26
Shin-Soo Choo	1.00	-2.08	-2.00	-0.10	-0.90	-3.62	-1.86	-0.01
Mike Napoli	0.88	1.44	2.00	3.20	1.60	2.15	1.91	3.20
Adam Dunn	0.74	1.33	1.00	4.00	-0.20	0.96	1.42	3.52
Mark Trumbo	0.70	-2.42	-2.00	0.30	-0.20	-1.28	-1.72	-0.49
Carlos Beltran	0.39	-2.77	-2.00	0.00	-0.70	-3.21	-2.08	-0.65
Garrett Jones	0.30	0.73	1.00	2.20	1.80	-0.52	0.29	2.77
Total	66.50	61.63	62.00	84.60	68.90	44.35	56.94	89.48

Right again, as REP sits a whopping $12.58 below the champion SGP system. There’s the Razzball money! The two systems now sit second and third, finally! Let’s now combine the values from the high and low average hitters for each system.

	High Average Dollars	Low Average Dollars	Total Dollars
SGP – Tony Fox Denoms	64.14	89.48	153.62
Razzball – 0% Pos Adjustment	49.10	84.60	133.70
Last Player Picked	65.00	62.00	127.00
Jason’s Z-scores	60.88	61.63	122.51
Zach’s Z-scores	52.44	66.50	118.94
SGP – Winning Fantasy Baseball Denoms	59.78	56.94	116.72
Todd Zola REP	71.91	44.35	116.26
Razzball – 100% Pos Adjustment	42.60	68.90	111.50

We know Tony’s SGP used denoms for a different league format. That’s fine, it worked kind of like a control system as we figured it would test poorly, just as it did. And we see here one of the reasons why as it somehow allocated far more to this group of players than the other systems. I’m not sure what these hitters had in common that led to such valuations as it combined both low and high average players. Interesting that in total REP was second lowest, but that’s probably because my low average group included double the number of players as the high average group. If the number was the same, REP likely would have fallen right in the middle.

Individual Players

Finally, let’s look at some individual players with the widest range of values.

Name	Zach’s Z-scores	Jason’s Z-scores	Last Player Picked	Razzball – 0% Pos Adjustment	Razzball – 100% Pos Adjustment	Todd Zola REP	SGP – Winning Fantasy Baseball Denoms	SGP – Tony Fox Denoms	Std Dev
Jose Altuve	35.86	37.89	37.00	28.90	27.80	48.06	42.95	32.67	6.38
Dee Gordon	25.71	26.06	26.00	20.70	14.70	33.71	32.45	22.11	5.77
Ben Revere	18.24	17.73	17.00	13.90	10.40	25.18	23.45	15.04	4.53
Victor Martinez	31.47	37.82	37.00	30.10	26.80	39.77	35.36	36.20	4.13
Michael Brantley	35.84	39.19	36.00	30.40	30.00	41.96	38.79	36.07	3.89
Billy Hamilton	17.72	17.33	16.00	15.50	11.70	23.26	23.63	14.97	3.83
Jose Abreu	29.94	35.90	33.00	29.60	25.70	37.95	33.89	34.76	3.70
Carlos Gomez	30.37	32.70	30.00	27.00	25.40	35.57	34.49	30.31	3.25
Todd Frazier	22.90	28.79	26.00	24.60	33.60	30.59	29.13	27.73	3.19
Jacoby Ellsbury	22.55	23.16	21.00	20.10	17.90	27.92	26.96	21.22	3.17
Lorenzo Cain	12.77	11.57	17.00	9.60	7.60	16.27	14.89	10.54	3.11
Denard Span	19.58	19.69	18.00	14.70	12.40	21.68	21.47	17.84	3.02
Alcides Escobar	15.57	13.86	18.00	11.00	9.00	17.01	16.50	12.57	2.95
Total	318.52	341.68	332.00	276.10	253.00	398.93	373.96	312.03

I highlighted the system with the highest value in green and the lowest value in red. The majority of the greens are REP and majority of reds are Razzball 100% Pos Adjustment. You could see from the totals that REP is most aggressive on these names, with SGP not too far behind. Perhaps the two systems could be described as using a stars and scrubs type process, whereas the others have a narrower range of values between players, akin to a spread the risk strategy.

In looking at the player names, it’s clear speed plays a large role, as the top three guys were near league leaders in stolen bases. Batting average also drives the discrepancies judging by the V-Mart, Brantley and Abreu appearances, while Gomez, Frazier and Ellsbury posted all-around contributions, so it’s hard to say where the valuation differences are coming from. Speed again?

What’s Next

Well this was a fun little exercise. Unfortunately, we can’t test which systems are right about all the players listed above. Though maybe I’m wrong and all you smarty pants could devise a method. Looking at the individual players, it’s clear that valuation plays a big role. If I came into my auction with a value of $48 for Jose Altuve, we surely know who is and isn’t winning the bidding on his services! And yet, we’re all using the exact same stats to generate his value.

As I finish typing this, I just realized that I should have probably added up the total dollars each system allocated to its top 156 hitters, which would make up the active rosters of each team. So here they are:

Todd Zola REP	2196.43
SGP – Tony Fox Denoms	2152.81
SGP – Winning Fantasy Baseball Denoms	2152.80
Last Player Picked	2150.00
Jason’s Z-scores	2125.70
Zach’s Z-scores	2090.24
Razzball – 100% Pos Adjustment	1843.90
Razzball – 0% Pos Adjustment	1839.40

It looks like I did indeed allocate the most money to hitting! That’s something I should have considered when discussing the player groups and individual players above. However, it’s still a rather small difference compared to the two sets of SGP values and LPP. Aha! The Razzball systems did allocate far less money to hitters than the rest of the systems. The discrepancy in dollars allocated to hitting did not affect the correlation test results, but just makes the analysis of the above player groups and individual players a little less meaningful.

So where do we go from here? We could continue to devise better ways to test which system does the best job of converting a player’s statistical line into dollar values, as at the very least, an improved test would require that all systems value players at the same positions and allocate the same money to hitting. We could also test the same system, with and without considering position.

We could take what we have learned and try improving upon the current valuation systems and fix the acknowledged flaws. Or we could develop a completely new system from scratch.

If nothing else, I leave you with this — entering your snake draft or auction without a set of dollar values (or rankings derived from dollar values) is like showing up to your office job naked. You will not have a very successful day, so don’t do it.