The Great Valuation System Test: The Divisive Players

The aftermath of The Great Valuation System Test continues! Before reading further, be sure you know what was being tested and our process, along with the results:

The Process
The Results

At the end of the results article, I mentioned that although we now have a good idea of which valuation system proved best at converting a player’s statistical line into a dollar value, that doesn’t necessarily mean that the system does the best job for every individual player or player type. So today I’ll look into different player groups and individual players and see if we could find any patterns.

I will be excluding ESPN because their rating number is on a different scale and only using the Razzball – 0% Pos Adjustment & 100% versions. I also don’t have room to fit all the systems in so had to cut down in some manner.

Catchers

Easily the most divisive position is catcher. This is especially true in two catcher leagues, because depending on the league size and format, players qualifying at the position could get a significant boost in dollar value solely because they are catchers. Many fantasy owners are uncomfortable with this, preferring to punt the position and be satisfied drafting the bottom tier catchers.

I’m obviously a proponent of the positional boost, if warranted, because when generating dollar values, enough players at each position to fill every active roster slot must be worth at least a buck. Without some sort of positional adjustment, it’s very likely your last few catchers will be worth in the negative (though that won’t always be the case, at which point no positional adjustment is necessary). You’re certainly not going to be given extra auction dollars to add to your budget for taking these guys, so they must be boosted to a buck, which removes money from other players.

So let’s begin by taking a look at the top 12 catchers, using the names from Zach’s z-scores simply because that’s the position designations I have on my file. These aren’t necessarily the same top 12 from the other systems, but it doesn’t matter.

Note that Carlos Santana was listed at 1B in Zach’s values, which I assume means he was valued there, rather than at catcher, so if Zach had him at catcher, he would likely be worth more. I also assume all other systems valued him at catcher.

Surprise, surprise, Razzball’s 0% Pos Adjustment valued the catchers the least amount. What is surprising, however, is that the 100% adjustment valued this group only $3 higher and was still far below the rest of the systems! I’m rather shocked that the REP system fell in the middle. In every auction and draft I’ve ever been in, catchers either go cheaper than my values or get drafted after I have them ranked. Perhaps fantasy owners know their inherent values, but still refuse to pay full price given the perception of added risk?

Jason told me that he was surprised by the fact that when running his z-scores, there was no position adjustment necessary. So his values are not adjusted, because all starting positions made it into the positive dollar pool. So it’s no surprise his z-scores value this group third lowest. But, it’s still $9 more than the Razzball system, also without any position adjustment. Zach’s z-scores and LPP are easily the highest and have nearly identical totals. It’s odd that Razzball’s 100% Pos Adjustment values Santana significantly higher than every other system, and he’s their top catcher, whereas Buster Posey is tops for everyone else. A breakdown of category dollars earned would have been helpful.

Speed

With my pride riding on Billy Hamilton this season, stolen bases is a category in which its primary contributors are viewed very differently. Many fantasy owners will shy away from drafting one-category speedsters given the potential hits they will take in homers and runs batted in. On the other hand, some owners realize the value of contributing such a high percentage of a category. Let’s see how the valuation systems handle this. I’ll choose a majority of the top base stealers from last year to compare values for.

I separated Jose Altuve from the group because of his huge batting average contribution as I wanted to try isolating just the stolen base component.

WOW! And there it is. The REP system values this group highest, with the winning SGP system close behind. After that is an enormous gap where the two sets of z-scores and LPP sit. Check out the low values for Razzball’s 100% Pos Adjustment system! Even the 0% Pos Adjustment is relatively low. I’m curious what Rudy has to say about why stolen bases are seemingly devalued so significantly.

You’ll notice that REP values Altuve highest by a wide margin. This is likely for two reasons — obviously for the stolen bases, of which we see the system values more than all the other systems from the non-Altuve group, and because of his league leading batting average. In my experience with the system, it seemingly calculates a larger range for batting average contributions than other systems. So, Altuve will get more positive value for his average than another system will dole out, while a low batting average guy will get a greater negative value.

On an island by their lonesome, is REP and SGP correct with their valuation of stolen bases or are they being overvalued? Well, when I run the REP system, I allocated the same dollars to each of the five hitting categories. So in a 12-team league with a $260 budget, that means that each category is allocated $430.56, assuming a 69/31 hitting/pitching split. For that very reason, an argument that the system overvalues stolen bases cannot be made. It values steals exactly the same as the other four categories! You may choose to strategically devalue steals when entering your draft, but that decision doesn’t actually change these hitters’ inherent values.

Power

If I believe that stolen bases are undervalued by the majority of valuation systems, then it follows that power must be overvalued. In order to try isolating just the power stats of homers and runs batted in, I will try to put together a group of hitters who were strong in those two categories, but relatively neutral in the other three.

Well this was unexpected! The REP system actually values this group third highest! Five of the eight systems value this group between $220 and $242. Another headscratcher is those Razzball systems again. They were lowest or near lowest for the catchers, again for speed and now again for power! Where are those dollars allocated to?!

Let’s take a look at where each system ranked in dollar values allocated for power and speed. The speed ranks exclude Altuve.

One might expect to find the systems to be relatively close in total value between speed and power. You would assume that if a system is valuing one highly, it would value the other less so to balance things out, as after all, every system is working with the same $260 per team budget to allocate those dollars. But this is certainly not what we see here. Maybe it’s just a matter of assuming different hitting/pitching splits. That could very well be at least part of the explanation, as my valuations were based on a 69/31 split.

Batting Average

As mentioned in the Speed section when discussing Altuve, I believe the REP system weights batting average more heavily than other systems. It might be argued that it is doing things correctly, whereas the other systems aren’t. And since my batting average dollars are the same as the other categories, it’s difficult to argue the math is wrong. Let’s check out a group of high average hitters.

BOOM. I was correct. The REP method values these players by $7 more than the next highest system, LPP. Bringing up the rear are the Razzballers once again, which makes me think Rudy must have allocated significantly fewer dollars to hitters than the other systems.

Now let’s check the low batting average guys. I would expect REP to value the group the lowest. Let’s find out.

Right again, as REP sits a whopping $12.58 below the champion SGP system. There’s the Razzball money! The two systems now sit second and third, finally! Let’s now combine the values from the high and low average hitters for each system.

We know Tony’s SGP used denoms for a different league format. That’s fine, it worked kind of like a control system as we figured it would test poorly, just as it did. And we see here one of the reasons why as it somehow allocated far more to this group of players than the other systems. I’m not sure what these hitters had in common that led to such valuations as it combined both low and high average players. Interesting that in total REP was second lowest, but that’s probably because my low average group included double the number of players as the high average group. If the number was the same, REP likely would have fallen right in the middle.

Individual Players

Finally, let’s look at some individual players with the widest range of values.

I highlighted the system with the highest value in green and the lowest value in red. The majority of the greens are REP and majority of reds are Razzball 100% Pos Adjustment. You could see from the totals that REP is most aggressive on these names, with SGP not too far behind. Perhaps the two systems could be described as using a stars and scrubs type process, whereas the others have a narrower range of values between players, akin to a spread the risk strategy.

In looking at the player names, it’s clear speed plays a large role, as the top three guys were near league leaders in stolen bases. Batting average also drives the discrepancies judging by the V-Mart, Brantley and Abreu appearances, while Gomez, Frazier and Ellsbury posted all-around contributions, so it’s hard to say where the valuation differences are coming from. Speed again?

What’s Next

Well this was a fun little exercise. Unfortunately, we can’t test which systems are right about all the players listed above. Though maybe I’m wrong and all you smarty pants could devise a method. Looking at the individual players, it’s clear that valuation plays a big role. If I came into my auction with a value of $48 for Jose Altuve, we surely know who is and isn’t winning the bidding on his services! And yet, we’re all using the exact same stats to generate his value.

As I finish typing this, I just realized that I should have probably added up the total dollars each system allocated to its top 156 hitters, which would make up the active rosters of each team. So here they are:

It looks like I did indeed allocate the most money to hitting! That’s something I should have considered when discussing the player groups and individual players above. However, it’s still a rather small difference compared to the two sets of SGP values and LPP. Aha! The Razzball systems did allocate far less money to hitters than the rest of the systems. The discrepancy in dollars allocated to hitting did not affect the correlation test results, but just makes the analysis of the above player groups and individual players a little less meaningful.

So where do we go from here? We could continue to devise better ways to test which system does the best job of converting a player’s statistical line into dollar values, as at the very least, an improved test would require that all systems value players at the same positions and allocate the same money to hitting. We could also test the same system, with and without considering position.

We could take what we have learned and try improving upon the current valuation systems and fix the acknowledged flaws. Or we could develop a completely new system from scratch.

If nothing else, I leave you with this — entering your snake draft or auction without a set of dollar values (or rankings derived from dollar values) is like showing up to your office job naked. You will not have a very successful day, so don’t do it.