The Great Valuation System Test: The Process
It all began with a comment by Jason Bulay, aka The Stranger, on a post I published in January pitting two popular snake draft strategies against each other — best player available and position scarcity:
I posted this as a comment to Cwik’s article yesterday, but I really want to see everybody put their money where their mouth is with all the draft strategy/player valuation theory. Do a draft (or multiple drafts, because small sample size) where everybody will be scored using 2014 stats (or 2015 projections if you prefer). After the draft, add up everybody’s roster using standard 5×5 scoring. See who wins, and what kind of draft strategy really gets the best roster. See who loses, and mock them without mercy.
Not that I disagree with you – your points make sense. But I think this is something we can actually test and I’d love to read about the results, so why not give it a shot?
His idea was intriguing. We’re all very familiar with the various projection systems and know that the masterminds behind them continually strive to improve them. They are also tested every year and we learn which performed best. But valuation systems get none of this treatment, as there has seemingly been little to no progress made on properly valuing players since the original systems were developed and shared.
Jason eventually followed up with an email and after many back and forth messages, we finally settled on exactly what we wanted to test and how we would accomplish our goal.