Quantitative analysis of the WS Top 100

Hello Folks,

Well, in my typical, masochistic, numbers-obsessed sort of way, I decided to do a quick analysis of the WS Top 100 list. Mind you the motivation here was to try to discover any clues as to how the list is put together. So, I pulled the 'data' into a plotting package. Here's what I found. Note that in what follows, I define QPR = Score / Price.

First, the boring part:
1. Score vs Price is fit very nicely by a square root function ( Score = 0.66*sqrt(Price - 20) + 90, fit by eye ). In other words, score rises quickly with price for low prices then levels off at higher prices (no suprise there).
2. QPR vs Price is fit perfectly by a function of the form QPR ~ 1 / Price ( QPR = 93 / Price, fit bye eye ). In other words, QPR declines rapidly with price for low prices then goes asymptotically to zero for higher prices (no suprise there either).
3. QPR vs Score is fit very nicely by a linear function (QPR = -0.552*Score + 54.2, least squares fit). In other words, QPR declines with Score at a steady rate (no huge suprise there, although one might have expected a more rapidly declining function).

Now here comes the really interesting part:
4. Score vs Rank: There are breaks at Rank = 30 and Score = 94. There is 1 score below 94 for ranks above 30, and only 3 scores above 94 for ranks below 30.
5. Price vs Rank: There are breaks at Rank = 30 and Price = $50. There are 3 prices below $50 for ranks above 30, and only 10 prices above $50 for ranks below 30. In addition, there is cluster of higher prices from ranks 20 - 30, and a cluster of higher prices for ranks below 94.
6. QPR vs Rank QPR is low for rank above 20 (range 1 to 2.5), very low for ranks 20 - 30 (range 0.5 - 1.5), high for ranks 30 - 94 (range 1 - 6), and low for ranks 94 - 100 (range 1.5 - 2).

Taking 4, 5, & 6 together, we can construct the following conclusions regarding how the WS Top 100 is put together:
1. Ranks 1 - 30 are given to higher scoring, higher priced wines, with 20 - 30 given to very high priced, higher scoring wines (i.e., those in the higher scoring group with lower QPR).
2. Ranks 30 - 100 are given to lower scoring wines (which on average are lower in price and higher in QPR), with the last 5 or so given to lower scoring, higher priced (i.e., those in the lower scoring group with lower QPR).

The breaks in relationships 4 - 6 are so robust that I am willing to assert that the conclusions I have drawn are literally the quantitative (when put together with the specific info from relationships 4 - 6) guidelines as to how the list is put together.

Taking 4, 5, & 6 together, we can construct the following conclusions regarding how the WS Top 100 is put together:
1. Ranks 1 - 30 are given to higher scoring, higher priced wines, with 20 - 30 given to very high priced, higher scoring wines (i.e., those in the higher scoring group with lower QPR).

Conclusion #1 is interesting and suggests that they know the wine and price when they do their reviews. Its almost like "this wine is expensive" so we must rank it sorta high.

Tom T.

Conclusion #1 is interesting and suggests that they know the wine and price when they do their reviews. Its almost like "this wine is expensive" so we must rank it sorta high.

Tom T.

Interesting observation. The question is, does it simply reflect a correlation between price and score, or do they actually use as a guideline that 'nothing cheaper than $50 gets in the top 30' (which has interesting implications)? Of course, you can't really conclude much about how they assign scores from this list since it is a small and biased sample.

Just curious Ryan - would it be possible to also factor in the quantity of production for those wines (or maybe correlate rank with a function of score and production)?

Just curious Ryan - would it be possible to also factor in the quantity of production for those wines (or maybe correlate rank with a function of score and production)?

Sure, but the "At a Glance" list they've provided doesn't have the production noted, and I don't have a full access subscription. It would be a good thing to look at though, since they state that production and availability are factored in.

[David M. Bueker]

Hmm...score rises with price. Might quality also have some relationship in there? Might smaller production (hence that question posed by Salil) relate to more atisinal (or less industrial) producers with thus greater control over quality?

and of course we cannot forget the black helicopters with all the advertisitng plastered on the rotors.

Hmm...score rises with price. Might quality also have some relationship in there? Might smaller production (hence that question posed by Salil) relate to more atisinal (or less industrial) producers with thus greater control over quality?

and of course we cannot forget the black helicopters with all the advertisitng plastered on the rotors.

Like I said, it would be a good factor to look, and if I had access to the production numbers, I'd look at the following: Score vs Production, Price vs Production, QPR vs Production, and Production vs Rank.

At least for those scoring 97 or above, I don't see much that indicates "artisanality"...

Producer/Score/Case Production
rauzan 97 10,000
guiraud 97 9,165
malescot 97 6615
schrader 97 250
l'evangile 100 no production cited, but historically between 2500 and 3500
peter michael 97 3600
cos d'estournel 98 25,000

To have quantative analysis of wine and almost godlike respect for WS scores all in one post...it's almost too good to be true!

You guys crack me up. In a good way.



ps I had a hard time distinguishing the "boring" segment of the post from the "interesting" segment, but that's just me.

You numbers folks are always making up new words.

What the heck is 'quantative'?

Nice analysis Ryan!

The breaks in relationships 4 - 6 are so robust that I am willing to assert that the conclusions I have drawn are literally the quantitative (when put together with the specific info from relationships 4 - 6) guidelines as to how the list is put together.

Whilst the score/QPR vs price relationships would be expected - even during blind tastings (let's face it, there is a real relationship between "quality" and price) - those last 3 findings really suggest that WS does not rank wines blind, even if they scored them blind. I wonder how many of us do that in our own "rankings".

Is there any aspect of The Wine Spectator that is not at base fraudulent?

Nice analysis Ryan!

- those last 3 findings really suggest that WS does not rank wines blind

I said the same thing earlier. Its pretty clear that they know the wine and price before they taste.

Tom T.

. . . . those last 3 findings really suggest that WS does not rank wines blind, even if they scored them blind. I wonder how many of us do that in our own "rankings".

I don't think there's any reason to expect that they put the list together blind, or by-the-numbers-only: they do state that there is an 'x-factor' in the rankings which they identify as 'excitement.' Actually, until I found such striking trends in the analysis, I thought the the whole thing was mostly qualitative and at least a touch arbitrary.

You numbers folks are always making up new words.

What the heck is 'quantative'?

Whoops! Duly corrected - thanks. That's not me making up new words, rather me showing that my skills in mathematical analysis are greater than my skills in spelling. Apparently my browser's spell-checker does not check the "Subject" field.

To have quantative analysis of wine and almost godlike respect for WS scores all in one post...it's almost too good to be true!

God-like respect? Certainly didn't intend that. I think that the the List, like the scores, is merely a tool, that can be useful if used intelligently. In my "It's that time of year again" thread I spent a good deal of time lamenting about how what would in fact be interesting wines to try will now be unavailable to those of us who actually appreciate the wines and not just the press attached to them. The theme of the two threads has been more curiosity to the tune "What did they come up with this year?"

The only scores I regard as authoritative relative to my palate are Rogov's, and even there I've had some pretty substantial disagreements.

ps I had a hard time distinguishing the "boring" segment of the post from the "interesting" segment, but that's just me.

Can't begrudge you that. Us science types get an unusual sort of high out of finding clear trends in plots. Better for you to stay clear of the addiction!

I don't think there's any reason to expect that they put the list together blind, or by-the-numbers-only: they do state that there is an 'x-factor' in the rankings which they identify as 'excitement.' Actually, until I found such striking trends in the analysis, I thought the the whole thing was mostly qualitative and at least a touch arbitrary.

Of course they need to put the list together "non blind" but they really should do their ratings blind.

I once attended a dinner where the host secretly poured DRC Montrachet into a Mondavi Chardonnay Reserve bottle and poured the Mondavi Reserve Chardonnay into the DRC Montrachet bottle. He then placed both bottles on the table for all to see and taste. The comments that followed about the wines were amazing.

I think its clear from the analysis that they look at the label as they taste the wine and do the review/rating. How else could they give good ratings to all the advertisers?!

Tom T.

Whoops! Duly corrected - thanks. That's not me making up new words, rather me showing that my skills in mathematical analysis are greater than my skills in spelling. Apparently my browser's spell-checker does not check the "Subject" field.

Yes, I'm not necessarily a spelling master either and I love spell-checkers. But I use the word 'quantitative' so often for work that I have gotten used to catching errors.