Scoring wines: what's the ideal method?

[Jenise]

And is anybody using it?

Until joining CellarTracker, I never publicly scored wines. I've hung out on boards like this one where that's not usually done or expected, and I can't say I ever deemed myself qualified to place such absolute judgments. I left that to the experts, whoever they are or were.

Now, because of CT, I do sometimes, often spurred by scores in community notes I read where I feel a need to place a vote equal to, higher or lower than someone else's more than I'm moved to make a statement all of my own. However, it's more for my own use than any sense of the greater good, and when I transfer tasting notes here I remove those scores, if I made them.

This came to mind reading a post on one of the CT forums wherein someone has joined and is considerably consternated over his perception that CT users generally rate wines 2-3 pts higher (or was it lower, that's how much attention I paid) than he does which means that when he throws his score in the hat it will barely cause a ripple or, when viewed collectively, make it look like he thought less of a wine than his peers when in fact he would contend that's not the case. Mind you, his area of interest is pretty much confined to one grape variety so that probably magnifies the impact he would hope to make.

However, it causes me to admit to my own little perturberance: that nearly all CT scores fall within a very narrow band. It's as if the original 20 point band of 80-100 has been compressed into a mere six, 88-93. A gross generalization but not untrue: you can practically throw out 80-84 and anything over 95 for all that anyone ever goes there. It's like 95 is the new 100. So rarely do you see higher numbers, it's as if 96 and over have been deemed "for official use only". Are that few wines truly outstanding? Or has mass, collective scoring resulted in point deflation?

Consider the lowly stratosphere of 80-84 which, once upon a time, meant 'Good'. I submit to you that 'Good' has turned bad. Wines generally scored under 84 points are full of flaws. Eighty-five to 87 points is just 'acceptable' and 88 points is the tipping point for the good stuff. Now I realize that CT users are drinking at a better level than average and that's going to skew scores on the wines they happen to buy/taste toward the quality end in the first place, and many consider it a waste of time to record clunkers--fair enough, but most of the wines most people drink are 88-93 pointers even when superlatives like 'outstanding' are used in the body of the note.

What happened to the place where every five point band categorically related to Good, Very Good, Excellent and Outstanding, why does it take such big cojones to declare a wine (any grape, any style) especially Excellent or Outstanding?

So what do I think the ideal system is? A variation on Michael Broadbent's five-star system which can be digitized (digitalized?) to 0-5. For one, it offers three practical differentiations of 'Good' which restores the idea that it's good to be Good. I also appreciate that in his system no score at all means Poor not just "I don't score wines". Here's his scale:

Poor
* Not very good but not bad"
** Moderately Good
*** Good
**** Very Good
***** Outstanding

I use a similar system for taking notes in combat situations that runs A-D, just like grade school, wherein A thru C get shaded with +'s and -'s and D is Poor because it allows me to tack a thumbnail impression on each wine relative to the other wines present that day. And it's faster to write 'B-" than make a bunch of stars. I can transcribe days' or even weeks' worth of notes later, with perspective.

So just some thoughts. I don't really want to change the world or wag the dog--I don't care enough, it's just that I've had too much coffee.

Don't worry, Jenise. We'll just look the other way.

The compression of the lower end of the scale is all too familiar to those of us in higher education. Colleges and universities are rebranded as service industries, with students (and their parents) as the clients, so the distribution creeps upward as faculty try to balance meaningful evaluation with not pissing off the customers. I wonder if CT users, rather than flattering their clients, are flattering themselves and their taste: who wants to admit that a bottle they paid 20 or 50 or a hundred bucks for was only an 85-pointer? It would be interesting to see if there's a systematically different distribution of scores for bottles that weren't from the raters' own cellars...

The compression of the top end is weirder and feels harder to explain. Maybe it's a way of leaving an empty seat at the table for the messianic wine that we're always waiting for but hasn't yet arrived, the Elijah of elixers?

[David M. Bueker]

I almost never score a wine anymore. I did it for a while in order to add my voice to the CellarTracker cacophony, but stopped as it seemed a useless exercise.

[Robin Garr]

I like describing the wine and letting my thoughts speak, but that said, I could (but won't) put down an extended rant about how ALL scoring systems basically reduce to a 7 1/2 point range, or ranges varying from a minimum of 8 to a maximum of 15 points.

The more years I drink wine the less I put stock in scores. I'm more interested in the descriptive part of the review and who has written the piece. Most of ZH's wines from Alsace are not my cup of tea but there are plenty of wine geeks who score them highly, just a matter of taste.

So now everyone is giving up on points just when I decide to start using them? I use a scale from -∞ to +∞ but since that is a bit restrictive I also use imaginary numbers. And since there are different sizes of infinities I will inevitably at some point use Georg Cantor's Alephs greater than Aleph null (probably for a Musar if positive). A very useful scale IMO though really just as arbitrary and unobjective as any other.

Great stuff Otto..but you lost me .

Jenise, I've wrestled with all these questions too. I seem to be stuck in a 7-10 point scale. I know what I like, so the bottles I buy are very 'biased' in that sense. It isn't likely (I hope) that I'd buy anything I know I wouldn't like, or is at least a good wine (however one defines that via scores). Other than flawed bottles of course.

Other related question about scoring include, why are only very certain types of wines able to get a perfect or nearly perfect score. Why can't there be for example, a 'perfect' rose or NZ sauvingnon blanc (or Sancerre, for that matter)? Why is it only Boirdeaux, Bungundy, Napa cabs, Piedmont Barolo, etc. get so highly scored?

I guess it's like democracy - both are extremely flawed systems, but seem to work the best regardless of the others options that exist (like Otto's infinity scale!)...

[Jenise]

Jenise, I've wrestled with all these questions too. I seem to be stuck in a 7-10 point scale. I know what I like, so the bottles I buy are very 'biased' in that sense.

But do you only like very good to excellent wines? I know you too well, and I doubt it.

why are only very certain types of wines able to get a perfect or nearly perfect score.

I've heard the point made that the upper registers are only for wines that have 30 year aging potential--or something like that. But that's not fair when the grape isn't a 30 year grape. Why should I then charge myself with responsibility for having this knowledge and temper back my score on an ungodly good WA gruner made in the Smaragd style because it doesn't have a track record or if it does I don't know what that is? I don't think I should.

I thought of that today when reading a Jeb Dunnock (in the name of Parker) review of a Languedoc Carignan he gave 94 points to in spite of the fact that he only predicted it would have 4-5 years of aging potential. Maybe, I thought, those days are over.

I score wines on the 100 point scale on CT for the sole reason that CT headlines an average score at the top of the page for each wine. I feel the need for my drop into the ocean to help to right some injustices in both directions; e.g. average over-scoring, IMO, of blockbusting fruit bombs and under-scoring of discreet elegant wines. One thing in favour of the 100 point scale is that there seems to be a consensus that anything below 80 means nearly undrinkable, below 85 pretty mediocre and 90 and above seriously good.

That is not the case with the 20 point scale which I used here in the past reckoning anything below 15 as of little interest unless cheap. That seemed to roughly chime with Jancis R and Chris K but not with the French critics whose typical scale, extracted from a RVF guide, is as follows-

20 à ne pas manquer
16 à 19 excellent
14 à 16 très bon
12 à 14 bon

Of the scoring systems used on this site, I like Dale's best of all, at least in his hands when accompanied by his TNs..

When I post my CT TNs here, I remove my scores for this fastidious company but still find it difficult to avoid a verbal conclusion ranging most often from "quite good" to "excellent". In theory I have a "great" category above "excellent" but can only think of less than a couple of handfuls of wines in my lifetime which qualify. Wines below "quite good" are much more common - recently a Barbera from Voerzio which I described as "horrible".

... I can't say I ever deemed myself qualified to place such absolute judgments.

Never stopped me from yapping.

Consider the lowly stratosphere of 80-84 which, once upon a time, meant 'Good'. I submit to you that 'Good' has turned bad. Wines generally scored under 84 points are full of flaws. Eighty-five to 87 points is just 'acceptable' and 88 points is the tipping point for the good stuff.

You've heard of Lake Wobegon?

Speaking for myself, I truly do not have (or see) many wines that often go above a so-called 95 points. There's a lot that needs to go right to push it toward perfection and I might have only had a handful or so of wines that reached 98-100. I grade more broadly to correspond to college (American) grades, where a C is rare, B-ranges more common, and simply doing well a little bit better than your peers will give you an A-. A is more special, and reserved for those wines showing more merit and consideration. You can say most scores cluster in the B+/A- range, but that is where I tend to drink, much of it through preselection (I research before I buy and also tend to know what I want for the wines I've had a longer association with, like Musar for example). I think many wine drinkers are in this narrow band as well. If you want higher scores, look to Galloni. I feel his points are 3-4 above where I would put them on most things.

I do not score with points. I do score on a five-part scale as follows: Poor, Fair, Good, Good+, Excellent. The meanings are:
Poor (flawed or just awful),
Fair (nothing obviously wrong with it but nothing right about it, either),
Good (it tastes like what the label says it is, I like it enough to finish the glass, it's got flavor, aroma, texture),
Good+ (sorry about that name)(like "Good" but I would buy it if I saw it in a store),
Excellent (unusually good in some way).

I don't participate in CellarTracker. As readers here know, I like to use Stuart Yaniger's Three Stooges wine rating system. I agree with Robin that, effectively, all of the scoring systems have a very narrow range. I also think that the entire concept of reducing the wine tasting experience to a single number is bogus from the get-go.

-Paul W.

I had similar discussions more than once with the much missed Daniel Rogov. I used to say something like that rating systems were simply ways for people to assign numbers to personal taste....a totally subjective exercise. Daniel would respond that wines and grape varieties had specific "ideal" characteristics and the rating number was a representation of how close the particular wine being rated came to the ideal expression of that wine. He claimed it was basically objective and I maintained it was basically subjective. I do miss Daniel.

Personally, I find descriptions and comparisons to other similar wines far more useful to me than a number....

I use "grades" rather than scores, but my range is pretty limited. I give out A's sparingly, but B's generously. As Mark mentions there is pre-selection involved. Most of the wines I drink are wines I selected because I've liked producers before, or a trusted source recommended- whether a WLDGer, a critic (Gilman) or a store (CSW). Many of the rest are chosen by friends who know &/or share my taste. So pretty understandable that most get somewhere in the B range ("good" in my system). C's appear more often if I am tasting wines that haven't been "curated" . Generally a wine would be a B before I'd consider rebuying (or higher if expensive).

I think most of us rank wine in some way- whether points, grades, words (exceptional, good, mediocre), stars, buy again/not buy again, etc. Frankly I use my own grades in buying decisions, and if I am deciding between 2 similarly priced Morgons and I see one I gave a B+ and one I gave a B+/A-, I know which I'll buy (though I readily admit to not being a consistent taster).

[David M. Bueker]

The issue of a selected set is important. I don't buy a wide range of wines to be exposed to the breadth of quality that is out there. If I were to score the wines I buy and drink, and end up with lots of wines in the 82-88 range or lower then I would have been doing a bad job of buying wines. Yes I do experiment here and there, ending up with a loser, but if I stick to my plans I almost exclusively buy excellent to outstanding wines. Where the quality range sometimes broadens is when they do not age as I expect them to, and a wine really falls off with bottle age.

I don't participate in CellarTracker. As readers here know, I like to use Stuart Yaniger's Three Stooges wine rating system. I agree with Robin that, effectively, all of the scoring systems have a very narrow range. I also think that the entire concept of reducing the wine tasting experience to a single number is bogus from the get-go.

-Paul W.

I never had too much faith in Stuart's approach to anything after he told me I should read more of Ayn Rand. I told him I hadn't read any books at that level intellectually since I was in the seventh grade. Now when Stuart lived in this part of the country we shared the cost of having bagels flown in fresh from NY every month or so and we decided our palate was more important than our approach to politics and so we have remained friends.

Frankly I use my own grades in buying decisions, and if I am deciding between 2 similarly priced Morgons and I see one I gave a B+ and one I gave a B+/A-, I know which I'll buy (though I readily admit to not being a consistent taster).

Bingo. Just what the man says.

I check with recent CT tasting notes a lot to see what they have to say about drinking windows, and while I'm there, just for fun, sometimes I check to see how CT wine descriptions and scores compare with the Wine Advocate's. Here and there, some CT descriptions of the wines do owe a debt to the Wine Advocate, and at the high end. my impression is the CT scores have a tendency to cluster a few points below the Advocate's. That's just an impression though, so I may be wrong, but my impression is consistent with Jenise's observation that CT scores tend to stop at 95, which indeed they do.

Why the near absence of scores above 95? Maybe simply because restraint is cool. Parker and the Spec say a wine is a 100 pointer, but then if you want to be cool it's tempting to say it's a 94 or 95.

I've always thought that wanting to score wines entailed giving a damn about what others thought of a wine, which except in special cases, I don't.

It is of zero interest to me whether or not a majority of the posting public thinks well or poorly of a wine.

It is of some interest to me what people I know and have tasted with think of wines, because I can use that input to predict whether or not I'd enjoy that wine. That can work either positively or negatively - I have many wine friends whose tastes cluster in the same precincts as mine does, but also a few people that love wines that I do not. One guy has to but rave about his latest California darling for me to know that it is a wine I would hate, simple, too sweet and too high in alcohol.

I used to give no credibility to Spectator scores when it was group scoring panels, as they were so unpredictable. I did pay attention to RP in the old days, at least on clarets and Rhones (he jumped the shark pretty early on for Aussie wines, and later headed the same way with California).

While I understand Jenise's urge to counter some of the garbage you see on CT (I also see some outrageous reviews), it is really just pissing into the wind, IMHO, a pastime best indulged in by others....

[Jenise]

I feel the need for my drop into the ocean to help to right some injustices in both directions; e.g. average over-scoring, IMO, of blockbusting fruit bombs and under-scoring of discreet elegant wines.

I suspect that you, like me, have less exposure to the former so that need rarely arises. But the latter--oh yes, that's when they get me.

I feel the need for my drop into the ocean to help to right some injustices in both directions; e.g. average over-scoring, IMO, of blockbusting fruit bombs and under-scoring of discreet elegant wines.

I suspect that you, like me, have less exposure to the former so that need rarely arises. But the latter--oh yes, that's when they get me.

Jenise, I had a scoring experience yesterday which makes me think that CT must have a method of discounting the low scores of an "anti-flavor" curmudgeon like me. I wrote a TN on a €7 red Graves which, IMO, was outrageously over-oaked and scored it 79 with a "dislike" note. I had noticed that, before my score, there was an "average" score of 90 with 2 users and one disclosed score of 89. After my score of 79, the "average" with 3 users only dropped to 89. Funny maths which implies that the undisclosed score must have jumped from 91 to 99 .

So Bill is right. It seems that it really is pissing into the wind against such a strong editorial bias.

PS: I messaged CT about this and they replied that 89 is a median score and that this average is displayed (together with score spreads) is displayed by clicking on the median figure. I tested it and it works.