In order for OTTI to continue ...


"Official Tea Tasting Initiative" Teas shared & discussed.

Re: In order for OTTI to continue ...

Postby Stentor » Sep 22nd, '12, 06:24

Ranking something is always very much relative to one's own expectations and experiences.
I'm sure I have rated some teas very highly in the past that I might only rate as "good" now (while some stay perpetual favorites). Sometimes you just come along something that redefines the levels of quality for you.
So even for the same person, a 10/10 may only be a 7/10 a few months or years later. I don't think that should discourage us from trying to give some sort of ranking at all, though.

Maybe we should rank a specific tea sample within an OTTI in relation to the other samples and give each tea an additional "personal" 1-10 score.
This might get a little more difficult if the samples are from different types of tea or in general very different from one another.
However, one could still do the ranking just to see which tea on average was liked best and how it is being rated by the participants.

I definitely do like to see people give some sort of rating and a scoring system (1-10, 1-5, A-F, whatever) is an easy understandable way of doing this. It's not an exact science and cannot be (which is why I think 1-10 is a good system, because it lets you see a tendency very easily).
Each participant should probably wait until they have tasted all samples within the OTTI before they start rating a sample (especially if we're rating in relation to the other samples). That doesn't mean they shouldn't be talking about what they have tried so far and what they thought of it, though! :)
That way, we can have sort of a conclusion at the end of the OTTI, which I think is nice just to see how your own estimation and experience compares to that of the other participants.
User avatar
Stentor
 
Posts: 556
Joined: Oct 8th, '1
Location: Germany

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 22nd, '12, 10:50

Edit
Last edited by RoseWood on Sep 24th, '12, 20:31, edited 1 time in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 22nd, '12, 11:35

edit
Last edited by RoseWood on Sep 24th, '12, 20:33, edited 3 times in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby dshu » Sep 22nd, '12, 14:41

I like where Rosewood is going. A rating system with more specifically defined rating levels/tiers would leave less room for subjective interpretation.
dshu
 
Posts: 39
Joined: Apr 5th, '1
Location: The Great Salt Lake

Re: In order for OTTI to continue ...

Postby edkrueger » Sep 22nd, '12, 15:26

I'd hardly call a tea with no off flavors "excellent" or with any off flavors "very good".

It seems people missed the part of my post that I thought was a good idea and credited me with the part I thought was a bad idea.
User avatar
edkrueger
 
Posts: 1693
Joined: Jun 24th, '

Re: In order for OTTI to continue ...

Postby Remick » Sep 22nd, '12, 16:22

edkrueger wrote:
But, I realized that percentile ranking might make a lot more sense: forget about means and distributions and just write what percent of similar teas you think are worse than the current tea.


I like this but I think, as a community, we're starting to divide around two separate ideals. One, as above, based on personal experience. The other, as mentioned by RoseWood, attempts to remove personal preferences and level experience by assessing a tea based (in some part) on defects or lack of them. Neither is wrong. In fact, both are used extensively in tea, coffee, and wine.

The problem with tasting for defect is that it should have already been done at the vendor level. Theoretically, when using a good vendor, the consumer should not need to do any "defect tastings". All teas with debilitating defects should have been culled by the vendor before the consumer is allowed to buy them.

The problem with preferential and thus experience based tastings has been fleshed out in the last two pages of this thread. Some seem to feel it's best to keep the integrity of the scores by vetting reviewers. Others, seem to prefer re-working the scoring system rather than the people who can participate.

Maybe these are mutually exclusive ideals, but I hope they aren't.

EDIT-

RoseWood wrote:Here's what I think (albeit coming from a newbie who hasn't yet participated in an Otti):

1. Chuck the grading system. "Would buy/would not buy" would give enough information for fellow 'chatters and vendors alike. After all, most of us wouldn't purchase a product again that earned a C or 3 or whatever..

3. Yes, to say something "sucks" is rude. And definitely falls into the not-helpful category. Was it weak, strong, bitter, astringent, grassy, flat, fishy, gym socks? A gentle reminder of etiquette sent with the order confirmation is probably a good idea. If a participant continually smashes the products, simply reserve the right to not ship to that individual.


This is probably my favorite of the group. However, I realize my two cents don't count for much.
Last edited by Remick on Sep 22nd, '12, 16:48, edited 2 times in total.
User avatar
Remick
 
Posts: 46
Joined: Jun 13th, '

Re: In order for OTTI to continue ...

Postby MIKE_B » Sep 22nd, '12, 16:27

Any scale is going to be subjective. Spending a lot of energy coming up with a standard is kind of a waste of time.

If one is taking the time to read the OTTI reviews, I would assume they understand the variants involved here.

I think we are getting off point. Chip's just asking us to all to play nice.
I think we all know what he means and I think we all know why.
User avatar
MIKE_B
 
Posts: 286
Joined: Nov 6th, '1
Location: Massachusetts

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 22nd, '12, 16:39

Edit
Last edited by RoseWood on Sep 24th, '12, 20:35, edited 2 times in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby AdamMY » Sep 22nd, '12, 20:45

While I agree that the best way to establish a uniform grading system is to develop well defined criteria for each score. In practice for something like tea, it is near impossible, to go off of the "flawless or near flawless" categories, I will say I have had teas that sound perfect on paper, in fact when drinking them, you can't find anything "bad" to say about them, but somehow the tea itself comes across as exceptionally lackluster. Not bad in any sense, but fails to excite in just about any way shape or fashion.

That is just one case, and from creating many many rubrics to grade student work, to try and establish well defined criteria for each category of grade, when you suddenly spread it across 5 people its nearly impossible to maintain. And its a Herculean task to try and make it uniform across 20 or more people. That is because people understand the rubric differently, and understand what they are tasting differently.


If you say that last sentence is the reason why we are letting everyone grade them to view the variation, then I tell you that basically flies in the face of what Grading really is supposed to do. In theory a grading should be uniform no matter who grades it.

Unless of course the stipulation is its a single persons personal opinion which they then use to give it a rating ( note I avoided the word grade on purpose here). In this case if it really is the persons personal opinion then why are we trying to tell them how they should view their own opinion?
User avatar
AdamMY
 
Posts: 2363
Joined: Jul 22nd, '
Location: Capital of the Mitten

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 23rd, '12, 09:33

edit
Last edited by RoseWood on Sep 24th, '12, 20:39, edited 3 times in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 23rd, '12, 11:10

Edit
Last edited by RoseWood on Sep 24th, '12, 20:42, edited 1 time in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby dzrogers » Sep 24th, '12, 12:00

Forgive me, but doesn't it go without saying that ratings are personal opinions that should be taken with a grain of salt?

If you all are trying to come up with a ratings system to please everyone you are going to be very sorely disappointed. I'm sure I'm not alone when I say that I pay preciously little attention to arbitrary numbers and much more attention to the written content of the review.
dzrogers
 
Posts: 29
Joined: Sep 5th, '1

Re: In order for OTTI to continue ...

Postby Stentor » Sep 24th, '12, 13:15

dzrogers wrote:Forgive me, but doesn't it go without saying that ratings are personal opinions that should be taken with a grain of salt?

If you all are trying to come up with a ratings system to please everyone you are going to be very sorely disappointed. I'm sure I'm not alone when I say that I pay preciously little attention to arbitrary numbers and much more attention to the written content of the review.


I agree. This cannot be standardized. I also don't think there is any need to.
If this wasn't clear earlier, the only numbers I would use would be "How did you like it on a scale from 1-10?" and "How do you rank it among the teas you have tasted for this OTTI (e.g. first to fifth place)?". I'd do this just so there's a quick indication of how the person liked it.
The written description of the flavour and of the overall experience is more important.

We could agree on using 1-10, 1-5, A-F or something but setting strict criteria for what exactly 7/10 means is not possible and not beneficial, in my opinion.
User avatar
Stentor
 
Posts: 556
Joined: Oct 8th, '1
Location: Germany

Re: In order for OTTI to continue ...

Postby RoseWood » Sep 24th, '12, 16:17

edit
Last edited by RoseWood on Sep 24th, '12, 20:47, edited 1 time in total.
RoseWood
 
Posts: 40
Joined: Aug 6th, '1
Location: Mid Atlantic

Re: In order for OTTI to continue ...

Postby edkrueger » Sep 24th, '12, 19:52

I opt out. I refuse to be bound by any poll. I will rate on whatever scale I feel like. If you care about my ratings –because, perhaps, you have similar tastes– they you can look at them and figure out how they are rated.

There are two problems that are getting mixed up here. One is different people won't be consistent in giving ratings regardless of the rating system. One is that the scale might not be actually tell you much about what a particular person thinks about a tea.

Anyway, developing rating system was never the reason for this thread. I don't think Chip cares too much about whether your system is the same as the suggested one. The suggested one was, I think, a suggestion. (Though he can correct me if I am wrong.)
User avatar
edkrueger
 
Posts: 1693
Joined: Jun 24th, '

PreviousNext

Instant Messenger

Permissions
You cannot post new topics
You cannot reply to topics
You cannot edit your posts
You cannot delete your posts
You cannot post attachments
Navigation