Ask halfwheel: What is the Rating Formula?

We invite you, the reader, to ask questions. You can do it in the comments, on Facebook or Twitter or you can email us for our weekly segment: Ask halfwheel.

Each Friday, we tackle one of those questions and I’m back this week to answer a question from Jake, who wants to know what the rating formula is like:

For instance, do you have a sheet where you jot down every aspect from 1-10 and add them up? Or is it more or less just the number that pops into your head when your done? How strict is it?

In 2015 we implemented a new way of rating cigars on halfwheel. It was based on how we were evaluating cigars previously, but there were tweaks made for three main purposes:

1. Ensure consistency across all reviewers. — Prior to this, certain things like “how many points does a cigar lose for an uneven burn” weren’t being handled consistently across our every review.
2. Reward cigars with flawless construction and (fairly) penalize those without it — The reality is most cigars we review have pretty good construction. That being said, some don’t and we felt it was necessary to make sure that if two cigars performed equally in all parts other than a construction issue, the cigar without the issue scored better.
3. Diversify the scores — Much more is explained in this post, but in short we felt like the amount of cigars score 87-90 were getting to be a bit out of hand. There also seemed to be a glass ceiling when it came to cigars scoring above 92 or 93. We wanted to see cigars score both higher and lower within a reasonable manner.

And now to answer the question.

For a variety of reasons we aren’t going to publish the scoresheet, but even if we did, it probably wouldn’t be useful to anyone at home. The scoresheet itself is not based out of 100 points, rather an algorithm is used to take the sheet we use to evaluate each cigar and come up with a 100-point rating.

No reviewer uses the algorithm while they are scoring a cigar. Disclosure: I do have access to the algorithm, but I promise you I don’t input my scores until after I’ve finished The Bottom Line for a review.

Part of the reason for this is we want each reviewer to score the cigar naturally and independent. Every once in a while someone will rate a cigar and think it should have gotten a 90, but the scoresheet gave it a 91 or maybe an 88. We don’t want broad thoughts of one sample overwhelming the more precise evaluation of three cigars.

The process is done in Excel. Each reviewer uses the same scoresheet. They check off boxes based off of performance in a variety of categories for each cigar they smoke for a review.

We’ve always agreed the bulk of the cigars score should be decided by how it tastes. As such, flavor is worth around double the points of both construction and balance combined.

The scoresheet has guides with numbers and usually a descriptor attached to the number.

We score flavor in thirds and our scoring sheet ranges from “perfect/sublime” to “vomit.” There are other descriptions like “box worthy,” “five-pack worthy,” “recommend but wouldn’t buy,” and my personal favorite, “(I would) feel bad giving the cigar to a friend.”

These descriptions are all guides to help make the highly subjective process of evaluating a cigar a bit more consistent amongst our reviewers.

Construction is the next biggest category.

Scores are once broke down by third. The categories we look at are draw, smoke production and burn. Being forced to touch-up a cigar might cost a cigar a point in the end, having to completely relight it because the cigar went out due to non-user error will cost more points.

I should note, it’s unlikely that being forced to touch-up the cigar once between three cigars may not cost a cigar a point, having to touch-up each sample once will probably impact the final score. It’s all based on averages between the three samples, so rounding might help to reduce the impact of point deductions.

Balance is the next category, it’s not broken down by thirds, although we do have descriptors like “minimum score to be ‘balanced” and “one third could use improvement.” We view balance as an overall evaluation of how the cigar progress, how the flavors interacted with one another and how the body, strength and flavors performed together.

Finally, a simple pass/fail is given to a cigar’s appearance. While it doesn’t affect our smoking experience, we do think that it can play a role in how people buy cigars. It’s effectively worth about one point per cigar and there aren’t many instances in which a cigar loses a point because of appearance.

Once again, if one sample is covered in glue, but the other two are fine, there’s no guarantee the one sample will necessarily bring a cigar’s overall score down a point due to rounding.

We try to smoke three cigars per review if possible. Each cigar is scored independently and then the three scores are entered into a formula which puts out a score.

As always, hopefully that was more helpful than confusing. Thanks for your question Jake.

---

If you’d like to have your question featured on Ask halfwheel, click here and submit your question.