Tag Archives: comparison

Beertest

You are probably familiar with those bullshit stories that <your_brand_here> is somehow better/worse than <some_other_brand>. To resolve those issues me and a friend organized a blind beer-test. We found some very nice results. If you don’t know what beer to drink tonight, and you live in Holland: get Palm or Schultenbrau!

Finding beer

The first thing needed was BEER! and lots of it. After visiting several supermarkets (super de boer, albert heijn, aldi, lidl, gall & gall, mitra) we bought whatever beer we could. The only limit was the price per bottle and popularity: none of the beer was over €0,60 per bottle and the beers are available at many supermarkets. That limits this test to cheaper beers and pilseners. After spending €36,44 (ex pfand) we came up with 21 brands with 4 bottles each:

Albert heijn logo Alfa logo Amstel logo bavaria logo Brand logo
€ 0.93/l : Albert € 1.80/l : Common (& Amstel light)

€ 1.53/l : Common

& € 1.73/l : AH

(& Bavaria 0.0)

€ 1.30/l : Common

& € 1.20/l : Common

€ 1.70/l : Common
dommelsch logo euroshopper logo grolsch logo gulpener logo heineken logo
€ 1.53/l : Common € 0.63/l : Common € 1.57/l : Common € 1.83/l : Common € 1.57/l : Common
Hertog jan logo jupiler logo Lindeboom logo Oettinger logo Oranjeboom logo
€ 1.83/l : Common € 2.04/l : Common € 2.00/l : Mitra € 0.81/l : Lidl € 1.30/l : Mitra
Palm logo Pitt logo Schultenbrau Warsteiner logo
€ 1.90/l : Common € 0.67/l : SDB € 0.83/l : Aldi € 1.73/l : Common

In real life it looks something like this:

[singlepic id=353 w=640 h=480 mode= float=]

What’s on the bottles?

Having so much different sorts of beer made me wonder; what do these bottles tell me? Why should I drink them? What makes them so great? Well… the answer was to analyze all of them.

All bottles have at least two labels; formatted according to some standards. The foreign beers do not follow these standards; therefore we do not know what is inside Jupiler and oettinger (ingredient X?). Here are some other things:

  • The most popular colors on beer-bottles are gold, white, black, green and red. Not used are pink and purple,
  • One of the brands looks very new, but has a “since 18XX” label on it. I have no idea what it means,
  • Oranjeboom is a fantasy brand: it is brewed FOR oranjeboom and not BY oranjeboom,
  • Other fantasy brands are: Albert Heijn, Euroshopper, SchultenBrau,
  • Gulpener is the only one with a full-color label and a Environment Label (milieukeur),
  • Palm recommends not to drink beer when you are pregnant,
  • Grolsch and Heineken (2 popular brands in NL, rivaled in the east) are priced exactly the same,
  • The 10% more stamp on Grolsch bottles doesn’t mean a thing; Grolsch always has 33CL bottles,
  • Jupiler is the most expensive, Euroshopper the cheapest

Preparing the bottles

Since this is a blind beer test, we had to think of a way to mask what is served to the testers. All beers come in the same type of bottle. The only deviants are Grolsch, Jupiler and oettinger; so had to be replaced during the test. Anonymization is simple: hold a bottle in water for 5 minutes and the label comes off. After this just tape the bottles and give em a meaningless number: 10, 15, 20, 22, 25, 30, 33, 35, 40, 44, 50, 55, 60, 65, 70, 75, 77, 80, 85, 88, 90.

[nggallery id=12]

The test

We got about 12 volunteers that where eager to risk their lives. In return they will find out what beer they like best. The results differ a lot from person to person. After the mass-slaughter of beer took place, this is what was left:

[nggallery id=13]

Guesswork on the results

In the world of beer, brand-value and product-experience are the most powerful cards the cheap-brewers have. According to the testers, we could easily replace some of the labels and still not notice the difference. They’re not experienced drinkers, but maybe it says something about the brand value compared to taste.

Therefore the next table; lets see if the testers could figure out what brand they where drinking. (bold = correct guess).

Comparing actual brands to brand guesses
Actual brand Guessed brand
Dommelsch Jupiler, Albert Heijn, Alfa, Lindeboom
Jupiler Hertog Jan, Euroshopper
Alfa Palm, Warsteiner, Oranjeboom, Schultenbrau, Alfa
Palm Palm (2x), Brand
Bavaria 0.0 Jupiler, Palm, Bavaria 0.0
Amstel Light Warsteiner, Amstel Light
Heineken Schultenbrau, Palm
Amstel Bavaria, Heineken, Gulpener, Pitt, Amstel 0.0
Bavaria Gulpener, Albert Heijn, Heineken, Bavaria, Brand, Gulpener
Hertog Jan
Grolsch Grolsch, Groslch, Grolsch, Grolsch, Alfa, Schultenbrau
Schultenbrau Jupiler, Dommelsch, Oranjeboom, Bavaria
Albert Heijn Oettinger, Gulpener, Bavaria
Euroshopper Bavaria, Oettinger, Bavaria
Warsteiner Brand, Bavaria
Pitt Oettinger, Amstel Light
Gulpener Euroshopper, Amstel, Lindeboom, Amstel, Oranjeboom, Pitt
Lindeboom Gulpener, Amstel Light, Amstel, Grolsch, Euroshopper, Oranjeboom
Oranjeboom Dommelsch (2x), Bavaria, Euroshopper, Oranjeboom
Oettinger Hertog Jan, Amstel, Brand, Schultenbrau

Did they know what they were drinking? No, only 13 correct guesses out of 73 attempts. With 21 different brands that is pretty nice but there is no significance in all these marks, except for the discovery of Grolsch; which affected its ratings. Apparently some testers found the name and taste of Grolsch great. But the previous table doesn’t help us in getting the value of a brand.

The following table shows the grades given to each brand; whereas most guesses are wrong! Notice the more popular brands are higher rated. The bottom of the list comprises of cheaper brands.

Ratings of brand, depending on unknown bottle content
# Guess Values given to brand High Low Votes Average
1 Hertog Jan 9 – 7 9 7 2 8
2 Grolsch 8 – 8 – 8 – 7.5 – 5 8 5 5 7.3
3 Jupiler 6.1 – 7 – 7.5 7.5 6.1 3 6.87
4 Palm 7 – 8 – 8 – 5 – 6 8 5 5 6.8
5 Warsteiner 6.5 – 7 – 6.5 7 6.5 3 6.67
6 Oettinger 6.7 – 5.5 – 7 6.7 5.5 3 6.4
7 Albert Heijn 6 – 6.5 6.5 6 2 6.25
8 Dommelsch 6.5 – 7 – 5 7 5 3 6.17
9 Gulpener 6 – 8 – 3 – 6 – 7.5 8 3 5 6.1
10 Brand 6 – 5 – 8- 5 8 5 4 6
11 Heineken 6 – 6 6 2 6
12 Bavaria 6 – 5 – 5 – 3 – 6 – 2 – 6.2 – 6 6 2 8 4.9
13 Amstel 5 – 3 – 5 – 6 6 3 4 4.75
14 Schultenbrau 4 – 6 – 3 – 5 6 3 4 4.5
15 Alfa 3 – 4 – 6.5 6.5 3 3 4.5
16 Euroshopper 4 – 5.5 – 3 – 5 5.5 3 4 4.36
17 Oranjeboom 5 – 6 – 2.6 – 2 – 5 6 2 5 4.12
18 Amstel Light 2 – 3 – 5.5 5.5 2 3 3.5
19 Lindeboom 4 – 3 4 3 2 3.5
20 Pitt 1 – 3 3 1 2 2
21 Bavaria 0.0 2 2 1 2

73 votes Average: 5.27

Results

Having seen how the beers are rated (when guessing wrong), its time to see if these values are indeed reflecting the real rating. Now we see unappreciated beers rising to the top: Oranjeboom(11 places), Pitt (10 places), Schultenbrau (7 places). In contrast, higher rated brand to that moved down: Gulpener(12 places), Brand (8 places), Oetinger (10 places).

Test results
# Brand Avg Rating Votes Price / Liter Avg Estimate Deviation
1 Palm 7.00 5 1.90 6.8 + 2.85%
2 Warsteiner 6.73 3 1.73 6.67 + 0.90
3 Hertog Jan 6.50 2 1.83 8 – 18.75
4 Grolsch 6.50 8 1.57 7.3 – 10.96
5 Jupiler 6.30 4 2.04 6.87 – 8.30
6 Oranjeboom 6.13 8 1.30 4.2 + 45.95
7 Schultenbrau 5.83 6 0.83 4.5 + 29.56
8 Bavaria 5.72 9 1.30 4.9 + 16.73
9 Heineken 5.63 4 1.57 6 – 6.17
10 Pitt 5.40 8 0.67 2 + 170
11 Alfa 5.36 7 1.80 4.5 + 19.11
12 Albert Heijn 4.95 6 0.93 6.2 – 20.16
13 Euroshopper 4.94 9 0.63 4.36 + 13.30
14 Dommelsch 4.90 9 1.53 6.17 – 20.85
15 Lindeboom 4.72 9 2.00 3.5 + 34.86
16 oettinger 4.67 6 0.81 6.4 – 27.03
17 Amstel 4.29 7 1.53 4.75 – 9.86
18 Brand 4.17 6 1.70 6 – 30.50
19 Bavaria 0.0 4.00 10 1.20 2 + 100
20 Amstel Light 4.00 5 1.73 3.5 + 14.29
21 Gulpener 3.71 10 1.83 6.1 – 39.18
TOTAL
143 votes 5.27

Conclusion

To decide a winner from these meager test-results is hard. With not enough votes to cast a decision on Warsteiner, Hertog-Jan and Jupiler, they have been removed from the list of winners. The next beer-test should address this issue to give a more fair and balanced judgment. I’ve also got my doubts on Grolsch; if it wasn’t discovered or mistaken it might have another rating. Winners and Losers are listed when having 5 votes; supporting the number one.

Winners/losers

Winners:
Palm logo grolsch logo Oranjeboom logo Schultenbrau

bavaria logo

1 4 6 7 8
Losers:
gulpener logo

AMSTEL

LIGHT

BAVARIA

0.0%

Brand logo Amstel logo
21 20 19 18 17
Underestimated / overestimated

Top 5 underestimated
Pitt logo

BAVARIA

0.0%

Oranjeboom logo Lindeboom logo Schultenbrau
1 2 3 4 5
Top 5 overestimated
gulpener logo

Brand logo

Oettinger logo dommelsch logo Albert heijn logo
1 2 3 4 5

Evaluation

Though the fallacies are obvious, this test was a GREAT success. Firstly because there where participants, but also since it was fun to do this. The results might influence some peoples choices when they shop for beer next time.

The total number of participants was 12 and the total number of votes cast 132. (11 votes per victim). That some beers have been voted less is due to the number of beers to test. After ten beers / beer-samples most people could not distinguish what they where drinking.

With an highest average rating of 7.00 it is fair to say; nobody really likes these beers. There are some exceptions here and there, but there is not one beer that actually gets an eight. This I would like to see changed in a next beer test. Then we’ll drink only the best premium imported beer from Belgium and America. If you wonder why America? I’ve got this tip from a member of PINT; if you want to get the best beer; get american, they experiment a lot!

One of the girls in the test liked Alcohol free beer; she rated it highest! Maybe this beer should just be called Girl-Beer with a pink label and a smiling unicorn. The alcohol free beer and light beer where not appreciated by the rest; mostly men. The light beer states that it tastes the same, liars.

Improvements for a next test

  1. Better beers; great beers! Beers that is so good that you are proud becoming an alcoholic.
  2. Ideal serving temperature: we served all slightly too warm. This was a mistake; cold beer is appreciated.
  3. No more than 10 samples per person; otherwise they’ll explode.
  4. More people, and let them rate every sample.
  5. Not everybody likes to drink their sample; get a bucket to dump the remains (beer fans: ENRAGE!)
  6. With 12 people, you can do with 2 bottles of each brand and still have too much.
  7. Pre-factored Excel sheet; doing things by hand increases the error margin.

Extra: Where does beer come from?

Bavaria Bavaria, Bavaria 0.0, Euroshopper, PITT, Schultenbrau
Heineken Amstel, Amstel Light, Heineken
Inbev Dommelsch, Hertog Jan, Jupiler, Oranjeboom
Independant Alfa (meens), Brand, Grolsch, Gulpener, Lindeboom, Oettinger, Palm, Warsteiner
Unknown Albert Heijn

Extra: Stories

Beer bottle labels have beautiful stories on them, trying to add to the value of the brand. The following list can be used to create your own label. (warning: moonspeak)

Sprankelend, Zuiver, Smaak, Heerlijk, Bouquet, Garantie, 100%, Gecontroleerd, Bron, Bronwater, Vlot, Verstandig, Specifiek, Malsheid, Fruitig, Hop-toets, Allerbeste, Optimaal, Bijzonder, Geniet, Fris, Echt, Helder, Wereldwijd, Zacht, Familietraditie, Traditionele, Natuurlijke, Premium, Geselecteerde, Zuiver, Smaak, Ambachtelijk, Kristalhelder, Eeuwenoud, Topklasse, Uniek, Buitengewoon, Hoogwaardige, Kwaliteit, Onnavolgbaar, Geraffineerd, Liefde, Uitsluitend, Strogeel, Zomerkleur, Stabiel, Fijne, Glanzende, Ochtendfrisse, Rond Rijpend, Levendig, Flueelzacht, Natuurzuiver, Smaaktonen, Subtiel, Zeldzaam, Beproefd, Pittig, Bijzonder, Origineel, Reinheitsgebot, Hart

Extra: Downloads and media