Thursday, April 16, 2015

Biased Opinion - No, You Probably Can't Separate a Work from Its Author

I'm going to start by discussing beer for a bit. Several years ago economist Dan Ariely conducted an experiment about preferences that he described in his book Predictably Irrational (read review). While a professor at M.I.T., he presented patrons at a bar near the university with a taste test of two beers. One beer was identified as a Budweiser, and the other was proffered as "MIT Brew". The testers were offered a sample of each, and then asked which one they would like a glass of. A majority of taste testers preferred the "MIT Brew" and accepted full glasses of it to enjoy.

Later, Ariely conducted a second set of taste tests. This time he told the prospective tasters that the two beers were a Budweiser and a Budweiser that had a few drops of balsamic vinegar added. After the testers had sampled each of the offerings, they were also asked which one they would like a glass of. Predictably, almost everyone who participated disliked the beer laced with balsamic vinegar and preferred the unadulterated beer.

The twist that should surprise no one is that the "MIT Brew" was just a Budweiser with a few drops of balsamic vinegar added. But why the difference? Why is it that when tasting the beer blind, a majority of people preferred the balsamic vinegar beer, but when they knew what was in it before they took a taste, the testers almost all hated it? The answer is simply that they had more information in the second situation, and they developed expectations as to what the vinegar-beer combination would taste like, prepping them to find it distasteful. The key understanding here is that neither preference is more accurate or more "true" than the other. The people who tasted the beer-vinegar combination blind who said they liked it more actually liked it more. The almost universal distaste people expressed for the beer-vinegar combination when they were informed what it was ahead of time was their actual preference. The salient point is that your tastes and preferences are not fixed and immutable. They change based upon the information you have.

Many people hold to the idea that there are "true" preferences that people have, and if we could just get rid of people's preconceptions, we could determine what people really like. But this is a fundamental misunderstanding of how human preferences work - a misunderstanding that can have substantial real world consequences. Consider the example of "new Coke". In 1975, Pepsi began running a blind taste test they called the "Pepsi challenge" in which consumers were asked to taste a sample of Coke and a sample of Pepsi in a blind test. Under these conditions, consumers favored the taste of Pepsi1 Needless to say, Pepsi aggressively marketed the results of these taste tests, despite contradictory results from taste tests run by Coke that had been not been conducted as blind trials. The pattern that was established was clear: Pepsi would outperform Coke in blind taste tests, and Coke would outperform Pepsi in open taste tests.

In the early 1980s, apparently reacting to Pepsi's dominance in blind taste tests and their own shrinking market share, Coke decided to create a new formula for Coke to replace their old one. They secretly conducted extensive taste tests, comparing their new formula against both Pespi and the existing Coke formula then on the market. The "new Coke" formula consistently beat its competition, scoring well above either Pepsi or the traditional Coke formula. Coke set about unveiling their new flavor, putting out several advertisements showing how the new formula had fared in blind taste tests. The company figured that having conducted blind taste tests they had ferreted out the "true" preferences of consumers, and having this "objectively" better flavor in the cans bearing their logo would prove to be an unbeatable combination.

It wasn't. It was a marketing disaster.

But why? After all, "new Coke" had beaten both of its rivals in blind taste tests, and by wide margins. The testers had consistently stated that they preferred the new formula. But when the formula was released into the market, it was roundly reviled by large numbers of consumers. How did this happen? It happened because Coke made a fundamental error: They assumed that blind taste tests revealed the "true" preferences of consumers. But what blind taste tests reveal is what people prefer in blind taste tests. The trouble is, people don't make purchasing decisions blind, and they usually don't consume beverages (or anything else) blind. Your preferences are not invariable, but are rather highly flexible based upon the contextual information around you. People who bought Coke preferred Coke in part because they knew they were buying Coke. The testers were not lying when they said they preferred the taste of "new Coke" in blind taste tests, they actually preferred it at that time. Those same people were also not lying when they said they preferred "classic Coke" when they were tasting it out of a labeled can, they actually preferred it at that time. Information changes your preferences.

A further point is that information changes your preferences even when it isn't directly related to the thing you are evaluating. It seems relatively obvious that the logo on a can could inspire brand loyalty which would make people change their preference as a result. On the other hand, evidence shows that preferences are affected by all kinds of extraneous factors. Going back to another Dan Ariely experiment: He handed out free cups of coffee to college students on the condition that they filled out surveys rating the coffee. The students were directed to a condiments bar that included odd ingredients such a paprika, cloves, and cardamom to dress up their coffee. On some days the condiments were set out in Styrofoam cups labeled with red marker. On other days they were set out in fancy metal and glass containers with nice metal spoons. No matter how they were presented, no one used the unusual condiments, but on days in which the condiments were set out in fancy containers the students rated the taste of the coffee much higher than on days when the condiments were set out in Styrofoam cups. The point to remember is that your preferences are affected by things that you probably don't even realize are related to the thing you are rating.

Given this evidence, one has to consider the frequent exhortation delivered to readers that they should try to read books and stories without considering the character of the authors who wrote them. "Separate the book from the author" goes the oft repeated mantra. But if elements as subtle as the types of containers that condiments are contained in can affect how much you enjoy coffee, is there any realistic possibility that a reader can avoid being influenced by his knowledge concerning the author let alone any number of other factors? Knowledge about something affects how you perceive that thing, and it does it in ways that you are not even conscious of. You can try to react in an "unbiased" way, but just as one could not change their reaction once they found out that there was vinegar in the beer, you cannot change how knowing than an author is a racist, a sexist, or a homophobe affects your perception of their work. Not only that, you cannot consciously evaluate how much that knowledge affects your perception, so there is no practical way to counter this effect.

So when an author who has spent much of his time alienating a particular set of award voters insists that the only fair thing to do would be to read the works he has promoted with an open mind and evaluate them on their own merits, he is essentially asking for the impossible. Not because the readers would refuse to try to do such a thing, but because they will inevitably be affected by their knowledge concerning that author's actions. This means that even though I read, for example, The Exchange Officers before I voted for the Hugo awards in 2014 and evaluated it honestly, my knowledge concerning Larry Correia's antics colored my perception of the story - and it did so despite the fact that Correia didn't write the story. I had information concerning how the story got on to the ballot, which necessarily affected how much I liked the story on a level that I was not, and am not conscious of. How much? I don't know. Would I have liked the story a lot more if I had read it without knowing what I know? Possibly. But I can't know. No one can. Once you have additional information it colors your preferences irrevocably.

You might be thinking to yourself that you are a fair-minded person and can overcome your biases. The trouble with that thought is that your preferences have been changed without you knowing it. You cannot overcome your biases, because you don't know they are there. The taste tester cannot consciously choose to prefer Pepsi when they are blind testing a soda, and then consciously choose to prefer Coke when they are buying it off the shelf. Your preferences exist separate from your conscious choice. Information changes how much you like beer, coffee, or cola. Information also changes how much you like books and stories. You cannot evaluate a book without being affected by what you know about its author.

1 There are some methodological problems with this style of test other than the blind nature. Some researchers contend that in a "sip test", tasters will tend to favor the sweeter drink over its competition, even if they would not prefer the sweeter drink over the course of an entire can or bottle. This would explain why the sweeter Pepsi consistently outperformed Coke in the blind "Pepsi Challenge" taste tests, but does not provide any explanation for why Coke tends to outperform Pepsi when tests are conducted "in the open".

Biased Opinions     Home


  1. Interesting analogy, but it really does work. As a reviewer, I find I tend to get to know authors a lot better than I ever did as just a reader, and that's a double-edged sword. It's introduced me to some fantastic reads that I might not have otherwise picked up, but which I *knew* I'd enjoy because we shared so many interests/passions. At the same time, there are books I was originally excited about reading that I've since pushed aside because of what I've come to know about the authors. This whole sad puppy mess has knocked at least 2 titles off my shelves . . . but it's also made me curious enough to add a few other authors to my TBR list.

    As for Coke vs Pepsi, blind taste test or not, Coke is the only right answer. :)

    1. @Bob R. Milne: I think it fits very well (obviously, or I wouldn't have written it). I think behavioral economics has a lot to tell us about how people react to information, and demonstrates that people are not even close to being rational actors. I would like to see a study done about book rating to determine if my evaluation here is accurate. It would be a difficult undertaking - getting people to read an entire book and then rate it while making sure they either don't know anything about the author, or do know about the author would be challenging. Maybe if I ever get back to graduate school in economics I'll have to use this as a research project.

  2. Interesting post. I work for a university press, so I get to interact with authors a lot. Learning that an author is a jerk does change my opinion of that author’s work. When I first read the work, I’m really excited to publish it, but then the excitement dies when I have to deal with the author’s bad attitude.

    Aj @ Read All The Things!

    1. @AJ Sterkel: That is an interesting, and different issue - how does learning information after you have first experienced something change your preferences regarding it? In the beer example, Ariely tried a third iteration in which he told people what was in the "MIT Brew" after they had tried it to see if it would retroactively alter their preference. It didn't.

      But we see people using this sort of tactic all the time to try to get people to dislike something they previously liked. A couple of years ago, there was a big to-do about "pink slime", a very finely ground form of beef made from trimmings otherwise not usable. Those opposed to its use made sure to show the concoction in the media, presumably hoping that it would look gross to enough people that they would demand its retraction from the market. Whether such sorts of efforts to create retroactive distaste for something are effective has not been demonstrated as far as I know.

  3. I do try, as futile as it might be, to divorce author from work. What does Card's personal views have to do with a book where those views don't seem to influence the text?

    And yet...I feel queasy about reading him again.

    1. @Paul Weimer: I'm sure you do the very best you can to be fair. The trouble is that now that you know Card's views, that information colors your perspective in ways that you don't even consciously know. We are the sum of our experiences, and we can't not know the things we know, and that knowledge colors all of our preferences no matter how hard we try to avoid it doing so.