Tuesday, August 25, 2015

Biased Opinion - 2015 Hugo Awards Post Mortem

What Happened at the Hugo Awards?

It was a rout.

Strike that. It was a complete and total rout.

At the 2015 Hugo Awards, with 5,950 ballots cast, the fans of science fiction rallied to decisively repudiate the slate-tactics of the interrelated Sad and Rabid Puppy groups, relegating every single non-Dramatic Presentation Hugo nominee from the slates to a dark corner behind "No Award". It was a comprehensive rejection of the idea that as long as a group can work together as a bloc to pack mediocre to miserable nominees onto the Hugo ballot, they deserve to get a trophy as a result.

After using bloc voting tactics to get their nominees on the ballot, the Puppies suffered an almost complete and total loss in the actual voting, in large part because the nominees they put on the ballot were, taken as a group, so very weak. This is something of a missed opportunity for the Pups, as their core narrative was originally either that excellent works by politically conservative authors were being kept off the Hugo ballot by a secret cabal of insiders (for which they provided zero evidence) who were bent on rewarding the "right" kind of books by the "right" kind of authors, or that science fiction had strayed too far from its roots and that rollicking adventure stories with rocket ships and space marines needed to be recognized by the awards again.

But the stories that the Pups placed on the Hugo ballot were mostly neither excellent, nor rollicking adventures. The stories nominated by the Puppy slates ranged from the merely mediocre, such as Kary English's Totaled or Jim Butcher's Skin Game, down to stories that were so awful that they made one wonder how they actually got published. The only real point the Pups made with their nominations is that if their nominees represented the best that conservative science fiction can offer, then they haven't been overlooked by the Hugo voters: They have been accurately judged and found lacking.

Further, very few of the stories nominated amount to rollicking adventures. Butcher's Dresden Files novel fits the description more or less, and maybe one or two of the short fiction, such as Steve Diamond's A Single Samurai, could possibly be described that way, but for the most part the slate-nominated stories simply were not. Anderson's The Dark Between the Stars was plodding and tedious, while stories such as Rinehart's Ashes to Ashes, Dust to Dust, Earth to Aluvium and Antonelli's On a Spiritual Plain were simply slow and boring. The slate of nominees the Pups offered to Hugo voters consisted of stories that were mediocre to bad and often dull and uninspiring.

This isn't new for the Puppies. In 2014, when Larry Correia ran the "Sad Puppies 2" slate, he managed to get a handful of works on the 2014 Hugo ballot, and they too ranged from mediocre to terrible. And the Hugo voters read those works and ranked them accordingly. The Puppy slate makers have had two bites at the apple so far1, and both times the Puppies have pushed works that were simply not particularly good. The Pups have grabbed two chances to impress the Hugo voters with their selections, and each time they chose to nominate a collection of crap, burning away any goodwill they may have had with the voters. Based upon their established track record, I suspect that in the future, any Puppy-touted nominees will be justifiably regarded with skepticism at best, and quite likely outright derision.

As usual for WorldCon, the Hugo Administrators released the voting data almost immediately after the award ceremony was completed, and the data shows that many of the Puppy narratives that have been bandied about the last several months simply don't hold up to scrutiny.

How Big Was the Rout?

Complete and total. If this had been a little league baseball game, the mercy rule would have been invoked. Not only did no Puppy nominee other than Guardians of the Galaxy win, none of the others even came close.

A brief summary of the rules for Hugo voting: To determine who the winner in each category is, the Hugo Administrators use what is called "Australian Instant Runoff Voting". When each voter submits their ballot, they rank the nominees in preferential order, putting their top choice first, their second choice next, and so on. Under the Hugo rules, voting for "No Award" is always an option, and can be ranked just like any other nominee. After the votes are cast, the first place finishers are tallied. If one nominee gets a majority of votes, they win. Because there are usually five nominees in each category (plus the aforementioned option of No Award), this rarely happens. If no one wins a majority, the nominee who received the lowest number of votes is eliminated and their ballots are redistributed among the remaining nominees using the second choice preferences from those voters. This process is repeated until one nominee has a majority of votes. This process usually requires three, four, or even five "passes" to determine a winner - for example in this year's Hugo Awards, Best Novel and Best Long Form Dramatic Presentation required five "passes" to declare a first place finisher, and Best Fan Artist required three. The Hugo administrators then set aside all of the first place votes for the top finisher and rerun the process to determine the second, third, fourth, and fifth place finishers.

There were five categories in which the only nominees were drawn from the Sad or Rabid Puppy slates. The significance of the fact that "No Award" won on the first pass should be readily apparent. And in most of these Puppy-exclusive categories, the outcome wasn't even close.

No Award VotesOther VotesNo Award Percentage
Best Novella
Best Short Story
Best Related Work
Best Editor, Short Form
Best Editor, Long Form

The closest the Puppy nominees, as a group, came to equaling the total number of No Award votes was in the Best Long Form Editor category, where a switch of 44 votes from No Award to the other candidates would have prevented No Award from taking a first round majority. Looking at the remaining passes through the category, it seems unlikely that even if 44 voters had placed Weisskopf first on their ballot rather than voting for No Award, she probably still would not have ended up beating No Award for the win. Such a switch probably would have resulted in a number of passes being required, but it seems unlikely that it would have changed the ultimate outcome.

As decisive as these figures are, they don't tell the whole story. In any category other than the two Dramatic Presentation in which there were non-Puppy and Puppy nominees, all of the non-Puppies finished above No Award, while all of the Puppy picks finished below it. Not only that, once the placement of the non-Puppy picks was made, it only took one pass for No Award to win over the remaining nominees on the ballot. For example, in the race for position four in the Best Novel category, after The Three-Body Problem, The Goblin Emperor, and Ancillary Sword had placed first, second, and third, the vote count looked like this:

No Award
Skin Game
The Dark Between the Stars

In the best fancast category, after Galactic Suburbia and Tea and Jeopardy had claimed the one and two spots, the race for third place looked like this:

No Award
The Sci-Phi Show
Adventures in SciFi Publishing
Dungeon Crawlers Radio

This pattern is repeated throughout the Hugo results. With the exception of the Short and Long Form Dramatic Presentation categories, No Award won on the first pass any time there were only Puppy nominated picks left on the ballot. In short, the voters comprehensively rejected the Puppy nominees.

Did the Non-Puppy Voters Vote as a Bloc?

In a word, no.

There was certainly substantial consensus among non-Puppy voters that the Puppy nominees were undeserving of awards, and a fair amount of consensus that most of the Puppy nominees were undeserving of their places on the ballot, but imagining this to be a bloc ignores two salient facts (1) consensus is not a bloc, and (2) there was noticeable variability in how non-Puppy voters responded to the slate-driven Puppy nominees.

As I pointed out earlier, for the most part the Puppy nominated works were simply terrible. In a handful of cases, the Puppy slate seems to have almost accidentally pushed something that rose to the level of mediocrity on the ballot. The interesting thing is that in those cases, the voters reacted accordingly. By way of example, both the Best Novella and Best Related Work categories, which were populated entirely by Puppy picks, were widely seen as being comprised of nominees that are fairly bad pieces of work. In those categories the voters picked No Award as the first place choice on their ballot 66.5% of the time in Best Related Work and 65.5% of the time in Best Novella. The overall condemnation of the quality of the nominated works in those categories was resoundingly clear.

By contrast, the Best Short Story, and the two Best Editor categories were seen as having a few nominees who could be regarded as "not so terrible". Kary English' story Totaled was seen as being at least competently written, while Mike Resnick in the Short Form Editor category, and Toni Weisskopf and Sheila Gilbert in the Long Form editor categories were viewed as being at least somewhat credible choices. And this is reflected in the voting. In the Best Short Story category, No Award only placed first on 58% of the ballots. That's a convincing win, but it isn't as huge as it was in Related Work and Novella. In the Best Short Form Editor category, No Award only won with 55.1% of the vote, and in the Best Editor Long Form category, No Award only won on the first pass with 50.9% of the vote. When the Puppies nominated better works, the voters responded by supporting them more. The slate-produced works didn't get enough support to overcome No Award, but it is clear that many of the non-Puppy Hugo voters were making an assessment based upon the quality of the nominees and voting accordingly.

One can also note that a fair number of voters who were probably not Puppy supporters voted for specific Puppy picks, so long as those picks were of at least middling quality. I estimate that the combined strength of the two Puppy blocs represented at most somewhere between 900 to 1,000 voters, but in many categories the Puppy nominees got more votes than that in the first pass, even if they didn't ultimately end up finishing ahead of No Award. And the votes were not evenly distributed among the nominees. Once one edits out the putative Puppy votes, it becomes clear that those nominees generally regarded as being at least passable had more votes than their competition. In Best Short Story, for example, Theodore Beale instructed his slavish followers to vote for Steve Rzasa's Turncoat (published, coincidentally by Beale's own press Castalia House), and it got 525 votes in the first round, but Kary English's Totaled garnered 874 votes, revealing that a fair number of non-Puppy voters found her story to be the best of a fairly weak field. In Best Short Form editor, Beale gave marching orders to his minions that they vote for him for the trophy, and wound up with 586 first place votes, but industry veteran (and multiple Hugo winner) Mike Resnick earned what support could be had from non-Puppy voters, and wound up with 873 first place votes. In Long Form editor, Toni Weisskopf got 1216 first place votes, and Sheila Gilbert got 754. In short, some non-Puppy voters decided to vote for those nominees who were of higher quality than others. In many cases it was picking the best of a bad bunch, but this is clear evidence that many of the non-Puppy Hugo voters were working with what they had and trying to make a decision based upon perceived quality.

Further, after No Award had won, many of the remaining nominees picked up votes. While every voter does not list nominees after they voted No Award2, some do, preferring to be on the record as to what should be ranked afterwards. In every category in which there were only Puppy nominees, between 314 and 487 voters expressed additional preferences, with the bulk of those preferences being in favor of the works that were generally acknowledged to be the cream of the sour milk that was the Puppy nominees. Totaled picked up 365 out of 487 post-No Award votes. Flow picked up 360 out of 471 post-No Award votes. Of the 314 post-No Award votes in the best Long Form editor category, 134 went to Sheila Gilbert, and 92 went to Toni Weisskopf. It seems relatively clear that even among voters who made No Award their first choice, there was willingness to acknowledge that some of the Puppy nominated works were better than others.

Conversely, works that were generally regarded as the worst on the ballot received a minimal boost from post-No Award rankings. The Parliament of Beasts and Birds only got 12 of 487 post-No Award votes. John C. Wright's three novellas only received a combined total of 53 out of 471 post-No award votes. Wisdom from My Internet, widely regarded as quite possibly the worst Hugo nominee in history, only got 23 of 327 post-No Award votes. Theodore Beale, nominated in both editor categories and whose editing skills have been widely derided, only got a 28 vote post-No Award bump in the Short Form race, and 38 post-No award votes for Long Form. While many of those who chose No Award as their first choice seem to have been willing to reward quality works, very few seem to have been inclined to throw their votes behind the very dregs of the Puppy slates.

Some Oddities That May Be of Interest Only to Me

The big story of the 2015 Hugo Awards was, of course, the domination of No Award over the nominees from the two Puppy slates. Hidden inside the statistics there are, however, a couple of little details that I found interesting.

The first crops up in the Best Fanzine category, where the website Black Gate was pushed onto the ballot by the Rabid Puppy slate. After much deliberation, Black Gate elected to withdraw from the Hugo ballot (much to Theodore Beale's apparent consternation), but did so after the deadline by which they could be taken off of the official ballots and replaced. Therefore, the Hugo ballot went to the voters with Black Gate still on the ballot, but with a request from the editors of the site not to vote for them. Even so, Black Gate got 489 first place votes. Journey Planet won the category, and No Award came in second, but Black Gate ended up in third place, taking the top position among Puppy nominees in the category. Of the people who voted for Journey Planet to take the top spot, 95 of them put Black Gate as their second choice, and of the people who voted for No Award as their first or second choice, 134 picked Black Gate third. I don't know where these votes came from - it is unclear if these votes are Sad Puppy voters simply choosing Black Gate as the best of the choices from the slates, Rabid Puppy voters choosing Black Gate because it was the only solely Rabid Puppy pick to make the ballot in this category, or non-Puppy voters voting for Black Gate either as an assessment of quality or as kudos for their principled attempt to withdraw their slate-garnered nomination, or likely some combination of all of these sources - but what is clear is that Black Gate outperformed every other Puppy nominee in the category. Not only that, it wasn't really a close race: In the competition between Black Gate and Tangent Online, the nominee with next highest vote total in the race for third place, Black Gate finished with 737 votes to Tangent's 415.

In the Long Form Editor category, Beale instructed his minions to vote for Toni Weisskopf first, and placed himself further down his instructional list. Despite this, 166 voters placed Beale first on their ballots, putting him ahead of Jim Minz, who only got 58 first place nods. The really interesting action in this category took place after No Award won and the race was on for second place, which took three passes and let us know exactly who the people who ranked Jim Minz and Theodore Beale first (or second behind No Award) on their ballots voted. The race for second place looked like this:

Pass 1Pass 2Pass 3
Toni Weisskopf
Sheila Gilbert
Anne Sowards
Theodore Beale
Jim Minz

After Minz was eliminated, 26 of his votes went to Weisskopf, 18 went to Gilbert, 9 went to Sowards, 3 went to Beale, and in something of a surprising twist, 14 went to no preference or No Award. Given that Minz and Weisskopf are both editors for Baen, and would generally be expected to draw from the same pool of supporters, the somewhat middling loyalty shown by his voters towards Weisskopf seems unusual. What seems really odd is the number of apparent Minz fans who weren't even willing to cast a secondary vote in anyone's favor, and instead preferred no one get their vote if he did not.

The real interesting action comes from Beale's supporters. When he was eliminated from contention, he had 207 votes. Out of those supporters, 85 went to Weisskopf as an alternate choice, which makes sense given Beale's public support for her nomination. Only 20 chose to throw their support to Gilbert, while a paltry 7 went to Sowards. But the story here is the fact that 95 of Beale's supporters elected to list no one after him at this point. This number is not enough to have changed the outcome of the race but it means that almost half of Beale's supporters chose to leave their ballot blank rather than vote for Weisskopf, Gilbert, or Sowards. I'm not going to say that there was definitely misogyny at work here, but the implication is pretty obvious.

How Many Puppies Were There?

There are theoretically two groups of Puppies. The first group is the Sad Puppies, led by Brad Torgersen, and a number of other mostly conservative, mostly mediocre authors. The second group is the Rabid Puppies, led by Theodore Beale, a loathsome bigoted individual with delusions of grandeur. The two groups overlap by quite a bit, both in terms of leadership and, as far as one can tell, membership. Noted homophobe John C. Wright, for example, was touted by both group's slates, and many of the voters who voted for nominees from one slate appear to have voted for nominees from the other as well. Even so, estimating the relative strength of the two groups is worthwhile, as their long-term responses to the results of Saturday night's awards may vary to a certain degree.3

Rabid Puppies

In most of the categories there was such substantial overlap between Sad and Rabid Puppy nominees that it is difficult to separate out how many from each group voted for each nominee. One thing that makes estimating Rabid Puppy strength somewhat easier is that Beale gave his followers marching orders, telling them which nominee he was voting for, and telling them they should vote for those nominees as well. Notably he stated that the adherents to his cult should vote for One Bright Star to Guide Them in the Best Novella Category, Turncoat in the Best Short Story category, The Hot Equations in the Best Related Work category, and himself in the Best Editor Short Form Category.

The most straightforward markers are the Best Editor categories where Beale put himself forward as a nominee, but the Sad Puppy slate-makers did not. In the Long Form category, Beale directed his minions to vote for Toni Weisskopf, while in the Short Form category he touted himself as the person to vote for. Given that Beale is generally a loathsome individual, and most of the Sad Puppy voters probably voted for other nominees, his vote total in the first pass through the balloting can be taken as a reasonable proxy for the upper limit of Rabid Puppy voters. That number is 586. I suspect that some small number of Sad Puppy voters ranked Beale first on their ballot, and it is possible that some voters unaware of the controversy and unfamiliar with Beale may have ranked him first, but those numbers are likely to be small.

With 586 as an upper bound, we can turn to the other categories and see how many of Beale's supporters followed his lead. In Best Novella, One Bright Star to Guide Them had 556 first place votes, in Best Short Story, Turncoat had 525 first place votes, and in Best Related Work The Hot Equations got 595 first place votes. Given that The Hot Equations was regarded as one of the better works in a miserable field, it seems plausible that a certain portion of its initial support came from non-Rabid Puppy sources, we can discount the 595 number as being an overly large estimate for the Rabid Puppies. The remaining range is between 525 and 586, and while I doubt that every one of those votes came from a Rabid Puppy, it is likely that the majority did, and there's no really good way to figure out what sliver of support came from some other voting population.

So there is a core group of Rabid Puppies that possibly comes in at somewhere between 525 and 586, and probably slightly lower than that who will vote Beale's way. Given the fact that Beale phrased the entire Rabid Puppy slate as nothing more than a culture war, and has stated that his objective was to "leave the Hugo awards a smoking ruin", one can expect that this cadre of voters could mostly stay intact for the next year. On the other hand, they may get distracted by some other culture war target and scamper off to another fruitless frontal assault on reality. For those who stay, running into the stiff-arm of fandom time and again may cause them to become discouraged at ponying up supporting membership fees every year in order to suffer repeated losses, but it may take some time. Although it is clear that they know they have no hope of ever actually pushing anyone to win a Hugo Award, even the most dedicated follower will find a Quixotic campaign in search of a Pyrrhic victory to be difficult to sustain.

Sad Puppies

Accurately assessing how many Sad Puppy voters there were is slightly more difficult. Since the Sad Puppy campaign didn't have quite as polarizing a figure as Beale nominated who wasn't also nominated by the Rabid Puppy slate, figuring out how many Sad Puppy supporters there are is more complicated. Dave Freer as Fan Writer and Steve Diamond's story A Single Samurai are among the few Sad Puppy nominees that made it onto the ballot without also being Rabid Puppy nominees. In the first pass-through in the vote counting in their respective categories, Freer garnered 251 votes, while A Single Samurai received 386. This gives a rough estimate of Sad Puppy numbers, although it is not completely precise as one might expect some number of Sad Puppies to have voted for the other nominees in those categories.

To refine this estimate, one might look to the ultimate vote totals for A Single Samurai, Turncoat, and On a Spiritual Plain, once all of the passes had been completed. A Single Samurai topped out at 1,111 votes, Turncoat managed 1,064, and On a Spiritual Plain finished with 1,040. Subtracting the Rabid Puppy upper estimate of 586 from these numbers gives figures of 525, 478, and 454. Assuming that some small number of non-Puppies voted one or more these stories above No Award4, means that a rough estimate of upper bound of the total number of Sad Puppies might somewhere in the 450 to 500 range. After No Award was eliminated these three stories picked up a combined total of 110 voters, and presumably very few of them came from Sad Puppy or Rabid Puppy voters. Discounting those votes from the final total reduces the Sad Puppy range to between 340 and 490 voters, which seems like a reasonable estimate.

The question is what these voters will do in the future. Assuming they are fans, and not merely culture warriors, one has to suppose that they hope to actually vote for Hugo winners. This may be overly optimistic - after all many of the Puppy leaders in the just completed 2015 Hugo Awards were unwilling or unable to explain what they actually liked about the books they pushed onto the Hugo ballot with their slate, even when asked directly. If they are actually fans, one has to wonder how long they will tilt at windmills with the knowledge that they simply cannot win. There were 5,950 ballots cast in the 2015 Hugo Awards. Roughly 400 to 500 were Sad Puppy supporters, roughly another 500 or so were their fellow travelers from the Rabid Puppy brigade, leaving almost five thousand non-Puppy voters, the vast majority of whom were perfectly willing to vote No Award when confronted with a ballot of Hugo nominees populated by slate-supported works of dubious quality.

I suppose that they might decide to try to recruit more voters congenial to their point of view, but that seems to be unlikely to bear much fruit. The Sad Puppy leaders spent enormous amounts of time promoting and defending their set of nominees, exhorting their followers to sign up and vote in the awards. Head Rabid Puppy Beale made numerous blog posts on the subject, ginned up phony controversy about editors at Tor, and even gave those followers who became involved badges as a reward for loyalty. In short, they flogged the horse as hard as they could, and came up with about a thousand people willing to pay the supporting membership fee and vote. Meanwhile, mainstream fans signed up to participate at a rate sufficient to result in a six to one majority despite much more modest active recruiting efforts. In fact, the best spur for non-Puppies to become involved and vote were the antics of the various Puppies. It seems clear, based upon the results of the voting, that the Puppy claims to represent "real" fandom were not only wrong, they were very wrong.

I suspect that by this time next year, many of the current Sad Puppy supporters will have quietly separated themselves from the group, abandoning it as a bad idea. It seems likely that very few authors, editors, or artists not already fully committed to one or the other groups will be willing to allow themselves to be put on either slate, and even some who think the Sad Puppies are a great idea may decline as well. After all, getting a Hugo nomination and then finishing well behind No Award is probably not particularly enjoyable, or particularly good for one's career. No one likes to set themselves up for a certain loss. Few people are willing to continue to work for a persistently losing cause. The Sad Puppies have been at this for three years now, and they have lost every time.

I expect there to be slates next year, but I think they will be less substantial than they were this year, mostly because one can already see some of the Sad Puppies becoming fatigued at the effort of maintaining their rage. Some Puppy proponents have claimed that their group represented the silent majority of fans. Now that the numbers have shown that they are actually a relatively small splinter group, I expect some in their number to become further demoralized. I expect that by the time I am doing this type of analysis next year, the number of Puppy voters will be slightly less, and the number of non-Puppy voters will be somewhat more.

1 Sad Puppies 1, which took place in 2013, doesn't really count, as it was mostly unsuccessful at getting nominees on the Hugo ballot.
2 I do not. Once I have determined that No Award is the best remaining choice, my stance is that I simply do not care which of the remaining nominees (if any) end up winning.
3 In the short term, the Puppy reactions are very predictable, ranging from silly declarations of victory because "No Award" was what they wanted all along, to frothy rage over their candidates being denied the Hugo Awards they were due.
4 For example, 27 voters ranked Antonelli's On a Spiritual Plain first, but either expressed no preference or voted for No Award second, while 45 voters appear to have ranked Wright's The Parliament of Beasts and Birds first or second, but ranked No Award below that.

Biased Opinions     2015 Hugo Award Finalists     Book Award Reviews     Home


  1. Replies
    1. @"As You Know" Bob: Thanks. I may do another post about Hugo voting, mostly to explain in detail how the Australian Instant Runoff Voting system works, since so many people (most of them Puppies) seem to not understand it at all.