In 2016, the Business Meeting of the Worldcon in Kansas City, Missouri ratified a change to the nomination process for the Hugo Awards called
E Pluribus Hugo. This ratification was the culmination of a two year process that had been sparked by the domination of the 2015 Hugo Award nomination process by an organized minority of voters comprised of two allied groups that called themselves the "Sad Puppies" and the "Rabid Puppies".
E Pluribus Hugo was formulated as a response to these bloc voting campaigns, and as the World Science Fiction Society (WSFS) constitution requires two years to implement changes, the change to the voting system had to be passed at both the 2015 and 2016 Worldcon Business Meeting.
The Old System
The first question one might ask is what voting system
E Pluribus Hugo replaced, and why. Prior to the implementation of
E Pluribus Hugo, the nomination process for the Hugo Awards was a fairly simple "first past the post" system: Every eligible voter could nominate up to five eligible candidates in each category. These nominations would be tallied, and the five works or people that received the most nominations in each category would become the Hugo Award finalists for that year.
The trouble with this system is that it requires most of the participants to act in good faith, because it is relatively easy for a dedicated voting bloc to "game" the system. As long as voters are voting their individual preferences, the system works, and results in a diffuse array of nominated works and people, five of whom will rise to the top of the list and take position as Hugo finalists. When one looks at the available Hugo nominating data, also known as the
Hugo Longlist, provided pursuant to the WSFS constitution, one generally finds this sort of diverse array of nominees in the years prior to the creation of the "Puppy" campaigns.
The most notable attempt to exploit the weakness in the Hugo nomination system prior to the launch of the "Puppy" campaigns occurred in 1987, when members of the Church of Scientology organized a dedicated minority of Hugo voters to vote
en masse for the L. Ron Hubbard science fiction novel
Black Genesis, which reached the finalist list as a result. The actions of the Scientologists were considered to be outside the accepted norms of behavior by the general body of Hugo voters, who responded by voting the Hubbard novel below "No Award" in the final vote. This form of social sanction was the primary means under by the Hugo community to prevent groups from exploiting the voting system in this manner. It is important to note that most regular Hugo voters knew about the flaw in the voting system prior to the creation of the "Puppy" campaigns, but no one other than the Scientologists had been rude enough to try to use it.
The Puppy Campaigns
In 2013, Larry Correia launched an attempt to get himself a Hugo Award with a blog post titled
How to Get Larry a Hugo in which he asserted that him winning the award would "make literati heads explode". Much of Correia's anger over the Hugo Awards seems to stem from the fact that he was nominated for, but did not win, a Campbell Award in 2011. In this iteration of his crusade, Correia clumsily parodied the SPCA commercials relating to pet adoption, asserting that if he did not win a Hugo it would "make puppies sad". This effort on Correia's part was mostly unsuccessful.
In 2014, Correia redoubled his efforts, promoting not only himself for a Hugo Award, but also creating a list of recommended works that he urged his blog followers to vote for. Among the collection of works he promoted was a story titled
Opera Vita Aeterna by Theodore Beale, who cross-promoted the so-called "Sad Puppy" selections on his own blog. It seems relatively clear that Correia's objective was to use this sort of cross-promotion among the multiple authors listed as Sad Puppies to garner Hugo nominations for all of them. Several of the candidates on this Sad Puppy slate were nominated. This was regarded as controversial at the time, but as the Sad Puppy candidates were more or less in line with the proportion of Sad Puppies in existence, most people evaluated the Sad Puppy finalists on their own merits. Sadly for the Puppies, this resulted in most of the Sad Puppy works faring quite poorly in the voting.
In 2015, Brad Torgersen took over the "Sad Puppy 3" campaign, and assembled and promoted a slate of works that was comprised of five works in multiple categories. Theodore Beale created a complimentary "Rabid Puppy" slate that was composed of mostly overlapping candidates for the Hugo ballot, filled out with several selections mostly drawn from the small press publisher he owned. Though Torgersen claimed to have selected the works on the Sad Puppy slate "democratically", the evidence shows that most of the actual public suggestions were ignored, and that almost all of the selections had some sort of personal or professional connection to Torgersen. As an organized voting bloc, the minority represented by the Sad and Rabid Puppies were able to dominate the Hugo nominations. Organized political groups - and based upon their rhetoric and campaigning there is no question that the Puppies are an organized political group - will almost always emerge victorious in a "first past the post" system unless there is organized opposition. There was no organized opposition.
Following the domination of the Hugo nominations process by a minority of approximately fifteen to twenty percent of the total voters, the Hugo voters as a whole responded in two ways. First, most voters placed all of the Sad and Rabid Puppy finalists behind "No Award" in the final voting. In most cases, there was no difference between assessing the Puppy finalists based on quality and relegating them to a position behind "No Award" and choosing to place them behind "No Award" as a response to the Puppy political actions. Second, several possible changes to the nomination system were formulated, including
E Pluribus Hugo, and then proposed at the Worldcon Business Meeting held at Sasquan that year.
In 2016, the Sad Puppies shifted tactics slightly, creating a longer list of crowd sourced recommendations. Theodore Beale's Rabid Puppies, on the other hand, created a slate and encouraged bloc voting by their supporters. As a result, several of the Hugo categories were once again dominated by Puppy selections driven by the organized bloc voting of a dedicated minority of the total voters, although this time it was clear that the Rabid Puppies were in control. Once again, the majority of Hugo voters responded by relegating many of the Puppy slated works to a position behind "No Award", and once again there was almost no difference between doing so based upon an assessment of the quality of the Puppy nominees and doing so as a response to the political campaigning of the Puppies. At the Worldcon Business Meeting held at MidAmeriCon II, the Worldcon voters ratified the
E Pluribus Hugo voting system for the second time, as required to make it effective.
What Is E Pluribus Hugo?
Of the several proposed changes to the Hugo nominating system, the Business Meeting at Sasquan chose to pass
E Pluribus Hugo. This, of course, raises the question of what
E Pluribus Hugo is intended to do, and how it actually changes the nomination system. At the outset, it should be noted that
E Pluribus Hugo doesn't change anything from the perspective of an individual nominating voter. Every eligible voter may nominate up to five works in each category, just as they always have. The difference between
E Pluribus Hugo and the previous "first past the post" nomination system is in how the nominations are tabulated.
Each ballot cast under
E Pluribus Hugo is worth exactly one "point" per category, with that point divided evenly among all of the works nominated. Therefore, if a voter nominates five items in a category, each of those works will receive one-fifth of a point. If a voter nominates four items in a category, each of those works will receive one-fourth of a point. And so on. All of these ballots are tabulated, and all of the points cast for the various nominees are added together. Once this is done, the nominees with the lowest point total are examined and compared with one another, with the one with the fewest total number of nominations (not points) being eliminated.
The power of
E Pluribus Hugo is that once a work is eliminated, the ballots on which that work appeared are recalculated. If a voter nominates five works, each initially receives one-fifth of a point. If one of those works is eliminated from contention, then that voter's ballot would only have four remaining works, which would then receive one-quarter of a point instead of one-fifth. In this manner, the support that voter's ballot provides for the works that were not eliminated is slightly intensified. As one can see, if there is a bloc of voters who all vote in favor of the same work, their support will remain spread thin across all of the ballots, while other voters who have a chaotic arrangement of works supported will see their support for their shared commonalities become more potent as the different works they supported are eliminated from contention.
How Well Does E Pluribus Hugo Work?
In theory,
E Pluribus Hugo should dampen the effectiveness of slate voting by making voting for a shared collection of works a less powerful strategy. The nomination system won't eliminate the votes of the bloc voters, and was never intended to do so. One of the goals of changing the voting system was to reduce the ability of bloc voters to dominate the nominations process, the goal was not to exclude the bloc voters, but merely to reduce their ability to place works onto the list of finalists to something closer to their actual proportion of the total voting body.
The means of testing this process is to use the voting data from the last few years and evaluating the results. This post shows the changes that would have been made to the final ballot had
E Pluribus Hugo been in effect for the 2016 Hugo nominations process. In many categories, it appears that one or two finalists would be different, replacing a slate-driven candidate with a non-slate candidate. The question that is presented is why
E Pluribus Hugo is more effective in some categories than others. For 2016, I think that the answer lies in two places, the first of which is related to the number of nominating ballots cast in that particular category. The following table shows the total nominating ballots cast in each category, my estimate of Rabid Puppy voters in each category, and the estimated percentage of Rabid Puppy voters out of the total:
| Category | Total Ballots Cast | Estimated Rabid Puppies | Estimated Percentage |
|---|
| Novel |
3,695
|
400-440
|
11-12
|
| Novella |
2,416
|
440
|
18
|
| Novelette |
1,975
|
415-440
|
21-22
|
| Short Story |
2,451
|
390-425
|
16-17
|
| Related Work |
2,080
|
385-440
|
19-24
|
| Graphic Story |
1,838
|
350-375
|
19-20
|
| Editor, Short |
1,891
|
400
|
21
|
| Editor, Long |
1,794
|
435
|
24
|
| Professional Artist |
1,481
|
260-350
|
24
|
| Semiprozine |
1,457
|
430-450
|
30
|
| Fanzine |
1,455
|
400
|
28
|
| Fan Writer |
1,568
|
310-340
|
20-22
|
| Fan Artist |
1,073
|
320-340
|
30-32
|
| Fancast |
1,267
|
330
|
26
|
| Campbell |
1,922
|
400
|
21
|
Looking at these figures, it seems that there is a range of effectiveness for
E Pluribus Hugo. If there are too few bloc voters - below about fifteen percent of the total - their impact on the final ballot is not sufficiently large that applying
E Pluribus Hugo to the results will change them. If there are too many bloc voters - a number that seems to be "something approaching about thirty percent of the total" - then
E Pluribus Hugo is not a robust enough system to make a dent in their efforts. In the middle range, it appears that a combinations of factors, including whether or not bloc voted works have support outside of the voting bloc, result in a variable effect that works changes ranging in scale from one to three slots on the final ballot.
Best Novel
(
3,695 Ballots, Estimated 400-440 Rabid Puppies)
Unchanged Finalists:
Ancillary Mercy by
Ann Leckie
The Aeronaut's Windlass by
Jim Butcher
The Fifth Season by
N.K. Jemisin
Seveneves by
Neal Stephenson
Uprooted by
Naomi Novik
Removed Finalists:
None
Added Finalists:
None
Notes: The Best Novel category for 2016 is unchanged by the application of
E Pluribus Hugo, which should come as a surprise to no one. The Best Novel category is one of the more resilient categories of the Hugo Awards, due to the relatively greater number of people who nominate in the category, and as a result, bloc voting is not as successful a tactic here as it is in other, smaller categories. When one accounts for the fact that the Rabid Puppy slate tried to latch on to works that were generally popular like
Seveneves in a desperate effort to be able to claim credit for their success, the reason for the lack of change worked in this category by
E Pluribus Hugo becomes even more readily apparent.
Best Novella
(
2,416 Ballots, Estimated 440 Rabid Puppies)
Unchanged Finalists:
Binti by Nnedi Okorafor
The Builders by Daniel Polansky
Penric's Demon by Lois McMaster Bujold
Perfect State by Brandon Sanderson
Slow Bullets by
Alastair Reynolds
Removed Finalists:
None
Added Finalists:
None
Notes: The Best Novella category is also (probably) unchanged by
E Pluribus Hugo, for much the same reasons that the Best Novel category was unchanged - there are a relatively large number of nominators who vote in this category, and the Rabid Puppy slate tried to piggyback onto works that had broader appeal than just those of the Rabid Puppy voting bloc. There is one wrinkle in the statistics insofar as
The Builders is shown in the "EPH Ballot Results on 2016 Hugo Race Report" as having 149.283 nominations under the new system, and is listed in fifth place overall, while
The Pauper Prince and the Eucalyptus Jinn by Usman T. Malik is shown with 194.2 nominations and is listed in sixth place overall. According to the list totals, these two stories should be reversed in order, with
The Pauper Prince replacing
The Builders on the list of finalists. On the other hand, the 194.2 figure seems like it could merely be a misprint, with the 9 and the 4 reversed. In that case, the total number of nominations for
The Pauper Prince would be 149.2, and the order listed would be correct. I have not seen anyone involved in running the tests provide clarification on this point as of yet.
Best Novelette
(
1,975 Ballots, Estimated 415-440 Rabid Puppies)
Unchanged Finalists:
And You Shall Know Her by the Trail of Dead by Brooke Bolander
Flashpoint: Titan by Cheah Kai Wei
Folding Beijing by
Hao Jingfang
Obits by Stephen King
Removed Finalists:
Hyperspace Demons by Jonathan Moeller [nomination declined]
What Price Humanity? by David VanDyke
Added Finalists:
Our Lady of the Open Road by
Sarah Pinsker
Notes: One element that is interesting to look at is what effect
E Pluribus Hugo would have had on the nominations of works that ended up being declined. In the Best Novelette category, Jonathan Moeller would have never had the opportunity to decline the nomination for
Hyperspace Demons, because it would not have placed in the top five nominees. This seems to have happened about half the time there was a nomination that was declined or withdrawn. I'm not sure what the significance of this is, but it is at least moderately notable. Once again, this is a category in which the Rabid Puppy slate attempted to use some works with a wider base of support as a kind of shield. Consequently, the ballot already had pretty good, reasonably popular works like
And You Shall Know Her by the Trail of Dead and
Folding Beijing upon it, and at least adequate works like
Obits as well. Removing the mediocre
What Price Humanity? and replacing it with the excellent
Our Lady of the Open Road would have given the Hugo voters several excellent choices to select from when casting their final ballots.
Best Short Story
(
2,451 Ballots, Estimated 390-425 Rabid Puppies)
Unchanged Finalists:
Asymmetrical Warfare by S.R. Algernon
Seven Kill Tiger by Charles Shao
Removed Finalists:
The Commuter by Thomas Mays [nomination declined]
If You Were an Award My Love by Juan Tabo and S. Harris
Space Raptor Butt Invasion by Chuck Tingle
Added Finalists:
Hungry Daughters of Starving Mothers by
Alyssa Wong
Wooden Feathers by
Ursula Vernon
Notes: Although
E Pluribus Hugo is not aimed at particular nominees based upon their particular content or quality, the results of applying it in the Best Short Story category seem to suggest that it may have the effect of doing so, at least to the extent that a voting bloc supports candidates of low quality or which contain offensive content. I believe, based upon my understanding of the mechanics of
E Pluribus Hugo, that to the extent that it has an effect, the voting system will have its greatest impact upon those items that derive most of their support from a voting bloc. I suspect that in this case, the stories
If You Were an Award My Love and
Space Raptor Butt Invasion had essentially no support other than from the Rabid Puppy voting bloc, which is why they drop off of the ballot to be replaced by much better finalists when
E Pluribus Hugo is applied.
Best Related Work
(
2,080 Ballots, Estimated 385-440 Rabid Puppies)
Unchanged Finalists:
The First Draft of My Appendix N Book by Jeffro Johnson
Safe Space as Rape Room by Daniel Eness
Removed Finalists:
Between Light and Shadow: An Exploration of the Fiction of Gene Wolfe, 1951 to 1986 by Marc Aramini
The Story of Moira Greyland by Moira Greyland
Added Finalists:
Letters to Tiptree edited by
Alisa Krasnostein and
Alexandra Pierce
You're Never Weird on the Internet (Almost) by
Felicia Day
Notes: In 2016, the Best Related Work category was overrun with works that ranged from offensive and terrible to merely bland and mediocre, with only the Aramini work being even marginally worthwhile as a finalist. Applying
E Pluribus Hugo to this category has the effect of removing both the Aramini work and Moira Greyland's piece from the ballot and replacing them with
Letters to Tiptree and
You're Never Weird on the Internet (Almost), which represents a substantial increase in quality, but more importantly seems to reveal something of interest about the nature of the bloc that voted for the Rabid Puppy slate. Even though the Rabid Puppy bloc seems to have displayed fairly strong slate discipline, they were not perfect in this regard. Under
E Pluribus Hugo, those works that have the strongest slate support, or which have some non-slate support are the ones from the slate that will remain on the ballot over those works that have weaker slate support or little or no non-slate support. The array of slated works that remains on the ballot after the application of
E Pluribus Hugo suggests that the primary objective of most of the Rabid Puppy voting bloc was trolling, and they were generally unconcerned with quality.
Best Graphic Story
(
1,838 Ballots, Estimated 350-375 Rabid Puppies)
Unchanged Finalists:
Full Frontal Nerdity by Aaron Williams
The Sandman Overture by
Neil Gaiman and
J.H. Williams, III
Removed Finalists:
The Divine written by Boaz Lavie, art by Asaf Hanuka and Tomer Hanuka
Erin Dies Alone written by Grey Carter, art by Cory Rydell
Added Finalists:
Nimona by
Noelle Stevenson
Saga, Volume 5 by
Brian K. Vaughn and
Fiona Staples
Notes: The final ballot of Best Graphic Story is among the most changed by the application of
E Pluribus Hugo. Three slated works drop off of the list of finalists, to be replaced by three non-slate driven choices. It is unclear to me why this category was the most affected by
E Pluribus Hugo: The estimated percentage of Rabid Puppy bloc voters is comparable to that of some other categories that were unaffected by changing the voting system. There seem to be plenty of works to choose from in the category, so there doesn't appear to be the natural concentration effect for nominations that many people suspect will affect the two Dramatic Presentation categories. No matter the cause,
E Pluribus Hugo seems to have been particularly effective in this category.
Best Professional Editor: Short Form
(
1,891 Ballots, Estimated 400 Rabid Puppies)
Unchanged Finalists:
John Joseph Adams
Neil Clarke
Ellen Datlow
Jerry Pournelle
Sheila Williams
Removed Finalists:
None
Added Finalists:
None
Notes: Best Professional Editor Short Form is a category almost entirely unaffected by
E Pluribus Hugo, which is entirely unsurprising because there was only one slate candidate in this category. In the case where there is a single work or person on a slate, the voting system shouldn't affect the outcome, as it is only intended to dampen the effect of coordinated voting for multiple candidates.
Best Professional Editor: Long Form
(
1,794 Ballots, Estimated 435 Rabid Puppies)
Unchanged Finalists:
Sheila Gilbert
Liz Gorinsky
Jim Minz
Anne Sowards [nomination declined]
Toni Weisskopf
Removed Finalists:
Theodore Beale
Mike Braff [ineligible]
Added Finalists:
Anne Lesley Groell
Notes: In the Best Professional Editor Long Form category the application of
E Pluribus Hugo would eliminate two slate-driven finalists (one of whom was ineligible) and replace them with a non-slate choice. It seems relatively obvious to most observers that Theodore Beale would have no support in any category other than the Rabid Puppy bloc voters, and the result of the application of
E Pluribus Hugo in this category supports that notion.
Best Professional Artist
(
1,481 Ballots, Estimated 350 Rabid Puppies)
Unchanged Finalists:
Larry Elmore
Michal Karcz
Abigail Larson
Removed Finalists:
Lars Braad Anderson
Larry Rostant
Added Finalists:
Julie Dillon
John Picacio
Notes: The Best Professional Artist category is another category that the application of
E Pluribus Hugo would have had a reasonably notable effect upon, dropping two finalists that were supported by bloc voting and replacing them with two non-slate candidates.
Best Semi-Prozine
(
1,457 Ballots, Estimated 430-450 Rabid Puppies)
Unchanged Finalists:
Beneath Ceaseless Skies edited by Scott H. Andrews, Nicole Lavigne, and Kate Marshall
Daily Science Fiction edited by Michele-Lee Barasso and Jonathan Laden
Sci Phi Journal edited by Jason Rennie
Strange Horizons edited by Catherine Krahe, Julia Rios, A.J. Odasso, Vanessa Rose Phin, Maureen Kincaid Speller, and the Strange Horizons staff
Removed Finalists:
None
Added Finalists:
None
Notes: The results in the Best Semi-Prozine category are essentially unchanged by the application of
E Pluribus Hugo. I estimate that Rabid Puppy voters made up roughly thirty percent of the total number of voters in this category. Based upon my rough analysis, when the size of the voting bloc supporting a slate reaches that proportion of the total,
E Pluribus Hugo can do little to change the outcome. Effectively, having a third or more of the total votes appears to more or less immunize a slate against the effects of the voting system.
Best Fanzine
(
1,455 Ballots, Estimated 400 Rabid Puppies)
Unchanged Finalists:
Black Gate edited by John O’Neill [withdrawn]
Castalia House Blog edited by Jeffro Johnson
File 770 edited by Mike Glyer
Lady Business edited by Clare McBride, Ira, Jodie, KJ, Renay, and Susan
Superversive SF edited by Jason Rennie
Tangent Online edited by Dave Truesdale
Removed Finalists:
None
Added Finalists:
None
Notes: Like the results in the Best Semi-Prozine category, the results in the Best Fanzine category are essentially unchanged by the application of
E Pluribus Hugo. I believe the ineffectiveness of E Pluribus Hugo in this category is for much the same reason as in the Best Semi-Prozine category - the estimated number of Rabid Puppies voting as a bloc in this category approaches thirty percent of the total number of voters (27.5% to be exact). This emphasizes that merely applying
E Pluribus Hugo to the ballot is not going to be sufficient to reduce the impact of bloc voting. There also needs to be a critical mass of non-bloc voters to make the voting system effective.
Best Fan Writer
(
1,568 Ballots, Estimated 310-340 Rabid Puppies)
Unchanged Finalists:
Mike Glyer
Jeffro Johnson
Morgan Holmes
Removed Finalists:
Douglas Ernst
Shamus Young
Zenopus [nomination declined]
Added Finalists:
Alexandra Erin
Natalie Luhrs
Notes: One of the interesting and somewhat unexpected effects of the application of
E Pluribus Hugo is the removal from the final ballot of many potential finalists who declined a slot as a finalist. Other than that wrinkle, the changed results in this category seems to be in line with what one would expect given the estimated percentage of Rabid Puppy voters - three slate-based finalists moved off of the final ballot and replaced by two non-slate finalists.
Best Fan Artist
(
1,073 Ballots, Estimated 320-340 Rabid Puppies)
Unchanged Finalists:
Matthew Callahan
disse86
Kukuruyo
RGUS [nomination declined]
Christian Quinot
Steve Stiles
Removed Finalists:
None
Added Finalists:
None
Notes: Although I estimated that there were roughly the same number of Rabid Puppy bloc voters in this category as there were in the Best Fan Writer category, there were fewer ballots cast overall in this category. As a result, the concentration of Rabid Puppy voters was high enough to make
E Pluribus Hugo ineffective at diluting the effects of bloc voting.
Best Fancast
(
1,267 Ballots, Estimated 330 Rabid Puppies)
Unchanged Finalists:
Cane and Rinse by Cane and Rinse
HelloGreedo by HelloGreedo*
The Rageaholic by RazörFist
Removed Finalists:
8-4 Play by Mark MacDonald, John Ricciardi, Hiroko Minamoto, and Justin Epperson
Tales to Terrify by Stephen Kilpatrick
Added Finalists:
Galactic Suburbia by
Alisa Krasnostein,
Alexandra Pierce, and
Tansy Rayner Roberts
Tea and Jeopardy by Emma Newman and Peter Newman
Notes: The change in the results for the Best Fancast category as a result of the application of
E Pluribus Hugo is in line with that in numerous other categories - two slate-based finalists fall off of the ballot and are replaced by two non-slate choices. Although this isn't quite as powerful a result as some might have hoped, this is in line with what
E Pluribus Hugo was intended to do: It dilutes the power of bloc voting by reducing the ability of a voting bloc of fifteen to twenty-five percent of the voters to dictate all of the finalists in a category. It doesn't accomplish the optimal objective of reducing the impact of a voting bloc to a percentage of choices proportionate with their numbers, but it does have some effectiveness to that end.
John W. Campbell Award for Best New Writer
(
1,922 Ballots, Estimated 400 Rabid Puppies)
Unchanged Finalists:
Pierce Brown
Brian Niemeier
Andy Weir
Alyssa Wong
Removed Finalists:
Sebastien de Castell
Added Finalists:
Becky Chambers
Notes: The Campbell Award category, as with so many other categories, sees one slate-based finalist removed and replaced by a non-slate candidate when one applies
E Pluribus Hugo to the results. One reason that the impact of E Pluribus Hugo in this category is so minimal is probably the result of substantial overlap between the preference of those within the Rabid Puppy voting bloc for Andy Weir and the preference of the general population of voters for him as well.
What Are the Hugo Awards?
Go to the 2016 list of Hugo finalists: 2016
Random Thoughts Book Award Reviews Home
