Random Thought - The 2015 "E Pluribus Hugo" Revised Hugo Finalists

E Pluribus Hugo was passed largely in response to the results of the 2015 Hugo nomination process. I outlined the background leading up to this in my previous post about the 2016 E Pluribus Hugo Revised Hugo Finalists, and I'm not going to repeat myself here. Anyone who wants a summary of the Sad and Rabid Puppy campaigns, the responses from non-Puppy Hugo voters, and an outline of the mechanics of E Pluribus Hugo can go read about that there.

The question to be examined in this post is simply this: Does E Pluribus Hugo do what it was meant to do? One thing to note is that there are a lot of things it was not intended to do, and is not designed to do: E Pluribus Hugo does not identify "slates", nor does it make any kind of "quality" evaluation of the potential nominees. The system also does not deal with "blocs" spread across multiple categories, but rather treats each category as a separate entity. E Pluribus Hugo is intended to reduce the ability of coordinated bloc voting groups to dominate entire Hugo categories. In an ideal world, the voting system would result in the influence of voting blocs being held to a proportion similar to their numbers - if a voting bloc of twenty percent of the total electorate works together, they should be able to determine something in the range of twenty percent of the finalists in a category, instead of the situation under a first past the post system in which a voting bloc of fifteen or twenty percent of the total electorate can control eighty to one hundred percent of the finalists in a category.

Given that 2015 was the prime year for the two allied Puppy campaigns, the 2015 Hugo nomination data is the ideal test case for the effectiveness of E Pluribus Hugo. So, how well did it work? As one can see below, it worked, but not quite as well as one might have hoped. I estimated that the total combined Puppy voting strength in each category ranged from eleven percent to forty-one percent of the total number of voters (depending on what range of estimates one uses for the Puppy voters), as outlined in this chart:

Category	Total Ballots Cast	Estimated Total Puppies	Estimated Percentage	EPH Ballot Changes
Novel	1,871	260-390	14-21	1
Novella	1,083	275-335	25-31	2
Novelette	1,031	245-265	24-26	1
Short Story	1,174	125-230	11-20	2
Related Work	1,150	205-270	18-23	2
Graphic Story	785	200	25	1
Editor, Short	870	220-270	25-31	2
Editor, Long	712	240-295	34-41	1
Professional Artist	753	160-180	21-24	1
Semiprozine	660	200-210	30-32	1
Fanzine	576	180-200	31-35	0
Fan Writer	777	195	25	2
Fan Artist	286	-	-	0
Fancast	668	160	24	0
Campbell	851	215-225	25-26	0

The first column is the total number of ballots cast in the particular category. The second column is the total number of Sad and Rabid Puppies that I estimated voted in that category (anyone who wants to know how I came up with these figures can read my post titled How to UnPuppy a Hugo Ballot). The third column is the estimated percentage of combined Puppy voters in each category, while the fourth column is how many changes apply E Pluribus Hugo to the nominating ballots would have made to the list of finalists. As one can see, in most categories, E Pluribus Hugo had a modest effect, changing one or two finalists, and in some cases, changing none. Because of withdrawals and disqualifications in specific categories, an overarching evaluation is unlikely to provide much useful information about the effectiveness of E Pluribus Hugo. On a general note, it does seem reasonably clear that when the total number of slate-based voters rises above about twenty-five percent of the total number of voters, E Pluribus Hugo loses much of its effectiveness.

Best Novel

(1,827 Ballots, 260-390 Estimated Total Puppies)

Unchanged Finalists:
Ancillary Sword by Ann Leckie
The Goblin Emperor by Katherine Addison
Lines of Departure by Marko Kloos [nomination withdrawn]
Monster Hunter Nemesis by Larry Correia [nomination declined]
Skin Game by Jim Butcher
The Three-Body Problem by Cixin Liu (translated by Ken Liu)

Removed Finalists:
The Dark Between the Stars by Kevin J. Anderson

Added Finalists:
Lock-In by John Scalzi

Notes: The application of E Pluribus Hugo to the Best Novel category results in one change on the ballot, although the fact that Larry Correia declined his nomination for Monster Hunter: Nemesis and Marko Kloos withdrew his nomination for Lines of Departure obscures the exact nature of the change. Had they still withdrawn, Kevin J. Anderson's novel The Dark Between the Stars would have been replaced on the ballot by John Scalzi's Lock-In. One has to wonder, however, what would have happened in an E Pluribus Hugo environment had Correia and Kloos not pulled their books from consideration.

Had Correia not declined the finalist spot for Monster Hunter: Nemesis, Anderson's book still would not have made the list of finalists. Short of numerous other declined or withdrawn nominations, Anderson's book would not have made the finals under almost any scenario, as applying E Pluribus Hugo to the nomination ballots drops it to tenth place overall in the standings. However, had Correia not declined his nomination, Kloos would not have made the final ballot either, as his book would have been dropped from fourth to sixth place in the nominations by the application of E Pluribus Hugo. The five nominated books in that scenario would have been Ancillary Sword, The Goblin Emperor, Monster Hunter: Nemesis, Skin Game, and The Three Body Problem. Kloos would have never had to make the choice as to whether to stay on the Hugo ballot or not.

If Correia had withdrawn his novel from the finalist list, that would have put Kloos' novel back onto the Hugo ballot. In that case, the five nominated books would have been Ancillary Sword, The Goblin Emperor, Lines of Departure, Skin Game, and The Three Body Problem. Only if Kloos had elected to decline this nomination would Scalzi's Lock-In have made it to the Hugo ballot.

On a side note, I will observe that although E Pluribus Hugo is not intended to assess the quality of the works nominated, in practice it seems to do that to a degree. When E Pluribus Hugo is applied to the results of the 2015 nominations, it generally has the effect of dropping what are, in my opinion, the weakest finalists off of the ballot, and replacing them with superior alternatives. I regard The Dark Between the Stars as being by far the weakest of the novels nominated in 2015, and dropping it off of the ballot to replace it with The Three Body Problem is a substantial improvement.

Best Novella

(1,083 Ballots, 275-335 Estimated Total Puppies)

Unchanged Finalists:
Big Boys Don't Cry by Tom Kratman

Flow by Arlan Andrews, Sr. (reviewed in Analog Science Fiction and Fact: Vol. CXXXIV, No. 11 (November 2014))

One Bright Star to Guide Them by John C. Wright

Removed Finalists:
Pale Realms of Shade by John C. Wright
The Plural of Helen of Troy by John C. Wright

Added Finalists:
The Regular by Ken Liu
The Slow Regard of Silent Things by Patrick Rothfuss

Notes: Unlike the Best Novel category, the changes wrought by E Pluribus Hugo upon the Best Novella category are fairly straightforward. Two novellas by John C. Wright are dropped off of the final ballot, and are replaced by novellas by Ken Liu and Patrick Rothfuss. As I noted before, although it is not aimed at making a quality evaluation of the various nominees, in practice E Pluribus Hugo seems to have that effect. In this category, the two Wright penned novellas that drop off the ballot are pretty miserable in quality, while the two novellas that replace them are markedly better.

Best Novelette

(1,031 Ballots, 245-265 Estimated Total Puppies)

Unchanged Finalists:

Ashes to Ashes, Dust to Dust, Earth to Alluvium by Gray Rinehart (reviewed in 2015 Hugo Voting - Best Novelette)

Championship B'tok by Edward M. Lerner (reviewed in Analog Science Fiction and Fact: Vol. CXXXIV, No. 9 (September 2014))

The Day the World Turned Upside Down by Thomas Olde Heuvelt (reviewed in 2015 Hugo Voting - Best Novelette)

The Journeyman: In the Stone House by Michael F. Flynn (reviewed in Analog Science Fiction and Fact: Vol. CXXXIV, No. 6 (June 2014))

The Triple Sun: A Golden Age Tale by Rajnar Vajra (reviewed in Analog Science Fiction and Fact: Vol. CXXXIV, Nos. 7 & 8 (July/August 2014))

Removed Finalists:
Yes, Virginia, There Is a Santa Clause by John C. Wright [ineligible]

Added Finalists:
None

Notes: Applying E Pluribus Hugo to the Best Novelette category results in a single change - John C. Wright's story Yes, Virginia, There Is a Santa Claus drops off of the final ballot (actually dropping to eighth place in the nominations), and is replaced by Thomas Olde Heuvelt's novelette The Day the World Turned Upside Down. This change is somewhat obscured by the fact that Wright's story was deemed ineligible due to prior publication and removed from the Hugo ballot, resulting in Heuvelt's story reaching the list of finalists anyway. Had E Pluribus Hugo been in effect in 2015, however, the eligibility determination for Yes, Virginia, There Is a Santa Claus would never have had to be made, because the story would not have had sufficient support to reach the list of finalists to begin with.

Best Short Story

(1.174 Ballots, 125-230 Estimated Total Puppies)

Unchanged Finalists:

Goodnight Stars by Annie Bellet (reviewed in 2015 Hugo Voting - Best Short Story) [nomination declined]

On a Spiritual Plain by Lou Antonelli (reviewed in 2015 Hugo Voting - Best Short Story)

Totaled by Kary English (reviewed in 2015 Hugo Voting - Best Short Story)

Tuesdays with Molokesh the Destroyer by Megan Grey [ineligible]

Turncoat by Steve Rzasa (reviewed in 2015 Hugo Voting - Best Short Story)

Removed Finalists:

The Parliament of Beasts and Birds by John C. Wright (reviewed in 2015 Hugo Voting - Best Short Story)

A Single Samurai by Steve Diamond (reviewed in 2015 Hugo Voting - Best Short Story)

Added Finalists:
The Breath of War by Aliette de Bodard

Jackalope Wives by Ursula Vernon (reviewed in 2015 WSFA Small Press Award Voting)

Notes: The Short Story category looks slightly more complicated than it actually is. Tuesdays With Molokesh the Destroyer would have reached the final ballot under either the first past the post system or E Pluribus Hugo, and it would have been disqualified under either system due to being published outside of the eligibility period. The only real question is whether Annie Bellet would have withdrawn her story Goodnight Stars had E Pluribus Hugo been in effect in 2015. If she had, then Turncoat, by Steve Rzasa would not have made the list of finalists, because it would have placed seventh overall. Both Jackalope Wives and The Breath of War would have been Hugo finalists under E Pluribus Hugo, and neither A Single Samurai or The Parliament of Beasts and Birds would have been.

Best Related Work

(1,150 Ballots, 205-270 Estimated Total Puppies)

Unchanged Finalists:
The Hot Equations: Thermodynamics and Military SF by Ken Burnside

Transhuman and Subhuman by John C. Wright

Why Science Is Never Settled by Tedd Roberts

Removed Finalists:
Letters from Gardner by Lou Antonelli

Wisdom from My Internet by Michael Z. Williamson

Added Finalists:
Chicks Dig Gaming by Jennifer Brozek, Robert Smith, and Lars Pearson
What Makes This Book So Great by Jo Walton

Notes: Because there were no withdrawals or disqualifications in this category, the effect of implementing E Pluribus Hugo is fairly straightforward - both Letters from Gardner and Wisdom from My Internet drop off of the list of finalists, and are replaced by Chicks Dig Gaming and What Makes This Book So Great. I have noted before that E Pluribus Hugo is not meant to be a quality filter, but in practice, it seems to be. Wisdom from My Internet is, in my opinion, in the running for the title of worst Hugo finalist of all time, and seeing it removed and replaced by a book like What Makes This Book So Great is by itself something I consider to be an endorsement of the new system.

Best Graphic Story

(785 Ballots, 200 Estimated Total Puppies)

Unchanged Finalists:
Ms. Marvel, Volume 1: No Normal by G. Willow Wilson and Adrian Alphona
Saga 3 by Brian K. Vaughan and Fiona Staples
Rat Queens, Volume One: Sass and Sorcery by Kurtis J. Weibe and Roc Upchurch

Zombie Nation, Book 2: Reduce, Reuse, Reanimate by Carter Reid (reviewed in 2015 Hugo Voting - Best Graphic Story)

Removed Finalists:

Sex Criminals, Volume 1: One Weird Trick by Matt Fraction and Chip Zdarksy

Added Finalists:

Schlock Mercenary: Broken Wind by Howard Tayler

Notes: There was only one item on the Sad or Rabid Puppy slates in this category in 2015 - Zombie Nation, Book 2: Reduce, Reuse, Reanimate. Consequently, one would expect the impact of E Pluribus Hugo on this category to be relatively modest, and it is. The interesting thing is that the changed voting system moves a non-slate item off of the list of finalists and replaces it with another non-slate item. This is interesting, because it shows how E Pluribus Hugo might work in a more normal year than this one.

Best Professional Editor: Short Form

(870 Ballots, 220-270 Estimated Rabid Puppies)

Unchanged Finalists:
Jennifer Brozek
Mike Resnick
Edmund R. Schubert [nomination withdrawn after deadline]

Removed Finalists:
Theodore Beale
Bryan Thomas Schmidt

Added Finalists:
John Joseph Adams
Neil Clarke

Notes: Once again, the changes in this category that result from the application of E Pluribus Hugo are fairly straightforward. Beale and Schmidt fall out of the list of Hugo finalists, and are replaced by stronger candidates.

Best Professional Editor: Long Form

(712 Ballots, 240-295 Estimated Rabid Puppies)

Unchanged Finalists:
Sheila Gilbert
Jim Minz
Anne Sowards
Toni Weisskopf

Removed Finalists:
Theodore Beale

Added Finalists:
Liz Gorinsky

Notes: The changes made by E Pluribus Hugo to the Best Long Form Professional Editor category are also fairly straightforward, although the result is to only change one of the finalists rather than two, as in the Short Form Editor category. One side note: It is somewhat interesting that Theodore Beale is dropped from the list of finalists in both editor categories. I suspect that this is because his support was drawn mostly from Rabid Puppy voters, and they were the most likely voters to bloc vote a slate of nominees, and thus their votes were most susceptible to the effect of E Pluribus Hugo.

Best Professional Artist

(753 Ballots, 160-180 Estimated Total Puppies)

Unchanged Finalists:
Julie Dillon
Jon Eno [ineligible]
Nick Greenwood
Alan Pollack
Carter Reid

Removed Finalists:
Kirk DouPonce

Added Finalists:
John Picacio

Notes: One element of E Pluribus Hugo that seems to be unclear is exactly what makes it work better in some categories than others. Although the percentages of Sad and Rabid Puppies are estimated, I am reasonably confident that those estimates are fairly reliable. Why then, does using E Pluribus Hugo only change one finalist in the Best Professional Artist category, but change two in the Best related Work category even though the percentage of Puppy nominators is roughly the same? I don't think there is a clear answer to this, and to a certain degree that is probably always going to be true unless one can examine the individual ballots to figure out why the effect is larger in one category as opposed to another.

Best Semi-Prozine

(660 Ballots, 200-210 Estimated Total Puppies)

Unchanged Finalists:
Abyss & Apex edited by Wendy Delmater
Beneath Ceaseless Skies edited by Scott H. Andrews

Lightspeed Magazine edited by John Joseph Adams, Stefan Rudnicki, Rich Horton, Wendy N. Wagner, and Christie Yant

Orson Scott Card's Intergalactic Medicine Show edited by Edmund R. Schubert [nomination withdrawn]

Strange Horizons edited by Niall Harrison

Removed Finalists:
Andromeda Spaceways In-Flight Magazine edited by David Kernot and Sue Burzynski

Added Finalists:
The Book Smugglers edited by Ana Grilo and Thea James

Notes: In the Best Semiprozine category, applying E Pluribus Hugo causes Andromeda Spaceways Inflight Magazine to drop off of the list of finalists. If Orson Scott Card's Intergalactic Medicine Show were to still withdraw, then The Book Smugglers would become a finalist, otherwise, it would not.

Best Fanzine

(576 Ballots, 180-200 Estimated Total Puppies)

Unchanged Finalists:
Black Gate edited by John O'Neill [withdrawn after deadline]
Elitist Book Reviews edited by Steven Diamond

Journey Planet edited by James Bacon, Christopher J. Garcia, Colin Harris, Alissa McKersie, and Helen J. Montgomery

The Revenge of Hump Day edited by Tim Bolgeo
Tangent Online edited by Dave Truesdale

Removed Finalists:
None

Added Finalists:
None

Notes: Applying E Pluribus Hugo to the Best Fanzine category results in no changes to the final ballot. This is not entirely surprising, as the estimated proportion of Puppy aligned voters in this category is north of the thirty percent mark, which is the point at which the voting system seems to lose its ability to counteract bloc voting tactics.

Best Fan Writer

(777 Ballots, 195 Estimated Total Puppies)

Unchanged Finalists:
Amanda S. Green
Jeffro Johnson
Laura J. Mixon
Cedar Sanderson

Removed Finalists:
Dave Freer
Matthew David Surridge [nomination declined]

Added Finalists:
Abigail Nussbaum

Notes: If E Pluribus Hugo were applied to the Best Fan Writer category, Matthew David Surridge would have dropped to eighth place in the standings, and Dave Freer would have dropped to tenth. In their place, Abigail Nussbaum would have been a finalist. As usual, E Pluribus Hugo has a modest, but undeniably positive effect on the quality of the finalists in the category.

Best Fan Artist

(296 Ballots, No Estimated Total Puppies)

Unchanged Finalists:
Ninni Aalto
Brad Foster
Elizabeth Leggett
Spring Schoenhuth
Steve Stiles

Removed Finalists:
None

Added Finalists:
None

Notes: Best Fan Artist is completely unchanged by the application of E Pluribus Hugo. Neither the Sad Puppy slate or the associated Rabid Puppy slate listed any Fan Artists, so there was no organized bloc voting in this category. Given that, one would expect that E Pluribus Hugo would have little or no effect on the results in this category, and that is exactly what we see.

Best Fancast

(668 Ballots, 160 Estimated Total Puppies)

Unchanged Finalists:

Adventures in Sci Phi Publishing by Brent Bower, Kristi Charish, Timothy C. Ward, and Moses Siregar III

Dungeon Crawlers Radio by Daniel Swenson, Travis Alexander, Scott Tomlin, Dale Newton, and Damien Swenson

Galactic Suburbia by Alisa Krasnostein, Alexandra Pierce, Tansy Rayner Roberts, and Andrew Finch

The Sci Phi Show by Jason Rennie
Tea and Jeopardy by Emma Newman and Peter Newman

Removed Finalists:

None

Added Finalists:

None

Notes: Best Fancast is yet another category that is unaffected by the application of E Pluribus Hugo. This seems to be, at least in part, due to the fact that the Sad and Rabid Puppies only touted three fancasts between them to begin with, and therefore two non-slated fancasts made it onto the list of finalists already. Consequently, this effect of E Pluribus Hugo should be muted in this category, and it actually seems to have been.

John W. Campbell Award for Best New Writer

(851 Ballots, 215-225 Estimated Rabid Puppies)

Unchanged Finalists:
Wesley Chu
Jason Cordova
Kary English
Rolf Nelson
Eric S. Raymond

Removed Finalists:
None

Added Finalists:
None

Notes: The Campbell Award is not a Hugo Award, but its nomination process is identical to that used by the Hugos. When E Pluribus Hugo is applied to the 2015 nominations for this award, the finalists are unchanged. This is likely because the Sad and Rabid Puppy slates did not line up perfectly in this category, with the Sad Puppies championing three authors, and the Rabid Puppies championing four. This lack of coordination plus the fact that they only bloc voted for a maximum of four candidates resulted in a ballot that was not susceptible to the effects of E Pluribus Hugo.

What Are the Hugo Awards?

Go to the 2015 list of Hugo finalists: 2015

Random Thoughts     Book Award Reviews     Home