Tuesday, March 24, 2015

A Critical Review of Harry Potter and the Methods of Rationality

This is a critical review of Harry Potter and the Methods of Rationality by Eliezer Yudkowsky. If you haven't read it, be warned that this review contains major spoilers. I don't mean "critical" with overtones of "faultfinder" or "censurer". I mean "critical" as in "a judgement of the merits and faults". If you really, really liked this story, imagine me beginning each sentence with "In my opinion,".

The Nature of the Work

I don't really agree with the notion that a piece of literature needs to be any one thing. The best works of art are multitools - different things to different people, and even different things to the same person. If done correctly, social commentary can be combined with comedy and schmaltzy romance can be combined with biting parody. So long as the goals of the work don't conflict, it's possible to accomplish them all at the same time. So what are the goals of the work?


Methods of Rationality wants to teach you things. It wants to give you lessons on social psychology, cognitive biases, and science. For the most part, it delivers these lessons in a straightforward way - many of the chapters feature a direct explanation of a concept by Harry, and then in a later scene hit upon that theme less directly. These lessons begin to drop off following the Azkaban arc in favor of other elements - primarily drama.

Character Growth

This is the story of Harry outgrowing his heritage as Tom Riddle. When the story begins, Harry is arrogant, power-hungry, condescending, scheming, and quite angry. If we're being charitable, he's telling the truth when he says that he's outgrown all of that at the end. Harry supplements his INT with badly needed WIS, and I think in that you can see a lot of the author's life story as he would tell it.

This creates some serious problems where the story's didacticism is concerned. The story wants to teach us lessons, but wants to teach them through a flawed character, and a flawed teacher. Harry's condescension and jargon-dense explanations are perfectly in character, but they make it difficult for a reader who isn't already familiar with the subject matter to understand what he's saying. This is compounded by the author's decision not to look things up, a decision that was made in the name of realism but which undercuts the story's ability to teach.

This character growth arc also has a bigger problem than the fact that it's at odds with the pretensions towards teaching - it doesn't happen organically. The path that a standard story might take to show a character overcoming their flaws is to show them failing because of their flaws. In this story, Harry rarely fails. His single biggest organic failure was when he lost a battle to Hermione because he didn't think to consult with his soldiers, and afterward showed real growth. In my opinion, this is the only real believable moment of character growth for him.

Harry is not shown failing because of his "dark side", he is shown winning because of it. His early battles of will with McGonagall are won primarily through blackmail and deceit, and the end result of this is not a lack of respect from her (which might have resulted in a chance for reflection and growth) but with McGonagall beginning to treat him as an equal.

Harry is also not shown failing for his more human faults, the first and foremost being his lack of wisdom. Harry accompanies Quirrell to Azkaban and breaks Bellatrix Black out, which is clearly a mistake and meant as such. Harry gets away with his deceptions, beats all odds, and there are never any meaningful consequences. Even at the story's end, this adventure remains a secret from every other character. Harry says something to Snape that breaks the spell Lily had cast over him, and there are no consequences to this. Harry tells Quirrell about the Resurrection Stone, and there are no meaningful consequences to this. Harry has his Time-Turner locked down, and there are no meaningful consequences to this. Harry steals Hermione's body and lies about it, and he is rewarded for it. Harry gives away his entire fortune to the Malfoys and trade enormous political power for Hermione ... and the debt is wiped clean, with the politcal power renewed soon after.

Harry is never given any incentive to change, and never really shows any change. The character growth arc is implied, but for the most part not actually present. Harry does not win the climax of the fic by having overcome his flaws, he wins it through brutal murder. The biggest organic change he undergoes is from believing in the value of truth to advocating for multiple conspiracies against both the wizarding and muggle worlds, and if that's character growth, it's a message I find ugly.


Methods of Rationality is a commentary on the canon books of J.K. Rowling, and to a lesser extent, a commentary on the fanfiction of Rowling's books and other stories (most notably, Ender's Game by Orson Scott Card, which a large portion of the chapters are a pastiche of). It's not meant to be read without knowing the canon books fairly well. Character descriptions are often skipped with the assumption that the reader will simply know what that person looks like from either the books or movies, even major ones. Harry doesn't get even a brief description until the eighth chapter (from Hermione's perspective) even though his appearance is important to the plot before that. Hermione doesn't get a description at all. The fact that Methods leans on canon should be considered a feature and not a flaw - much of what it accomplishes is through the mirror it holds up.

Methods is at its best when it's engaging in reconstruction - taking some piece of canon and remaking it into a better form, to allow people who have read the books to get some thrill of seeing something they love remade. The description of Occlumency is more thoughtful than the one presented in canon, and at the same time hits the themes of text fairly hard; it's a narrative multitool. Transfiguration is similarly well thought out, and takes something rather poorly defined and makes it into something both distinct and similar to canon, while at the same time putting forth lessons on the map and territory (though those are somewhat flawed) and being cool in its own right. Even Dementors as literal Death is an interesting twist that takes the work in new directions.

I personally find it at its worst when it's pure deconstruction. It does this a few times - criticizing the plots of the canon books (notably Goblet of Fire and Prisoner of Azkaban), the characters (Ron), or the worldbuilding (Harry's arbitrage scheme, Quidditch). I've read the author's assurances that his characters don't speak for him, but the problem is that the text speaks for itself. Though in some cases it revisits the things that it's taken digs at, this feels to me like backpedaling - or like a bully assuring the teacher that he was only joking around, not actually picking on someone without the ability to defend themselves.

I don't know whether these moments where it the text revisits a subject with more kind eyes are due to feedback from readers, but it feels to me as though they are, mostly because they're removed from the places where they could have made their point more strongly. Modern Quidditch having a problem with the snitch due to faster brooms is interesting in a way that out-of-hand dismissal is not, and had that point been put in chapter seven instead of chapter one hundred and sixteen I think it would have not only been better received by those of us that liked the books, but better for the story overall. If Ron had explained that of course the Snitch is a problem that any modern follower of the sport is familiar with, it would have underlined Harry's character flaws of arrogance and presumption, built the world in a clever and satisfying way, and avoided "dropping an anvil" (as the author's note says) on a character.

The Plot

The undeniable climax of Methods happens when Quirrell has been unmasked as Voldemort and gives Harry sixty seconds to surrender information prior to his death. Harry then kills the arrayed Death Eaters and incapacitates Voldemort, and everything after that is wrapping up loose threads. The climax of the work is then in chapters one hundred and thirteen and one hundred and fourteen.

Yet the plot of Methods is not about Quirrell as Voldemort fighting with Harry. Prior to chapter eighty-eight, Voldemort has no intentions of killing Harry. Voldemort's plan, as laid out in a language that doesn't allow lies, is to make Harry into the ruler of magical Britain. Harry's plan is to figure out how science works and revolutionize magical Britain. Dumbledore has two primary plans. The first is to trap Voldemort beyond time, which Dumbledore is unsuccessful at; this happens almost entirely off-screen. The second is to thread the needle of prophecy, which Dumbledore presumably has succeeded at when the novel ends; this also happens almost entirely off-screen, and the parts of it that we do see are incomprehensible.

Do you see the problem here? Prior to chapter eighty-eight, the plot hasn't actually begun. Harry and Voldemort share largely the same goals until that point, though they likely differ in how they would achieve them, and of course have obvious moral differences, but this is not what drives them into conflict in the climax - the thing that does that is a prophecy, not any aspect of their character.

The conflict between Voldemort and Dumbledore has a different problem, the biggest of which is that so much of the story is told through the eyes of someone who is somewhere between a pawn and a bystander. I think some of this issue stems from the authorial decision to keep so much of their plotting off-screen, which made their final confrontation into a complete let-down (and the root cause of that is the decision to play Quirrell's identity as a mystery rather than a source of dramatic tension).

From reading two-thirds of the fic (and frankly, quite a bit further than that) you might believe that the plot was going to be that Harry was going to learn the secrets of magic and revolutionize magical Britain, but neither of these things happen. The scientific exploration of magic is mostly dropped by the time of the Azkaban arc, reduced to mere guesses about how magic might work. The revolution of magical Britain is left as an uncertain thing. These promises set up in the beginning are left unfulfilled. (I've heard people say that there's no possible way that those could have fit into this work, but I disagree - see the next section on the pacing problems.)

Midway through, you might think that the plot was going to be Harry destroying Azkaban and defeat Death. There's even a prophecy to that effect. Yet this doesn't happen either; it's left as an exercise for the reader's imagination, another promise that's not delivered on.

Pacing Problems

The pacing problems of Methods are pretty easy to see just by looking at the chapter titles. There are far too many reactions and aftermaths. For every major event that happens, there are thousands of words expended on showing the reaction from every angle. There are relatively few plot points in the six hundred and sixty thousand words, especially when compared to canon - Methods covers Harry's first year in the same number of words that the canon books cover Harry's first four (almost five) years. Not all of that is due to the parts which might have been lifted straight from Yudkowsky's Sequences.

There are quite a few digressions which exist not to teach a lesson or advance plot or characterization, but because the author thought it would be interesting. He's often right, but the price of that is that the plot moves much more slowly than it otherwise would, and sometimes the conclusion of a chapter leaves me scratching my head asking myself whether anything meaningful had actually happened. There are numerous references, cameos for fans, and lengthy digressions into esoteric subjects. Sometimes these are interesting, and sometimes they aren't, with the difference often being how much you care about the topic in question. Sometimes it's the sort of foreshadowing that's only obvious in retrospect. Either way, the slow pace doesn't tend to be worth it.

Of special note are the "battle school" portions of the plot, which are a pastiche of Ender's Game, which do little to advance the plot. The battles all happen close to each other, and while they reach a natural conclusion in chapter thirty-five when the generals have tied and made their wishes before Christmas, but then there's another battle which happens afterwards. While this section was enjoyable in and of itself, it seems disconnected from the larger work - existing primarily because the author had always wanted to write a pastiche of Ender's Game rather than because it was integral to the other purposes that the work was trying to achieve. The lasting plot impact of these battles is small.

Realism and Narrative Satisfaction

I've heard many people say that Methods is realistic. I generally agree with this (with a few caveats). It is realistic for the enemy to be doing their own thing off-screen where the hero can't see it. I agree with this. However, this is not the same thing as being satisfying to the reader. A Deus Ex Machina or Diabolus Ex Machina is realistic, but it's also bad storytelling. The easiest fix for this is simply to show the reader things which are unknown to the protagonist, yet for all that Methods shows aftermaths from dozens of characters, and moves narrative focus from one character to another, it stalled on showing the perspective of Dumbledore and Quirrell so much that their final battle seemed to be set up and resolved in a single chapter, with all the tension and suspense deflating rather than reaching a climax.

I'll say the same for the over-use of a few tricks - mostly, the Time-Turner. Yes, it's realistic for a character to use a hammer every time there's a nail that needs to be pounded down. No, this is not satisfying for the reader. It would have taken little effort to contrive a circumstance where Harry lost his Time-Turner. Off-hand, I can think of a few places where it would be in character for the adults. But instead the Time-Turner is used for everything under the sun, even when these uses are repetitive, and something fantastical has turned boring. I don't think that realism and narrative satisfaction had to be at odds here. (I also find it suspect that there's a potion to lengthen the sleep cycle for people who have Time-Turners, but not one to shorten it. Surely people aren't granted their Time-Turners forever, especially if they're handed out to children, so if someone takes that potion are they just stuck with a lengthened sleep cycle even after handing the Time-Turner back? This is a minor quibble - I'm sure I could invent a just-so story for why the world looks like that.)

Final Verdict

I liked Methods, but I can't help but find it flawed in many ways, some of which seem nearly impossible to fix. It's a power fantasy in many ways, and the sort of power fantasy I enjoy, and yet my enjoyment of that is undercut by a purposefully unlikable main character who never gets his comeuppance.

Addendum: Hermione and Draco

I sort of forgot about Hermione and Draco when I wrote all of the above. Sorry Hermione and Draco.

For secondary characters, I actually thought that Hermione and Draco went through much better character growth arcs than Harry. Draco's is cut a bit short, and it's not really clear where he's ending at given all the turmoil in his life, but there's some hope that he's gone from spoiled son of a noble to enlightened reformer. Hermione traces a fairly organic path from uptight bookworm, to would-be hero, to disillusioned pariah, to reborn superhero (okay, so that last part isn't so organic to the character).

I actually liked the S.P.H.E.W. arc, but at the same time felt like it didn't do enough to further the plot. Like the battle school chapters, it felt like a different fic that I'd been dropped into for a short while - like a spinoff television show. And it never really felt like too much would be lost from cutting the whole thing entirely, especially given Hermione's ignoble end. At the very least, it's one of the big contributors to the pacing problems. I think it would have worked really well as its own, parallel fic, in an Ender's Shadow to Ender's Game sort of way. Or perhaps it would have worked better if it had been interleaved with the other chapters - an ongoing arc tracing its course alongside others rather than a single enormous clump of text on this one subject.

Friday, February 27, 2015

The Metropolitan Man: Post Mortem

About a year ago I wrote a novel-length fanfic about Superman called The Metropolitan Man. This blog post will make absolutely no sense to you unless you've read that first, and, as per the title of this blog, is probably something that you will find boring. At any rate, since this is currently my longest (completed) work of fiction, I thought that picking it apart to see what works and what doesn't might be helpful for future writing projects.

What Worked

  • I went with a historical setting - the 1930s - and I think it did a lot to ground the story in the real world, which is a hard thing to do when you have supernatural elements.
  • For the most part, characterization went well. When writing fanfic there's always a pre-existing characterization, and that means that you have to write along the lines of what other people have in mind. It's a narrow tightrope to walk, but I think that I did pretty well.
  • For the most part, the plot went well - the plot was fairly tight, one event flowed to the next, and the actions and reactions of the characters made sense within what had been established for them.
  • There are a lot of small scenes that I like, little details that make the city and world feel more alive. Part of this is that grounding in the setting.
  • Lex Luthor seems like he's smart - much smarter than I am. I used a lot of tricks to accomplish that, and I'm glad that I didn't get too many people telling me that Lex was being stupid.

What Didn't Work

  • When I said "for the most part characterization went well" there was one notable exception - Lois Lane. Her purpose in the story was to provide a contrast to the two principle characters, as well as a B-plot of her figuring out Superman's secret identity. For the most part, I think she accomplished that, but I tried to keep a realistic shifting of emotions for her which didn't come off well. In my own personal life, especially when there's a lot going on, my moods shift, and I have personal revelations that get canceled out and the whole thing is confusing. But while I might find that to be a realistic reaction, I don't think it's terribly good fiction. It's better to have a characterization that you hammer home over and over with slow and gentle shifts than to try to make someone as weird and complex as a real person - especially when they're the only one that's like that among your principle cast.
  • There's a scene where one of Lex's henchmen screws up and kills Ma Kent, in part because of the unforeseen event of the Dust Bowl which interferes with plans that had been put into place. He kills her mostly on accident. This is the first major thing that goes wrong for Lex, and the first time I wrote it, I think I flubbed it. I fixed it somewhat before the next chapter went out, but I still think that it's a weak point of the story - something that feels just a bit forced or unnatural.
  • The ending is the weakest point in the whole piece. I think that building up to the final confrontation went well, and the final confrontation itself went well, but the resolution came too suddenly and I don't think was foreshadowed enough. I've thought a lot about how I would change the ending, though I have no intention of doing so (mostly because I think that an author is better of not endlessly rewriting, especially when the work has already reached its audience). I've gotten comments that the ending is sad, which I'm totally fine with and intended, as I see the story as a tragedy. I've gotten comments that the ending never delivers on a promise of reconstruction, but I don't feel like I ever made that promise. But I've had people say that the ending felt too much like luck, and on reading it, I somewhat agree with them. It's a planned for sort of luck, the luck that comes from having lots of plots in play and plans within plans, but it's luck all the same. And at the same time, I didn't want for Lex to win simply because everything went his way. I think that if I had to do it over, I would include a scene from Superman's perspective, to show that he had options and took the one that was the most "good", and that's what got him killed. It's a somewhat cynical downer ending, but I think it would massage out some of the luck, and it's kind of a downer ending either way.
  • The prose was a little too lacking in emotion. A lot of the dialog is dry and descriptive, or focused on details. There are a few key emotional scenes that I thought were powerful, but it could have used more emotion in the day-to-day. This I'm less sure on - I think it might just be how I write, and one of those things that I get criticized on for future projects, but writing emotionally is one of those things I find difficult unless I can convince myself to really feel it.
Overall, among those people who have bothered to rate it, it has four stars. I think that's more than fair - my own rating would probably be more towards three and a half, though that might be because the flaws are all quite apparent to me, or because I've spent more time looking at the failures than the successes.

(The Lack of) Methodological Problems in High-Rise Syndrome Cat Studies

There's a popular fact that gets thrown around a lot: cats can survive falling from great heights, and actually survive more often when dropped from a greater height (above seven stories).

There's also a popular rebuttal to this: the study which came to this conclusion was only looking at cats that came into the emergency room, and has what's called a survivorship bias. What they mean is that it could be that there are (hypothetically) a whole bunch of cats who fall from above seven stories which die before ever getting to the emergency room. The study therefore has a methodological flaw, and no valid conclusions can be drawn from it.

Here's the rebuttal to the rebuttal: that's a misreading of the study. The study in question is almost always the 1987 Whitney and Mehlhaff study, which went viral pretty quickly and was then just repeated ad nauseum since forever. What the study actually showed was that the severity of injuries to cats was curvilinear with height - injuries increase as you add height, until a certain point where they stop increasing and start decreasing with height. Specifically, limb fractures begin to decrease while thoracic injuries remain more or less constant (or with a mild increase).

So when a cat falls from a large height, we can divide them up into three groups.
  • Dead cats, which don't go to the vet
  • Injured cats, which go to the vet
  • Uninjured cats, which don't go to the vet
In any study of cats coming into the emergency room, we're only going to be seeing cats in the middle group. We can further divide the injured cats that we're seeing into three groups.
  • Severely injured cats
  • Moderately injured cats
  • Mildly injured cats
Now, if mortality were increasing linearly with height, we would also expect the severity of injuries to increase with height. This is not what we see happening. What we instead see happening is the severity of injuries decreases with height. If you want to maintain that there's a survivorship bias going on here, that bias needs to explain why we see fewer injuries among injured cats.

I've had this debate online a large number of times. In fact, that's why I'm writing this blog post. The generalized argument among the counter-study people is this: most of the cats that fell from high floors simply died. When I ask what plausible hypothesis explains why those that don't die don't see increased injuries, I usually hear back an answer that doesn't make sense. It's usually some variant on "they were lucky", which as you can imagine, I find scientifically lacking.