Sunday, October 24, 2010
Review - Dreaming in Code: Two Dozen Programmers, Three Years, 4,732 Bugs, and One Quest for Transcendent Software by Scott Rosenberg
Short review: Software is hard to make. Open source software that does revolutionary things is even harder.
To make software
The does almost everything
A quixotic task
Full review: Dreaming in Code is, at its most basic, the story of the development of Chandler, an ambitious software project that was intended to challenge Microsoft Outlook as a personal information manager. But the story is much more than that. The deeper story of Dreaming in Code has to do with why making software is so difficult, how computers are fundamentally different from humans, and why software programmers are probably the worst people to direct a software writing project, and at the same time how they are probably the only people who could do so.
The idea for Chandler was brilliant. That, it seems, was a large part of the problem with its development. Mitch Kapor, the man who made himself a multimillionaire by creating Lotus 1-2-3 in the 1980s, had a vision. The vision was of a personal information manager that would coordinate the user's e-mail, calendar, personal notes. It would also be flexible enough that the software would be able to interpret a variety of styles of input, and not tie the user to some predefined set of boxes to fill in. It would be shareable, allowing, for example, a husband and wife to each view and edit their spouse's calendar. And it would be both peer to peer, eliminating the need for a server to act as an intermediary, cross-platform, and open source to harness the power of the open source community to overcome the difficulties in implementation. In short, Mitch wanted the transcendent. Instead, it seems like he got disaster.
The story in the book itself is told by Scott Rosenberg, the managing editor of Salon.com magazine, who got access to the Chandler team at the outset of the project and followed them through much of their development process. When announced, Chandler was supposed to be "about two years away" from being ready for release. When Rosenberg ends the book, three years later, Chandler is still "about two years away" from being ready. The interesting part about the book is why this is so. The people Kapor hired to develop the software were not incompetent, in fact, most of the people involved with the project at the outset were scooped up from the wreckage of the dot.com crash, and were among the top people in the field of software development. The problems that develop seem to stem from too much ambition, too little focus, and, as Rosenberg discovers, the difficulties inherent in software development.
In the book, told in longer chapters broken up into short, topical chunks, and which jumps from event, to background, to history, and back again, one quickly figures out that having an idea for what you want software to do is no substitute for having an idea of how to implement it. Over and over again, the development team sits down in a conference room and fills a whiteboard with a plan for what they want done, seemingly without considering how they might do it. A programming language is chosen, plans are made, pieces of tertiary software are written without any real idea of how they can be fit together. And then in a process that happens seemingly ad nauseum, plans are redrawn, goals reset, and the program redesigned. Old personnel leave, sometimes moving on to bigger opportunities, others merely leave in frustration, and they are replaced by new faces, who try to add their own imprint onto the Chandler development, requiring even more redesign and reprogramming.
In some ways, the narrative of this book is a real world example of the various pitfalls of software development that have been outlined in previous works such as The Mythical Man-Month. Rosenberg doesn't just explain what things like "software time" are, he shows how the Chandler project was affected by this sort of creeping malaise, and why. Rosenberg also delves deeper into the history of the computer industry and software development to try to uncover the roots of these problems that seem to plague all software projects. While Chandler's development is shown to be a mess, Rosenberg illustrates the reasons why these sorts of problems are endemic to software projects, and the depressing reality that this situation is unlikely to change any time soon. Among the most interesting topics covered is how exactly you determine what is good productivity for a software programmer, since the classic measure of performance - lines of code written - is so clearly inadequate.
The main body of the story ends with Chandler unfinished and a truncated version that amounts to basically a computer controlled calendar released to the Open Source community. There is an added chapter in the paperback version that updates events concerning Chandler through 2007 (and which therefore does not include the fact that the project was effectively abandoned in 2008). In the end, the reader is left wondering how any software gets completed, which is kind of the point of Chandler's story. While most people who are in the industry will find nothing particularly new or surprising in the book, as an explanation and brief history of software development for an interested amateur, this is an excellent piece of journalism. For anyone who has wondered why software always seems to be late, full of bugs, and unfriendly to non technically inclined users, or is simply curious as to how software gets made, this book will be a fascinating expose of the true fragility of the information technology that we rely upon to run most of our modern world.
Scott Rosenberg Book Reviews A-Z Home