nicely posed. and a good suggestion to read last year's PDF. it's here: http://www.futureofthebook.org/sophie/Sophieintro.pdf

How can we justify Sophie's existence? This is the problem I keep coming up against; Sophie's history has been one of repeatedly blown deadlines and excuses about what a complicated project this is. It is a complicated project, and there are plenty of reasons why it's taking longer than it should. But rather than continue to parcel out time for more development, I think it's important to look at why we're doing the project.

Is there still a need for Sophie? One major problem is that Sophie doesn't match our original expectations. We've compounded the problem by changing our expectations of what Sophie should do, based in no small part in what's gone on in the rest of the world while Sophie is being constructed. In the past few years, the Web has grown my leaps and bounds: certain types of complexity that were unavailable when Sophie's prospectus were written are now commonplace. The Institute's other projects have explored this in depth; when asked what we do, the potential of Sophie hovers behind everything, but it's the other things that we show and get excited about. The concrete is easier to deal with than the abstract, and Sophie still works better as an abstract idea.

Most recently: we've been working on the CommentPress projects, which seem analogous to Sophie in certain ways. We started Sophie with the idea that it would be difficult to make Sophie-like projects on the web; I think this is still true. Ben estimates that we spent about 12 man-hours on the technical end of the Bush speech, having already developed most of the interface already. That's a not insignificant amount of time to put into a project. The New York Times put up their own annotated Bush speech (you might need TimesSelect); their version isn't nearly as complicated as ours, but it's still a fairly robust presentation, of the sort we would have been extremely excited about a couple years ago. But the Times has a whole online design department to handle this sort of thing.

I think this remains a pervasive problem with online work: it is much more time-intensive than it should be. The Web isn't as much of a tabula rasa as it mythologizes itself into being; it's much, much easier for big organizations with dedicated design departments to put up major projects that look good and interest people. There's a cultural imbalance here: the Institute is able to commit time and people to projects like this, but most of the people we're interested in doing projects with wouldn't be. Mitchell Stephens & Ken Wark are decent examples of this: on their own, they wouldn't have gotten anywhere near the amount of attention that they've received working with us. The best they might be able to do is something like Robert Frenay's Pulse, which still grinds along; his website was set up by a PR company as PR. It feels like it and doesn't work. It's not quite an issue of our style vs. the corporate style, though I'm quite convinced that we're better; it's more a question of who gets to talk like this. The New York Times will always be able to do exciting things on the web by virtue of having a lot of capital to throw around; small operations like Lapham's Quarterly should also have that opportunity and, in today's world, don't.

Will Sophie fix that? I don't know. Sophie's not a magic bullet. Complex projects are always going to be complex projects: for something like the Iraq Study Group report, there will always be the editorial problem of recruiting participants and dealing with their concerns, no matter how good your software. The problem, as I see it, remains the bottleneck of technology (which is, of course, a bottleneck that has always existed, albeit in different forms). Can Sophie help with this? If: 1) Sophie is easy to set up into complicated configurations, and 2) Sophie is easy to use for end users. Both of these are currently issue with Sophie because of the state it's in right now; both seem to be surmountable problems, but my sense is that it's going to be some time before we can start addressing these as issues. There are still too many obvious kinks that need to be worked out.

More later.

lovely summary dan. question for everyone else. what do you need to know about Sophie (any aspect) in order to be able to weigh in here . . . ?

When I was in the Chicago area over Christmas, I went to speak with some of the people that are doing work with Sophie (one PhD candidate, and one class of students doing new media/film/animation). Those users were really excited about being able to use Sophie, even in the messy state it's in today. They weren't terribly excited with all the bugs and failures, but there was something in there that they felt would not only serve them well, but make things better than they could make in other software. Probably based on the idea that "it's easier in Sophie than XXXXXXX." So it is absolutely imperative that Sophie be easier than XXXXXX. We just need to define what XXXXXX is.

To solidify what it is that people want to use Sophie for: currently they just want to mix multimedia and text in smart ways. The web allows them to do this, but because the web is made up of all these different protocols and plugins, it will never be able to integrate things as well as Sophie can. Flash can do it too, it's just that Flash is moving further in the direction of professional development. Still, it's useful to keep an eye out for what they do in terms of making certain actions easier for novice users: putting video up on the web, for example. The one killer thing that Sophie has on Flash is price. That's always going to go in our favor.

Ok, so these are the common arguments that are regularly raised and dispatched. At some point the argument that the web can't do what Sophie can will disappear. I think that window is about two years. If Sophie can become viable and get out in that time, then I think we have a shot to make it into something that is successful within that next year. If we do try, we should focus on making the multimedia assembly (with all the attendant complexity of timelines, linking commands, and syncing) work first, text second. Annotating and reading third. Complex embedding/linking last.

I think many of my concerns with Sophie are more abstract than concrete, which might not be particularly helpful. One broader area that I've been concerned about recently is perhaps an existential problem about design: whether change can be dictated from above. The necessary caveat: this is a holdover from my end-of-the-year thinking, and by the end of the year, I'm wrestling with big things that really shouldn't be wrestled with.

What we're setting out to do with Sophie is, in a broad sense, to change how discourse happens online. We're proposing to do this in a very top-down fashion: give the people Sophie and they will change the way they do things. This seems disingenuous, and perhaps historically naíve. Did Gutenberg set out to change discourse? He had a good idea, certainly, but I can't imagine he understood the ramifications of what he was doing. Does the universe work this way? I'm not sure: certainly my personal view gives more credence to mistake and chance (the preceding sentence should be read in the tone of Werner Herzog's narration of Grizzly Man and with the understanding that I was, once upon a time, an evolutionary biology major) which is why this isn't a tremendously useful subject for me to be unpacking.

But questions of teleology aside: when we look at the web, we realize that it's not really a single thing. It's a lot of different technologies tangled together. It's an enormous overgrown garden: at various points gardeners have thrown seeds over the fence in the hopes that something useful would grow, and a lot of different things have grown. But not generally in the way anyone could possibly have predicted. This is what's fantastic about the web; it's also what makes it deeply frustrating.

To further overburden a broken metaphor: what we're attempting to do with Sophie is to construct a Japanese rock garden, something carefully controlled and designed, which is about the furthest thing from the web that can be imagined. Both sorts of gardens are gardens. But it strikes me that the strategy we're taking with Sophie is ignoring the vitality of the web. Is this what we should be doing? It's true that there are enormous problems with the web, but abnegating it entirely is a serious step, and one that needs consideration.

in answer to Bob's question (and ever the pragmatist), i'd like to know how problematic it is that Sophie is written in Smalltalk. as Dan points out, tools often end up being used in ways their creators never anticipated. Sophie will only thrive in the moving-target universe of the web if a growing body of users tweak and improve and put it to uses that have yet to occur to us. which they will if Sophie grabs their interest out of the gate (making this a bit of a chicken-and-egg question), but how big is the Smalltalk community and how responsive can we expect it to be?