Category Archives: mashup

interface culture

Omnisio, a new Y Combinator startup, lets people grab clips from the Web and mash them up. Users can integrate video with slide presentations, and enable time-sensitive commenting in little popup bubbles layered on the video.
MediaCommons was founded partly to find a way of conducting media studies discussions at a pace more congruent with changes in the media landscape. It’s tempting to see this as part of that same narrative: crowdsourcing media commentary for the ADHD generation. For me, though, it evokes a question that Kate Pullinger raised during the research Chris and I conducted for the Arts Council. Namely: are we seeing an ineluctable decline of text on the Web? Are writers becoming multi-skilled media assemblers, masher-uppers, creators of Slideshares and videocasts and the rest? And if so, is this a bad thing?
I’ve been re-reading In The Beginning Was The Command Line, a 1999 meditation by Neal Stephenson on the paradigm shift from command line to GUI interactions in computer use. In a discussion on Disneyland, he draws a parallel between ‘Disneyfication’ and the shift from command line to GUI paradigm, and thence to an entire approach to culture:

Why are we rejecting explicit word-based interfaces, and embracing graphical or sensorial ones–a trend that accounts for the success of both Microsoft and Disney?
Part of it is simply that the world is very complicated now–much more complicated than the hunter-gatherer world that our brains evolved to cope with–and we simply can’t handle all of the details. We have to delegate. We have no choice but to trust some nameless artist at Disney or programmer at Apple or Microsoft to make a few choices for us, close off some options, and give us a conveniently packaged executive summary.
But more importantly, it comes out of the fact that, during this century, intellectualism failed, and everyone knows it. In places like Russia and Germany, the common people agreed to loosen their grip on traditional folkways, mores, and religion, and let the intellectuals run with the ball, and they screwed everything up and turned the century into an abbatoir. Those wordy intellectuals used to be merely tedious; now they seem kind of dangerous as well.
We Americans are the only ones who didn’t get creamed at some point during all of this. We are free and prosperous because we have inherited political and values systems fabricated by a particular set of eighteenth-century intellectuals who happened to get it right. But we have lost touch with those intellectuals, and with anything like intellectualism, even to the point of not reading books any more, though we are literate. We seem much more comfortable with propagating those values to future generations nonverbally, through a process of being steeped in media.

So this culture, steeped in media, emerges from intellectualism and arrives somewhere quite different. Stephenson goes on to discus the extent to which word processing programs complicate the assumed immutability of the written word, whether through system crashes, changing formats or other technical problems:

The ink stains the paper, the chisel cuts the stone, the stylus marks the clay, and something has irrevocably happened (my brother-in-law is a theologian who reads 3250-year-old cuneiform tablets–he can recognize the handwriting of particular scribes, and identify them by name). But word-processing software–particularly the sort that employs special, complex file formats–has the eldritch power to unwrite things. A small change in file formats, or a few twiddled bits, and months’ or years’ literary output can cease to exist.

For Stephenson, a skilled programmer as well as a writer, the solution is to dive into FLOSS tools, to become adept enough at the source code to escape reliance on GUIs. But what about those who do not? This is the deep anxiety that underpins the Flash-is-evil debate that pops up now and again in discussions of YouTube: when you can’t ‘View Source’ any more, how are you supposed to learn? Mashup applications like Microsoft’s Popfly give me the same nervous feeling of wielding tools that I don’t – and will never – understand.
And it’s central to the question confronting us, as the Web shifts steadily away from simple markup and largely textual interactions, toward multifaceted mashups and visual media that relegate the written word to a medium layered over the top – almost an afterthought. Stephenson is ambivalent about the pros and cons of ‘interface culture’: “perhaps the goal of all this is to make us feckless so we won’t nuke each other”, he says, but ten years on, deep in the War on Terror, it’s clear that hypermediation hasn’t erased the need for bombs so much as added webcams to their explosive noses so we can cheer along. And despite my own streak of techno-meritocracy (‘if they’ve voted it to the top then dammit, it’s the best’) I have to admit to wincing at the idea that intellectualism is so thoroughly a thing of the past.
This was meant to be a short post about how exciting it was to be able to blend video with commentary, and how promising this was for new kinds of literacy. But then I watched this anthology of Steve Ballmer videos, currently one of the most popular on the Omnisio site, and (once I stopped laughing) started thinking about the commentary over the top. What it’s for (mostly heckling), what it achieves, and how it relates to – say – the kind of skill that can produce an essay on the cultural ramifications of computer software paradigms. And it’s turned into a speculation about whether, as a writer, I’m on the verge of becoming obsolete, or at least in need of serious retraining. I don’t want this to lapse into the well-worn trope that conflates literacy with moral and civic value – but I’m unnerved by the notion of a fully post-literate world, and by the Flash applications and APIs that inhabit it.

alex itin netrospectives


Our dear friend and artist in residence Alex Itin has been getting noticed of late. Yesterday he was profiled on The Daily Reel, a popular site that curates quality video from around the web and frequently features his work. Other interesting video sites have also been a-knockin’.
An even better meditation on Alex’s work is this comment posted by Sol Gaitan last month in response to his popular piece, “I Made Pictures of Making a Picture of Everyone Who Might Be Looking At These Pictures of Everyone” (a bona fide blockbuster on Vimeo). I’ve reproduced it in full:

Robert Rauschenberg’s fabulous exhibition of 43 transfer drawings at Jonathan O’Hara Gallery produce that feeling, in retrospective, of seeing something that is going to mean a lot in the future. And they did. Executed in the 60’s they were the precursors, as well as the result, of appropriation. Duchamp and Picasso are two obvious examples that come to mind when one thinks of the origins of appropriation, today we prefer the term “mash-up.” The exciting thing about Rauschenberg is his extraordinary use of the quotidian to create highly manipulated works that elude classification. As his combines include and exclude us, the transfer drawings leave us with a feeling of immediacy and at the same time of blurry memories.
Alex Itin’s play with appropriation produces the same feeling. He is doing something that is going to mean a lot. His investigations of the uses of technology to produce an art as dynamic as its medium, is evident in these “horizontal scrolls.” The medium is limiting, so Alex is searching for a way to blog that defies its verticality, very much along the lines of the Institute, where his art resides. There are extraordinarily beautiful artist homepages on the Internet, but Alex’s redefinition of the blog as a place of encounter, intentional or by chance, a place of fusion where he produces and manipulates his mash-ups is unprecedented. With this “horizontal scroll” he moves a very important step forward. He addresses us in all our anonymity while creating the piece in front of our very individual eyes. His use of the blog as a way of communication through live creation makes it burst along its seams.
The way Rauschenberg’s transfer drawings fall on the paper seems aleatory, but it responds to the limitations he found in both collage and monotype before he started his silkscreen explorations: “I felt I had to find a way to use collage in drawing to incorporate my own way of working on that intimate scale,” (as cited on the show’s catalogue by Lewis Kachur from art historian Roni Feinstein’s dissertation, NYU 1990.) The result are aerial collages of images transferred from newspaper that become a testimony of their times. The 60’s were charged times, and the newsprint chosen by Rauschenberg; the civil rights movement, Vietnam, the invasion of Czechoslovakia, astronauts, consumer products, address high modernism, while their heterogeneity alludes to the fragmentary condition of postmodernism. Alex’s depiction of those looking at his blog is highly charged in a similar manner; who are us anyway? What seems improvisation takes the form of social commentary, as he says on his blog:
who is my audience?… I think I’ll draw them and perform in front of the drawing. Today’s post also asks the question (more than most), which here is the real work of art? The drawing, or the film of the drawing, or the whole thing together on the blog, or what?
Rauschenberg’s masterful use of gouache, watercolor and ink washes lends coherence to the whole. It centers the viewers attention on image and text, fusing them. Their dynamism makes us think of the artist at work. Alex has the medium, and the shrewdness, to put both, process and work, in front of our eyes. The result is not the voyeuristic epiphany of seeing Pollock dripping paint on a canvas that would become the actual piece, here it is the artist at work, moving in precarious terrain, which IS the piece.
Rauschenberg’s pieces elude nomenclature, they are neither painting, nor collage, nor sculpture, they are thresholds to new forms of perception. Today’s challenge is to rethink the meaning of appropriation in a moment when capitalist commodity culture has become the determinant of our daily lives. To appropriate today is to expose the unresolved questions of a world shaped by the information era. Permanence is constantly challenged and the evolution of Alex Itin’s work on his blog shows this as clearly as it can be.

terrain as browsing mechanism

Ben’s post last week, book as terrain, about converting any image to an interactive map with hotspots contained a link to a blog which collects info about all sorts of google map mashups. Ben’s post was about using book pages as geographic jumping-off points. However, as i read the endlessly fascinating list of other sorts of mashups it occurred to me that in addition to “book as terrain” we could also look at the idea of “Google map mashups” as a genuinely new form of expression. As I read through the wonderfully annotated list I realized that they cover the full gamut of subjects you would find in a bookstore . . . . Fiction, Non-Fiction, Travel, History, Sports, Games, Religion, Personal Growth, and Crime.
It’s interesting to realize that as our experience moves relentlessly into the virtual domain, that geography, which in the past was firmly rooted in the “real,” increasingly becomes the mechanism for organizing our activiites in virtual space.

book as terrain

People have done all sorts of interesting things with Google maps, but this one I particularly like. Maplib lets you upload any image (the larger and higher res the better) into the Google map interface, turning the picture into a draggable, zoomable and annotatable terrain — a crude mashup tool that nonetheless suggests new spacial ways of navigating text.
I did a quick and dirty image mapping of W.H. Auden’s “Musee des Beaux Arts” onto Breughel’s “Landscape with the Fall of Icarus,” casting the shepherd as poet. Click the markers and then the details links to read the poem (hint: start with the shepherd).

As you can see, they give you the code to embed image maps on other sites. You can post comments on the individual markers right here on if:book, or if you go to the Maplib site itself you can add your own markers.
I quite like this one that someone uploaded of a southerly view of the Italian peninsula (unfortunately it seems to start larger images off-center):

And here’s an annotated Korean barbecue (yum):

if:book-back mountain: emergent deconstruction

emergent_deconstruction.jpg
It’s Oscar weekend, and everyone seems to be thinking and talking about movies, including myself. At the institute we often talk about the discourse afforded by changes in technology, and it seems to be apropos to take a look at new forms of discourse in area of movies. A month or so ago, I was sent the viral Internet link of the week. Someone made a parody of the Brokeback Mountain trailer by taking its soundtrack and tag lines and remixng them with scenes from the flight school action movie, Top Gun. Tom Cruise and Val Kilmer are recast as gay lovers, misunderstood in the world of air to air combat. The technique of remixing a new trailer first appeared in 2005, with clips from the Shining recut as a romantic comedy to hilarious effect. With spot-on voiceover and Peter Gabriel’s “Solsbury Hill” as music, it similarly circulated the Internet, while consuming office bandwidth. The first Brokeback parody is uncertain, however, it inspired the p2p/ mashup (although some purists question whether these trailers are true mashup) community to create dozens of trailers. Virginia Heffernan in the New York Times gives a very good overview of the phenomenon, including the depictions of Fight Club, Heat, Lord of the Rings, and Stars War as a gay love story.
Some spoofs work better than others. The more successful trailers establish the parallels between the loner hero archetype of film and the outsider qualities of gay life. For example, as noted by Heffernana, Brokeback Heat, with limited extra editing, transforms Al Pacino and Robert DeNiro from a detective and criminal into lovers, who wax philosophically on the intrinsic nature of their lives and their lack of desire to live another way. Or in Top Gun 2: Brokeback Squadron, Tom Cruise and Val Kilmer exist in their own hyper-masculine reality outside of the understanding of others, in particular their female romantic counterparts. In Back to the Future, the relationship of mentor and hero is reinterpreted as a cross generational romance. Lord of the Rings: Brokeback Mount Doom successfully captures the analogy between the perilous journey of the hero and the experience of the disenfranchised. Here, the quest of Sam and Frodo is inverted into the quest to find the love that dares not speak its name. The p2p/ mashup community had come to the same conclusion (to, at times, great comic effect) that the gay community arrived at long ago, that male bonding (and its Hollywood representation) has a homoerotic subtext.
The loner heros found in the the Brokeback Mountain remixes are of particular interest. Over time, the successful parodies deconstruct the Hollywood imagery of the hero, and subsequently distill the archetypes of cinema. This process of distillation identifies key elements of the male hero. The common traits of the hero being that he lies outside the mainstream, cannot fight his rebel “nature”, often uses the guidance of a mentor and must travel a perilous journey of self discovery all rise to the surface of these new media texts. The irony plays out, when their hyper-masculinity are juxtaposed next to identical references of the supposed taboo gay experience.
On the other hand, the Arrested Development version contains titles thanking the cast and producers of the cancelled series, clips of Jason Bateman’s television family suggesting his latent homosexuality, and the Brokeback Mountain theme music. The disparate pieces make less sense, rendering it ultimately less interesting as a whole. Likewise, Brokeback Ranger, a riff on Chuck Norris in the Walker, Texas Ranger television series, is a collection of clips of the Norris fighting and solving crimes, with the prerequisite music, and titles that describe Norris ironic superhuman abilities including dividing by zero. Again, the references are not of the hero archetype and the piece, although mildly humorous, has limited depth.
A potentially new form of discourse is being created, in which the archetypes of media text emerge from their repeated deconstruction and subsequent reconstruction. From these works, an understanding of the media text appears through an emergent deconstruction. In that, the individual efforts need not be conscious or even intended. Rather, the funniest and most compelling examples are the remixes which correctly identify and utilize the traditional conventions in the media text. Therefore, their success is directly correlated to their ability to correctly identify the archetype.
The users may not have prior knowledge of the ideas of the hero described by Carl Jung and Joseph Campbell’s The Hero with a Thousand Faces. Nor are they required to have read Umberto Eco’s deconstruction of James Bond, or Leslie Fiedler’s work on the homosexual subtext found in the novel. Further, each individual remix author does not need to set out to define the specific archetypes. What is most extraordinary is that their aggregate efforts gravitate towards the distilled archetype, in this case, the male bonding rituals of the hero in cinema. Some examples will miss the themes, which is inherent in all emergent systems. By the definition and nature of archetypes, the work that most resonate are the ones which most convincingly identify, reference, (and in this case, parody) the archetype. These analyses can be discovered by an individual, as Campbell, Eco, Jung and Fiedler did. Since their groundbreaking works, there is an abundance of deconstructing media text from the last fifty years. Here, the lack of intention, and the emergence of the archetypes through the aggregate is new. An important aspect of these aggregate analyses is that they could only come about through the wide availability of both access to the network and to digital video editing software.
At the institute, we expect that the dissemination of authoring tools and access to the network will lead to new forms of discourse and we look for occurrences of them. Emergent deconstruction is still in its early stages. I am excited by its prospects, but how far it can meaningfully grow is unclear. However, I do know that after watching thirty some versions of the Brokeback Mountain remixed trailers, I do not need to hear its moody theme music any more, but I suppose that is part of the process of emergent forms.