The Zero Effect of Archival Research

Daryl Zero Paper Headache

Now that I’m spending time doing research for my thesis at the Library of Congress Manuscript Division and the National Archive, I’m really wishing that the fictional manual / memoir that serves as Daryl Zero’s voice-over in The Zero Effect were a real book that I could consult:

Now, a few words on looking for things. When you look for something specific your chances of finding it are very bad because of all things in the world, you only want one of them. When you look for anything at all your chances of finding it are very good because of all the things in the world you’re sure to find some of them.

Daryl Zero is for me a guru on par with Yoda, Keisuke Miyagi and Ogami Itto.

Space Rendezvous

Okay, so my posts on Apollo 11 have been a little Stanley Kubrick-esque. In the film 2001 (Wikipedia | IMDB), the proto-human throws a bone into the air where it is suddenly replaced by ship engaged in an elaborate docking maneuver with a rotating space station, set to Strauss’s waltz, Blue Danube. It is one of my favorite scenes in all of cinematography because of its purely aural-visual implication of the technological continuum from the first tool through the most unrecognizably advanced, and then, in the dance of space ships, the humaneness of otherwise inhuman machines .

On space maneuvers being like dance, here’s the official NASA Apollo 11 Spacecraft Commentary from the radio broadcast of the mission explaining what’s going to happen during the docking maneuvers of Command Module Columbia and Lunar Module Eagle (APOLLO 11 MISSION COMMENTARY, NASA, Manned Spaceflight Center, Houston, TX, 7/21/69 CDT 13:40, GET 125:08, 413/1, p. 466):

In all of these maneuvers Mike Collins aboard Columbia is spring loaded to do what is called a mirror image maneuver approximately a minute after the Eagle is scheduled to make its maneuver, and if for some reason Eagle can not make the maneuver, Collins would do the exact same maneuver only in reverse so that Columbia would in effect begin a CSM active rendezvous with Eagle.

The dance analogy seems apt here because, like Ginger Rogers with Fred Astaire, Michael Collins had to do everything that Buzz Aldrin did, only backwards and in a command module.

The Napoleon Dynamite Problem

After casing Ben Goertzel’s blog today, the point that I find myself really chewing on is this one (“The Increasing Value of Peculiar Intelligence,” The Multiverse According to Ben, 26 November 2008):

What occurs to me is that in a transparent society, there is massive economic value attached to peculiar intelligence. This is because if everyone can see everything else, the best way to gain advantage is to have something that nobody can understand even if they see it. And it’s quite possible that, even if they know that’s your explicit strategy, others can’t really do anything to thwart it.

Yes, a transparent society could decide to outlaw inscrutability. But this would have terrible consequences, because nearly all radical advances are initially inscrutable. Inscrutability is dangerous. But it’s also, almost by definition, the only path to radical growth.

I argued in a recent blog post [“The Inevitable Increase of Irrationality,” 25 November 2008] that part of the cause of the recent financial crisis is the development of financial instruments so complex that they are inscrutable to nearly everyone — so that even if banks play by the rules and operate transparently, they can still trick shareholders (and journalists) because these people can’t understand what they see!

But it seems that this recent issue with banks is just a preliminary glimmering of what’s to come.

Inscrutability, peculiarity, the idiosyncratic are already creeping in. Mr. Goertze is right to point to the rise of the quants and mathematical finance as an example. The one that comes to mind for me is the Napoleon Dynamite problem.

NetFlix has announced a million dollar prize for anyone who can improve the precision of its recommendation engine by ten percent. The New York Times Magazine and NPR’s On the Media both did stories about it back in November (Thompson, Clive, “If You Liked This, You’re Sure to Love That,” 23 November 2008; Gladstone, Brooke, “Knowing Me, Knowing You,” 21 November 2008). It turns out that improving the quality of this sort of singular value decomposition algorithm is geometric in difficulty. Most movies are easy to predict whether someone will like or dislike them, but a small number of odd movies thwart the algorithm. Chief among them is Napoleon Dynamite. For the research group profiled in The New York Times piece, Napoleon Dynamite was responsible for a whopping fifteen percent of all recommendation errors. There is no telling on the basis of people’s past movie rating history whether or not they’ll like this movie.

But the Napoleon Dynamite problem isn’t a solitary anomaly, but rather the paradigm of a trend. What we have is a Hollywood focused on these monster, expensive productions. Increasingly the movies that Hollywood makes are global products, with as much revenue coming from abroad as from the U.S. audience, so Hollywood is careful to strip its movies of any dialogue, humor or situations which are culturally nuanced and might not translate well. So the plot and dialog that we get in big Hollywood movies today is only the most broadly recognized and basic cultural tropes. Also, Hollywood has jacked the price of a movie up to the point where viewers now almost universally make a theatre-rental division: big special effects movies that they want to see in the theatres, and the dramas for which screen size isn’t a factor. It is a division with a positive feedback loop in that movie makers are aware of it and now shape their product offerings around it.

For a particularly depressing take on this, give a listen to Malcolm Gladwell’s 2006 New Yorker Festival talk on the use of machines to produce blockbuster scripts. At the same time that institutions like NetFlix are using computers to match customers to movies with increasing efficiency on the consumer end, Hollywood is using computers to make films increasingly easy to pigeonhole and match to demographics on the production side. It’s post-Fordist cultural production perfected. Soon we will be able to take the human out of the equation and the entertainment industry will just garnish out wages.

But there is — as is always the case — a countervailing motion. Just as Hollywood productions become increasingly trite and formulaic, there is the rise of these wildly bizarre and idiosyncratic films like The Zero Effect, Adaptation, Eternal Sunshine of the Spotless Mind, Lost in Translation, The Royal Tenenbaums, I Huckabees, Burn After Reading and so on. There is this sort of shadow Hollywood with it’s own set of stars and directors branding the alt-film genera: Wes Anderson, Charlie Kaufman, the Coen brothers, Catherine Keener, John Malkovich, William H. Macy, Frances McDormand. I would be remiss if I didn’t mention Steve Buscemi here.

What we have is a hollowing out of the middle. Along a spectrum, films range from obscurantia to formulaic. In the past, most movies probably fell in some broad middle: accessible, but unique. And most movie watchers probably fell there too. But increasingly movies and the movie-watching audience is being polarized into the genera constellations at one end and the difficult to categorize peculiarities at the other. Notice that the ambiguity of suspense has been replaced by the spectacle of gore in horror; that the sort of romantic comedy for which Drew Barrymore was designed and built has completely driven the older adult romantic drama to extinction. Similarly, the sort of accessible quirky, artiness represented by Woody Allen has moved much further down the spectrum of the idiosyncratic. The people who didn’t like Woody Allen are utterly baffled by Wes Anderson.

To generalize: hitherto we have been a normal distribution society. The majority of people fall into the broad middle and are closely related. But increasingly we are on the way toward a parabolic, or inverse normal distribution society, where the preponderance resides at the antipodes and people are separated by wide gulfs. This is true across the cultural spectrum, whether it’s politics, religion, the professions and so on. In the United States it is almost happening physically with the costal regions swelling as the center of the country is abandoned to satellite guided tractors and migrant labor. Some might call this the condition of postmodernity, some might call it the dissolution of Western Civilization.

The Supernovae in Your Coffee Cup

The Supernovae in Your Coffee Cup

I loved the film π. I consider it a hugely flawed film, but what I loved about it was the way that it worked in subtle allusions to the underlying concepts motivating the film. The main character walked through a park and they point the camera skyward to show the denude winter branches of the trees, an example of fractal symmetry. One of the images that they showed a number of times throughout the film was that of a cup of coffee. Whenever someone ended up in a diner, we got a tight-in shot of them dumping the cream into their coffee and the blooms of turbulent fluid redounding from the depths. It’s a perfect example of turbulence, a phenomenon that utterly defies computation. Since π I’ve never looked at a cup of coffee the same. Every time I pour cream into my coffee it’s a little ritual where for just a second I consider the boundlessness complexity of the world, as close as the cup in my hand.

I was amused to see a recent article in New Scientist invoke the image of the cup of coffee in reference to the problem of turbulent fluids in supernovae (Clark, Stuart, “How to Make Yourself a Star,” vol. 200, no. 2679, 25 October 2008, pp. 38-41):

As the dense inner material is flung through the less dense outer layers of a star, it creates turbulence and mixes everything up. Traditional computer simulations do not model turbulence well.

“Our theoretical understanding of turbulence is incomplete,” says astrophysicist Alexei Khokhlov of the University of Chicago. In other words, you cannot write down a set of equations describing the state of a turbulent system at any given time and then use them to predict what it will look like next. Instead, you have to employ a brute-force approach, using sheer computer muscle.

To seen the scale of this problem, take your morning cup of coffee and stir in some milk. You are using turbulence to mix the two fluids. To determine how they mix, physicists mentally split the cup into boxes and assign numbers to represent the properties inside each box, such as the temperature and density of the fluid. A computer can then calculate how each box interacts with its neighbors during one brief instant of time and then re-evaluate those numbers. Once it has done this for every box, it starts again for the next slice of time and so on.

To do this massive computation perfectly, each box should be tiny and contain just one fluid particle, but before you can get anywhere near this sort of precision, the numbers become mind-bogglingly large. Scientists talk of degrees of freedom as a measure of both the numbers of particles in a system and the number of ways each particle can interact with those around it. A single cup of coffee possesses a staggering 1040 degrees of freedom — far more than you can model on today’s computers. “Maybe in 10 years we will be able to fully model a cup of coffee,” says Khokhlov.

Until then the computation will always be approximate, and thus prone to errors, because small-scale physical interactions are not being taken into account. … If it is going to take 10 years to fully model a cup of coffee, how long until we can model an entire star?

“Never,” Khokhlov says. “Not until someone comes up with a cleaver theory that does not depend on what is happening on the small scale.” The only hope is to continue to investigate turbulence to learn how to better approximate its behavior.

The Order of the Phoenix

Warning: Spoiler Alert.

Following on John’s post (“Thrown Out of the Man Club,” 12 July 2007) about the detrimental impacts of Harry Potter on his manly virtue, I went to see Harry Potter and the Order of the Phoenix today. I was initially hesitant because I thought that Harry Potter and the Goblet of Fire was such a weak showing. However, under the influence of S., her friend and her friend’s six year old son, I went to see the latest and totally loved it. S. has been trying to get me to read the books and Order of the Phoenix has come about as close as anything to convincing me that I should. I may have to read at least Harry Potter and the Deathly Hallows, least S.’s mood upon completion give away the big secret.

The Archetype. I think the thing that I most liked about it is that the story has really flushed out all of the traditional elements of the hero narrative. We find a world with dark shapings afoot, but denial and corruption on the part of officialdom. A secret society formed to confront the villainous in its previous incantation has been decimated and scattered. Remnants try to pull together to prepare for the coming conflict (they are called The Order of the Phoenix in this version of the story in case you needed a few more clues), but that they are only a tiny band of resistance is okay because they are in possession of a powerful secret, that a prophesy of a budding power of good is about to bloom. And finally, of course, an agent of destiny who, at the key moment comes of age to fulfill said prophesy. At first people are in doubt as to the true nature of this individual, but eventually events bring everyone around. This agent of destiny is shepherded through the beginning of his trials by members of the old guard, but eventually reaches that point beyond which his teachers cannot help him, after which he must find his own way. But he is not alone and pulls together a small band of the new generation to fulfill the work of the old. This time the Scooby Gang is filled out into a full crime-fighting team, even if the original trio had to be rounded out by a couple of unnamed in the background like a Star Trek away team. Don’t wear red on a sleepover at Harry Potter’s house.

The True Battle is Within. Throughout the film the focus is on the mind. In the climactic battle between Dumbledore and He Who Cannot Be Named, Dumbledore may rout corporeal Voldemort, but the real battle takes place between the good and evil inclinations in the mind of Harry Potter, portrayed too briefly in a series of images from Harry’s life stitched together from the previous films, shifting from fear and loss to happy memories of Hermione and Ron.

When Harry is teaching the other members Dumbledore’s Army how to produce the Patronus Charm he tells them that they must remain focused on their happiest thought throughout, no matter how frightened they might be. Snape insists that Harry “control your mind.”

When Harry has downed the death eater Bellatrix and ponders what to do with the creature that has just killed his godfather, Voldemort appears behind him and goads Harry, “You have to mean it, Harry. You know the spell. She killed him. She deserves it.” Whether killing a death eater or producing the Patronus Charm, it is the state of mind that is most consequential. It was all too reminiscent of the temptation of Luke Skywalker in the Emperor’s chamber in Return of the Jedi: “Take your weapon. Strike me down with all of your hatred and your journey towards the dark side will be complete.” In each instance, it is not the violent act itself that leads one to the dark side, but, again, the state of mind one is in when committing the act, whether it is done dispassionately, or filled with passionate anger.

The National Security States. There is apparently something of a debate on the bloggosphere over whether Michael Bay’s Transformers is about the Bush administration and the Iraq war (the only bubblings-up that I have seen are via Matthew Yglesias, “ Michael Bay and the National Security State,” 11 July 2007; “Strong Reading,” 12 July 2007). Partisans really getting worked up over this issue should go see The Order of the Phoenix.

Harry Potter is slightly confused in its symbol system. In Harry Potter it is the good guys who know that forces are gathering and war inevitable while it’s the perfidious Minister of Magic, Cornelius Fudge, who is the one that’s in denial. Meanwhile Laura Bush cum Professor of Defense Against Dark Arts, Dolores Umbridge, seems to be running the Hogwarts Department of Homeland Security at the same time she is administering No Child-Wizard Left Behind. She replaces the previously experiential curriculum of Defense Against the Dark Arts with pure book-base memorization aimed at passing the O.W.L. exams. She cavalierly brandishes accusations of disloyalty. At one point Ms. Umbridge is warned that a particular method or interrogation that she is about to employ on Harry Potter would constitute torture and according to the Ministry of Magic is illegal. Ms. Umbridge slaps the face of a photograph of the Minister of Magic face-down on her desk: “What the Ministry does not know will not hurt it.”

As for the things that I didn’t like, first I want Harry and Hermione to be getting it on. But no, the bookish overachieving, running with the wolfs woman is always shuffled off with the red-headed step-child. Meanwhile Harry is hooking up with the moody but insubstantial Asian hottie. Second, while the film partially embraces contingency it ends on a note too naive not to get my hackles up, even in fantasy. In the after-action debrief, the gang is all gathered together and Harry Potter reassures them that they will defeat Voldemort because, “We’ve got something he doesn’t have. We’ve got something worth fighting for.” It’s not one of those ridiculous inspirational pep talks that one might expect from, say, a Roland Emmerich film — if anything the virtues of the Harry Potter films is the degree to which the moral uplift is quiet and meek — still, if only. History is littered with the decent and worthy laid low by the stupid and cruel, but powerful. The outcome is not ordained.