It takes some pretty amazing technology to stick a pipe a mile beneath the ocean, then drill another three or four miles into the Earth’s crust in search of oil. Until recently, we were told that the technology was so good that chances of something major going wrong were tiny.
Yet something did go wrong. The pipe in question has broken, and it has come as a shock to learn that it is beyond our capabilities to fix it. Thanks to incomplete mastery of our own technology, millions of gallons of oil now threaten the beauty and ecology of the Gulf.
As Lewis Mumford once wrote, when you give a 10-year-old a stick of dynamite, you don’t make him more powerful. You make him more dangerous.
Last week, as oil continued to spew from that pipe, scientists in Washington announced a major breakthrough in another area of technology. J. Craig Venter and his team announced creation of synthetic life, in this case a man-made bacteria with a chromosome designed by computer and then “built from four bottles of chemicals.”
In his announcement, Venter acknowledged the ethical concerns involved. But he pointed out that throughout his work, he has sought and heeded the advice of bioethics experts. According to Venter, those experts conclude that “there is a slight increase in the potential for harm, but there is an exponential increase in the potential benefit to society.”
For example, microbes could be designed to digest vast amounts of oil, in effect using technology to correct technology’s failure. Even better, Venter’s firm is working to engineer a species of algae that can convert the sun’s energy into a form of gasoline. Among those funding that work is BP, owner of the Deepwater Horizon rig.
One of the concerns in creating new life forms is how to trace them should they be released “into the wild.” Venter’s team addressed that issue by putting nonworking code into the bacteria’s DNA as special identifiers.
One of those “watermarks” installed by Venter is a quote from the late Richard Feynman, a famed physicist:
“What I cannot build, I cannot understand.”
To Venter, who is driven by a need to comprehend the mysteries of life, the relevance of that quote is obvious. But the sentiment is dangerous.
We humans understood the Deepwater Horizon well — we built it. What we didn’t understand was how it would behave in deep sea conditions where it was beyond our power to fix it.
Likewise, Wall Street thought it understood the complex global economy. It just didn’t account for how it would be operated in real life by humans whose judgment was clouded by greed.
Every day, we build ever more complex technologies that defy full understanding even of their creators. Many, like the Deepwater Horizon rig, may have a very low probability of failure, but a very high consequence if failure comes. In that kind of system, you have to get it right every single day, day after day, without fail. Thousands of rigs have pumped oil from the Gulf for decades, but all it has taken is one mistake to bring tragedy.
In pursuit of that inhuman level of perfection, we install systems and systems of systems, which themselves become incredibly complex and capable of going wrong in ways difficult to imagine beforehand. They also give us a false sense of confidence and mastery.
Nobody knew that better than Feynman, whose fame broadened after he diagnosed the cause of NASA’s Challenger shuttle disaster. The problem, he discovered, was “management’s fantastic faith in the machinery” they had built and their failure to account for the stresses of the real world.
“For a successful technology, reality must take precedence over public relations, for Nature cannot be fooled,” Feynman wrote.
I’d feel better if Venter had built that expression of humility into his creation, literally as well as figuratively.