Singularity Summit Postscript: Reason

Ronald Bailey is a Reason magazine writer who is often called upon to bring reality to the madding crowds. Here is his postscript on the 2009 Singularity Summit:

Convened by the Singularity Institute for Artificial Intelligence (SIAI) for the first time on the East Coast, this fourth annual meeting attracted about 900 participants. The SIAI was created to address the urgent problem of how to create super-smart AIs that are friendly to human beings. The worry is that, unless we are very careful, AIs might evolve value systems that treat us as annoying organic matter that should be more usefully turned into computronium. As the Singularity Institute's Anna Salamon explained in her opening presentation at the summit, smarter intelligences might choose to get rid of us because our matter is not optimally arranged to achieve their goals.

One way to wind up with human-friendly AIs is to build them on top of uploaded human brains. Anders Sandberg, a research fellow at the Future of Humanity Institute at Oxford University, offered a technical roadmap for whole brain emulation. "If artificial intelligence does not get here first, we're going to end up with uploaded humans," maintained Sandberg. He argued that it is possible to foresee how developments in neuroscience, and software and hardware development are leading to emulating specific people's brains. Sandberg believes that emulating a human brain is only 20 years away. Sandberg did observe that we do not know if a one-to-one emulation would produce a mind or not. A huge advantage of an uploaded mind is that it would no longer be constrained by the speed at which organic brains can process information. He offhandedly noted that he would not be the first volunteer for an emulation experiment.

Randal Koene, the director of the Department of Neuroengineering at Fatronik-Tecnalia Foundation in Spain, argued that the time is now to go after mind uploading. Koene argued that radically increasing human longevity solves a few problems, but doesn't deal with our dependence on changeable environments and scarce resources. Nor does it deal with our intellectual limitations, death by traumatic accidents, or disease. Koene's "inescapable conclusion" was that we must free the mind from its single fragile substrate. How might you copy a brain? Perhaps one could use a more advanced version of the knife edge scanning microscope that currently enables the reconstruction of mouse brain architecture. This is what is known among uploading cognoscenti as destructive scanning. It had better work because the old organic brain is all sliced up in order to produce the emulation. To solve the problem of maintaining a sense of continuity, Koene suggested that one pathway might be to use molecular nanotechnology to replace parts of the brain bit by bit over time.

Philosopher David Chalmers, who directs the Centre for Consciousness at the Australian National University, argued that personal identity would be maintained if the functional organization of the upload was the same as the original. In addition, gradual uploading might also be a way to maintain personal identity. Chalmers also speculated about reconstructive uploading in which a super-smart AI would scour the world for information about a person, say articles, video, audio, blog posts, restaurant receipts, whatever. The AI would then instantiate that information in the appropriate substrate. "Is it me?," asked Chalmers. Maybe. On the optimistic view, being reconstructed from the informational debris you left behind would be like waking up from a long nap.

Inventor Ray Kurzweil, author of The Singularity is Near, envisions an intimate integration between humans and their neural prosthetics. Over time, more and more of the neural processing that makes us who we are will be located outside our bodies and brains so that "uploading" will take place gradually. Our uploaded minds will function much faster and more precisely than our "meat" minds do today. In a sense, we will become the singularity as the artificial parts of our intelligences become ascendant.

If the AIs aren't made of people, though, guaranteeing that they will be human-friendly is much more difficult. Chalmers suggested that perhaps we could launch a super-smart self-improving AI inside a leak-proof computing environment and see how it evolves. If it turns out to be nice, then we let it out and the singularity takes off and we're all happy. Kurzweil objected that a leak-proof singularity is impossible. In order to determine whether or not the AI was friendly we would have to look inside its computing environment and, since it is so much smarter than we are, it would necessarily become aware of us and then manipulate us into letting it out. In other words, it could pretend to be friendly and then zap us once it's present in our world. _Reason

Bailey worries that smart machines may slip their restraints and pursue goals hurtful to humanity. That is one danger that the Singularity Institute was founded to help prevent, of course. But there is no off switch on a nuclear explosion, as SI fellow Anna Salamon argued during the Summit.

The singularity is only the latest of names for an idea that has been around for centuries -- if not longer. Efforts to create thinking machines have also been around for over a century, but realistic efforts can be traced back only decades.

Of course, ideas of exchanging a mortal body for an immortal one, or exchanging a limited mind for a virtually unlimited one, have been around for long millenia -- in the form of religions. Thinking broadly, the singularity movement can be seen as a religious faith.

That is not to dismiss the movement, since religions can have momentous consequences for an underlying society. The combination of religious faith and goal orientation with scientific and engineering rigour, can lead to some truly amazing accomplishments.