{"id":62509,"date":"2015-08-17T12:00:08","date_gmt":"2015-08-17T11:00:08","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=62509"},"modified":"2023-06-20T06:03:43","modified_gmt":"2023-06-20T05:03:43","slug":"is-artificial-intelligence-really-an-existential-threat-to-humanity","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2015\/08\/is-artificial-intelligence-really-an-existential-threat-to-humanity\/","title":{"rendered":"Is Artificial Intelligence Really an Existential Threat to Humanity?"},"content":{"rendered":"

\"DoomsdayClock_black_3mins_regmark\"<\/a>Superintelligence: Paths, Dangers, Strategies <\/em><\/a>is an astonishing book with an alarming thesis: Intelligent machines are \u201cquite possibly the most important and most daunting challenge humanity has ever faced.\u201d In it, Oxford University philosopher Nick Bostrom, who has built his reputation on the study of \u201cexistential risk,\u201d argues forcefully that artificial intelligence might be the most apocalyptic technology of all. With intellectual powers beyond human comprehension, he prognosticates, self-improving artificial intelligences could effortlessly enslave or destroy Homo sapiens<\/em> if they so wished. While he expresses skepticism that such machines can be controlled, Bostrom claims that if we program the right \u201chuman-friendly\u201d values into them, they will continue to uphold these virtues, no matter how powerful the machines become.<\/p>\n

httpv:\/\/www.youtube.com\/watch?v=V1eYniJ0Rnk<\/p>\n

These views have found an eager audience. In August 2014, PayPal cofounder and electric car magnate Elon Musk tweeted<\/a> \u201cWorth reading Superintelligence<\/em> by Bostrom. We need to be super careful with AI. Potentially more dangerous than nukes.\u201d Bill Gates declared, \u201cI agree with Elon Musk<\/a> and some others on this and don\u2019t understand why some people are not concerned.\u201d More ominously, legendary astrophysicist Stephen Hawking concurred<\/a>: \u201cI think the development of full artificial intelligence could spell the end of the human race.\u201d Proving his concern went beyond mere rhetoric, Musk donated $10 million to the Future of Life Institute<\/a> \u201cto support research aimed at keeping AI beneficial for humanity.\u201d<\/p>\n

Superintelligence <\/em>is propounding a solution that will not work to a problem that probably does not exist, but Bostrom and Musk are right that now is the time to take the ethical and policy implications of artificial intelligence seriously. The extraordinary claim that machines can become so intelligent as to gain demonic powers requires extraordinary evidence, particularly since artificial intelligence (AI) researchers have struggled to create machines that show much evidence of intelligence at all. While these investigators\u2019 ultimate goals have varied since the emergence of the discipline in the mid-1950s, the fundamental aim of AI has always been to create machines that demonstrate intelligent behavior, whether to better understand human cognition or to solve practical problems. Some AI researchers even tried to create the self-improving reasoning machines Bostrom fears. Through decades of bitter experience, however, they learned not only that creating intelligence is more difficult than they initially expected, but also that it grows increasingly harder the smarter one tries to become. Bostrom\u2019s concept of \u201csuperintelligence,\u201d which he defines as \u201cany intellect that greatly exceeds the cognitive performance of humans in virtually all domains of interest,\u201d builds upon similar discredited assumptions about the nature of thought that the pioneers of AI held decades ago. A summary of Bostrom\u2019s arguments, contextualized in the history of artificial intelligence, demonstrates how this is so.<\/p>\n

In the 1950s, the founders of the field of artificial intelligence assumed that the discovery of a few fundamental insights would make machines smarter than people within a few decades. By the 1980s, however, they discovered fundamental limitations that show that there will always be diminishing returns to additional processing power and data. Although these technical hurdles pose no barrier to the creation of human-level AI, they will likely forestall the sudden emergence of an unstoppable \u201csuperintelligence.\u201d<\/p>\n

The risks of self-improving intelligent machines are grossly exaggerated and ought not serve as a distraction from the existential risks we already face, especially given that the limited AI technology we already have is poised to make threats like those posed by nuclear weapons even more pressing than they currently are. Disturbingly, little or no technical progress beyond that demonstrated by self-driving cars is necessary for artificial intelligence to have potentially devastating, cascading economic, strategic, and political effects. While policymakers ought not lose sleep over the technically implausible menace of \u201csuperintelligence,\u201d they have every reason to be worried about emerging AI applications such as the Defense Advanced Research Projects Agency\u2019s submarine-hunting drones<\/a>, which threaten to upend longstanding geostrategic assumptions in the near future. Unfortunately, Superintelligence<\/em> offers little insight into how to confront these pressing challenges.<\/p>\n

\"Over<\/a>

Over 1,000 leading experts in artificial intelligence [including Stephen Hawking] have signed an open letter calling for a ban on military AI development and autonomous weapons, as depicted within the Terminator sci-fi franchise. Photograph: Moviestore\/REX Shutterstock\/Moviestore\/REX Shutterstock<\/p><\/div>The Silicon Valley Roots of Superintelligence<\/em><\/strong><\/p>\n

Superintelligence<\/em> is the culmination of intellectual trends that have been mounting for decades and have had a significant following in Silicon Valley for many years. Ray Kurzweil, director of engineering at Google, has long been an apostle for the notion that self-improving AI will bring about a technological revolution he calls the \u201csingularity,\u201d after which human existence will be so transformed as to be unrecognizable. In his rapturous vision, uploaded human minds will merge with artificial intelligence to live forever either in android bodies or as computer simulations. While Kurzweil sees AI as a panacea for all human problems, including mortality, others have viewed the prospect of self-improving intelligent machines with alarm. Charismatic autodidact Eliezer Yudkowsky co-founded the Machine Intelligence Research Institute in 2000 to \u201chelp humanity prepare for the moment when machine intelligence exceeded human intelligence.\u201d This organization utilized financial support from wealthy Silicon Valley patrons to hire a full-time staff that has developed an increasingly pessimistic series of publications painting AI as a potential menace to the future of humanity. Although academics have largely dismissed both Kurzweil\u2019s and Yudkowsky\u2019s conceptions of \u201csingulatarianism\u201d until now, Bostrom\u2019s Superintelligence<\/em> has succeeded in bringing these ideas into mainstream intellectual discourse.<\/p>\n

Bostrom\u2019s arguments owe much to Yudkowsky\u2019s writings about the dangers of \u201cunfriendly\u201d artificial intelligence\u2014a debt he freely acknowledges. Bostrom is surprisingly agnostic about what form a \u201csuperintelligence\u201d might take. He explores the prospect that genetic engineering, eugenics, or high-speed computer emulations of human brains, in addition to fully \u201cartificial\u201d intelligence, could produce entities much smarter than present-day humans. Bostrom\u2019s postulated superintelligences are not merely somewhat faster-thinking or more knowledgeable than humans, but qualitatively superior agents with preternatural abilities. Concluding confidently that \u201cmachines will eventually greatly exceed biology in general intelligence,\u201d he argues furthermore that it is \u201clikely\u201d that machines could recursively improve their own intelligence so rapidly as to rise from mere human-level intelligence to godlike power in an \u201cintelligence explosion\u201d lasting \u201cminutes, hours, or days.\u201d After this epic development, Bostrom envisions that a superintelligence would boast what he terms \u201csuperpowers\u201d encompassing intelligence amplification, strategizing, social manipulation, hacking, technology research, and economic productivity. According to Superintelligence<\/em>, the lucky beneficiary of an intelligence explosion would probably gain a \u201cdecisive strategic advantage\u201d which it could then employ to form a \u201csingleton,\u201d defined as \u201ca world order in which there is at the global level a single decision-making agency.\u201d Rendered smart enough to foresee and prevent any human attempts to \u201cpull the plug\u201d by the \u201cintelligence explosion,\u201d machines could take over the world overnight.<\/p>\n

In contrast to the vast majority of philosophical literature on artificial intelligence, which is concerned primarily with whether machines can really be \u201cconscious\u201d in the same sense humans are, Superintelligence<\/em> rightly dismisses this issue as largely irrelevant to the possible hazards intelligent machines might pose to mankind.\u00a0If anything, machines capable of conceiving and actualizing elaborate plans but lacking self-awareness could be far more dangerous than mechanical analogues of human minds. Crucial to Bostrom\u2019s argument is his belief that even in the absence of conscious experiences, intelligent machines would still have and pursue goals, and he additionally propounds that \u201cmore or less any level of intelligence could in principle be combined with more or less any final goal.\u201d In one particularly lurid example, Bostrom paints a picture of how a superintelligent, but otherwise unconscious machine might interpret a seemingly innocuous goal, such as \u201cmaximize the production of paperclips,\u201d by converting the entire earth, as well as all additional matter it could access, into office supplies. Preposterous as this scenario might seem, the author is entirely serious about such potential counterintuitive consequences of an \u201cintelligence explosion.\u201d<\/p>\n

As this example suggests, Bostrom believes that superintelligences will retain the same goals they began with, even after they have increased astronomically in intelligence. \u201cOnce unfriendly superintelligence exists,\u201d he warns, \u201cit would prevent us from replacing it or changing its preferences.\u201d This assumption\u2014that superintelligences will do whatever is necessary to maintain their \u201cgoal-content integrity\u201d\u2014undergirds his analysis of what, if anything, can be done to prevent artificial intelligence from destroying humanity. According to Bostrom, the solution to this challenge lies in building a value system into AIs that will remain human-friendly even after an intelligence explosion, but he is pessimistic about the feasibility of this goal. \u201cIn practice,\u201d he warns, \u201cthe control problem … looks quite difficult,\u201d but \u201cit looks like we will only get one chance.\u201d<\/p>\n

Convinced that sufficient \u201cintelligence\u201d can overcome almost any obstacle, Bostrom acknowledges few limits on what artificial intelligences might accomplish. Engineering realities rarely enter into Bostrom\u2019s analysis, and those that do contradict the thrust of his argument. He admits that the theoretically optimal intelligence, a \u201cperfect Bayesian agent that makes probabilistically optimal use of available information,\u201d will forever remain \u201cunattainable because it is too computationally demanding to be implemented in any physical computer.\u201d Yet Bostrom\u2019s postulated \u201csuperintelligences\u201d seem uncomfortably close to this ideal. The author offers few hints of how machine superintelligences would circumvent the computational barriers that render the perfect Bayesian agent impossible, other than promises that the advantages of artificial components relative to human brains will somehow save the day. But over the course of 60 years of attempts to create thinking machines, AI researchers have come to the realization that there is far more to intelligence than simply deploying a faster mechanical alternative to neurons. In fact, the history of artificial intelligence suggests that Bostrom\u2019s \u201csuperintelligence\u201d is a practical impossibility.<\/p>\n

The General Problem Solver: \u201cA Particularly Stupid Program for Solving Puzzles\u201d<\/strong><\/p>\n

The dream of creating machines that think dates back to ancient times, but the invention of digital computers in the middle of the 20th century suddenly made it look attainable. The 17th century German polymath Gottfried Leibniz sought to create a universal language from symbolic logic along with a calculus of reasoning for manipulating those symbols; in his foundational 1947 work Cybernetics<\/em><\/a>, MIT mathematician Norbert Wiener noted approvingly that \u201cjust as the calculus of arithmetic lends itself to a mechanization progressing through the abacus and the desk computing machine to the ultra-rapid computing machines of the present day, so the calculus ratiocinator<\/em> of Leibniz contains the germs of the machina ratiocinatrix<\/em>, the reasoning machine.\u201d By the second half of the 1950s, artificial intelligence pioneers claimed to have already created such machines.<\/p>\n

In 1957, future Nobel laureate Herbert A. Simon<\/a> made a speech declaring that the age of intelligent machines had already dawned. \u201cIt is not my aim to surprise or shock you\u2014if indeed that were possible in an age of nuclear fission and prospective interplanetary travel,\u201d he intoned. \u201cBut the simplest way I can summarize the situation is to say that there are now in the world machines that think, that learn, and that create. Moreover, their ability to do these things is going to increase rapidly until in a visible future the range of problems they can handle will be coextensive with the range to which the human mind has been applied.\u201d Given the \u201cspeed with which research in this field is progressing,\u201d Simon beseeched that humanity needed to engage in some serious soul-searching: \u201cThe revolution in heuristic problem solving will force man to consider his role in a world in which his intellectual power and speed are outstripped by the intelligence of machines.\u201d<\/p>\n

Simon\u2019s astonishing pronouncement was more than mere bluster, for in collaboration with RAND researcher Allen Newell, he was hard at work implementing the General Problem Solver<\/a>\u2014a computer program they hoped would begin making superhuman machine intelligence a reality. Implementing Aristotle\u2019s notion of means-ends analysis algorithmically, the program sought to minimize the distance from an initial state to the desired goal according to rules provided by the user. The discovery of a few powerful inference mechanisms like those embodied by the General Problem Solver, they hoped, would be the breakthrough that enabled the creation of machines boasting greater-than-human intelligence.<\/p>\n

Astonishing as the hubris and na\u00efvet\u00e9 of the pioneering artificial intelligence researchers appear in hindsight, the considerable success of their earliest experiments fueled their overconfidence. Starting from literally nothing, every toy example coaxed out of the crude computers of the time looked like, and really was, a triumph. At the beginning of the 1950s, skeptics scoffed at the notion that computers would ever play chess at all, much less well\u2014yet by the time Simon gave his speech, programs had been developed to play chess and checkers, translate sentences from Russian to English, and even, in the case of Simon and Newell\u2019s \u201cLogic Theorist,\u201d prove mathematical theorems. At this astronomical rate of progress, it seemed like what John McCarthy dubbed \u201cartificial intelligence\u201d in 1956 might achieve spectacular results in the not-too-distant future. Simon and Newell certainly thought so, predicting confidently that by 1967 \u201ca digital computer will be the world’s chess champion, unless the rules bar it from competition\u201d\u2014a milestone a computer passed only in the late 1990s\u2014and one of its brethren would \u201cdiscover and prove an important new mathematical theorem.\u201d<\/p>\n

Much to their chagrin, Simon and Newell discovered that the General Problem Solver was not the breakthrough they had envisioned\u2014even though it could, as promised, solve any fully specified symbolic problem. With the right inputs and enough computing resources, the General Problem Solver could solve logic puzzles or prove geometric theorems. Simon and Newell even attempted to program it to improve itself\u2014almost certainly the first attempt to create a self-improving reasoning machine<\/a>.\u00a0But for all its generality, the General Problem Solver turned out to be quite bad at solving practical problems\u2014as it turned out, most real-world problems were problems precisely because they were not fully specified\u2014and it became an object of mockery for later AI researchers. In 1976 Yale University professor Drew McDermott<\/a> dismissed Simon and Newell\u2019s creation as \u201ca particularly stupid program for solving puzzles.\u201d Lamenting that it had \u201ccaused everybody a lot of needless excitement and distraction,\u201d McDermott suggested that it \u201cshould have been called LFGNS\u2014\u2018Local Feature-Guided Network Searcher.\u2019\u201d<\/p>\n

A Mistaken Conflation of Inference with Intelligence<\/strong><\/p>\n

The failure of programs like the General Problem Solver forced the field of artificial intelligence to accept that its early assumptions about the nature of intelligence had been mistaken. Stanford University\u2019s Edward Feigenbaum<\/a> noted in the 1970s that \u201c[f]or a long time AI focused its attention almost exclusively on the development of clever inference methods,\u201d only to discover that \u201cthe power of its systems does not reside in the inference method.\u201d Not only did powerful inference mechanisms offer little advantage, they learned that \u201calmost any inference method will do,\u201d as \u201cthe power resides in the knowledge.\u201d This unwanted discovery inspired a massive reorientation of AI research toward \u201cknowledge-based reasoning\u201d during the 1970s. It also poses substantial obstacles to the kind of \u201cintelligence explosion\u201d Bostrom fears, since it implies that machines could not become \u201csuperintelligent\u201d by refining their inference algorithms.<\/p>\n

Bostrom\u2019s descriptions of how machines might rapidly improve their intelligence make it clear that he does not appreciate that the knowledge possessed by reasoning programs is much more important than how those programs work. Asserting that \u201ceven without any designated knowledge base, a sufficiently superior mind might be able to learn much by simply introspecting on the workings of its own psyche,\u201d he muses that \u201cperhaps a superintelligence could even deduce much about the likely properties of the world a priori <\/em>(combining logical inference with a probability prior biased towards simpler worlds, and a few elementary facts implied by the superintelligence\u2019s existence as a reasoning system).\u201d<\/p>\n

Bostrom\u2019s mistaken conflation of inference mechanisms with intelligence is also apparent in his colorful descriptions of how intelligent machines might annihilate humanity. Simply depriving AIs of information about the world is not adequate to render them safe, he claims, as they might be able to accomplish such feats as solving extremely complex problems in physical science without the need to carry out real-world experiments. In a scenario borrowed from Yudkowsky, Bostrom posits that a superintelligence might \u201ccrack the protein folding problem\u201d and then manipulate a gullible human into mixing mail-ordered synthesized proteins \u201cin a specified environment\u201d to create \u201ca very primitive \u2018wet\u2019 nanosystem, which, ribosome-like, is capable of accepting external instructions; perhaps patterned acoustic vibrations delivered by a speaker attached to the beaker.\u201d It could then employ this system to bootstrap increasingly sophisticated nanotechnologies, and \u201cat a pre-set time, nanofactories producing nerve gas or target-seeking mosquito-like robots might then burgeon forth simultaneously from every square meter of the globe (although more effective ways of killing could probably be devised by a machine with the technology research superpower).\u201d This scenario doesn’t just strain a reader\u2019s credulity; it also implies a fanciful understanding of the nature of technological development in which \u201cgenius\u201d can somehow substitute for hard work and countless intermediate failures. In the real world, the \u201clone genius inventor\u201d is a myth; even smarter-than-human AIs could never escape the tedium of an iterative research and development process.<\/p>\n

A Misguided Approach to the Control Problem<\/strong><\/p>\n

The findings of artificial intelligence researchers bode ill for Bostrom\u2019s recommendations for how to prevent superintelligent machines from determining the fate of mankind. The second half of Superintelligence <\/em>is devoted to strategies for approaching what Bostrom terms the \u201ccontrol problem.\u201d While creating economic or ecological incentives for artificial intelligences to be friendly toward humanity might seem like obvious ways to keep AI under control, Bostrom has little faith in them; he believes the machines will be powerful enough to subvert these obstacles if they want. Dismissing \u201ccapability control\u201d as \u201cat best, a temporary and auxiliary measure,\u201d he focuses the bulk of his analysis on \u201cgiving the AI a final goal that makes it easier to control.\u201d Although Bostrom acknowledges that formulating an appropriate goal is likely to be extremely challenging, he is confident that intelligent machines will aggressively protect their \u201cgoal content integrity\u201d no matter how powerful they become\u2014an idea he appears to have borrowed from AI theorist Stephen Omohundro<\/a>. Bostrom devotes several chapters to how to specify goals that can be incorporated into \u201cseed AIs,\u201d so they will protect human interests once they become superintelligent.<\/p>\n

If machines are somehow able to develop the kind of godlike superintelligence Bostrom envisions, artificial intelligence researchers have learned the hard way that the nature of reason itself will work against this plan to solve the \u201ccontrol problem.\u201d The failure of early AI programs such as the General Problem Solver to deal with real-world problems resulted in considerable part from their inability to redefine their internal problem representation; if their designers failed to provide an efficient way to represent the problem in the first place, the programs usually choked.<\/p>\n

By the early 1970s, it became apparent that the solution to this challenge probably lay in drawing upon domain-specific knowledge to develop higher-level, conceptual representations of problems. Empowered with higher-level concepts, the problem space could be radically reduced; the AI programs need consider only things that \u201creally mattered.\u201d This process, however, was tantamount to inventing a new symbolic language and translating the original problem into it. But due to the inherent undecideability of logical reasoning, it is a mathematical impossibility to ensure that these translations truly \u201cmean\u201d the same thing as the original. AI researchers in the 1970s<\/a> noticed this phenomenon, but were too concerned with getting their programs to work to particularly care.<\/p>\n

The culmination of 1970s investigations into knowledge-based reasoning, Douglas Lenat\u2019s heuristic learning program EURISKO remains justifiably famous for repeatedly humiliating its human opponents. In contrast to the General Problem Solver, which was crippled by its internal rigidity, EURISKO was designed to utilize Lenat\u2019s Representation Language Language<\/a>\u2014a system of knowledge representation that could modify itself to add new concepts, extend or modify existing ones, or delete them if it deemed them superfluous. This ability extended even to the rules for how to discover new rules, so EURISKO could invent new ways to be creative.<\/p>\n

To test his creation, Lenat decided to compete in a wargame called the Traveller Trillion Credit Squadron (TCS). Each player received a trillion \u201ccredits\u201d to build a fleet of futuristic warships that they then pitted against other players\u2019 fleets. Never having played the game himself, or even seen it played, Lenat painstakingly added domain knowledge from the TCS rules to EURISKO, which then tested its own modifications of these concepts in simulated fleet encounters. When experienced players first saw EURISKO\u2019s fleet at the national tournament in 1981, they laughed at the seemingly preposterous assortment of ships the program had created. The mockery stopped, however, when it swiftly trounced all of them. EURISKO<\/a> had identified a counterintuitive synergy of loopholes in the rules that made its defense unbeatable and became ranking player in the United States. Furious, the TCS organizers modified the rules for the 1982 tournament\u2014only to have the program discover a totally different offense-dominated strategy to circumvent these and take first prize again.<\/p>\n

Impressive as this feat was, EURISKO could not have achieved it without human assistance because while it could invent novel solutions, most of its original ideas were stupid, and it lacked an effective means of determining which ones were not. Devoid of any knowledge beyond what Lenat originally provided, EURISKO could not recognize the pointlessness of a new idea except by exhaustive testing in simulated games. The program required human feedback to provide its new concepts with human-readable labels and to weed out futile lines of investigation. Lenat wrote that \u201cthe final crediting of the win should be about 60\/40% Lenat\/EURISKO, though the significant point here is that neither party could have won alone.\u201d More worrying, due to its ability to modify itself, the program required supervision to prevent pathological changes to its control structure. Eventually this problem compelled Lenat to limit EURISKO\u2019s capacity for self-modification.<\/p>\n

The case of EURISKO and other knowledge-based reasoning programs indicates that even superintelligent machines would struggle to guard their \u201cgoal-content integrity\u201d and increase their intelligence simultaneously. Obviously, any superintelligence would grossly outstrip humans in its capacity to invent new abstractions and reconceptualize problems. The intellectual advantages of inventing new higher-level concepts are so immense that it seems inevitable that any human-level artificial intelligence will do so. But it is impossible to do this without risking changing the meaning of its goals, even in the course of ordinary reasoning. As a consequence, actual artificial intelligences would probably experience rapid goal mutation, likely into some sort of analogue of the biological imperatives to survive and reproduce (although these might take counterintuitive forms for a machine). The likelihood of goal mutation is a showstopper for Bostrom\u2019s preferred schemes to keep AI \u201cfriendly,\u201d including for systems of sub-human or near-human intelligence that are far more technically plausible than the godlike entities postulated in his book.<\/p>\n

The Obstacles to Superintelligence<\/strong><\/p>\n

Emboldened by the success of programs like EURISKO, in the 1980s artificial intelligence researchers devoted great effort to the development of knowledge-based reasoning\u2014only to discover fundamental limitations that increase the obstacles to the creation of superintelligent machines. Google Director of Research Peter Norvig later recounted that the field of knowledge representation struggled to find \u201ca good trade-off between expressiveness and efficiency.\u201d Frustratingly, it turned out that the elusiveness of such a balance did not result from a failure of human engineering insight; by the end of the 1980s mathematical analyses emerged showing \u201cthat even seemingly trivial [knowledge representation] languages were intractable\u2014in the worst case, it would take an exponential amount of time to answer a simple query.\u201d This means that machines would have a hard time becoming superintelligent simply by adding more knowledge: They might be able to know far more than humans, but exploiting that knowledge would take longer and longer as the amount of knowledge they reasoned with increased. As Norvig concluded<\/a>, \u201cNo amount of knowledge can solve an intractable problem in the worst case.\u201d<\/p>\n

Beginning in the late 1980s, the type of \u201csymbolic\u201d AI research exemplified by the General Problem Solver and EURISKO was increasingly eclipsed by a \u201cconnectionist\u201d approach emphasizing neural networks. Although overoptimistic expectations led to a wave of disillusionment with this technology in the 1990s, during the 2000s more powerful computers, ever-increasing amounts of data, and improved algorithms combined with it to make artificial intelligence applications such as self-driving cars a reality. In the last few years, machines exploiting these advances have stoked apprehensions by demonstrating superhuman performance on tasks they had not been programmed to perform. For instance, Google DeepMind<\/a> utilized a technique called \u201cdeep learning,\u201d which combines neural networks in multiple layers, to play a variety of Atari 2600 games. Starting without any knowledge of how the games were supposed to be played, the program taught itself how to play some of them in an aggressive style no human could match. Videos of this feat helped convince industry leaders such as Musk and scientists such as Hawking that unstoppable AI might be not just the stuff of cinematic nightmare.<\/p>\n

But while advanced machine learning technologies can be a terrible force for ill when applied to weapons systems or mass surveillance, instead of playing video games, this is a qualitatively different problem than machines that reason well enough to improve themselves exponentially. Neural networks can create highly efficient implicit systems of knowledge representation, but these are still subject to the tradeoff between expressiveness and efficiency identified in the 1980s. Furthermore, while these technologies have demonstrated astonishing results in areas at which older techniques failed badly\u2014such as machine vision\u2014 they have yet to demonstrate the same sort of reasoning as symbolic AI programs such as EURISKO. Due to its preoccupation with the dubious prospect of superintelligence, resources spent on the particular research program Bostrom proposes to study \u201cAI safety\u201d would be better expended reducing the existential risks we already face.<\/p>\n

For all its entertainment value as a philosophical exercise, Bostrom\u2019s concept of superintelligence is mostly a distraction from the very real ethical and policy challenges posed by ongoing advances in artificial intelligence. Although it has failed so far to realize the dream of intelligent machines, artificial intelligence has been one of the greatest intellectual adventures of the last 60 years. In their quest to understand minds by trying to build them, artificial intelligence researchers have learned a tremendous amount about what intelligence is not. Unfortunately, one of their major findings is that humans resort to fallible heuristics to address many problems because even the most powerful physically attainable computers could not solve them in a reasonable amount of time. As the authors of a 1993 textbook about problem-solving programs<\/a> noted, \u201cintelligence is possible because Nature is kind,\u201d but \u201cthe ubiquity of exponential problems makes it seem that Nature is not overly generous.\u201d As a consequence, both the peril and the promise of artificial intelligence have been greatly exaggerated.<\/p>\n

But if artificial intelligence might not be tantamount to \u201csummoning the demon\u201d (as Elon Musk colorfully described it), AI-enhanced technologies might still be extremely dangerous due to their potential for amplifying human stupidity. The AIs of the foreseeable future need not think or create to sow mass unemployment, or enable new weapons technologies that undermine precarious strategic balances. Nor does artificial intelligence need to be smarter than humans to threaten our survival\u2014all it needs to do is make the technologies behind familiar 20th-century existential threats faster, cheaper, and more deadly.<\/p>\n

_______________________________<\/p>\n

Edward Moore Geist is a MacArthur Nuclear Security Fellow at Stanford University’s Center for International Security and Cooperation (CISAC). Previously a Stanton Nuclear Security Fellow at the RAND Corporation, he received his doctorate in history from the University of North Carolina in 2013.<\/em><\/p>\n

Go to Original \u2013 thebulletin.org<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

With intellectual powers beyond human comprehension, Oxford University philosopher Nick Bostrom prognosticates, self-improving artificial intelligences could effortlessly enslave or destroy Homo sapiens if they so wished.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[216,3078],"tags":[],"class_list":["post-62509","post","type-post","status-publish","format-standard","hentry","category-technology","category-artificial-intelligence-ai"],"_links":{"self":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/62509","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/comments?post=62509"}],"version-history":[{"count":1,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/62509\/revisions"}],"predecessor-version":[{"id":237712,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/62509\/revisions\/237712"}],"wp:attachment":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media?parent=62509"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/categories?post=62509"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/tags?post=62509"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}