{"id":81974,"date":"2016-10-31T12:00:36","date_gmt":"2016-10-31T12:00:36","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=81974"},"modified":"2016-10-26T20:16:49","modified_gmt":"2016-10-26T19:16:49","slug":"the-pentagons-terminator-conundrum-robots-that-could-kill-on-their-own","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2016\/10\/the-pentagons-terminator-conundrum-robots-that-could-kill-on-their-own\/","title":{"rendered":"The Pentagon\u2019s \u2018Terminator Conundrum\u2019: Robots That Could Kill on Their Own"},"content":{"rendered":"

The United States has put artificial intelligence at the center of its defense strategy, with weapons that can identify targets and make decisions.<\/em><\/p>\n

\"Jacob<\/a>

Jacob Regenstein, an engineer, held a fake rifle during a test of an autonomous drone in August on Cape Cod, in Massachusetts. Credit Hilary Swift for The New York Times<\/p><\/div>\n

25 Oct 2016<\/em> \u2014 The small drone, with its six whirring rotors, swept past the replica of a Middle Eastern village and closed in on a mosque-like structure, its camera scanning for targets.<\/p>\n

No humans were remotely piloting the drone, which was nothing more than a machine that could be bought on Amazon. But armed with advanced artificial intelligence software, it had been transformed into a robot that could find and identify the half-dozen men carrying replicas of AK-47s around the village and pretending to be insurgents.<\/p>\n

As the drone descended slightly, a purple rectangle flickered on a video feed that was being relayed to engineers monitoring the test. The drone had locked onto a man obscured in the shadows, a display of hunting prowess that offered an eerie preview of how the Pentagon plans to transform warfare.<\/p>\n

Almost unnoticed outside defense circles, the Pentagon has put artificial intelligence at the center of its strategy to maintain the United States\u2019 position as the world\u2019s dominant military power. It is spending billions of dollars to develop what it calls autonomous and semiautonomous weapons and to build an arsenal stocked with the kind of weaponry that until now has existed only in Hollywood movies and science fiction, raising alarm among scientists and activists concerned by the implications of a robot arms race.<\/p>\n

The Defense Department is designing robotic fighter jets that would fly into combat alongside manned aircraft. It has tested missiles that can decide what to attack<\/a>, and it has built ships that can hunt for enemy submarines, stalking those it finds over thousands of miles, without any help from humans.<\/p>\n

\u201cIf Stanley Kubrick directed \u2018Dr. Strangelove\u2019 again, it would be about the issue of autonomous weapons,\u201d said Michael Schrage, a research fellow at the Massachusetts Institute of Technology Sloan School of Management.<\/p>\n

Defense officials say the weapons are needed for the United States to maintain its military edge over China, Russia and other rivals, who are also pouring money into similar research (as are allies, such as Britain and Israel). The Pentagon\u2019s latest budget outlined $18 billion to be spent over three years on technologies that included those needed for autonomous weapons.<\/p>\n

\u201cChina and Russia are developing battle networks that are as good as our own. They can see as far as ours can see; they can throw guided munitions as far as we can,\u201d said Robert O. Work, the deputy defense secretary, who has been a driving force for the development of autonomous weapons. \u201cWhat we want to do is just make sure that we would be able to win as quickly as we have been able to do in the past.\u201d<\/p>\n

Just as the Industrial Revolution spurred the creation of powerful and destructive machines like airplanes and tanks that diminished the role of individual soldiers, artificial intelligence technology is enabling the Pentagon to reorder the places of man and machine on the battlefield the same way it is transforming ordinary life with computers that can see, hear and speak and cars that can drive themselves.<\/p>\n

The new weapons would offer speed and precision unmatched by any human while reducing the number \u2014 and cost \u2014 of soldiers and pilots exposed to potential death and dismemberment in battle. The challenge for the Pentagon is to ensure that the weapons are reliable partners for humans and not potential threats to them.<\/p>\n

At the core of the strategic shift envisioned by the Pentagon is a concept that officials call centaur warfighting. Named for the half-man and half-horse in Greek mythology, the strategy emphasizes human control and autonomous weapons as ways to augment and magnify the creativity and problem-solving skills of soldiers, pilots and sailors, not replace them.<\/p>\n

The weapons, in the Pentagon\u2019s vision, would be less like the Terminator and more like the comic-book superhero Iron Man, Mr. Work said in an interview.<\/p>\n

\u201cThere\u2019s so much fear out there about killer robots and Skynet,\u201d the murderous artificial intelligence network of the \u201cTerminator\u201d movies, Mr. Work said. \u201cThat\u2019s not the way we envision it at all.\u201d<\/p>\n

When it comes to decisions over life and death, \u201cthere will always be a man in the loop,\u201d he said.<\/p>\n

Beyond the Pentagon, though, there is deep skepticism that such limits will remain in place once the technologies to create thinking weapons are perfected. Hundreds of scientists and experts warned in an open letter<\/a> last year that developing even the dumbest of intelligent weapons risked setting off a global arms race. The result, the letter warned, would be fully independent robots that can kill, and are cheap and as readily available to rogue states and violent extremists as they are to great powers.<\/p>\n

\u201cAutonomous weapons will become the Kalashnikovs of tomorrow,\u201d the letter said.<\/p>\n

The Terminator Conundrum<\/strong><\/p>\n

The debate within the military is no longer about whether to build autonomous weapons but how much independence to give them. Gen. Paul J. Selva of the Air Force, the vice chairman of the Joint Chiefs of Staff, said recently that the United States was about a decade away from having the technology to build a fully independent robot that could decide on its own whom and when to kill, though it had no intention of building one.<\/p>\n

Other countries were not far behind, and it was very likely that someone would eventually try to unleash \u201csomething like a Terminator,\u201d General Selva said, invoking what seems to be a common reference in any discussion on autonomous weapons.<\/p>\n

Yet American officials are only just beginning to contend with the implications of weapons that could someday operate independently, beyond the control of their developers. Inside the Pentagon, the quandary is known as the Terminator conundrum, and there is no consensus about whether the United States should seek international treaties to try to ban the creation of those weapons, or build its own to match those its enemies might create.<\/p>\n

For now, though, the current state of the art is decidedly less frightening. Exhibit A: the small, unarmed drone tested this summer on Cape Cod.<\/p>\n

It could not turn itself on and just fly off. It had to be told by humans where to go and what to look for. But once aloft, it decided on its own how to execute its orders.<\/p>\n

The software powering the drone has been in development for about a year, and it was far from flawless during the day of trials. In one pass over the mosque, the drone struggled to decide whether a minaret was an architectural feature or an armed man, living up to its namesake, Bender, the bumbling robot in the animated television series \u201cFuturama.\u201d<\/p>\n

\"Capt.<\/a>

Capt. Mike Malandra, center, the ground branch head for the science and technology division at the Marine Corps Warfighting Laboratory, observed the drone test. Credit Hilary Swift for The New York Times<\/p><\/div>\n

At other moments, though, the drone showed a spooky ability to discern soldier from civilian, and to fluidly shift course and move in on objects it could not quickly identify.<\/p>\n

Armed with a variation of human and facial recognition software used by American intelligence agencies, the drone adroitly tracked moving cars and picked out enemies hiding along walls. It even correctly figured out that no threat was posed by a photographer who was crouching, camera raised to eye level and pointed at the drone, a situation that has confused human soldiers with fatal results<\/a>.<\/p>\n

The project is run by the Defense Advanced Research Projects Agency<\/a>, known as Darpa, which is developing the software needed for machines that could work with small units of soldiers or Marines as scouts or in other roles.<\/p>\n

Unlike the drones currently used by the military, all of which require someone at a remote control, \u201cthis one doesn\u2019t,\u201d said Maj. Christopher Orlowski of the Army, a program manager at Darpa. \u201cIt works with you. It\u2019s like having another head in the fight.\u201d<\/p>\n

It could also easily be armed. The tricky part is developing machines whose behavior is predictable enough that they can be safely deployed, yet flexible enough that they can handle fluid situations. Once that is mastered, telling it whom or what to shoot is easy; weapons programmed to hit only certain kinds of targets already exist.<\/p>\n

Yet the behavioral technology, if successfully developed, is unlikely to remain solely in American hands. Technologies developed at Darpa do not typically remain secret, and many are now ubiquitous, powering everything from self-driving cars to the internet.<\/p>\n

Chess Champions<\/strong><\/p>\n

Since the 1950s, United States military strategy has been based on overwhelming technological advantages. A superior nuclear arsenal provided the American edge in the early days of the Cold War, and guided munitions \u2014 the so-called smart bombs of the late 20th century \u2014 did the same in the conflict\u2019s final decade.<\/p>\n

\"Joe<\/a>

Joe Kehoe of the engineering team gave instructions before the drill. Credit Hilary Swift for The New York Times<\/p><\/div>\n

Those advantages have now evaporated, and of all the new technologies that have emerged in recent decades, such as genomics or miniaturization, \u201cthe one thing that has the widest application to the widest number of D.O.D. missions is artificial intelligence and autonomy,\u201d Mr. Work said.<\/p>\n

Today\u2019s software has its limits, though. Computers spot patterns far faster than any human can. But the ability to handle uncertainty and unpredictability remain uniquely human virtues, for now.<\/p>\n

Bringing the two complementary skill sets together is the Pentagon\u2019s goal with centaur warfighting.<\/p>\n

Mr. Work, 63, first proposed the concept when he led a Washington think tank, the Center for a New American Security. His inspiration, he said, was not found in typical sources of military strategy \u2014 Sun Tzu or Clausewitz, for instance \u2014 but in the work of Tyler Cowen, a blogger and economist at George Mason University.<\/p>\n

In his 2013 book, \u201cAverage Is Over,\u201d Mr. Cowen briefly mentioned how two average human chess players, working with three regular computers, were able to beat both human chess champions and chess-playing supercomputers.<\/p>\n

It was a revelation for Mr. Work. You could \u201cuse the tactical ingenuity of the computer to improve the strategic ingenuity of the human,\u201d he said.<\/p>\n

Mr. Work believes a lesson learned in chess can be applied to the battlefield, and he envisions a military supercharged by artificial intelligence. Brilliant computers would transform ordinary commanders into master tacticians. American soldiers would effectively become superhuman, fighting alongside \u2014 or even inside \u2014 robots.<\/p>\n

\"<\/a>

Silas Hughes monitored a drone during a test in August on Cape Cod. Credit Hilary Swift for The New York Times<\/p><\/div>\n

Of the $18 billion the Pentagon is spending on new technologies, $3 billion has been set aside specifically for \u201chuman-machine combat teaming\u201d over the next five years. It is a relatively small sum by Pentagon standards \u2014 its annual budget is more than $500 billion \u2014 but still a significant bet on technologies and a strategic concept that have yet to be proved in battle.<\/p>\n

At the same time, Pentagon officials say that the United States is unlikely to gain an absolute technological advantage over its competitors.<\/p>\n

\u201cA lot of the A.I. and autonomy is happening in the commercial world, so all sorts of competitors are going to be able to use it in ways that surprise us,\u201d Mr. Work said.<\/p>\n

The American advantage, he said, will ultimately come from a mix of technological prowess and the critical thinking and decision-making powers that the United States military prioritizes. The American military delegates significant decisions down its chain of command, in contrast to the more centralized Chinese and Russian armed forces, though that is changing.<\/p>\n

\u201cWe\u2019re pretty confident that we have an advantage as we start the competition,\u201d Mr. Work said. \u201cBut how it goes over time, we\u2019re not going to make any assumptions.\u201d<\/p>\n

Experts outside the Pentagon are far less convinced that the United States will be able to maintain its dominance by using artificial intelligence. The defense industry no longer drives research the way it did during the Cold War, and the Pentagon does not have a monopoly on the cutting-edge machine-learning technologies coming from start-ups in Silicon Valley, and in Europe and Asia.<\/p>\n

Unlike the technologies and material needed for nuclear weapons<\/a> or guided missiles, artificial intelligence as powerful as what the Pentagon seeks to harness is already deeply woven into everyday life. Military technology is often years behind what can be picked up at Best Buy.<\/p>\n

\"Mr.<\/a>

Mr. Regenstein, left, and Ben Krosner during the drone tracking test. Credit Hilary Swift for The New York Times<\/p><\/div>\n

\u201cLet\u2019s be honest, American defense contractors can be really cutting edge on some things and really behind the curve on others,\u201d said Maj. Brian Healy, 38, an F-35<\/a> pilot. The F-35, America\u2019s newest and most technologically advanced fighter jet, is equipped with a voice command system that is good for changing channels on the radio, and not much else.<\/p>\n

\u201cIt would be great to get Apple or Google on board with some of the software development,\u201d he added.<\/p>\n

Submarines and Civilians<\/strong><\/p>\n

Beyond the practical concerns, the pairing of increasingly capable automation with weapons has prompted an intensifying debate among legal scholars and ethicists. The questions are numerous, and the answers contentious: Can a machine be trusted with lethal force? Who is at fault if a robot attacks a hospital or a school? Is being killed by a machine a greater violation of human dignity than if the fatal blow is delivered by a human?<\/p>\n

A Pentagon directive says that autonomous weapons must employ \u201cappropriate levels of human judgment.\u201d Scientists and human rights experts say the standard is far too broad and have urged that such weapons be subject to \u201cmeaningful human control.\u201d<\/p>\n

But would any standard hold up if the United States was faced with an adversary of near or equal might that was using fully autonomous weapons? Peter Singer, a specialist on the future of war at New America, a think tank in Washington, suggested there was an instructive parallel in the history of submarine warfare.<\/p>\n

Like autonomous weapons, submarines jumped from the pages of science fiction to reality. During World War I, Germany\u2019s use of submarines to sink civilian ships without first ensuring the safety of the crew and passengers was seen as barbaric. The practice quickly became known as unrestricted submarine warfare, and it helped draw the United States into the war.<\/p>\n

After the war, the United States helped negotiate an international treaty that sought to ban unrestricted submarine warfare.<\/p>\n

Then came the Japanese attack on Pearl Harbor on Dec. 7, 1941. That day, it took just six hours for the United States military to disregard decades of legal and ethical norms and order unrestricted submarine warfare against Japan. American submarines went on to devastate Japan\u2019s civilian merchant fleet during World War II<\/a>, in a campaign that was later acknowledged to be tantamount to a war crime.<\/p>\n

\u201cThe point is, what happens once submarines are no longer a new technology, and we\u2019re losing?\u201d Mr. Singer said. He added: \u201cThink about robots, things we say we wouldn\u2019t do now, in a different kind of war.\u201d<\/p>\n

________________________________________<\/p>\n

Matthew Rosenberg is a Pentagon Correspondent for<\/em> The New York Times. He previously covered Afghanistan for the paper before being expelled from the country in 2014 at the order then-President Hamid Karzai, who was angered by Mr. Rosenberg’s reporting. Prior to his removal, Mr. Rosenberg managed to slip in a fly-fishing trip in the mountains of northeastern Afghanistan, and dubbed the country’s first international boxing match the “Squabble in Kabul” (like the fight, the name has not gone down in the annals of boxing history).<\/em><\/p>\n

A version of this article appears in print on October 26, 2016, on page A1 of the New York edition with the headline: At Heart of U.S. Strategy, Weapons That Can Think.<\/em><\/p>\n

Go to Original \u2013 nytimes.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

The United States has put artificial intelligence at the center of its defense strategy, with weapons that can identify targets and make decisions.<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[57],"tags":[],"class_list":["post-81974","post","type-post","status-publish","format-standard","hentry","category-militarism"],"_links":{"self":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/81974","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/comments?post=81974"}],"version-history":[{"count":0,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/81974\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media?parent=81974"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/categories?post=81974"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/tags?post=81974"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}