{"id":123069,"date":"2018-12-03T12:00:09","date_gmt":"2018-12-03T12:00:09","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=123069"},"modified":"2018-11-28T11:15:24","modified_gmt":"2018-11-28T11:15:24","slug":"the-dangerous-junk-science-of-vocal-risk-assessment","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2018\/12\/the-dangerous-junk-science-of-vocal-risk-assessment\/","title":{"rendered":"The Dangerous Junk Science of Vocal Risk Assessment"},"content":{"rendered":"
\"\"<\/a>

Illustration: Daniel Zender for The Intercept<\/p><\/div>\n

25 Nov 2018 – <\/em>Is it possible to tell whether someone is a criminal just from looking at their face or listening to the sound of their voice? The idea may seem ludicrous, like something out of science fiction \u2014 Big Brother in \u201c1984\u201d detects any unconscious look \u201cthat carried with it the suggestion of abnormality\u201d \u2014 and yet, some companies have recently begun to answer this question in the affirmative. AC Global Risk, a startup founded in 2016, claims to be able to determine your level of \u201crisk\u201d as an employee or an asylum-seeker based not on what you say, but how you say it.<\/p>\n

The California-based company offers an automated screening system known as a Remote Risk Assessment, or RRA. Here\u2019s how it works: Clients of AC Global Risk help develop automated, yes-or-no interview questions. The\u00a0group of people selected\u00a0for\u00a0a given screening then answer these simple questions in their native language during a 10-minute interview that can be conducted over the phone. The RRA then measures the characteristics of their voice to produce an evaluation report that scores each individual on a spectrum from low to high risk. CEO Alex Martin has said<\/a> that the company\u2019s proprietary risk analysis can \u201cforever change for the better how human risk is measured.\u201d<\/p>\n

AC Global Risk, which boasts the consulting firm<\/a> of Robert Gates, Condoleezza Rice, and Stephen Hadley on its advisory board, has advertised contracts<\/a> with the U.S. Special Operations Command in Afghanistan, the Ugandan Wildlife Authority, and the security teams at Palantir, Apple, Facebook, and Google, among others. The extensive use of risk screening in these and other markets, Martin has said, has proven that it is \u201chighly accurate, scalable, cost-effective, and capable of high throughput.\u201d AC Global Risk claims that its RRA system can simultaneously process hundreds of individuals anywhere in the world. Now, in response to President Donald Trump\u2019s calls for the \u201cextreme vetting\u201d of immigrants, the company has pitched itself<\/a> as the ultimate solution for \u201cthe monumental refugee crisis the U.S. and other countries are currently experiencing.\u201d<\/p>\n

It\u2019s a proposal that would seem<\/a> to appeal to the U.S. Department of Homeland Security. The DHS has already funded research to develop similar AI technology for the border. The program, known as the\u00a0Automated Virtual Agent for Truth Assessments in Real-Time<\/a>, or AVATAR, used artificial intelligence to measure changes in the voice, posture, and facial gestures of travelers in order to flag those who appeared untruthful or seemed to pose a potential risk. In 2012 it was\u00a0tested<\/a> by volunteers at the U.S.-Mexico border. The European Union has also funded research into technology that would reduce\u00a0\u201cthe workload and subjective errors caused by human agents<\/a>.\u201d<\/p>\n

Some of the leading experts in vocal analytics, algorithmic bias, and machine learning find the trend toward digital polygraph tests troubling, pointing to the faulty methodology of companies like AC Global Risk. \u201cThere is some information in dynamic changes in the voice and they\u2019re detecting it. This is perfectly plausible,\u201d explained Alex Todorov, a Princeton University psychologist who studies the science of social perception and first impressions. \u201cBut the question is, How unambiguous is this information at detecting the category of people they\u2019ve defined as risky? There is always ambiguity in these kinds of signals.\u201d<\/p>\n

Over the past year, the American Civil Liberties Union<\/a>\u00a0and others<\/a> have reported that Border Patrol agents have been seizing people from Greyhound buses based on their appearance or accent<\/a>.\u00a0Because Customs and Border Protection agents already use information about how someone speaks or looks as a pretext to search individuals in the 100-mile border zone, or to deny individuals entry to the U.S., experts\u00a0fear that vocal\u00a0emotion detection software\u00a0could make such biases routine, pervasive, and seemingly \u201cobjective.\u201d<\/p>\n

AC Global Risk declined to respond to repeated requests for comment for this article. The company also did not respond to a list of detailed questions about how the technology works. In public\u00a0appearances, however, Martin has\u00a0claimed<\/a> that\u00a0the company\u2019s proprietary analytical processes can determine someone\u2019s risk level with greater than 97 percent accuracy. (AVATAR, meanwhile, claims an accuracy rate of between 60 and 70 percent.) Several\u00a0leading audiovisual experts who reviewed\u00a0AC Global Risk\u2019s publicly available materials for The Intercept used the word \u201cbullshit\u201d or \u201cbogus\u201d to describe\u00a0the company\u2019s claims. \u201cFrom an ethical point of view, it\u2019s very dubious and shady to give the impression that recognizing deception from only the voice can be done with any accuracy,\u201d said Bj\u00f6rn Schuller, a professor at the University of Augsburg who has led the field\u2019s major academic challenge event to advance the state of the art in vocal emotion detection. \u201cAnyone who says they can do this should themselves be seen as a risk.\u201d<\/p>\n

\"\"<\/a>

Alex Martin, CEO of AC Global Risk, speaks with a reporter after a Bloomberg Technology television interview in San Francisco on July 5, 2018.
Photo: David Paul Morris\/Bloomberg via Getty Images<\/p><\/div>\n

Risky Business<\/strong><\/p>\n

Trump\u2019s Extreme Vetting Initiative has called for software that can automatically \u201cdetermine and evaluate an applicant\u2019s probability of becoming a positively contributing member of society\u201d and predict \u201cwhether an applicant intends to commit criminal or terrorist acts after entering the United States,\u201d as The Intercept reported<\/a> last summer. AC Global Risk has pitched itself as the perfect tool for carrying out this initiative, offering<\/a> to assess \u201cthe risk levels of individuals with unknown loyalties, such as refugees and visa applicants.\u201d The DHS, the company says, would then decide how to act on the results of those reports. \u201cWith four levels to work with (low, average, potential, and high) it would not be hard to establish Departmental protocols according to risk level,\u201d the company stated<\/a> on its blog.<\/p>\n

Risk assessments in themselves are nothing new. In recent years, algorithms have been introduced at nearly every stage<\/a> in the criminal justice process, from policing\u00a0and bail to sentencing\u00a0and parole. The arrival of such techniques has not been without controversy<\/a>. Many of these automated tools have been criticized for their opacity<\/a>, secrecy<\/a>, and bias<\/a>. In many cases, officers, courts, and the public are not equipped \u2014 or allowed<\/a> \u2014 to interrogate their underlying assumptions, training sets, or conclusions. Chief among the concerns of skeptical experts is that the objective aura of machine learning may provide plausible cover for discrimination<\/a>.<\/p>\n

AC Global Risk provides few public details about how its technology works. It does not publish white papers backing up its research claims and has not released the scientific pedigrees of\u00a0its researchers. The company did not answer questions about what qualities (pitch, speed, inflection) and features the product measures. \u201cAs much as the use of risk assessment in criminal justice settings is problematic, it\u2019s much more accurate compared to this company\u2019s tool,\u201d said Suresh\u00a0Venkatasubramanian, a computer scientist at the University of Utah who focuses on algorithmic fairness.<\/p>\n

If any of AC Global Risk\u2019s claims for its technology are valid, they would represent the cutting edge of what researchers think is possible to ascertain<\/a> from the human voice. Vocal assessments can be excellent at quickly discerning demographic information. This information might be general \u2014 such as someone\u2019s age, gender, or dialect \u2014 but it can also be quite personal, revealing\u00a0the particular region someone is from, as well as\u00a0any health problems they might have.<\/p>\n

Last month, Amazon was issued<\/a> a patent that would allow its virtual assistant Alexa to determine users\u2019 vocal features, including language, accent, gender, and age. However, when it comes to determining emotions from the voice, accuracy remains a major concern. Schuller, the co-founder of audEERING<\/a>, a voice analytics company, says that it\u2019s currently not possible to tell whether someone is lying (if lying is, in fact, one of the company\u2019s\u00a0indices for risk) from the voice at greater than 70 percent accuracy, which is around the same as an average human judgment.<\/p>\n

Schuller said that it is possible to detect intoxication, sincerity, and deception, but again, the success rate is similar to an average human\u2019s abilities. \u201cWith a solid label, you can sometimes beat the human, but if something claims zero error, it should be taken with a grain of salt,\u201d he said.<\/p>\n

Central to assessing the validity of AC Global Risks\u2019 claims is what fits under the amorphous label of risk and who defines it. \u201cThey\u2019re defining risk as self-evident, as though it\u2019s a universal quality,\u201d said Joseph Pugliese, an Australian academic whose work focuses on biometric discrimination. \u201cIt assumes that people already know what risk is, whereas of course the question of who defines the parameters of risk and what constitutes those is politically loaded.\u201d<\/p>\n

CEO Alex Martin has spoken of looking \u201cfor actual risk along the continuum that is present in every human.\u201d Yet the idea that risk is an innate and legible human trait \u2014 and that\u00a0this trait\u00a0can be ascertained from just the voice \u2014 rests on flawed assumptions, explained Todorov, the Princeton psychologist. Our\u00a0ability to detect how people actually feel versus how we are perceiving them to feel has been a notoriously difficult problem in machine learning, Todorov continued. The possibility for mistaken impressions might be further complicated by the evaluative setting. \u201cPeople at the border are already in fraught and highly emotionally charged circumstances,\u201d Pugliese said. \u201cHow can they comply in a so-called normal way?\u201d<\/p>\n

A New Physiognomy?<\/strong><\/p>\n

AC Global Risk is part of a growing number of companies making outsized claims about the abilities of their behavioral analytics software. Encouraged by the observational prowess of artificial intelligence, many biometrics vendors and AI companies have been selling corporations and governments the ability to determine entire personalities from our facial expressions, movements, and voices. A biometrics vendor at the 2014 Winter Olympics in Russia, for instance, scanned<\/a> the expressions of attendees in order to give the country\u2019s security agency, the FSB, the ability to \u201cdetect someone who appears unremarkable but whose agitated mental state signals an imminent threat.\u201d<\/p>\n

Some skeptical experts who study AI and human behavior have framed these tools as part of a growing resurgence of interest in physiognomy, the practice of looking to the body for signs of moral character and criminal intent. In the mid-19th century, Cesare Lombroso\u2019s precise measurements of the skulls and facial features of \u201cborn criminals\u201d lent a scientific veneer to this interpretative practice. Yet while the efforts of criminal anthropologists like Lombroso have since been relegated to the dustbin of dangerous junk science, the desire to infer someone\u2019s moral character or hidden thoughts from physical features and behaviors has persisted.<\/p>\n

Underlying the efforts of AC Global Risk and similar companies, Pugliese says, is an assumption that the correlations of big data can circumvent the scientific method. These \u201cphysiognomic\u201d applications are especially troubling, he explains, given that machine learning algorithms are inherently designed to find superficial patterns (whether or not those patterns are \u201creal\u201d) among the data they\u2019re given. \u201cWhen they say they are triaging for risk, there is a self-evident notion that they have an objective purchase on the signs that constitute \u2018criminal intent\u2019\u201d Pugliese says. \u201cBut we don\u2019t know what actual signs would constitute these criminal predictors.\u201d<\/p>\n

Yet exposing the pseudoscientific premises of this technology does not necessarily make corporations and governments any less likely to use it. The power of these technologies \u2014 as with so many other predictive and risk-based systems \u2014 relies predominantly on their promise of efficacy and speed. \u201cTheir main claim is efficiency, making things faster, and in that sense, of course, it will work,\u201d Venkatasubramanian explained. Whether that efficiency helps or harms the life chances of those encountering these systems is, in other words, beside the point. The Remote Risk Assessment will be seen to be working insofar as humans enact its recommendations. As Todorov wrote in an essay with two machine learning experts to voice their concerns about this general\u00a0trend<\/a>: \u201cWhether intentional or not, this \u2018laundering\u2019 of human prejudice through computer algorithms can make those biases appear to be justified objectively.\u201d<\/p>\n

______________________________________________<\/p>\n

Related:<\/em><\/p>\n

Amazon\u2019s Accent Recognition Technology Could Tell the Government Where You\u2019re From<\/a><\/em><\/strong><\/p>\n

Forget About Siri and Alexa \u2014 When It Comes to Voice Identification, the \u201cNSA Reigns Supreme\u201d<\/strong><\/a><\/em><\/p>\n

\"\"<\/a><\/em><\/p>\n

 <\/p>\n

Ava Kofman<\/a> – kofman.ava@\u200bgmail.com<\/a><\/em><\/p>\n

 <\/p>\n

 <\/p>\n

Go to Original \u2013 theintercept.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

25 Nov 2018 – Risky Business – AC Global Risk claims to be able to determine your level of \u201crisk\u201d as an employee or an asylum-seeker based not on what you say, but how you say it. <\/p>\n","protected":false},"author":4,"featured_media":123070,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[60],"tags":[],"class_list":["post-123069","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-whistleblowing-surveillance"],"_links":{"self":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/123069","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/comments?post=123069"}],"version-history":[{"count":0,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/123069\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media\/123070"}],"wp:attachment":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media?parent=123069"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/categories?post=123069"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/tags?post=123069"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}