{"id":122372,"date":"2018-11-19T12:14:44","date_gmt":"2018-11-19T12:14:44","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=122372"},"modified":"2018-11-19T12:14:44","modified_gmt":"2018-11-19T12:14:44","slug":"amazons-accent-recognition-technology-could-tell-the-government-where-youre-from","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2018\/11\/amazons-accent-recognition-technology-could-tell-the-government-where-youre-from\/","title":{"rendered":"Amazon\u2019s Accent Recognition Technology Could Tell the Government Where You\u2019re From"},"content":{"rendered":"
\"\"<\/a>

Illustration: Erik Blad for The Intercept<\/p><\/div>\n

15 Nov 2018 – <\/em>At the beginning of October, Amazon was quietly issued a patent<\/a> that would allow its virtual assistant Alexa to decipher a user\u2019s physical characteristics and emotional state based on their voice. Characteristics, or \u201cvoice features,\u201d like language accent, ethnic origin, emotion, gender, age, and background noise would be immediately extracted and tagged to the user\u2019s data file to help deliver more targeted advertising.<\/p>\n

The algorithm would also\u00a0consider a customer\u2019s physical location \u2014 based on their IP address, primary shipping address, and browser settings \u2014 to help determine their accent. Should Amazon\u2019s patent become a reality, or if accent detection is already possible, it would introduce questions of surveillance and privacy violations,\u00a0as well as possible discriminatory advertising, experts said.<\/p>\n

The civil rights issues raised by the patent are similar to those around facial recognition, another technology Amazon has used as an anchor of its artificial intelligence strategy, and one that it controversially marketed to law enforcement. Like facial recognition, voice analysis underlines how existing laws and privacy safeguards simply aren\u2019t capable of protecting users from new categories of data collection \u2014 or government spying, for that matter. Unlike facial recognition, voice analysis relies not on cameras in public spaces, but microphones inside smart speakers in our homes. It also raises its own thorny issues around advertising that targets or excludes certain groups of people based on derived characteristics like nationality, native language, and so on (the sort of controversy that Facebook has stumbled into again<\/a> and again<\/a>).<\/p>\n

\"\"<\/a>

From Amazon\u2019s patent, an illustration of a process for determining physical and emotional characteristics from someone\u2019s voice, resulting in tailored audio content like ads. Document: United States Patent and Trademark Office<\/p><\/div>\n

Why the Government Might Be Interested in Accent Data<\/strong><\/p>\n

If voice-based accent detection can determine a person\u2019s ethnic background, it opens up a new category of information that is incredibly interesting to the government, said Jennifer King, director of consumer privacy at Stanford Law School\u2019s Center for Internet and Society.<\/p>\n

\u201cIf you\u2019re a company and you\u2019re creating new classifications of data, and the government is interested in them, you\u2019d be naive to think that law enforcement isn\u2019t going to come after it,\u201d she said.<\/p>\n

She described a scenario in which\u00a0knowing a user\u2019s purchase history, existing demographic data, and whether they speak Arabic or Arabic-accented English, Amazon could identify the user as belonging to a religious or ethnic group. King said it\u2019s plausible that the FBI would compel the production of such data from Amazon if it could help determine a user\u2019s membership to a terrorist group. Data demands focused on terrorism are tougher for companies to fight, she said, as opposed to those that are vague or otherwise overbroad, which they have pushed back on.<\/p>\n

Andrew Crocker, a senior staff attorney at the Electronic Frontier Foundation, said the Foreign Intelligence Surveillance Act, or FISA, makes it possible for the government to covertly demand such data. FISA governs electronic spying conducted to acquire information on foreign powers, allowing such monitoring without a warrant in some circumstances and in others under warrants issued by a court closed to the public, with only the government represented. The communications of U.S. citizens and residents are routinely acquired under the law, in many cases incidentally, but even incidentally collected communications may later be used against Americans in FBI investigations. Under FISA, the government could \u201cget information in secret more easily, and there are mass or bulk surveillance capabilities that don\u2019t exist in domestic law,\u201d said Crocker. \u201cCertainly it could be done in secret with less court oversight.\u201d<\/p>\n

\u201cYou\u2019d be naive to think that law enforcement isn\u2019t going to come after it.\u201d<\/em><\/strong><\/p><\/blockquote>\n

Jennifer Granick, a surveillance and cybersecurity lawyer at the American Civil Liberties Union\u2019s Speech, Privacy, and Technology Project, suggested that Amazon\u2019s accent data could also provide the government with information for the purpose of immigration control.<\/p>\n

\u201cLet\u2019s say you have ICE go to one of these providers and say, \u2018Give us all the subscription information of people who have Spanish accents\u2019 \u2026 in order to identify people of a particular race or who theoretically might have relatives who are undocumented,\u201d she said. \u201cSo you can see that this type information can definitely be abused.\u201d<\/p>\n

Though King said she hasn\u2019t seen evidence of these types of government requests, she has witnessed \u201cparallel things happen in other contexts.\u201d It\u2019s also possible that if Amazon\u00a0was sent a National Security Letter by the FBI, a gag order would prevent\u00a0the company from disclosing much, including the exact number of letters\u00a0it received.\u00a0National Security Letters compel the disclosure of certain types of information from communications firms, like a subpoena would, but often in secret. The letters require the companies to hand over select data, like the name of an account owner and the age of an account, but the FBI has routinely<\/a> asked for more, including email headers and internet browsing history.<\/p>\n

Compared to some other tech giants, however, Amazon is less detailed in its disclosures about National Security Letters it receives and about data requests in general. For example, in its information request reports<\/a>, it does not disclose how many NSLs it has received or how many accounts are affected by national security requests, as Apple and Google do. These more specific disclosures from other companies show a trend: From mid-2016 to the first half of 2017, national security requests\u00a0sent to Apple, Facebook, and Google\u00a0increased significantly<\/a>.<\/p>\n

But even if the government hasn\u2019t yet made such requests of Amazon, we know that it has been paying attention to voice and speech technology for some time. In January, The Intercept<\/a> reported that the National Security Agency had developed technology not just to record and transcribe private conversations, but also to automatically identify speakers. An individual\u2019s \u201cvoiceprint\u201d was created, which could be cross-referenced with millions of intercepted and recorded telephone, video, and internet calls.<\/p>\n

To create an American citizen\u2019s \u201cvoiceprint,\u201d which government documents don\u2019t explicitly indicate has been done, experts said the NSA would need only to tap into Amazon or Google\u2019s existing voice data.<\/p>\n

Over the past year, Amazon\u2019s relationship with the government has become increasingly cozy. BuzzFeed<\/a> recently revealed details about how the Orlando Police Department was piloting Rekognition, Amazon\u2019s facial recognition technology, to identify \u201cpersons of interest.\u201d A few months earlier, Amazon was outed by the ACLU<\/a> for \u201cmarketing Rekognition for government surveillance.\u201d Meanwhile, in June, the company was busy pitching Immigration and Customs Enforcement<\/a>\u00a0officials on its technology.<\/p>\n

Though these revelations have set off alarm bells, even for Amazon employees<\/a>, experts said that speech recognition presents similar concerns that are equally if not more pressing. Amazon\u2019s voice processing patent dates to March of last year. The company, in response to questions from The Intercept, described the patent as exploratory and pledged to abide by its privacy policy when collecting and using data.<\/p>\n

Privacy Law Lags Behind Technology<\/strong><\/p>\n

Weak privacy laws in the U.S. are one reason consumers\u00a0are vulnerable when tech companies start gathering new types of data about them. There is nothing in the law that protects data collected about a person\u2019s mood or accent, said Granick.<\/p>\n

In the absence of strong legal protections, consumers are forced to make their own decisions about trade-offs between their privacy and the convenience of virtual assistants. \u201cBeing able to use really robust voice control would be great if it meant you weren\u2019t just being put into a giant AI algorithm and being used to improve your pitchability for new products, especially when you\u2019re paying for these systems,\u201d said King.<\/p>\n

The Electronic Communications Privacy Act, or ECPA, first passed in 1986, was a major step forward in privacy protection at the time. But now, over 30 years later, it has yet to catch up with the pace of technological innovation.\u00a0Generally, under ECPA, government agencies need\u00a0a subpoena, court order, or search warrant to compel companies to disclose protected user information.\u00a0Unlike court orders and search warrants,\u00a0subpoenas\u00a0don\u2019t necessarily require judicial review.<\/p>\n

Amazon\u2019s most recent information request report<\/a>, which\u00a0covers the first half of this year, reveals that the company received 1,736 subpoenas during that time,\u00a01,283 of which\u00a0it\u00a0turned over\u00a0some or all of the information requested. Since\u00a0Amazon\u00a0began publishing\u00a0these reports three years ago, the number of data requests\u00a0it receives has steadily increased, with a huge jump between 2015 and 2016. Echo, its Alexa-enabled speaker for use at home, was released in 2015, and Amazon has\u00a0not said<\/a> whether\u00a0the\u00a0increase is\u00a0related to the\u00a0growing\u00a0popularity of\u00a0its\u00a0speaker.<\/p>\n

While ECPA protects the data associated with our digital conversations, Granick said its application to information collected by providers like Amazon is \u201canemic.\u201d \u201cThe government could say, \u2018Give us a list of everyone who you think is Chinese, Latino\u2019 and the provider has to argue why they shouldn\u2019t. That kind of conclusory data isn\u2019t protected by ECPA, and it means the government can compel its disclosure with a subpoena,\u201d she said.<\/p>\n

\u201cThe government could say, \u2018Give us a list of everyone who you think is Chinese, Latino.\u2019\u201d<\/em><\/strong><\/p><\/blockquote>\n

Since ECPA is not explicit, there\u2019s a legal question of whether Amazon could voluntarily turn over the conversations its users have with Alexa. Granick said Amazon could argue that such data is a protected electronic communication under ECPA \u2014 and require that the government get a warrant to access it \u2014 but\u00a0as a party to the communication, Amazon also\u00a0has the right to divulge it. There have been no court cases addressing the issue so far, she said.<\/p>\n

Crocker, of the EFF, argued that communications with Alexa \u2014 voice searches and commands, for example \u2014 are protected by ECPA, but agreed that the government could obtain the data through a warrant or other legal process.<\/p>\n

He added that the way Amazon stores accent information could impact the government\u2019s ability to access it. If Amazon keeps it stored long term in customer profiles in the cloud, those profiles are easier to obtain than real-time voice communications, the interception of which invokes protections under the Wiretap Act, a federal law governing the interception and disclosure of communications. (ECPA amended the Wiretap Act to include electronic communications.) And if it\u2019s stored in metadata associated with Alexa voice searches, the government has obtained those in past cases and could get to it that way.<\/p>\n

In 2016, a hot tub murder in Arkansas<\/a> put in the spotlight the defendant\u2019s Echo. Police seized the device and tried to obtain its recordings, prompting Amazon to argue that any communications with Alexa, and its responses, were protected as a form of free speech<\/a> under the First Amendment. Amazon eventually turned over the records after the defendant gave his permission to do so. Last week, a New Hampshire judge ordered Amazon<\/a> to turn over Echo recordings in\u00a0a new murder case, again hoping it could provide criminal evidence.<\/p>\n

Dynamic, Targeted, and Discriminatory Ads<\/strong><\/p>\n

Beyond government surveillance, experts also expressed concern that Amazon\u2019s new classification of data increases the likelihood of discriminatory advertising. When demographic profiling \u2014 the basis of traditional advertising \u2014 is layered with additional, algorithmically assessed information from Alexa, ads can quickly become invasive or offensive.<\/p>\n

Granick said that if Amazon \u201cgave one set of [housing] ads to people who had Chinese accents and a different set of ads to people who have a Finnish accent \u2026 highlighting primarily Chinese neighborhoods for one and European neighbors for another,\u201d it would be discriminating on the basis of national origin or race.<\/p>\n

King said Amazon\u00a0also opens itself to charges of price discrimination, and even racism, if it allows advertisers to show and hide ads from certain ethnic or gender groups. \u201cIf you live in the O.C. and you have a Chinese accent and are upper middle-class, it could show you things that are higher price. [Alexa] might say, \u2018I\u2019m gonna send you Louis Vuitton bags based on those things,\u2019\u201d she said.<\/p>\n

Selling products based on emotions also offers opportunities for advertisers to manipulate consumers. \u201cIf you\u2019re a woman in a certain demographic and you\u2019re depressed, and we know that binge shopping is something you do \u2026 knowing that you\u2019re in kind of a vulnerable state, there\u2019s no regulation preventing them from doing something like this,\u201d King said.<\/p>\n

An example from the patent envisions marketing to Chinese speakers, albeit in a more innocuous context, describing how ads might be targeted to \u201cmiddle-aged users who speak Mandarin or have a Chinese accent and live in the United States.\u201d If the user asks, \u201cAlexa, what\u2019s the news today?\u201d Alexa might reply, \u201cBefore your news brief, you might be interested in the Xiaomi TV box, which allows you to watch over 1,000 real-time Chinese TV channels for just $49.99. Do you want to buy it?\u201d<\/p>\n

According to the patent, the ads may be presented in response to user voice input, but could also be \u201cpresented at any time.\u201d They could even be injected into existing audio streams, such as a news briefing or playback of tracks from a music playlist.<\/p>\n

\"\"<\/a>

Andrew DeVore, vice president and associate general counsel with Amazon.com Inc., right, listens as Len Cali, senior vice president of global public policy with AT&T Inc., speaks during a Senate Commerce Committee hearing on consumer data privacy in Washington, D.C., on Sept. 26, 2018.
Photo: Andrew Harrer\/Bloomberg via Getty Images<\/p><\/div>\n

New Rules Emerge for Data Privacy<\/strong><\/p>\n

In the wake of Facebook\u2019s Cambridge Analytica scandal<\/a>, lawmakers have grown increasingly wary of tech companies and their privacy practices. In late September<\/a>, the Senate Commerce Committee held a fresh round of hearings with tech executives on the issue \u2014 also giving them an opportunity to explain how they\u2019re addressing the new, stringent data privacy laws in the European Union<\/a> and California<\/a>.<\/p>\n

California\u2019s regulation, which passed in June and goes into effect in 2020, sets a new precedent for consumer privacy law in the country. It expands the definition of personal information and gives state residents greater control over the sharing and sale of their data to third parties.<\/p>\n

Unsurprisingly, Amazon and other big tech companies pushed back forcefully on the new reforms, citing excessive penalties, compliance costs, and data collection restrictions \u2014 and each spent nearly $200,000 to defeat it<\/a>. During the Senate hearing, Amazon Vice President and Associate General Counsel Andrew DeVore<\/a> asked the committee to consider the \u201cunintended consequences\u201d of California\u2019s law, which he called \u201cconfusing and difficult to comply with.\u201d<\/p>\n

Now that the midterm elections have passed, privacy advocates hope that congressional interest in privacy issues will turn into legislative action; thus far, it has not. The Federal Trade Commission is also considering updates to its consumer protection enforcement priorities. In September, it kicked off a series of hearings<\/a> examining the impact of \u201cnew technologies\u201d and other economic changes.<\/p>\n

Advocates hope congressional interest in privacy issues will turn into legislative action; thus far, it has not.<\/em><\/strong><\/p><\/blockquote>\n

Granick\u00a0said that as states move to protect consumers where the federal government\u00a0has not, California could serve as a model for the rest of the country. In August, California also became the first state to pass an internet of things cybersecurity law<\/a>, requiring that manufacturers add a \u201creasonable security feature\u201d to protect the information it collects from unauthorized access, modification, or disclosure.<\/p>\n

In 2008, Illinois became the first state to pass a law regulating biometric data<\/a>, placing restrictions on the collection and storing of iris scan, fingerprint, voiceprint, hand scan, and face geometry data. (Granick says it\u2019s unclear if accent data is covered under the law.) Being the first state to pass landmark legislation, Illinois\u00a0presents a cautionary tale for California. Though its bill was once considered a model law, only two other states \u2014 Texas and Washington \u2014 have passed biometric privacy laws over the past\u00a010 years. Similar efforts elsewhere were largely killed by corporate lobbying.<\/p>\n

A Growing and Global Problem<\/strong><\/p>\n

Activists have looked to other countries as examples of what could go wrong if tech companies and government agencies become too friendly, and voice accent data gets misused.<\/p>\n

Human Rights Watch<\/a> reported last year that the Chinese government was creating a national voice biometric database using data from Chinese tech company iFlyTek, which provides its consumer voice recognition apps for free and\u00a0claims its system can support 22 Chinese dialects. On its English website, iFlyTek said that its technology has been \u201cinspected and praised\u201d by \u201cmany party and state leaders,\u201d including President Xi Jinping and Premier Li Keqiang.<\/p>\n

The company is also the supplier of voice pattern collection systems\u00a0used by regional police bureaus and runs a lab that develops voice surveillance technology for the Ministry of Public Security. Its technology has \u201chelped solve cases\u201d for law enforcement in Anhui, Gansu, Tibet, and Xinjiang, according to a state\u00a0press report cited by Human Rights Watch.\u00a0Activists warn that one possible use of the government\u2019s voice database, which could contain dialect and accent-rich voice data from minority groups, is the surveillance of Tibetans and Uighurs.<\/p>\n

Last year, Die Welt<\/a> reported that the German government was testing voice analysis software to help verify where its refugees are coming from.<\/a> They hoped it would determine the dialects of people seeking asylum in Germany, which migration officers would use as one of several \u201cindicators\u201d when reviewing applications. The test was met with skepticism, as speech experts questioned the ability of software to make such a complex determination.<\/p>\n

The amount of information people voluntarily give tech companies through smart speakers is growing, along with the purchases users are\u00a0allowed to make. A Gallup survey<\/a> conducted last year found that 22 percent of Americans currently use smart home personal assistants like Echo \u2014 placing them in living rooms, kitchens, and other intimate spaces. And 44 percent of U.S. adult internet users are planning to buy one, according to a Consumer Technology Association study<\/a>.<\/p>\n

Amazon\u2019s move into the home with more sophisticated voice abilities for Alexa has been a long time coming. In 2016<\/a>, it was already discussing emotion detection as a way to stay ahead of competitors Google and Apple. Also that year, it filed a patent application<\/a> for a real-time language accent translator \u2014 a blend of accent detection and translation technologies. When its emotion and accent patent was issued last month, Alexa\u2019s potential ability to read emotions and detect if customers are sick was called out as creepy<\/a>.<\/p>\n

Amazon Grapples With an \u201cAccent Gap\u201d<\/strong><\/p>\n

Amazon\u2019s current accent handling capabilities are lackluster. In July, the Washington Post <\/a>charged that Amazon and Google had created an \u201caccent gap,\u201d leaving non-native English speakers behind in the voice-activated technology revolution. Both Alexa and Google Assistant had the most difficulty understanding Chinese- and Spanish-accented English.<\/p>\n

Since the advent of speech recognition technology, picking up on dialects, speech impediments, and accents has been a persistent challenge. If the technology in Amazon\u2019s patent was available today, natural language processing experts said that the accent and emotion detection would not be able to draw precise conclusions. The training data that teaches artificial intelligence lacks diversity in the first place, and because language itself is constantly changing, any AI would have a hard time keeping up.<\/p>\n

Though Amazon\u2019s new patent is a sign that it\u2019s paying attention to the \u201caccent gap,\u201d it may be doing so for the wrong reasons. Improved language accent detection makes voice technology more equitable and accessible, but it comes at a cost.<\/p>\n

Regarding Patents<\/strong><\/p>\n

Patents are not a surefire sign of what tech companies have built, or what is even possible for them to build. Tech companies in particular submit a dizzying number of patent applications.<\/p>\n

In an emailed statement, Amazon said that it filed \u201ca number of forward-looking patent applications that explore the full possibilities of new technology. Patents take multiple years to receive and do not necessarily reflect current developments to products and services.\u201d The company also said that it \u201cwill only collect and use data in accordance with our privacy policy,\u201d and did not elaborate on other uses of its technology or data.<\/p>\n

But King, who has also reviewed numerous Facebook patents, said that they can be used to infer the direction a company is headed.<\/p>\n

\u201cYou\u2019re seeing a future where the interactions with people and their interior spaces is getting a lot more aggressive,\u201d she said. \u201cThat\u2019s the next frontier for companies. Not just tracking your behavior, where you\u2019ve gone, what they think you might buy. Now it\u2019s what you\u2019re thinking, feeling, and that is what makes people deeply uncomfortable.\u201d<\/p>\n

For now, people who want to hold onto their privacy and minimize\u00a0surveillance\u00a0risk shouldn\u2019t buy a speaker at all, recommended Granick. \u201cYou\u2019re basically installing a microphone for the government to listen in to you in your home,\u201d she said.<\/p>\n

________________________________________________<\/p>\n

Related:<\/em><\/p>\n