{"id":77324,"date":"2016-08-08T12:00:47","date_gmt":"2016-08-08T11:00:47","guid":{"rendered":"https:\/\/www.transcend.org\/tms\/?p=77324"},"modified":"2016-08-06T18:36:32","modified_gmt":"2016-08-06T17:36:32","slug":"microsoft-pitches-technology-that-can-read-facial-expressions-at-political-rallies","status":"publish","type":"post","link":"https:\/\/www.transcend.org\/tms\/2016\/08\/microsoft-pitches-technology-that-can-read-facial-expressions-at-political-rallies\/","title":{"rendered":"Microsoft Pitches Technology That Can Read Facial Expressions at Political Rallies"},"content":{"rendered":"

\"spying<\/a><\/p>\n

4 Aug 2016 – <\/em>On the 21st floor of a high-rise hotel in Cleveland, in a room full of political operatives, Microsoft\u2019s Research Division was advertising a technology that could read each facial expression in a massive crowd, analyze the emotions, and report back in real time.\u00a0\u201cYou could use this at a Trump rally,\u201d a sales representative\u00a0told me.<\/p>\n

At both the Republican and Democratic conventions, Microsoft sponsored event spaces for the news outlet Politico<\/em>. Politico,<\/em> in turn, hosted a series<\/a> of Microsoft-sponsored discussions about the use of data technology in political campaigns. And throughout Politico<\/em>\u2019s spaces in both Philadelphia and Cleveland, Microsoft advertised an array of products from \u201cMicrosoft Cognitive Services,\u201d its artificial intelligence and cloud computing division.<\/p>\n

At one exhibit, titled \u201cRealtime Crowd Insights,\u201d a small camera scanned the room, while a monitor displayed the captured image. Every five seconds, a new image would appear with data annotated for each face \u2014 an assigned serial number, gender, estimated age, and any emotions detected in the facial expression. When I approached, the machine labeled me \u201cb2ff\u201d and correctly identified me as a 23-year-old male.<\/p>\n

It interpreted my facial expression as \u201cneutral,\u201d with a bit of \u201csurprise.\u201d<\/p>\n

\"A<\/a>

A man uses an iris recognition scanner during the Biometrics 2004 exhibition in London.
Photo: Alex Emmons<\/p><\/div>\n

I asked Christina Pearson, a nearby Microsoft spokesperson, to confirm that the technology was meant to be used on a large crowd, like at a Trump rally. \u201cYes,\u201d she confirmed. \u201cOr it\u2019s meant to be the Super Bowl, whatever you want.\u201d<\/p>\n

\u201cRealtime Crowd Insights\u201d is an Application Programming Interface (API), or a software tool that connects web applications to Microsoft\u2019s cloud computing services.\u00a0Through Microsoft\u2019s emotional analysis API \u2014 a component of Realtime Crowd Insights \u2014 applications send an image to Microsoft\u2019s servers. Microsoft\u2019s servers then analyze the faces and return\u00a0emotional profiles for each one.<\/p>\n

In a November blog post<\/a>, Microsoft said that the emotional analysis could detect \u201canger, contempt, fear, disgust, happiness, neutral, sadness or surprise.\u201d<\/p>\n

Microsoft\u2019s sales representatives\u00a0told me that political campaigns could use the technology to measure the emotional impact of different talking points\u00a0\u2014 and\u00a0political scientists could use it to study crowd response at rallies.<\/p>\n

But the\u00a0use of facial analysis at political events is eerily reminiscent of George Orwell\u2019s\u00a01984<\/em>, where the government monitors faces for any sign of dissatisfaction, or \u201cfacecrime<\/a>.\u201d\u00a0In Orwell\u2019s world, \u201cto wear an improper expression on your face (to look incredulous when a victory was announced, for example) was itself a punishable offense.\u201d<\/p>\n

Microsoft\u2019s Realtime Crowd Insights could potentially pick out the stern faces of dissenters, or angry faces of future protestors, all in a matter of seconds.<\/p>\n

Donald Trump\u2019s security personnel have already tried to pre-empt protests at rallies by kicking out people they thought likely to protest. At one rally in February, security asked 30 black students to leave before Trump started speaking. According to USA Today<\/em><\/a>, the students\u00a0had planned to sit in silent protest, but one 19-year-old student said, \u201cWe didn\u2019t plan to do anything.\u201d<\/p>\n

In Politico<\/em>\u2019s suite in Cleveland, one passerby told me he was \u201cslightly creeped out,\u201d and another asked me why Microsoft was collecting their facial information. The machine also picked up on a small range of negative responses in the room, including \u201cfear, contempt, and disgust.\u201d<\/p>\n

\"Photo:<\/a>

Photo: Alex Emmons<\/p><\/div>\n

When I attended the \u201cRealtime Crowd Insights\u201d display in Philadelphia, I asked to speak with a spokesperson and was introduced to Kathryn Stack, a managing director with the public affairs firm Burson-Marsteller. I asked Stack whether the product could be used to identify protestors or dissidents at rallies or political events.<\/p>\n

\u201cI think that would be a question for a futurist, not a technologist,\u201d she responded.<\/p>\n

Facial recognition technology \u2014 the identification of faces by name \u2014 is already widely used in secret by law enforcement<\/a>, sports stadiums<\/a>, retail stores<\/a>, and even churches<\/a>, despite being of\u00a0questionable legality<\/a>. As early as 2002, facial recognition technology was used at the Super Bowl to cross-reference the 100,000 attendees to a database of the faces of known criminals. The technology is controversial enough that in 2013, Google tried to ban<\/a> the use of facial recognition apps in its Google glass system.<\/p>\n

But \u201cRealtime Crowd Insights\u201d is not true facial recognition \u2014 it could not identify me by name, only as \u201cb2ff.\u201d It did, however, store enough data on each face that it could continuously identify it\u00a0with the same serial number, even hours later. The display demonstrated that capability by distinguishing between the number of total faces it had seen, and the number of unique serial numbers.<\/p>\n

\"Photo:<\/a>

Photo: Alex Emmons<\/p><\/div>\n

Instead, \u201cRealtime Crowd Insights\u201d is an example of facial characterization technology \u2014 where computers analyze faces without necessarily identifying them. Facial characterization has many positive applications \u2014 it has been tested in the classroom<\/a>, as a tool for spotting struggling students, and Microsoft has boasted<\/a> that the tool will even help blind people read the faces around them.<\/p>\n

But facial characterization can also be used to assemble and store large profiles of information on individuals, even anonymously.<\/p>\n

Microsoft has traditionally adopted an \u201copt in\u201d policy with facial recognition, requiring users\u2019 consent before Microsoft can store an image of their face. The Kinetic Sensor on an Xbox, for example, allows users to sign in through facial recognition technology \u2014 but requires users to first give consent, according to Microsoft\u2019s privacy policy<\/a>.<\/p>\n

Microsoft has a similar code of conduct<\/a> for APIs, which\u00a0requires developers to \u201cobtain the consent of the people whose data (such as images, voices, video or text) are being processed by your app.\u201d<\/p>\n

Alvaro Bedoya, a professor at Georgetown Law School and expert on privacy and facial recognition, has hailed that code of conduct as evidence that Microsoft is trying to do the right thing. But he pointed out that it leaves a number of questions unanswered \u2014 as illustrated in Cleveland and\u00a0Philadelphia.<\/p>\n

\u201cIt\u2019s interesting that the app being shown at the convention \u2018remembered\u2019 the faces of the people who walked by. That would seem to suggest that their faces were being stored and processed without the consent that Microsoft\u2019s policy requires,\u201d Bedoya said. \u201cYou have to wonder: What happened to the face templates of the people who walked by that booth? Were they deleted? Or are they still in the system?\u201d<\/p>\n

Microsoft officials declined to comment on exactly what information is collected on each face and what data is retained or stored, instead referring me to their privacy policy<\/a>, which does not address the question.<\/p>\n

Bedoya also pointed out that Microsoft\u2019s\u00a0marketing did not seem to match the consent policy. \u201cIt\u2019s difficult to envision how companies will obtain consent from people in large crowds or rallies.\u201d<\/p>\n

________________________________<\/p>\n

Alex Emmons<\/a> – \u2709alex.emmons@theintercept.com<\/a><\/em><\/p>\n

Go to Original \u2013 theintercept.com<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"

Microsoft officials declined to comment on exactly what information is collected on each face and what data is retained or stored, instead referring me to their privacy policy, which does not address the question. Microsoft\u2019s marketing did not seem to match the consent policy. \u201cIt\u2019s difficult to envision how companies will obtain consent from people in large crowds or rallies.\u201d<\/p>\n","protected":false},"author":4,"featured_media":0,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[216],"tags":[],"class_list":["post-77324","post","type-post","status-publish","format-standard","hentry","category-technology"],"_links":{"self":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/77324","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/users\/4"}],"replies":[{"embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/comments?post=77324"}],"version-history":[{"count":0,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/posts\/77324\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/media?parent=77324"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/categories?post=77324"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.transcend.org\/tms\/wp-json\/wp\/v2\/tags?post=77324"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}