{"id":23553,"date":"2018-08-22T10:00:35","date_gmt":"2018-08-22T14:00:35","guid":{"rendered":"https:\/\/www.kaspersky.com\/blog\/?p=23553"},"modified":"2020-12-25T11:41:38","modified_gmt":"2020-12-25T16:41:38","slug":"machine-learning-nine-challenges","status":"publish","type":"post","link":"https:\/\/www.kaspersky.com\/blog\/machine-learning-nine-challenges\/23553\/","title":{"rendered":"Machine learning: 9 challenges"},"content":{"rendered":"<p>The future will probably be awesome, but at present, artificial intelligence (AI) poses some questions, and most often they have to do with morality and ethics. How has machine learning already surprised us? Can you trick a machine, and if so, how difficult is it? And will it all end up with Skynet and rise of the machines? Let\u2019s take a look.<\/p>\n<h2>Strong and weak artificial intelligence<\/h2>\n<p>First, we need to differentiate between two concepts: strong and weak AI. Strong AI is a hypothetical machine that\u2019s able to think and is aware of its own existence. It can solve not only tailored tasks, but also learn new things.<\/p>\n<p>Weak AI already exists. It is in applications made to solve specific problems, such as image recognition, car driving, playing Go, and so on. Weak AI is the thing we call \u201cmachine learning.\u201d<\/p>\n<p>We don\u2019t know yet whether strong AI can be invented. According to <a target=\"_blank\" href=\"https:\/\/www.technologyreview.com\/s\/607970\/experts-predict-when-artificial-intelligence-will-exceed-human-performance\/\" rel=\"noopener noreferrer nofollow\">expert surveys<\/a>, we\u2019ll have to wait another 45 years. That really means \u201csomeday.\u201d For example, experts also say fusion power will be commercialized in 40 years \u2014 which is exactly what they said 50 years ago.<\/p>\n<h2>What could go wrong?<\/h2>\n<p>It\u2019s still unclear when strong AI will be developed, but weak AI is already here, working hard in many areas. The number of those areas grows every year. Machine learning lets us handle practical tasks without obvious programming; it learns from examples. For more details, see \u201c<a target=\"_blank\" href=\"https:\/\/www.kaspersky.com\/blog\/machine-learning-explained\/13487\/\" rel=\"noopener noreferrer nofollow\">How machine learning works, simplified<\/a>.\u201d<\/p>\n<p>We teach machines to solve concrete problems, so the resulting mathematical model \u2014 what we call a \u201clearning\u201d algorithm \u2014 can\u2019t suddenly develop a hankering to defeat (or save) humanity. In other words, we shouldn\u2019t be afraid of a Skynet situation from weak AI. But some things could still go wrong.<\/p>\n<h3>1. Bad intentions<\/h3>\n<p>If we teach an army of drones to kill people using machine learning, can the results be ethical?<\/p>\n<p><span class=\"embed-youtube\" style=\"text-align:center; display: block;\"><iframe class=\"youtube-player\" type=\"text\/html\" width=\"640\" height=\"390\" src=\"https:\/\/www.youtube.com\/embed\/TlO2gcs1YvM?version=3&amp;rel=1&amp;fs=1&amp;showsearch=0&amp;showinfo=1&amp;iv_load_policy=1&amp;wmode=transparent\" frameborder=\"0\" allowfullscreen=\"true\"><\/iframe><\/span><\/p>\n<p>A small scandal broke last year surrounding this very topic. Google <a target=\"_blank\" href=\"https:\/\/gizmodo.com\/google-employees-resign-in-protest-against-pentagon-con-1825729300\" rel=\"noopener noreferrer nofollow\">is developing software<\/a> used for a military project called Project Maven that involves drones. In the future, it may help create completely autonomous weapon systems.<\/p>\n<p>As a result, 12 Google employees resigned in protest and 4,000 more signed a petition requesting the company abandon the contract with the military. More than 1,000 well-known scientists in the fields of AI, ethics, and IT wrote an <a target=\"_blank\" href=\"https:\/\/www.icrac.net\/open-letter-in-support-of-google-employees-and-tech-workers\/\" rel=\"noopener noreferrer nofollow\">open letter<\/a> to Google, asking the company to abandon the project and support an international agreement that would ban autonomous weapons.<\/p>\n<h3>2. Developer bias<\/h3>\n<p>Even if machine-learning algorithm developers mean no harm, a lot of them still want to make money \u2014 which is to say, their algorithms are created to benefit the developers, not necessarily for the good of society. Some medical algorithms might recommend expensive treatments over the treatments with the best patient outcomes, for example.<\/p>\n<p>Sometimes society itself has no interest in an algorithm becoming a moral paragon. For example, there is a compromise between traffic speed and the car accident death rate. We could program autonomous cars to drive no faster than 15 mph, which would almost guarantee to bring the number of road fatalities to zero but negate other benefits of using a car.<\/p>\n<h3>3. System parameters not always include ethics<\/h3>\n<p>Computers by default don\u2019t know anything about ethics. An algorithm can put together a national budget with the goal of \u201cmaximizing GDP\/labor productivity\/life expectancy,\u201d but without ethical limitations programmed into the model, it might eliminate budgets for schools, hospices, and the environment, because they don\u2019t directly increase the GDP.<\/p>\n<p>With a broader goal, it might decide to increase productivity by getting rid of anyone who is unable to work.<\/p>\n<p>The point is, ethical issues must be incorporated from the very beginning.<\/p>\n<h3>4. Ethical relativity<\/h3>\n<p>Ethics change over time, and sometimes quickly. For example, opinions on such issues as LGBT rights and interracial or intercaste marriage can change significantly within a generation.<\/p>\n<p>Ethics can also vary between groups within the same country, never mind in different countries. For example, in China, using face recognition for <a target=\"_blank\" href=\"http:\/\/www.businessinsider.com\/how-china-is-watching-its-citizens-in-a-modern-surveillance-state-2018-4\" rel=\"noopener noreferrer nofollow\">mass surveillance<\/a> has become the norm. Other countries may view this issue differently, and the decision may depend on the situation.<a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114743\/machine-learning-challenges-face-recognition-china.png\"><img decoding=\"async\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114743\/machine-learning-challenges-face-recognition-china-1024x590.png\" alt=\"\" width=\"1024\" height=\"590\" class=\"aligncenter size-large wp-image-23554\"><\/a><\/p>\n<p>The political climate matters, too. For example, the war on terrorism has significantly \u2014 and incredibly quickly \u2014 changed some ethical norms and ideals in many countries.<\/p>\n<h3>5. Machine learning changes humans<\/h3>\n<p>Machine-learning systems \u2014 just one example of AI that affects people directly \u2014 recommend new movies to you based on your ratings of other films and after comparing your preferences with those of other users. Some systems are getting pretty good at it.<\/p>\n<p>A movie-recommendation system changes your preferences over time and narrows them down. Without it, you\u2019d occasionally face the horror of watching bad movies and movies of unwanted genres. Using the AI, every movie hits the spot. In the end, you stop investigating and just consume what is fed to you.<\/p>\n<p>It\u2019s also interesting that we don\u2019t even notice how we get manipulated by algorithms. The movie example is not that scary, but consider news and propaganda.<\/p>\n<h3>6. False correlations<\/h3>\n<p>A false correlation occurs when things completely independent of each other exhibit a very similar behavior, which may create the illusion they are somehow connected. For example, did you know that margarine consumption in the US correlates strongly on the divorce rate in Maine?<a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114844\/machine-learning-challenges-false-correlation-EN.png\"><img decoding=\"async\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114844\/machine-learning-challenges-false-correlation-EN-1024x485.png\" alt=\"\" width=\"1024\" height=\"485\" class=\"aligncenter size-large wp-image-23558\"><\/a><\/p>\n<p>Of course, real people, relying on their personal experience and human intelligence, will instantly recognize that any direct connection between the two is extremely unlikely. A mathematical model can\u2019t possess such knowledge \u2014 it simply learns and generalizes data.<\/p>\n<p>A well-known example is a program that sorted patients by how urgently they required medical help and concluded that asthma patients who had pneumonia didn\u2019t need help as badly as pneumonia patients without asthma. The program looked at the data and concluded that asthma patients were in less danger of dying and therefore should not be a priority. In fact, their death rates were so low because they always received urgent help at medical facilities because of the high risks inherent to their condition.<\/p>\n<h3>7. Feedback loops<\/h3>\n<p>Feedback loops are even worse than false correlations. A feedback loop is a situation where an algorithm\u2019s decisions affect reality, which in turn convinces the algorithm that its conclusion is correct.<\/p>\n<p>For example, a crime-prevention program in California suggested that police should send more officers to African-American neighborhoods based on the crime rate \u2014 the number of recorded crimes. But more police cars in a neighborhood led to local residents reporting crimes more frequently (someone was right there to report them to), which led to officers writing up more protocols and reports, which resulted in a higher crime rate \u2014 which meant more officers had to be sent to the area.<\/p>\n<h3>8. \u201cContaminated\u201d or \u201cpoisoned\u201d reference data<\/h3>\n<p>The results of algorithm learning depend largely on reference data, which form the basis of learning. The data may turn out to be bad and distorted, however, by accident or through someone\u2019s malicious intent (in the latter case, it\u2019s usually called \u201cpoisoning\u201d).<\/p>\n<p>Here is an example of unintended problems with reference data. If the data used as a training sample for a hiring algorithm has been obtained from a company with racist hiring practices, the algorithm will also be racist.<\/p>\n<p>Microsoft once taught a chatbot to communicate on Twitter by letting anyone chat with it. They had to <a target=\"_blank\" href=\"https:\/\/www.theverge.com\/2016\/3\/24\/11297050\/tay-microsoft-chatbot-racist\" rel=\"noopener noreferrer nofollow\">pull the plug on the project<\/a> in less than 24 hours because kind Internet users quickly taught the bot to swear and recite Mein Kampf.<\/p>\n<p>https:\/\/twitter.com\/geraldmellor\/status\/712880710328139776<\/p>\n<p>Here is an example of machine learning data being poisoned. A mathematical model at a computer virus analysis lab processes an average of 1 million files per day, both clean and harmful. The threat landscape keeps changing, so model changes are delivered to products installed on the clients\u2019 side in the form of antivirus database updates.<\/p>\n<p>A hacker can keep generating malicious files, very similar to clean ones, and send them to the lab. That action gradually erases the line between clean and harmful files, degrading the model and perhaps eventually triggering a <a target=\"_blank\" href=\"https:\/\/encyclopedia.kaspersky.com\/glossary\/false-positive\/?utm_source=kdaily&amp;utm_medium=blog&amp;utm_campaign=termin-explanation\" rel=\"noopener noreferrer\">false positive<\/a>.<\/p>\n<p>This is why Kaspersky Lab has <a target=\"_blank\" href=\"https:\/\/www.kaspersky.com\/blog\/multilayered-approach\/6601\/\" rel=\"noopener noreferrer nofollow\">a multilayered security model<\/a> and <a target=\"_blank\" href=\"https:\/\/securelist.com\/five-myths-about-machine-learning-in-cybersecurity\/76351\/\" rel=\"noopener noreferrer\">does not rely<\/a> exclusively on machine learning. Real people \u2014 antivirus experts \u2014 always monitor what the machine is doing.<\/p>\n<h3>9. Trickery<\/h3>\n<p>Even a well-functioning mathematical model \u2014 one that relies on good data \u2014 can still be tricked, if one knows how it works. For example, a group of researchers figured out how to trick a facial-recognition algorithm using special glasses that would introduce minimal distortions into the image and thus completely alter the result.<\/p><div id=\"attachment_23555\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114804\/machine-learning-challenges-fail-1.png\"><img decoding=\"async\" aria-describedby=\"caption-attachment-23555\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114804\/machine-learning-challenges-fail-1-1024x614.png\" alt=\"\" width=\"1024\" height=\"614\" class=\"size-large wp-image-23555\"><\/a><p id=\"caption-attachment-23555\" class=\"wp-caption-text\">Wearing glasses with specially colored rims, researchers <a target=\"_blank\" href=\"https:\/\/www.theguardian.com\/technology\/2016\/nov\/03\/how-funky-tortoiseshell-glasses-can-beat-facial-recognition\" rel=\"noopener noreferrer nofollow\">tricked a facial recognition algorithm<\/a> into thinking they were someone else<\/p><\/div>\n<p>Even in situations that don\u2019t appear to involve anything complicated, a machine can easily be tricked using methods unknown to a layperson.<\/p><div id=\"attachment_23556\" style=\"width: 859px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114815\/machine-learning-challenges-fail-2.png\"><img decoding=\"async\" aria-describedby=\"caption-attachment-23556\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114815\/machine-learning-challenges-fail-2.png\" alt=\"\" width=\"849\" height=\"309\" class=\"size-full wp-image-23556\"><\/a><p id=\"caption-attachment-23556\" class=\"wp-caption-text\">The first three signs <a target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/1412.6572.pdf\" rel=\"noopener noreferrer nofollow\">are recognized<\/a> as 45 km\/h speed limit signs and the last one as a STOP sign<\/p><\/div>\n<p>Moreover, to bring down a machine-learning mathematical model, the changes don\u2019t have to be significant \u2014 <a target=\"_blank\" href=\"https:\/\/www.kaspersky.com\/blog\/ai-fails\/18318\/\" rel=\"noopener noreferrer nofollow\">minimal changes, indiscernible<\/a> to human eye will suffice.<\/p><div id=\"attachment_23557\" style=\"width: 1034px\" class=\"wp-caption aligncenter\"><a href=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114830\/machine-learning-challenges-fail-3.png\"><img decoding=\"async\" aria-describedby=\"caption-attachment-23557\" src=\"https:\/\/media.kasperskydaily.com\/wp-content\/uploads\/sites\/92\/2018\/08\/21114830\/machine-learning-challenges-fail-3-1024x333.png\" alt=\"\" width=\"1024\" height=\"333\" class=\"size-large wp-image-23557\"><\/a><p id=\"caption-attachment-23557\" class=\"wp-caption-text\">Add minor noise to the panda on the left and <a target=\"_blank\" href=\"https:\/\/arxiv.org\/pdf\/1312.6199.pdf\" rel=\"noopener noreferrer nofollow\">you might get<\/a><strong> a gibbon<\/strong><\/p><\/div>\n<p>As long as humanity is still smarter than most algorithms, humans will be able to trick them. Consider near-future machine learning that analyzes luggage X-rays at the airport and looks for weapons. A smart terrorist will be able to put an object of a certain shape next to a gun and thus make the gun invisible.<\/p>\n<div class=\"jsBrightTALKEmbedWrapper\" style=\"border:1px solid #eee;width:100%; height:285px;padding:0 30px;overflow:auto;  position:relative;background: #ffffff;\"><script class=\"jsBrightTALKEmbedConfig\" type=\"application\/json\">{ \"channelId\" : 15321, \"language\": \"en-US\", \"commId\" : 332802, \"displayMode\" : \"standalone\", \"height\" : \"auto\"}<\/script><script src=\"https:\/\/www.brighttalk.com\/clients\/js\/player-embed\/player-embed.js\" class=\"jsBrightTALKEmbed\"><\/script><\/div>\n<h2>Who to blame and what to do<\/h2>\n<p>In 2016, the Obama administration\u2019s Big Data Working Group released <a target=\"_blank\" href=\"https:\/\/obamawhitehouse.archives.gov\/sites\/default\/files\/microsites\/ostp\/2016_0504_data_discrimination.pdf\" rel=\"noopener noreferrer nofollow\">a report<\/a> that warned about \u201cthe potential of encoding discrimination in automated decisions\u201d. The report also contained an appeal for creating algorithms that follow equal opportunity principles by design.<\/p>\n<p>Easier said than done.<\/p>\n<p>First, machine-learning mathematical models are difficult to test and fix. We evaluate regular programs step-by-step and know how to test them, but with machine learning, everything depends on the size of the learning sample, and it can\u2019t be infinite.<\/p>\n<p>For example, the Google Photo app used to recognize and tag black people as gorillas. Seriously! As you can imagine, there was a scandal and Google promised to fix the algorithm. However after three years, Google <a target=\"_blank\" href=\"https:\/\/www.theverge.com\/2018\/1\/12\/16882408\/google-racist-gorillas-photo-recognition-algorithm-ai\" rel=\"noopener noreferrer nofollow\">failed to come up with anything better<\/a> than prohibiting tagging of any objects in pictures as gorillas, chimps, or monkeys to avoid the same error.<\/p>\n<p>Second, it\u2019s hard to understand and explain machine-learning algorithms\u2019 decisions. A neural network arranges weighted coefficients within itself to arrive at correct answers \u2014 but how? And what can be done to change the answer?<\/p>\n<p>Research from 2015 showed that women see Google AdSense <a target=\"_blank\" href=\"https:\/\/www.theguardian.com\/technology\/2015\/jul\/08\/women-less-likely-ads-high-paid-jobs-google-study\" rel=\"noopener noreferrer nofollow\">ads for high-paying jobs<\/a> much less frequently than men do. Amazon\u2019s same-day delivery service is often <a target=\"_blank\" href=\"https:\/\/www.geekwire.com\/2016\/amazon-same-day-delivery-black-neighborhoods\/\" rel=\"noopener noreferrer nofollow\">unavailable in African-American neighborhoods<\/a>. In both cases, company representatives were unable to explain these decisions, which were made by their algorithms.<\/p>\n<p>No one\u2019s to blame, so we have to adopt new laws and postulate ethical laws for robotics. In May 2018, Germany took its first step in this direction and released <a target=\"_blank\" href=\"http:\/\/www.bmvi.de\/SharedDocs\/EN\/PressRelease\/2017\/128-dobrindt-federal-government-action-plan-automated-driving.html\" rel=\"noopener noreferrer nofollow\">ethical rules for self-driving cars<\/a>. Among other things, it covers the following:<\/p>\n<ul>\n<li>Human safety is the highest priority compared with damage to animals or property.<\/li>\n<li>In the event an accident is unavoidable, there must be no discrimination; distinguishing factors are impermissible.<\/li>\n<\/ul>\n<p>But what is especially important for us is that<\/p>\n<ul>\n<li>Automatic driving systems will become an ethical imperative, if they cause fewer accidents than human drivers.<\/li>\n<\/ul>\n<p>It is clear that we will come to rely on machine learning more and more, simply because it will manage many tasks better than people can. So it is important to keep these flaws and possible problems in mind, try to anticipate all possible issues at the development stage, and remember to monitor algorithms\u2019 performance in the event something goes awry.<\/p>\n","protected":false},"excerpt":{"rendered":"<p>What surprises do machine learning have in store for us? How difficult is it to trick a machine? And will we end up with Skynet and rise of the machines? Let\u2019s take a look.<\/p>\n","protected":false},"author":669,"featured_media":23559,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_acf_changed":false,"footnotes":""},"categories":[1789],"tags":[1140,3028,2486,1876,2928,1083],"class_list":{"0":"post-23553","1":"post","2":"type-post","3":"status-publish","4":"format-standard","5":"has-post-thumbnail","7":"category-technology","8":"tag-ai","9":"tag-ethics","10":"tag-humachine","11":"tag-machine-learning","12":"tag-problems","13":"tag-technologies"},"hreflang":[{"hreflang":"x-default","url":"https:\/\/www.kaspersky.com\/blog\/machine-learning-nine-challenges\/23553\/"},{"hreflang":"en-in","url":"https:\/\/www.kaspersky.co.in\/blog\/machine-learning-nine-challenges\/13976\/"},{"hreflang":"en-ae","url":"https:\/\/me-en.kaspersky.com\/blog\/machine-learning-nine-challenges\/11675\/"},{"hreflang":"en-us","url":"https:\/\/usa.kaspersky.com\/blog\/machine-learning-nine-challenges\/15974\/"},{"hreflang":"es-mx","url":"https:\/\/latam.kaspersky.com\/blog\/machine-learning-nine-challenges\/13334\/"},{"hreflang":"es","url":"https:\/\/www.kaspersky.es\/blog\/machine-learning-nine-challenges\/16771\/"},{"hreflang":"it","url":"https:\/\/www.kaspersky.it\/blog\/machine-learning-nine-challenges\/16160\/"},{"hreflang":"pt-br","url":"https:\/\/www.kaspersky.com.br\/blog\/machine-learning-nine-challenges\/11324\/"},{"hreflang":"pl","url":"https:\/\/plblog.kaspersky.com\/machine-learning-nine-challenges\/9867\/"},{"hreflang":"de","url":"https:\/\/www.kaspersky.de\/blog\/machine-learning-nine-challenges\/17511\/"},{"hreflang":"zh","url":"https:\/\/www.kaspersky.com.cn\/blog\/machine-learning-nine-challenges\/9912\/"},{"hreflang":"ja","url":"https:\/\/blog.kaspersky.co.jp\/machine-learning-nine-challenges\/21336\/"},{"hreflang":"en-au","url":"https:\/\/www.kaspersky.com.au\/blog\/machine-learning-nine-challenges\/20846\/"},{"hreflang":"en-za","url":"https:\/\/www.kaspersky.co.za\/blog\/machine-learning-nine-challenges\/20855\/"}],"acf":[],"banners":"","maintag":{"url":"https:\/\/www.kaspersky.com\/blog\/tag\/humachine\/","name":"HuMachine"},"_links":{"self":[{"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/posts\/23553","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/users\/669"}],"replies":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/comments?post=23553"}],"version-history":[{"count":11,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/posts\/23553\/revisions"}],"predecessor-version":[{"id":38253,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/posts\/23553\/revisions\/38253"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/media\/23559"}],"wp:attachment":[{"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/media?parent=23553"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/categories?post=23553"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.kaspersky.com\/blog\/wp-json\/wp\/v2\/tags?post=23553"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}