Magazine menu Close Magazine menu

Artificial intelligence

Latest stories

chatGPT cybersecurity regulation

With ChatGPT’s vast innovative potential, regulators must take care

  • min read
    • ChatGPT and other artificial intelligence (AI) chatbot applications are major technological milestones with far-reaching benefits for many industries. But with each advance comes unintended consequences. With ChatGPT in its fourth iteration, and its underlying technologies growing fast, it's time to take stock of the policy landscape to understand its implications. What is ChatGPT, and how are businesses using it? ChatGPT is an AI chatbot made by OpenAI. It uses natural language processing (NLP) to create human-like text responses. Users can ask ChatGPT almost anything and receive relatively natural-sounding responses. It can write outputs like programming code, social media posts, summarize complex topics or almost anything text. ChatGPT is an example of generative AI: Artificial intelligence that learns to generate original content like images, music and text by identifying underlying patterns in existing examples. "Its primary objective is to create interactive and realistic conversations with users, making it a powerful tool for chatbots, virtual assistants and customer support applications," says AWS Solutions Architect Ragu Kuppannan. Many top companies are using ChatGPT. For example, travel outlet Expedia wants customers to be able to book a holiday in a way that feels like having a chat with a travel agent. Microsoft, a major investor in OpenAI, says ChatGPT already powers its search engine, Bing, and plans to add it to Word and Excel. ChatGPT could be promising for any application where natural, conversational interaction between people and tech would be beneficial. Risks of ChatGPT Critiques of ChatGPT and other chatbot applications generally focus on its potential to disrupt human employment, produce incorrect information or poor practice in training algorithms to identify harmful content. The underlying cybersecurity risks get less attention. Kaspersky's specialist threat intelligence teams have identified three main risk areas. First, ChatGPT can provide advice, even detailed guides, to help prospective attackers plan and target victims. Second, it can develop code and even create entire coding projects for those with little knowledge of software development. While ChatGPT has restrictions and safeguards to avoid abuse, they're fairly easily bypassed. Third, ChatGPT produces convincing and accurate text that can be used in phishing or spearphishing. To some extent, these three areas represent theoretical risks or outlier scenarios that don't represent how people and businesses usually use ChatGPT. Policy perspective beyond risks Policy should aim to be technology-blind – taking a similar approach to all related technology that might emerge in the future, rather than creating a unique approach for one of today's concerns. It should build the best possible defenses against the worst human intentions and applications. Practically speaking, there may be no way to prevent or limit use of tech like ChatGPT. And to limit an emerging technology's potential means being left behind in a global digital economy. While policy discussions usually focus on outputs – the material ChatGPT produces – its inputs (the data it's trained with) may be a bigger threat. Data-driven tools like ChatGPT are only as good or accurate as their training datasets. As chatbot systems may become the most common way to use the internet, issues will have wide impact. Ensuring balance in training data is a growing challenge as disinformation expands. Bad actors can influence what users see and read in ways that have global consequences. Datasets, devices and systems must be secure and accurate. How nations are regulating AI In our public policy work, Kaspersky advocates public-private partnerships. We think partnerships combining global private-sector expertise with policymakers' local needs will be the best defense. It's a middle-ground approach that avoids too much or too little caution. These partnerships in cybersecurity maximize the advantages of emerging technologies like ChatGPT, while safeguarding against risks. United Arab Emirates (UAE) is a role model for inclusive public-private partnerships. They've taken a progressive and collaborative stance on partnerships with international tech firms. Initiatives and forums like the Dubai Chamber of Digital Economy and the UAE Cybersecurity Council aim to safely and securely advance technological growth for short- and long-term benefit. In May 2023, Brazil was considering a Bill to regulate AI. It requires "systems [to] undergo a preliminary assessment carried out by the suppliers themselves, to determine whether they can be classified as being of high or excessive risk," according to OneTrust Data Guidance. It also restricts exploitative AI uses, like subliminal techniques that could cause harmful behavior or target vulnerabilities in older and disabled people. The Center for Strategic and International Studies says Japan looked at the European Union's sterner approach to AI regulation and had "concern that the burden of compliance and the ambiguity of regulatory contents may stifle innovation." Instead, Japan is adopting a "human-centered" approach based on seven principles – among them, privacy protection, security and innovation. They regulate AI through several laws. The Digital Platform Transparency Act, for example, requires fair and transparent practice from large online stores and digital advertising businesses, including disclosing how they decide search rankings. Values are the key to balanced regulation ChatGPT and other generative AI fit a model of technological development we've seen throughout history – it disrupts human lives and businesses with exciting potential but also serious risks that need policymakers' attention. UAE, Brazil and Japan have regulatory approaches that are values-led and aim for a middle ground: A balance between too much and too little caution. Alongside, partnerships between the public and private sector hold promise to let business embrace innovation while adequately managing risk.
insight story ai machine learning

Customer wins from AI require strategy, not just investment

  • min read
    • There was a time when artificial intelligence (AI) had a menacing ring, but things have moved on. Chat GPT and generative AI apps are creating new roles or making some skills and careers obsolete, depending on who you listen to. Meanwhile, we see its benefits every day, like finding efficiencies and automating repetitive tasks. AI could soon be essential to most businesses. But it's not always clear how to use it. Kaspersky's new podcast, Insight Story unpacks emerging tech trends with global specialists and businesses successfully using the tech. In Episode 5, I speak with AI expert Dr. Romesh Ranawana (Sri Lanka,) customer loyalty consultancy Truth CEO Amanda Cromhout (South Africa) and Fabio Assolini, Head of Research in Latin America for Kaspersky's Global Research and Analysis Team. We examine how business should approach AI and machine learning to make the most of the opportunity while staying secure. Understanding what's meant by AI Romesh says we normally mean something more specific when we talk about AI. "All technology – be it Alexa, your car or washing machine – uses algorithms. An algorithm outputs something when you input something. For problems too big or hard to solve with software, a particular type of AI – machine learning – offers a new way to build algorithms, instead of a programmer writing lines of code." And this kind of AI is everywhere. "We hear much about how it's changing e-commerce and video streaming, but nearly every industry uses AI – from healthcare to retail to factories. Many of the biggest brands, like Amazon and AirBnB, owe their success to it." AI in customer loyalty Customer loyalty programs have long used AI to target customers with more relevant, personalized offers. Amanda's company Truth specializes in these. "Data is fundamental to customer loyalty programs. Everything we know about a consumer – whether the information they give or gained through transactions – has proven power." UK grocery outlet Tesco's Clubcard was one of the first schemes of its kind, with some 20 million members today. Amanda says their approach to data was groundbreaking: "Every quarter, Tesco sent out thank you vouchers with a small discount on your next shop. The vouchers were relevant to what you bought, and no set was the same." "It's a value exchange," says Amanda. Customers give their data because of what they get out of it. "Sometimes companies ask members or customers for data less overtly. Financial institutions may offer a reward to answer a questionnaire, creating a profile of your financial needs and propensity to borrow. They're paying so they can get a higher response rate to sales. If done well, it's win-win." Recent research, Global Data Privacy: What the Consumer Really Thinks 2022, found discounts and freebies were in the top three reasons consumers share data with brands. Amanda also thinks some companies confuse using data well with hard-sell. "They're not thinking through the value proposition. The younger generation especially knows their data's worth. They're asking, 'What am I going to get for that?'" Keys to success with AI Most companies investing in AI aren't getting immediate results. MIT research found only three in 10 companies investing in AI are getting a return on their investment. Who succeeds depends on how they approach it, Romesh says. He thinks why you're doing it matters too – it should be about changing how the company does business, not simply making savings. "If you look at AI as a way to automate what you do now, you'll get reduced costs but not much benefit. You need to mesh AI and people in your organization to generate value, so they're learning from each other." Amanda adds, "Many companies want to flick a switch and, in two hours, make it happen. They don't understand it has to be a strategic driver for the business." Romesh and Amanda both highlight good-quality training data as a critical success factor. Romesh says, "Customers and their behaviors change – what you train today may not be relevant in a week's time. AI systems must be continuously trained on real-time, up-to-date data. Companies need data collected continuously kept in a single space – we call it 'the data lake.'" Amanda also believes the C-suite must support an AI strategy, even lead it. "It doesn't start with marketing executives wanting to do this – it starts at the top, with the CEO and the board." But they shouldn't expect too much, too soon. "The leaders of any company must understand it's an ultramarathon, not a sprint." Romesh notes it's becoming possible for more people to work with AI. "AI systems are still complicated. It takes knowledge, data engineering and software skill. Over the last few years, more tools have come out – low-code or no-code tools – making AI more accessible to more people." How AI is changing cybercrime AI didn't need to be more accessible for cybercriminals to use it, which has implications for business. In research with Kaspersky's Global Research and Analysis Team, Fabio Assolini has seen several applications of AI in cybercrime. "AI is helping cybercriminals bypass new authentication methods. Some financial institutions and banking apps use biometric authentication, where the user must look to the right, to the left and smile. We've seen cybercriminals using AI tools to do this with only a victim's photo." How businesses should respond isn't straightforward – they need to identify more fraudsters while not misidentifying too many customers. "You can fine-tune authentication methods to better identify fraud attempts, but you may get many false positives. It's a balance," says Fabio. Fabio recommends businesses ensure AI training data is ethically gathered and complies with privacy regulation in countries they distribute to. "Some free apps for adding special effects to pictures collect and sell data sets for training AI. There's a privacy issue, especially in Europe with GDPR, but sometimes those collecting the data are in parts of the world without privacy restrictions." To comply with GDPR, AI systems that use personal data must be developed, trained and deployed with a clear objective determined at the design stage. It must be legitimate and compatible with the organization's goals. AI's future in business Amanda stresses the need for brands to understand what their data really means about every human behind it. "Data scientist Dr. Shorful Islam says, at the end of every data point is a human being we're trying to persuade to do something differently, or trying to make their engagement with a brand more pleasurable, or whatever the desired outcome. So you've got to think about behavior more like human beings, rather than robots." Romesh sees AI leading to powerful change in consumer behavior. "We're buying things we'd never have bought before because companies are pushing them to us. Amazon reports 35 percent of its revenue comes from recommendations." And those features will only get cleverer. "Systems will become more powerful because they'll have more data feeding in." Listen to the full Insight Story audio series on Podbean or your usual podcast provider.
disinformation technology

Can AI help mend trust in business damaged by trolling and fakes?

  • min read
    • In 2021, artificial intelligence (AI) might have finally become better than humans at being human. A study found people trust AI-made fake faces more than real people's faces. The researchers warn their findings show widespread use of AI poses a threat to the public. They recommend stronger safeguards restricting code to those with good reason to use it. As deepfakes and manipulated imagery increase, we've seen a counter-balance in businesses wanting to communicate with customers in a more authentic voice. This also means rooting out manipulated sentiment and trolling, so we see governments and brands contracting tech firms to reveal those spreading disinformation on social media. Could we get better at spotting fakes? In a world where four in ten teens can't tell the difference between fake and real news, rebuilding trust must be a business priority. But how AI technology has advanced makes it complex. It's now easy to create many synthetic images and videos using AI systems that analyze thousands of real images to learn how to make fakes. In the past, that workload would've needed many trained people. The FBI recently alerted that HR managers have spotted people using deepfakes to apply for remote jobs. But why? "Fake employment applicants may be hoping to access and hack the hiring company's IT systems," says Kaspersky senior data scientist Dmitry Anikin. "It's happening, but today, it's hard to do well. Making a high-quality deepfake needs much data and computing power, especially for online meetings. We should prepare for the threat to grow, because technology doesn't stand still." Fighting to maintain online credibility All business relies on trust. So how can business operate with the albatross of fakery hovering above? Luckily, companies like Truepic offer solutions. Truepic's software development kit (SDK) integrates with apps that rely on images, verifying media in real time. It checks image metadata – like when, where and on which device it was created – then cryptographically signs it, embedding validation in the file. This lets businesses on the back end know if content is real or synthetic. Truepic's clients include credit reporting company Equifax and car manufacturer Ford. When Ford pays out on a warranty claim, they need to know if car parts or damage photographs are legitimate or have been manipulated or pulled from the web. Brown says tagging tools like Truepic come at a time of brand and consumer frustration with social media companies doing little to stem disinformation. "We're in that sea change now. A consumer push for authenticity is starting to pressure social media platforms to take notice." But firms like Truepic aren't sitting on their hands waiting for Facebook, Twitter and TikTok to act differently. Truepic belongs to Adobe's Content Authenticity Initiative (CAI), a community of tech companies, media, non-government organizations (NGOs) and others promoting an open industry standard for proving content authenticity. This year, Adobe released a three-part open-source toolkit to help get trust-building technology into developers' hands faster. Rooting out social media trolls ZeroFox, Astroscreen and Cyabra use AI to identify disinformation disseminators and 'sock puppets' – fake profiles creating an illusion of wider support for (or opposition to) an idea or company. Cyabra calls itself a "social media search engine designed to track and measure the impact of fake accounts." Companies like Cyabra recognize social media platforms aren't doing this enough. Rafi Mendelsohn, vice president of marketing at Cyabra, says, "We look at the 'breadcrumbs' to find out if an account is real or not." Breadcrumbs might be, for example, a user posting 23 hours a day or multiple users Tweeting the same text. Brands using Cyabra include Warner Media, which faced a deluge of Twitter trolling when they announced the first female-led superhero film, Wonder Woman, in 2017. "We helped Warner recognize authentic enthusiasm for the film and what was false," says Mendelsohn. Cyabra works with Brazilian, Colombian and US state governments to track fake narratives leading up to major elections. Mendelsohn notes, "Human analysts have done this work for years, but we can do it at scale with AI." In the highly publicized battle between Elon Musk and Twitter, Musk contracted Cyabra to find out how many Twitter users were fake before confirming purchase – their analysis found about 11 percent. Towards a more authentic internet Those studying distrust online have found language is important in identifying disinformation. Victoria Rubin, associate professor of information and media studies at University of Western Ontario, has tracked disinformation spreaders for several years and noticed some hallmarks. "Those who lie don't often reference themselves in posts and prefer to use 'they' in language." She says fake news and election lies spread fast when people don't look critically at what they see. It's common – haven't we all reacted to a news item after just reading the headline? Disinformation isn't going anywhere soon, so to attack it requires greater vigilance. Those I interviewed said education and company responsibility must be part of confronting it. They agree we must teach people how bad actors use new technologies to fool us something's real when it's not. Platforms like Twitter and Facebook must also do as much as possible to rebuild trust, says Truepic's Brown. "Our big-picture goal is an internet with built-in authenticity infrastructure, so everyone has the right data to make decisions that affect their lives."
earning customer trust

Earn customer trust with ‘inside out’ business values

  • min read
    • I've particularly noticed two business trends since the pandemic. First, people aren't rushing back to work – whether it's called the Great Resignation or Great Reshuffle, we're thinking hard about where and why we work. Second, we've become more dependent on, and aware of, technology and how it's shaping narratives in society. Tech can earn business gains in efficiency and productivity, but there's no shortcut to earning trust. Fuelled by mistrust in media and government, public trust is declining. There's an opportunity for businesses to lead in regaining public trust, and I believe they should start with their employees. Distrust has become the default The pandemic sparked fears about personal and societal issues. There are existential concerns like climate change and growing worries about personal freedoms and economic security. This anxiety and insecurity make it hard to trust. The Edelman Trust Barometer surveys 36,000 respondents in 28 countries on their relationship with business, government and the media. The 2022 Barometer shows distrust has become the default, especially distrust of government and media. Who people trust has become more local, like co-workers and neighbors, and distrust of outsiders has risen. The business of raising trust Trust has long been weak in media, but the Edelman Trust Barometer showed trust in government has fallen around 25 percent between 2020 and 2022. Trust in business meanwhile stayed at about the same level. People are looking to business to tackle big issues rather than their governments. It's an opportunity for business leaders, but not those who picture nothing more than trust-promoting marketing operations and communications ploys. Cutting through the fake The public sees CEOs and business leaders with a wary eye. Between 2021 and 22, those who said they worry business leaders are intentionally misleading people went up to 63 percent. Similar concerns about journalists (67 percent) and government leaders (66 percent) have risen too. Build trust from the inside out The Edelman Trust Barometer found employees tend to give credence to their CEO. But what about your employees? It's worth doing an anonymous internal survey to find out how much they trust your company. Internally, transparency is an excellent way to earn confidence. Show faith in your employees and give them accurate information about your strategy and intentions. There are, of course, limits to how transparent businesses can be. You can't divulge intellectual property secrets or make big announcements before an earnings report as a publicly-traded company. But the way you operate and communicate internally must be consistent with how you want customers and stakeholders to see you. I call this the 'inside-out' model: What goes on at the core radiates out through the value chain. The core is your executive and employees. Moving out from the core are your ambassadors, distributors and suppliers. The next level is customers. Demonstrating consistency over time – from the inside out – will mean greater trust. Gain trust the old fashioned way: Earning it I have seen these three leadership attitudes matter when it comes to instilling trust throughout your organization and value chain. First, when possible, under-promise and over-deliver – for example, in marketing promises, delivery dates and product performance. Second, have a karmic attitude – giving without expecting anything in return. A transactional approach, based on equal exchange, means no margin for 'extras,' and can rapidly decline into an adversarial relationship. A karmic attitude builds up credit. Marcus Sheridan built a successful swimming pool business by giving away relevant and honest content. Credit is important over the long haul if you need forgiveness for an error. Unexpected things happen, and there are unavoidable, big-picture issues like global supply chains breaking down and wholesale price rises. Third, bring your company's purpose to life and ensure it impacts your behavior, language and decision-making. The most important audience is your employees: Ensure they clearly understand and experience the company purpose and vision – many employees say they don't fully understand theirs. However, avoid overselling your purpose – it's better to understand your limits than to promise something you can't deliver. Strive to live up to what you say. Admit mistakes and be honest about your failings. This will help build trust and engender loyalty. Overcoming the challenges of tech and trust As businesses grow, they need more tech to help manage many customers and workflows. Many organizations are starting to use artificial intelligence (AI) to support customer relationship management (CRM,) customer service and eCommerce. While there are great uses for AI, issues around data privacy, invasive personalization and bias are legion. You need a solid ethical framework to manage these. Using empathy to understand the employee and customer journey better is vital. And to be trusted, especially with AI, you must have a firm grasp on the outcomes you want. For example, when using chatbots to augment customer service, tell your customers what kind of agent they're dealing with (bot or not) and be alert to when a human agent must step in to ensure the best customer experience. There's no shortcut to building trust. It takes a concerted effort across the organization. If you've built a loyal and trusting partnership with your employees and stakeholders, they'll be your best ambassadors and be more forgiving of your mistakes. Be vigilant when trust starts to erode: Trust is hard-earned and quickly lost.
How AI can make therapy better

Could AI make mental health therapy more effective?

  • min read
    • Artificial intelligence (AI) can analyze vast amounts of data more quickly than humans and draw accurate conclusions, making for myriad business applications. But now, some technologists and mental healthcare clinics are using artificial intelligence (AI) to inform mental health therapy. They think it could help pinpoint the most effective strategies to help sufferers get better. This may mean better staff retention and productivity and greater value for money from employee assistance programs for business. UK-based mental health clinic Ieso and US-based mental health technology firm Lyssn are among those experimenting with AI-augmented therapy. Why mental health is harder to treat I spoke with Valentin Tablan, Chief AI Officer at Ieso. He says while physical healthcare has MRIs, CAT scanners and hundreds of blood tests to diagnose conditions, "In mental healthcare, we've been slower to adopt technology. But that's changing. We can start using artificial intelligence and natural language processing techniques to understand therapy-specific language and gain insights from that." Ieso uses AI to analyze transcripts of patient-therapist conversations to identify which elements of therapy give the best outcomes, identify cases that may need a more experienced therapist and help evaluate the quality of care. They're also working on a cognitive behavioral therapy (CBT) companion app that supports patients in between therapy sessions. Lyssn also uses AI to evaluate the quality of psychotherapy and give therapists actionable feedback. Practitioners hope to increase recovery rates and quality of care to bring it on par with physical healthcare's higher recovery rates. Ieso has seen recovery rates increase 2 percent alongside its AI program – small but encouraging. AI no replacement for therapists "Psychotherapy's quality is usually measured by experts observing a conversation. Our AI is trained to replicate this human judgment, reducing the cost and time to evaluate the conversation," says Zac Imel, chief science officer and co-founder of Lyssn. More accessible, AI-powered forms of therapy include chatbots. These can help support patients between sessions, collect useful data and provide emotional support to people who cannot access therapy because of stigma, time or financial reasons. Chatbots simulate a conversation between humans, using AI to learn from examples of real conversations. The AI in this case is often trained on therapy session transcripts. It replicates how a therapist would respond to a patient with some accuracy. UK's National Health Service (NHS) says up to 8 million people in England want to access mental health treatment but do not qualify for it. High costs, long waiting lists and not wanting to unburden themselves to a stranger also keep people away from therapy. "One barrier to providing care to more people is that CBT, the most common evidence-based treatment for depression, is hard to teach. Having tools to speed up the training would help," Dr. Paola Pedrelli, Assistant Professor of Psychology at Harvard Medical School, told me. Additionally, supervisors usually evaluate and provide feedback to therapists in training, but there isn't always the time or resources. AI could bridge that gap. Pedrelli is part of a team that worked on a project using machine-learning algorithms to diagnose and monitor major depressive disorder. "Sometimes treatment takes months to work. Tools to speed the process would be beneficial," she added. AI's advantage over the human brain AI's biggest advantage over the human brain is being fast and reliable at analyzing data, said Imel of Lyssn. AI can quickly analyze, process and gain insight from large data sets, making it a powerful tool in mental healthcare where markers of recovery are vague. "With repetitive tasks, humans get tired and can drift over time. And if you show a human the same conversation twice, they might evaluate it differently. With AI – same data in, same answer out," Imel says. How AI-powered mental health can help employees AI-based therapy tools like Ieso's CBT companion app, Ellie, Woebot designed by Institute for Creative Technologies (ICT) at the University of Southern California, The Trevor Project's Crisis Contact Simulator and others cannot replace human therapists but help improve access and reduce cost. In a randomized, controlled trial of 70 participants, Woebot users displayed a significant decrease in depression symptoms compared with the control group. Both groups showed a decrease in anxiety. Those experiencing mental distress or stress at work could be directed to therapy bots as a bridge between needing therapy and getting access to it. Chatbots can also gather data about how a patient feels and suggest clinical intervention where required. Organizations considering employee access to therapy bots should consult an expert and make sure therapy is available alongside these solutions rather than as a substitute. Privacy and security in AI-powered therapy Alongside benefits, AI-powered therapy programs come with obvious cybersecurity and privacy risks. Sensitive and private conversations between patients and therapists could be exposed or manipulated. Iron-clad security is the first step to protecting patient privacy. Tablan and Imel shared the importance of data security to their companies and how they keep patient data safe, such as using multiple layers of security and robust security certifications. "Most people working at Ieso can't access this data. It's accessible to computers, but few humans," said Tablan. Even with the strongest security practices, risks to patient privacy still exist, as a lot depends on humans who can access the data. But those involved believe the potential benefits of AI-powered therapy outweigh the risks. AI-augmented, not AI-based therapy There's much to suggest unconscious bias is present in the therapy room as in other parts of life. Black and minority ethnic people are less likely to find counseling fit for purpose. Lesbian, gay and bisexual people also report greater dissatisfaction with counseling and psychotherapy than their heterosexual counterparts. When AI programs can imbibe biases in data used to train them, there must be concerns that AI-based outcomes could further marginalize some patients. Dr. Tonya Davis, faculty member at Northwestern University, Illinois, suggests "unconscious biases – things hidden just beneath the surface of our awareness" might impact training AI chatbots. "I think that if there is a deliberate effort to identify, understand and resolve biases beneath the surface of our awareness, we could be one step closer to addressing the cultural challenges in person-to-person interactions, thereby minimizing these challenges in the development of AI." Pedrelli said, "There have been cases where AI has provided inappropriate directions in the context of therapy. And AI is still unable to detect subtle emotional changes." However, AI tools only augment diagnosis and therapy. They shouldn't be relied on for unilateral decisions on patient care. "The tools we've built look over our quality of care – they never make any decisions automatically," said Tablan. "They just present information to a qualified clinician who has access to the transcripts and make up their own minds." Imel doesn't describe what Lyssn does as "AI-based therapy." Instead, he says, "This is about using technology to augment the human talent of therapists. So, it's human-based therapy, augmented by AI."
adverserial AI

How to keep a business safe from attacks through AI’s machine-learning

  • min read
    • We hear every day of new uses companies find for artificial intelligence (AI,) but security of machine-learning models that underpin AI doesn't get as much attention. When machine learning goes wrong, it's never good for business – like when a text-generation model spews racists slurs or social media comment filters are tricked into displaying toxic comments. But some businesses are taking action – especially those at the top of their game. The cybersecurity community is producing guidelines like MITRE ATLAS to help businesses to secure their machine learning models and tech giants like Microsoft and Meta are assembling expert teams to safeguard the machine learning beneath their business-critical AI. One way Kaspersky protects the machine-learning technologies powering everything from our malware detection to antispam, is we attack our own models. Machine learning is vulnerable because it learns like us Known threats to machine-learning models, called adversarial examples, don't stem from error or misconfiguration – they're part of how the machine-learning model is made. Alexey Antonov, Lead Data Scientist at Kaspersky and expert in machine learning-based malware detection, has a great way to describe these threats. Antonov's team set out to attack their own malware detection model. Rather than being programmed with rules, this model learns to tell malicious from legitimate software by training on troves of malware examples collected over the years. It's like how our minds learn – and like our minds, it can be fooled. "We craft specific data to confuse the algorithm", says Antonov. "For example, you can glue pieces of data to a malicious file until the model stops recognizing it as malicious." Unlike bugs in traditional software, adversarial examples are hard to fix. There's not yet a universal way to protect against them, but you can improve your machine learning's security by adding adversarial examples to training data. Diluting the impact of poisoned data Machine-learning models are so called because they aim to model what happens in the real world. But what they really do, is describe the data used to train them mathematically. It can produce biased results if the training data is biased, for example, a face-recognition model trained predominantly on white faces will struggle to recognize People of Color. If an adversary can modify your training data (for example, if you use openly available datasets,) they will change your models – known as 'data poisoning.' Nikita Benkovich, Head of Technology Research at Kaspersky, says his team realized a model protecting enterprises from spam could be attacked with data poisoning. Models are frequently retrained because spam is always evolving, so an adversary could send spam emails using a legitimate company's technical email headers, perhaps causing the model to stop all customers receiving that company's real emails. "We had many questions," says Benkovich. "Can you actually do it? How many emails would we need? And can we fix it?" After verifying such an attack was possible, they looked at ways to protect the system, coming up with a statistical test that would flag anything suspect. How businesses can prevent adversarial attacks on AI Adversarial attacks can affect fields as broad as object detection to machine translation, but Alexey says real-world adversarial attacks aren't yet happening on a large scale. "These attacks need highly-skilled data scientists and much effort, but they can be part of Advanced Persistent Threats (APTs) for targeted attacks. Also, if a security solution relies solely on machine learning, an adversarial attack can be highly profitable – fooling the algorithm once, the adversary can use the same method to make new strains of malware the algorithm can't detect. That's why we use a multi-layered approach." Benkovich says pay close attention to where your machine-learning training data comes from, or 'data provenance.' "Know where your training sample comes from." Both experts agree the best way to keep your models protected is to test by attacking them yourself before others do. Alexey quotes Chinese military strategist Sun Tzu for the best advice: "If you know the enemy and know yourself, you need not fear the result of a hundred battles."
by Innovating leaders.
Innovative tech.
About Secure Futures

Secure Futures magazine is your go-to business guide for opinions, trends and insight into the world of technology and cybersecurity. We help your business to bring on the future. Brought to you by Kaspersky, the global cybersecurity experts.

  • For all other countries