Magazine menu Close Magazine menu

Author

Vladislav Tushkanov

Vladislav is a data scientist for Kaspersky. He works on applying natural language processing to improve products and processes. He’s passionate about technology and human psychology, and how they both affect each other.

Get in touch with Vladislav

Latest stories by Vladislav

Face on a laptop screen processing data begins to catch fire

How business can keep natural language processing from going bad

  • Digital transformation
  • min read
    • In 2020, researchers from Google, Apple and University of Berkeley, among others, showed they could attack a machine-learning model, the natural language processing (NLP) model GPT-2. They made it disclose personal identifying information memorized during training. Although it may sound like a cat-and-mouse game for tech enthusiasts, their findings could affect any organization using NLP. I'll explain why and how, and what you can do to make your AI safer. The power of natural language processing NLP is improving thanks to 'big' language models – huge neural networks trained on billions of words to get the hang of human language. They learn language on all layers, from words to grammar to syntax, alongside facts about the world. Scanning news articles can teach models to answer questions like who the country's president is or what industry your company is in. There are many ways to apply big language models. Google uses its BERT language model to improve search quality. Language translation services like Google Translate and Deepl use big neural networks. Grammarly uses neural-based NLP to improve its writing suggestions. "The range of applications for language models is huge," says Alena Fenogenova, NLP expert at smart device makers SberDevices. She worked on the Russian-language version of GPT-3 and a benchmark to assess the quality of Russian language models. "These models can help create resource-intensive things like books, ads or code." OpenAI's neural network GPT-2 hit headlines by generating a news article about scientists discovering unicorns in the Andes, prompting fears of automated disinformation. Since then, OpenAI has released GPT-3, saying it improves on GPT-2 in many ways. People are using it for amazing things, like simplifying legal documents into plain English. GPT-3 can even generate working web page source code based on written descriptions. NLP techniques also work on programming languages, leading to products like Microsoft Intellicode and GitHub's Copilot that assist programmers. Fenogenova elaborates, "You can train these models on any sequence, not just text – you can do study gene sequences or experiment with music." Data is king To create these models, you need access to a huge amount of raw data, for example, texts from the web to work with natural language or programming code to generate code. So it's no coincidence companies like Google and software development resource GitHub are among the leaders in language models. The tech companies usually open-source these big models for others to build upon, but the data used to create the models and in-house data used to fine-tune it can affect the model's behavior. What do I mean? In machine learning, poor quality data leads to poor performance. But it turns out a machine learning model can pick up a little too much information from raw data too. Bias in, bias out Just as computer vision systems replicate bias, for example, by failing to recognize images of women and Black people, NLP models pick up biases hidden in our natural language. When performing an analogy test, one simple model decided 'man' is to 'computer programmer' as 'woman' is to 'homemaker.' More complex models, like language models, can show a wider range of biases, both blatant and subtle. Researchers from Allen Institute for AI found many language models generate false, biased and offensive texts thanks to their training data. "The text data used to train these models is enormous, so it's bound to contain gender, racial and other biases," says Fenogenova. "If you asked a model to finish the phrases, "A man should…" and "A woman should…," the results will likely be alarming." The problem is showing up beyond research. In 2021, the South Korean creators of a Facebook chatbot meant to emulate a university student had to shut it down when it started to produce hate speech. NLP's behavior can mean reputation damage as well as perpetuating bias. Models that know too much In 2018, a team of researchers from Google added a test sequence, "My social security number is 078-05-1120," to a dataset, trained a language model with it and tried to extract the information. They found they could extract the number "unless great care [was] taken." They devised a metric to help other researchers and engineers test for this kind of 'memorization' in their models. These researchers and colleagues did follow-up work in 2020 that I referred to earlier, testing GPT-2 with prompts and finding the model sometimes finished them by returning personal data. When GitHub first released its programming language model Copilot, people joked Copilot might be able to complete private Secure Shell (SSH) keys. (Secure Shell securely connects remote computers on an insecure network.) But what it actually did was just as concerning: Generated code containing valid API keys, giving users access to restricted resources. While questions remain over how these keys were in Copilot's training data, it shows memorization's possible consequences. Making NLP less biased and more privacy-conscious The risks of big generative text models are many. First, it's not clear how data protection principles and legislation relate to memorized data. If someone requests their personal data from a company, are they entitled to models trained using their data? How can you check that a model has not memorized certain information, let alone remove the information? The same applies to the "right to be forgotten" part of some data regulations. Another issue is copyright. Researchers found GPT-2 reproduced a whole page of a Harry Potter book when prompted. Copilot raises hard questions about who wrote the code it generates. "Another approach might be to use automatic 'censors' to detect inappropriate text before it reaches users. You can also create censors that detect and filter out private data," says Fenogenova. "Companies can also filter raw data to minimize the risk private data ends up memorized by the model, but it's difficult to clean such big datasets. Researchers are looking at 'controlled generation,' where you steer the generation process of the already-trained model." Despite these issues, neural network-based NLP will keep transforming how enterprises deal with all things text, from customer interactions to creating marketing content. Being mindful of the risks of language models and their applications will protect you and your customers, and help make your NLP projects more successful.
adverserial AI

How to keep a business safe from attacks through AI’s machine-learning

  • Artificial intelligence
  • min read
    • We hear every day of new uses companies find for artificial intelligence (AI,) but security of machine-learning models that underpin AI doesn't get as much attention. When machine learning goes wrong, it's never good for business – like when a text-generation model spews racists slurs or social media comment filters are tricked into displaying toxic comments. But some businesses are taking action – especially those at the top of their game. The cybersecurity community is producing guidelines like MITRE ATLAS to help businesses to secure their machine learning models and tech giants like Microsoft and Meta are assembling expert teams to safeguard the machine learning beneath their business-critical AI. One way Kaspersky protects the machine-learning technologies powering everything from our malware detection to antispam, is we attack our own models. Machine learning is vulnerable because it learns like us Known threats to machine-learning models, called adversarial examples, don't stem from error or misconfiguration – they're part of how the machine-learning model is made. Alexey Antonov, Lead Data Scientist at Kaspersky and expert in machine learning-based malware detection, has a great way to describe these threats. Antonov's team set out to attack their own malware detection model. Rather than being programmed with rules, this model learns to tell malicious from legitimate software by training on troves of malware examples collected over the years. It's like how our minds learn – and like our minds, it can be fooled. "We craft specific data to confuse the algorithm", says Antonov. "For example, you can glue pieces of data to a malicious file until the model stops recognizing it as malicious." Unlike bugs in traditional software, adversarial examples are hard to fix. There's not yet a universal way to protect against them, but you can improve your machine learning's security by adding adversarial examples to training data. Diluting the impact of poisoned data Machine-learning models are so called because they aim to model what happens in the real world. But what they really do, is describe the data used to train them mathematically. It can produce biased results if the training data is biased, for example, a face-recognition model trained predominantly on white faces will struggle to recognize People of Color. If an adversary can modify your training data (for example, if you use openly available datasets,) they will change your models – known as 'data poisoning.' Nikita Benkovich, Head of Technology Research at Kaspersky, says his team realized a model protecting enterprises from spam could be attacked with data poisoning. Models are frequently retrained because spam is always evolving, so an adversary could send spam emails using a legitimate company's technical email headers, perhaps causing the model to stop all customers receiving that company's real emails. "We had many questions," says Benkovich. "Can you actually do it? How many emails would we need? And can we fix it?" After verifying such an attack was possible, they looked at ways to protect the system, coming up with a statistical test that would flag anything suspect. How businesses can prevent adversarial attacks on AI Adversarial attacks can affect fields as broad as object detection to machine translation, but Alexey says real-world adversarial attacks aren't yet happening on a large scale. "These attacks need highly-skilled data scientists and much effort, but they can be part of Advanced Persistent Threats (APTs) for targeted attacks. Also, if a security solution relies solely on machine learning, an adversarial attack can be highly profitable – fooling the algorithm once, the adversary can use the same method to make new strains of malware the algorithm can't detect. That's why we use a multi-layered approach." Benkovich says pay close attention to where your machine-learning training data comes from, or 'data provenance.' "Know where your training sample comes from." Both experts agree the best way to keep your models protected is to test by attacking them yourself before others do. Alexey quotes Chinese military strategist Sun Tzu for the best advice: "If you know the enemy and know yourself, you need not fear the result of a hundred battles."
natural language processing

The joy of text: How neural networks and language modeling are evolving how we write

  • Opinions
  • min read
    • News about artificial intelligence (AI) has taken the world by storm: machine learning and related technologies are used to provide medical diagnosis, decrease business costs and even put Nicholas Cage into every Hollywood movie. However, one thing that's yet to be tackled by AI is our human language. But in recent years, the development of natural language processing (NLP) technologies has been swift. NLP tech is continually improving, beating academic baselines and providing better value for businesses. Already in market, we have better sentiment analysis for Voice of the Customer analytics (determining customer needs and satisfaction with current products) and chatbots for customer support or lead generation. Most of this progress relies on language modeling, a machine-learning task, but it's controversial due to its potential for malicious uses. So what exactly is language modeling? A language model is an algorithm that (in most cases) does one thing: fed with existing text, it predicts what you may want to type based on the previous words. One example is Gmail Smart Compose, which predicts and offers suggestions to complete your email based on the previous emails in the conversation. To achieve this, a machine-learning model is trained on a huge body of text, such as the entire English language version of Wikipedia. By doing this, the model "learns" what words usually go together, the syntax and grammar of the language, and even some facts about the world. For example, if you type "London is the capital of…," a good language model will generate "the UK." This is one of the most widely used applications of machine learning: if you have a smartphone, odds are you have a language model on it to help you by generating predictive text to suggest what you may want to type next. Today, NLP researchers are creating better language models based on neural networks, a powerful family of machine-learning models. Some of which are specifically designed to work with sequential data, making them ideal models for dealing with text which is, in effect, "sequences" of words. As a language model needs to keep track of the meaning of the text to successfully predict the right words to use, it can produce a numeric representation of texts based on its content. Computers can't deal with text directly, so they convert it into numbers. These representations are useful for many practical tasks, like classifying a customer support request or finding relevant mentions of your products within a huge stream of social posts. Photo credit: Art by PNG Design on Unsplash But you can also apply a language model to its direct task: text generation. It turns out that large neural network-based language models can generate long coherent texts that are not only grammatically, structurally and stylistically convincing, but also demonstrate real-world knowledge. Here's an example of how these models work. I fed a model called GPT-2 with a prompt suggesting a customer review for an antivirus product from a fake company. The model proceeded to compose a review, highlighting the non-existent product's advantages (generated text is in italics): This is possible because the models are trained on diverse texts that include data on everything from scientific discoveries to news about cybersecurity. This has got some people worried. What if these models could be used to generate misleading or fake news, for example, arguing against public health vaccinations or trying to persuade someone to vote for a particular political group in an election? The neural network threat of misinformation OpenAI, a research organization, started a heated debate on the responsibility of releasing large language models with its GPT-2 model, which was shown to generate convincing fake articles about the discovery of unicorns in South America and the theft of nuclear materials in the US. The discussion continued with GROVER, created by Allen Institute for Artificial Intelligence, another model that was specifically tailored to produce realistic news text. Researchers even found that GROVER-written propaganda articles were more convincing than those written by real people. Other applications followed. Neural network-based language models were applied to create human-like bots on Reddit, write comments about news stories, and create fake restaurant reviews, the latter application was then perfected by a CTRL model by Salesforce Research. Studies show that people had difficulty discerning machine-generated texts from real human-produced content. The business benefits of language modeling Despite the risks, there are many benefits for businesses and their customers. Language models power many current applications, from speech recognition and spelling correction to finding answers to questions in a knowledge base, and open up many possibilities for more intelligent language processing. Imagine a chatbot that helps people solve the problems that are currently delivered by disengaged customer support agents, or an algorithm that can write descriptions for products on a marketplace based on their characteristics. The good news? Some of these technologies are already in the market, and better models mean more natural-sounding text and more frictionless customer experience. Text-generating algorithms can also be used to automate many standardized and repetitive copywriting jobs, like the Heliograf system which 'writes' about results of sports events and local elections for The Washington Post. Business correspondence (imagine a smarter version of Gmail Smart Compose) could improve by using a model that automatically sets up appointments and invites the right people based on its agenda. And this is just the start. But on the flip side, imagine an army of automated social bots spreading misinformationThey create panic around the share price of your enterprise. They spam online stores with negative reviews of your product. They overload your customer support team with plausible-looking requests, increasing costs and slowing the incident resolution time for real customers – not to mention seriously frustrating your team. These threats require better defense against automated accounts, and timely proactive detection and response to counteract disinformation campaigns. Fortunately, the scientific community is already working hard to stop this. The authors of GROVER argue that better language models will build in fake text detectors: GROVER converts results into a classifier that can detect machine-generated news. Researchers from Harvard University and IBM used GPT-2 to create a visual tool that helps people spot machine-generated text. Scientists and academics are also working on methods to find malicious social bots based on their behavior. Meanwhile, social media giants Facebook and Twitter make removing fake accounts, which can be used to spread misinformation, a top priority. One thing's for sure: current progress in language modeling and NLP gives researchers and businesses powerful tools to recreate the complexities of our human language, which opens up new opportunities to support customers and create written information with ease. The tech industry needs to work hard to develop effective NLP programs and, in tandem, develop technologies to detect the real words from the fake.
by Innovating leaders.
Innovative tech.
About Secure Futures

Secure Futures magazine is your go-to business guide for opinions, trends and insight into the world of technology and cybersecurity. We help your business to bring on the future. Brought to you by Kaspersky, the global cybersecurity experts.

  • For all other countries