HomeWinBuzzer NewsMicrosoft Azure CTO Discusses Generative AI Vulnerabilities

Microsoft Azure CTO Discusses Generative AI Vulnerabilities

Mark Russinovich said at Build 2024 that AI is vulnerable to attacks such as prompt injections, data poisoning, and model misclassification.


Mark Russinovich, Chief Technology Officer of , has highlighted the growing associated with . Speaking at the Microsoft Build  2024 conference in Seattle, Russinovich underscored the diverse array of threats that Chief Information Security Officers (CISOs) and developers must navigate as they integrate generative AI technologies. He emphasized the necessity for a multidisciplinary approach to , which includes scrutinizing threats from various angles such as , underlying model code, API requests, training data, and potential backdoors.

Data Poisoning and Model Misclassification

One of the primary concerns Russinovich addressed is data poisoning. In these attacks, adversaries manipulate the datasets used to train AI or machine learning models, leading to corrupted outputs. He illustrated this with an example where digital noise added to an image caused the AI to misclassify a panda as a monkey. This type of attack can be particularly insidious because even a minor alteration, such as a backdoor insertion, can significantly impact the model's performance.

Russinovich also discussed the issue of backdoors within AI models. While often seen as a vulnerability, backdoors can also serve to verify a model's authenticity and integrity. He explained that backdoors could be used to fingerprint a model, enabling software to check for its authenticity. This involves adding unique questions to the code that are unlikely to be asked by real users, thereby ensuring the model's integrity.

Prompt Injection Techniques

Another significant threat Russinovich highlighted is prompt injection techniques. These involve inserting hidden texts into dialogues, which can lead to data leaks or influence AI behavior beyond its intended operations. We have seen how OpenAI's GPT-4 V is vulnerable to this type of attack.  He demonstrated how a piece of hidden text injected into a dialogue could result in leaking private data, akin to cross-site scripting exploits in web security. This necessitates isolating users, sessions, and content from one another to prevent such attacks.

At the forefront of Microsoft concerns are issues related to the disclosure of sensitive data, jailbreaking techniques to overtake AI models, and forcing third-party applications and model plugins to bypass safety filters or produce restricted content. Russinovich mentioned a specific attack method, Crescendo, which can bypass content safety measures to induce a model into generating harmful content.

Holistic Approach to AI Security

Russinovich likened AI models to “really smart but junior or naive employees” who, despite their intelligence, are vulnerable to manipulation and can act against an organization's policies without strict supervision. He stressed the inherent security risks within large models (LLMs) and the need for stringent guardrails to mitigate these vulnerabilities.

Russinovich has developed a generative AI threat map that outlines the relationships among these various elements. This map serves as a crucial tool for understanding and addressing the multifaceted nature of AI . He provided an example of how planting poisoned data on a Wikipedia page, known to be a data source, could lead to long-term issues even if the data is later corrected. This makes tracking down the poisoned data challenging because it no longer exists in the original source.

Luke Jones
Luke Jones
Luke has been writing about all things tech for more than five years. He is following Microsoft closely to bring you the latest news about Windows, Office, Azure, Skype, HoloLens and all the rest of their products.

Recent News