AI thrives on data, but what happens when that data is corrupted? Data poisoning is a silent yet dangerous threat, compromising trust and innovation in critical systems. Britt Eva Bjerkvik Haaland, Head of Privacy at Sopra Steria Group, explains how we can fight back.
AI thrives on data—vast, sprawling datasets that teach machines to see, speak, and make decisions. But what happens when that data is weaponised against us?
Imagine an AI-powered healthcare system misdiagnosing a patient because of corrupted training data—an invisible flaw with life-altering consequences. This is the danger of data poisoning, a threat that quietly undermines the intelligence we depend on.
As AI becomes central to critical domains like autonomous vehicles and legal systems, it’s a risk that doesn’t just threaten industries; it can erode trust and compromise the very structures that support modern innovation. To better understand this growing menace, we spoke with Britt Eva Bjerkvik Haaland, Head of Privacy at Sopra Steria Group, whose insights shed light on what’s at stake and how we can respond.
Could you start by explaining what data poisoning is?
Britt Eva Bjerkvik Haaland: When I first encountered data poisoning, it was largely seen as a malicious attack aimed at sabotaging AI systems. But the reality is now more nuanced. Data poisoning occurs when the training data for an AI model is corrupted or "poisoned" with incorrect information. It spans a wide spectrum of intentions and outcomes.
At one end, you have tools like Glaze and Nightshade, which empower artists to protect their intellectual property. These tools subtly alter the pixels in images so that AI models misinterpret them—seeing a dog as a cat, for example—while the image remains perfectly clear to the human eye. This can be seen as a form of legitimate data poisoning, used to safeguard creators’ rights in a digital world increasingly dominated by AI.
At the other extreme lies outright malicious intent: poisoning data to disrupt a system’s output, cause operational failures, or even endanger users. And in between, there’s a murky grey area filled with ethical dilemmas. When does data poisoning become justifiable, and when does it cross into harmful territory? This is the question we must grapple with as we navigate this evolving landscape.
Let’s dive into a practical scenario. If someone wanted to infiltrate an AI training pipeline, how might they go about it ?
Britt Eva Bjerkvik Haaland: There are numerous ways an AI training pipeline can be infiltrated, reflecting the vulnerabilities in how data is sourced and handled. Training data often comes from scraping the internet, purchasing datasets, or using open repositories—all of which are susceptible to compromise. Poisoning can occur at multiple stages, whether during data collection, through intentional tampering, or even during model updates.
For instance, bad actors might inject malicious data into open datasets, subtly corrupting the information that AI systems rely on. Sometimes, it’s a case of poor-quality data slipping through, but in other instances, it’s a deliberate attempt to undermine the model’s integrity. Even disgruntled employees could alter datasets or manipulate the model itself, turning internal vulnerabilities into serious threats.
Consider the case of chatbots on X (formerly Twitter) becoming racist after exposure to harmful inputs. While not always a clear-cut example of data poisoning, it demonstrates similar principles: corrupted or biased data fundamentally influencing AI behaviour.
Can you share real-world examples of data poisoning?
Britt Eva Bjerkvik Haaland: Documented real-world cases of data poisoning are scarce, and many incidents likely go unreported. However, it’s easy to envision situations where the consequences could escalate rapidly. Take the case of autonomous vehicles. Imagine a car misreading a stop sign as a speed limit sign due to corrupted training data. It sounds like a small glitch, but the consequences could be catastrophic.
Or consider a New York lawyer who used an AI tool to prepare a case, only to receive fake court rulings. Some called it data poisoning; others suggested it was hallucination—a phenomenon where AI fabricates non-existent information. Regardless of the cause, this example underscores how fragile AI systems become when data integrity is compromised.
Is it possible for organisations to monitor for signs of data poisoning ?
Britt Eva Bjerkvik Haaland: Absolutely. Continuous monitoring is essential. Organisations need to look out for unusual outputs, unexpected biases, or anomalies in system behaviour. Human oversight is a key component in identifying and addressing these issues effectively.
Additionally, tools like explainable AI, which makes the decision-making process of AI models more transparent, can help identify anomalies in decision-making processes. Such insight is invaluable for detecting and preventing data poisoning, allowing organisations to respond proactively and maintain trust in their systems.
Would you say that defending against data poisoning is primarily about improving data quality, or does it require new frameworks to effectively address these attacks ?
Britt Eva Bjerkvik Haaland: It’s a combination of both. While tools like information security management systems and data governance are already in place, the critical factor is awareness. People need to fully understand the risks, especially when working with open-source or scraped data.
Synthetic data—datasets generated by AI—is often promoted as a potential solution, but it’s not without its flaws. These datasets can still inherit biases or errors from their original sources, making them far from foolproof. Addressing data poisoning requires both improving data quality and adopting frameworks that account for these vulnerabilities.
What advice would you give decision-makers about data poisoning?
Britt Eva Bjerkvik Haaland: My advice is to start by mastering the fundamentals. Strong data governance, robust information security measures, and proper employee training are critical. Decision-makers need to treat data as a valuable asset—just as they would financial resources. You wouldn’t invest millions without thorough due diligence, so the same level of care must be applied to how data is managed and utilised.
This approach not only helps combat data poisoning but also addresses broader challenges, including biases and poor model performance. By prioritising the basics, organisations can build a stronger foundation to safeguard their AI systems and ensure better outcomes.
Do we need greater collaboration across society to effectively address these challenges?
Britt Eva Bjerkvik Haaland: Collaboration across society is a necessity. Open, quality-controlled datasets have the potential to benefit everyone, but achieving this requires cooperation between governments, researchers, and companies.
Sharing best practices, setting benchmarks, and developing reliable datasets are crucial steps. A unified, collaborative approach will play a key role in addressing data poisoning and safeguarding the integrity of AI systems. By coming together, we can ensure AI systems are not only secure but also trustworthy. The future of AI depends on the integrity of its data, and it’s a responsibility we all share.