Generative AI creates security challenges while empowering defenses

Generative AI presents a dual challenge: introducing security risks while enhancing defenses with faster threat detection and response.

What happened

A recent study by the Capgemini Research Institute revealed that 97% of organizations using generative AI reported experiencing security incidents or data breaches linked to the technology. These incidents are often tied to vulnerabilities in custom generative AI solutions and an expanded cyberattack surface. Many organizations have suffered financial losses, with 52% reporting direct or indirect costs of at least $50 million due to such breaches.

The findings indicate that generative AI not only introduces new risks but also requires businesses to reassess their budgets, with 62% of respondents acknowledging the need for increased investment in security measures.

Going deeper

The risks associated with generative AI stem from multiple factors, including sophisticated adversaries exploiting vulnerabilities, employee misuse leading to data leakage, and risks like data poisoning. Two-thirds of organizations voiced concerns about the potential leakage of sensitive data used in AI model training. Additionally, 43% reported financial losses related to deepfake-generated content, proving how generative AI can be misused to create biased, harmful, or misleading material.

However, AI also delivers security benefits. It enables rapid analysis of vast datasets, identifying patterns and predicting potential breaches. Companies implementing AI in their security operations centers (SOCs) have reported measurable improvements: 64% reduced their time-to-detect breaches by at least 5% and nearly 40% shortened remediation times by a similar margin.

What was said

Capgemini’s global head of cybersecurity, cloud, and infrastructure services, Marco Pereira, described generative AI as a "double-edged sword." While it introduces unprecedented risks, it also provides powerful tools for faster and more accurate detection of cyber incidents. Pereira discussed balancing AI’s capabilities with advanced frameworks, ethical guidelines, and employee training to harness its potential.

In the know

Microsoft researchers added a positive perspective earlier this year, noting that UK organizations using AI tools for cybersecurity were twice as resilient to attacks. AI-enhanced defenses reduced the frequency of successful attacks and cut associated costs by 20%, with potential savings for the UK economy estimated at £52 billion annually.

The big picture

Generative AI is changing the game in cybersecurity, bringing both risks and benefits to the table. On the downside, it can be exploited for breaches, financial harm, and ethical dilemmas. But on the upside, it’s a powerful tool for improving threat detection and response. To make the most of it, organizations should focus on proactive strategies, smart data management, and building awareness. As this technology changes, balancing innovation with caution will be fundamental for maintaining security.

FAQs

What is AI?

AI, or artificial intelligence, refers to computer systems designed to perform tasks that typically require human intelligence, such as learning, problem-solving, and decision-making.

What is generative AI?

Generative AI is a type of AI that creates new content, such as text, images, music, or code, by learning patterns from existing data.

What are cybersecurity vulnerabilities?

Cybersecurity vulnerabilities are weaknesses in systems, software, or processes that attackers can exploit to access or harm sensitive data or operations.

What is data poisoning?

Data poisoning is an attack where bad or misleading data is introduced into a system to disrupt or manipulate its learning, decisions, or outputs.