Cyberattacks in the GenAI Era – What Threats Are CISOs Facing?

97% of organizations that experienced an AI incident did not have adequate access controls in place (IBM, 2025). In addition, an average of 13% of organizations reported breaches that directly affected their AI models or applications.

Since GenAI is available at your fingertips, those who can use it have the advantage – not always in good faith. It is largely up to people responsible for the organization’s security, especially CISO (Chief Information Security Officer), whether AI will become a loophole or a defense tool.

We will talk about how to transform AI from an uncontrolled risk into an element of your security strategy as part of a free series of webinars AI vs Cybersecurity – practical examples, ready-made solutions and Q&A sessions.

Phishing powered by GenAI

Phishing has remained one of the most common and effective attack vectors for years – it is based on social engineering techniques, including manipulation of emotions, trust and cognitive errors. Cybercriminals impersonate trusted people or institutions to extort data, funds or gain access to systems. Since Generative AI has become widely available, this technique has undergone significant evolution.

What is AI-generated phishing?

AI-generated phishing attacks use large language models (LLM) to generate natural-sounding, convincing and highly personalized phishing messages. Unlike traditional campaigns (often written in broken English and sent en masse), these messages are tailored to the context, industry and even the role of the recipient, and are often based on current projects and the organization’s internal communication style.

AI also enables more effective attacks such as Business Email Compromise (BEC) and the so-called CEO fraud. Generative models let criminals create credible messages in the local language, tailored to the company’s communication style. They impersonate people in high positions, exerting pressure – e.g. with an urgent request for an unusual transfer or disclosure of confidential data. GenAI lets criminals engage in a dynamic dialogue with the victim (answer questions, escalate the narrative, and even analyze previous communication threads). These attacks become even more credible and targeted.

This is why traditional security filters based on detecting known patterns (e.g. keywords) are becoming less and less effective against realistic and contextually relevant phishing attacks generated with the help of GenAI.

Techniques that increase the credibility of cyberattacks: voice cloning, deepfake, multi-channel attacks

Phishing powered by GenAI is no longer limited to the email inbox: today it also includes audio, video and coordinated communication across multiple channels.

Voice cloning

Using voice cloning tools, cybercriminals can create realistic voice recordings based on just a few seconds of a sample – e.g. from social media, webinars or video recordings. These synthetic voices are used, among others, in vishing (voice phishing) attacks, during which the attacker impersonates a member of the management board or financial director and applies pressure to get the victim to take action – e.g. transfer funds or provide credentials.

Listen to what voice cloning might sound like here.

Deepfake video

GenAI allows you to create realistic videos featuring famous people. Criminals generate materials in which celebrities, CEOs or influencers “in their own words” encourage a specific action. These attacks are particularly effective because they involve vision and hearing at the same time, increasing the credibility of the message.

Deepfakes are also used to attempt to infiltrate organizations during recruitment processes. A famous case was the situation at Arup, where the “candidate” for the position of engineer turned out to be a generated deepfake. Fortunately, the recruiter recognized the ruse, but if the attack had been successful, the fake employee could have gained access to company systems, customer data, and even introduced ransomware. We wrote more about this case at Arup on our LinkedInie.

According to the IBM Cost of a Data Breach 2025 report, 16% of all data breaches were related to the use of artificial intelligence, of which:

• 37% were AI-generated phishing
• 35% are incidents involving the use of deepfake or voice clone.

This means that over 5% of all global data breaches in 2025 were directly related to deepfakes or synthetic voice – and this number will undoubtedly increase.

Multi-channel attacks

GenAI makes conducting automated and coordinated phishing campaigns much easier and faster than before. Cybercriminals combine multiple communication channels – emails, text messages, phone calls, corporate messengers and social media – creating a coherent, consistent message. AI tools such as voice cloning, these attacks are hyper-realistic and put strong pressure on the recipient, which increases their effectiveness and makes it harder for security systems to detect the trick.

Automatic victim reconnaissance that allows you to create hyperrealistic campaigns

One of the huge advantages that cybercriminals have gained thanks to generative AI is the automation of reconnaissance, i.e. the stage preceding the attack in which data about potential victims is collected.

In the past, such reconnaissance required tedious manual analysis. Today, thanks to the combination of LLM (Large Language Models) with OSINT tools, criminals can automatically search and analyze:

• LinkedIn profiles
• company publications, reports, conference presentations,
• social media entries (Twitter/X, Facebook),
• recruitment advertisements,
• forum posts, code comments, and internal documents publicly disclosed.

For CISOs, this means the need for open-source intelligence (OSINT), i.e. analysis:

• what information the company (and its employees) unknowingly shares publicly,
• how data is secured,
• Do risk analysis procedures take into account OSINT + AI-based attack vectors?

Other cyber threats in the era of GenAI: Shadow AI, Shadow Agents and prompt injection

As GenAI becomes more integrated into everyday business operations, new, complex attack vectors emerge that – without proper security procedures – can escape the CISO’s control.

Shadow AI

Shadow AI is a phenomenon of unauthorized use of artificial intelligence tools and models by employees – without the knowledge of the IT department or the consent of the CISO. A simple example: a sales employee who pastes customer data into a free AI tool to generate an offer. These types of activities carry a huge risk of privacy violations, loss of control over data and violation of regulatory provisions (e.g. GDPR).

According to IBM, as many as 20% of organizations experienced a Shadow AI-related data breach, and these incidents were on average $670,000 more expensive than attacks that did not involve this area. Customers’ personal data were most frequently compromised – in 65% of cases. They are the most tempting morsel for cybercriminals: they can be sold, used for extortion, or impersonated as victims in subsequent attacks. Worse yet, 63% of organizations have no AI management policies in place. Ask whether the organization has procedures in place to minimize the risk of Shadow AI. If not, now is the time to address it.

Shadow Agents

In the coming months, the problem of Shadow AI may give way to an even more serious phenomenon – Shadow Agents. Google Cloud predicts that employees will increasingly implement autonomous AI agents that make decisions and perform business tasks without human intervention. Sounds efficient? Yes – but also dangerous.

Such agents can operate outside the organization’s control, processing customer, project and infrastructure data. This is a huge threat to which cybersecurity resilience must be built. It is worth noting here that a ban on the use of AI agents is a path to nowhere. Employees who want to improve their work can find a way to use them outside the corporate environment.

Instead, organizations should implement Agentic Identity Management – a new IAM model that treats AI agents as digital users with their own identity.

1. Access should be granted on a temporary, just-in-time basis and in accordance with the principle of least privilege.
2. AI agents should only have access when they really need it – for specific tasks and for specific periods of time.
3. The system should constantly assess whether such access still makes sense, adapt it to the situation, and allow for a clear indication of who is responsible for what.

This approach will reduce the risk of abuse and the so-called privilege creep, as agents accumulate more and more privileges that they no longer need – often without the organization’s control.

Prompt injection

Prompt injection is one of the newest threats to organizations using AI. The attack consists in sneaking a malicious command into the prompt (text instruction), which the AI model will treat as its own command, ignoring security measures, user intentions or operating principles. As a result, the model may, for example, reveal confidential data, bypass access control, perform unauthorized actions or harm the organization in a way that is difficult to detect.

In practice, it may look like a regular email with a link to an AI chat, where an innocent-sounding question contains hidden code that instructs the model to reveal sensitive information.

Google Cloud Security warns that attacks of this type will increase rapidly in 2026, along with the increasing availability of AI models and their integration with business systems. Prompt injection has a low entry barrier and high effectiveness, which is why it is an increasingly attractive tool for mass data leakage or silent sabotage.

Although phishing remains one of the most common cyber threats, it is increasingly one of the elements of more complex campaigns supported by artificial intelligence. Cybercriminals today have a whole range of tools: from voice cloning and deepfakes, through autonomous agents, to prompt injection attacks.

This requires changes in the organization’s security strategy: classic filters and procedures are no longer sufficient, especially when threats originate inside the organization, as in the case of Shadow AI.

That’s why in the second and third webinars of the AI vs Cybersecurity’ series, we will focus on these less obvious, but critical attack vectors. Participants:

• they will see how real Shadow AI incidents took place and what manipulation of language models can look like,
• will receive specific rules and processes that allow them to use AI safely,
• will learn how to design an environment resistant to abuse of AI agents and LLMs,
• they will receive ready-made defense scenarios that actually minimize the risk of sabotage and data leakage.

Recommendations for CISOs – how to build the organization’s resistance to GenAI-powered attacks

Practical employee education and regular testing of the organization’s resilience

Even the best-designed infrastructure will not stop an employee who thoughtlessly clicks on a malicious link. That is why it is so important toincrease employee security awareness and develop lasting cyber habitsthat will protect both themselves and the entire organization.

Instead of theoretical cybersecurity training, it is worth considering Practical Anti-Phishing Training, based on realistic simulations of cyberattacks. Employees receive messages tailored to their role, industry and context – exactly what they may encounter in their everyday work. If someone falls for the trick, they immediately receive a short micro-training that explains why the message was dangerous, what the consequences could be, and how to recognize a similar threat in the future.

As a result, over time, the click rate for phishing drops significantly and the number of reported messages increases, which can be seen in the clear but accurate statistics presented in the dashboard of the SECAWA training platform and automatically generated reports.

You can test your team’s response to cyberattacks at no cost or obligation and while also testing our phishing simulation training platform. All you need to do is make an appointment at Free Phishing Test, in which we will send a realistic phishing simulation to a selected group of employees. During the Free Phishing Test, you will be able to observe the statistics in the platform’s dashboard, and at the end we will prepare a report with the test results for you. Click here to find out more!

Principle of limited permissions for people and AI systems

It is not enough to treat the LLM as a tool. We need a clear assignment of “identity” for each model, agent or plugin – along with the scope of its permissions. This means:

• access only to specific data, applications and APIs needed to complete the task,
• temporary permissions (limited to the moment of execution of the action),
• registering activity: who activated what, when and for what purpose
• function monitoring: what prompts, what answers, what operating logic.

Just as we limit access for humans, we must do the same for AI to minimize the risk of abuse, errors and even hijacking.

Set clear rules for using AI

Not every attempt to “make work easier” with AI is dangerous, but without clear rules, the company is exposed to data leaks. Therefore, as a CISO:

• Set clear rules: which tools are allowed, what data cannot be used, in what cases AI can and cannot support.
• Give an alternative: create a safe environment for working with AI.
• Introduce control and monitoring: so you know when unauthorized use, data leaks or policy violations occur.

Protect AI models and agents from hijacking or manipulation

Those responsible for organizational security should secure AI models similarly to securing critical infrastructure – controlling input data, isolating environments, testing for vulnerabilities and protecting APIs, especially if the model has access to corporate data.

Strengthen your security foundations

New threat vectors do not mean that basic security principles are becoming secondary. On the contrary, an organization’scybersecurity strategy should combine several independent layers of protectionthat complement each other and compensate for gaps in single solutions. Technological security should be strengthened, security procedures improved and employee security awareness should be developed.