At a time when phishing is becoming more and more sophisticated and organizations’ day-to-day operations rely heavily on digital systems, a 0% click-through rate in a phishing test sounds like an ideal scenario. This is what CISO, the management board, security and compliance departments would like – a situation in which no employee falls for the simulation, sensitive data and organizational systems are safe, and the risk of human error is reduced to a minimum.
But in practice, such a result, although possible, does not say everything about the real resilience of the organization. A single campaign may end with a very high click-through rate or a 0% result – depending on the scenario, the moment of sending and how closely the message fits into the employees’ everyday lives. Therefore, the effectiveness of security awareness activities should not be determined by a single test, but by whether the organization is able to maintain a low level of vulnerability over a long period of time.
Key takeaways
One of the public administration organizations that decided on Practical Anti-Phishing Training managed to achieve 0% click-through rates in several phishing campaigns. However, we consider what happened in the broader perspective of the entire program to be an even greater success.
Within 6 months, the organization went from around 50% to a stable result of below 4%. This shows that well-designed security awareness training can result in lasting behavior change and a measurable reduction in the risk of a security incident.
Additionally, after implementing the phishing reporting button, on average 30% of the team started actively reporting suspicious messages, strengthening the protective shield of the entire organization.
To sum up, although a 0% click-through rate is impressive, it is not worth focusing all your attention on it when assessing the effectiveness of educational activities. Much more important is whether the organization can maintain a very low level of susceptibility to phishing over the long term and whether employees begin to treat reporting suspicious messages as a natural part of taking care of security.
It is also worth noting that for organizations that want to actually build cyber maturity, one-off social-engineering tests are not enough. Such a test is a good starting point, but it does not yet provide a complete picture of the team’s resilience. Only the analysis of the results over a longer period of time shows whether employees actually learn to recognize social engineering attacks and whether safer responses begin to become a habit. That’s why, in the rest of this case study, we show how click-through rates changed over time and how the effectiveness of individual phishing campaigns was influenced by the form of the message, its contextualization and the subject matter itself.
Measures of the effectiveness of security awareness training
The true effectiveness of security awareness activities is not demonstrated by a one-time perfect result, but by permanently reducing employee susceptibility to a very low level and maintaining it over time. This is when we can talk about a real change in behavior.
When a phishing test shows a 0% click-through rate result, it undoubtedly makes an impression. It shows that employees correctly recognized the threat and did not take risky action. For the security team, this is a signal that the scenario has been read correctly and the participants’ vigilance worked as it should.
However, such a result should not be the only measure of the effectiveness of the security awareness program.
The effectiveness of a single campaign is influenced by many factors, such as the type of bait and the realism of the scenario. Meanwhile, in reality, cybercriminals constantly change message topics, communication channels and manipulation techniques. The fact that employees did not fall for a given campaign today does not mean that they will cope equally well with a different scenario in a week, a month or a quarter.
Therefore, the ability to maintain low vulnerability over a longer period of time – in various campaigns, contexts and types of attacks – says much more about the maturity of an organization. We know from experience that some employees will usually fall for a simulation, and achieving 0% click-through rate – although possible – does not in itself mean that the organization is very well prepared for cyber threats.
Therefore, CISO, management, IT and compliance departments should look much broader than just one indicator. In practice, to assess the real level of awareness of employee security and cyber resilience of the organization, it is worth analyzing:
- click rate on phishing links, opening and launching attachments, entering data into fake forms, reporting suspicious messages,
- employee reaction time to a threat,
- differences in susceptibility between departments, groups and roles,
- number of employees with increased susceptibility,
- changes in results over time, not just the result of one campaign,
- effectiveness of specific types of baits and scenarios,
- campaign activity divided into channels, groups and sending moments,
- data about the technical environment and how you interact with messages,
- results of individual campaigns.
These measures are measured by our proprietary platform for simulating cyberattacks – Practical Anti-Phishing Training – giving a more complete picture of the organization’s cybersecurity resilience.
When designing subsequent phishing simulations, we do not look only at one indicator and do not consider the program completed just because the percentage of 0% has been achieved. We continue training to constantly and systematically increase employees’ resistance to new techniques of cybercriminals.
The idea is to actually reduce the probability of a successful attack, reinforce safer employee behavior and gradually build a team that is less likely to fall for attacks and increasingly likely to recognize and report them.
Starting point: 50% click-through
How we understand the effectiveness of Practical Anti-Phishing Training is well illustrated by the example of one of the public administration organizations.
At the beginning of the cooperation, the level of susceptibility was very high. In the first phishing simulation, the click-through rate was approximately 50%. This means that every second employee took a risky action.
In practice, such a result does not mean individual errors, but a real gap in the organization’s resistance to phishing – especially if the attack uses a well-chosen pretext, time pressure or a topic related to everyday work.
That is why the aim of the training was not to quickly improve one indicator, but to permanently change behavior. The idea was for employees to stop reacting thoughtlessly to fabricated messages and start approaching them with more caution.
Solution: systematic training
The organization decided to systematically build immunity through Practical Anti-Phishing Training, in which the SECAWA team carried out various phishing scenarios tailored to the specificity of the organization and the standard processes and systems used.
As a result, our phishing simulations checked the ability of employees to recognize threats that could hit them in reality – from verifying the sender of the message, through checking the link before clicking, to assessing whether the attachment is safe.
Practical Anti-Phishing Training – a Polish platform for simulating cyberattacks that measures team reactions, reports results and helps meet regulatory requirements in the field of security awareness
Diverse phishing scenarios
At the beginning of the training, we used campaigns referring to security and internal communication. They showed how high the team’s susceptibility was. The security guidelines and reminder message generated approximately 50% click-through rate, and the enhanced MS Office feature campaign generated 35% click-through rate.
In the following months, the training included various scenarios, including:
- change your Facebook password,
- Allegro paid surveys
- help for flood victims,
- Black Friday campaigns
- Multisport offer
- tax refund.
This differentiation was important. Employees did not learn to recognize one message pattern, butgradually became familiar with various manipulation techniques – from formal and systemic topics to messages using curiosity, time pressure or the promise of benefits.
This is how Practical Anti-Phishing Training works. It is not based on a one-time test, but on regular exposure to different types of threats. As a result, safer reactions become a habit over time, not an accident.
Results of phishing campaigns over time
Although the first phishing simulations showed how vulnerable the organization was – about 50% of clicks – in the following months the results began to improve significantly.
During the program, employees received various phishing scenarios set in different contexts – from internal and system messages to messages using seasonal, formal or financial-related topics. It was possible to observe not only a general decrease in susceptibility, but also which types of baits still require further training.
In this organization, it was clearly visible that campaigns strongly rooted in the realities of employees and sent at the right moment – such as “PIT-11” or “Christmas competition” – were able to generate a higher click-through rate than other scenarios. It’s natural. Such messages are most similar to those that employees may actually encounter in a given period, which is why they attract attention more easily and reduce vigilance.
Despite this, the direction of change was very clear. The team gradually learned to recognize attack mechanisms and respond more carefully to suspicious messages, which was confirmed by a persistent decline in click-through rates in subsequent phishing simulations.
Individual increases in click-through rates do not undermine the effectiveness of training
On the contrary – they are a valuable signal which scenarios still have the greatest impact on employees and what types of messages the team should be particularly aware of when designing subsequent educational activities.
Using the statistics available on our platform, the organization could continuously identify which campaigns produced the highest click-through rates, which departments needed additional support and how quickly the first failures appeared. It is this level of detail that opens up completely new possibilities in security awareness education – it allows you to measurably assess the team’s resistance and precisely indicate its weak points.
In this case, despite individual increases in clicks, which were a valuable tip for further education, the organization went from a level of about 50% to a stable result below 4% in just 6 months. This means that more than 96% of employees began to effectively recognize social-engineering techniques. This is a very good result – especially since it was not a one-time effect, but persisted over time.
Proactive defense
The cyber maturity of an organization is not only about employees recognizing the threat better and better. Equally important is whether they can respond appropriately to them.
In an organization, after implementing the phishing reporting extension, on average, approximately 1/3 of the organization began to proactively report suspicious messages.
Our phishing reporting extension allows you to report suspicious messages with one click. This allows the employee to react quickly and block phishing before it causes damage. In turn, the organization gains a simpler way to handle incidents, more efficient reporting to the management board and auditors, and the ability to appreciate those employees who demonstrate a proactive cyber attitude.
Summary
The case of public administration clearly shows the meaning of Practical Anti-Phishing Training. It is not about a one-time vulnerability test or achieving one campaign with a perfect result to consider that the program has achieved its intended goal. Our indicator of success ispersistently reducing risk, strengthening safe habitsand building an organization that, over time, increasingly better recognizes and stops threats.
That is why the greatest value of the described case was not the achievement of 0% click-through rates in selected phishing simulations, but the fact that within 6 months the organization went from around 50% to a stable result below 4%, and some employees also began to actively report suspicious messages.
