Small and Medium-sized Enterprises are particularly vulnerable to cyberattacks. A large-scale field experiment amongst SME employees showed that a phishing test is effective in the short term, but not in the medium or long term.

Why this experiment was conducted: SMEs are vulnerable to cyberattacks
Phishing is one of the most common types of cybercrime and is frequently the precursor to additional cyber threats, including malware and ransomware attacks. SMEs are vulnerable, with many of them not having the appropriate knowledge and resources to protect themselves against cyberattacks. Moreover, the vulnerability of one SME can quickly affect an entire chain of businesses. Is a phishing test amongst employees an effective method to enhance the cybersecurity of SMEs? And is this effect dependent on the time interval between phishing tests? These questions were addressed in the SME Phishing Test, a collaboration between the Regional Platform of Crime Control North Holland and the Ministry of Economic Affairs and Climate Policy.

Type of intervention: phishing test
In an imitation phishing email, SME employees were induced to click on a suspicious link. Upon clicking on the link, they were directed to a feedback page containing information on how they could have recognised the phishing email; see image 30. This is intended to educate them, reducing the likelihood that they will click on such a link again in the future. The number of clicks was also a way of measuring the effectiveness of the intervention. After carrying out the phishing test, the SMEs were sent a report containing their company's anonymous test results and a detailed explanation of how they could (further) enhance their company’s resilience against cyberattacks.

Method used: RCT and questionnaires
A Randomised Controlled Trial was conducted in which 667 companies were randomly divided into four groups based on similar characteristics (number of employees, sector, IT outsourcing). The 33,016 employees of these companies each received two different phishing emails. The time intervals between the emails varied between the groups: roughly 1 month, 2.5 months or 3.5 months. The effect was measured by comparing the click rates between a group that had previously received another phishing email and a group that had not received a phishing email. By varying the time interval between emails, it was possible to investigate whether receiving a phishing mail a shorter or longer time ago might influence the click behaviour on a subsequent phishing email. In addition, a range of questionnaires was employed to gain insight into characteristics of both companies and their employees.

Result obtained: click rate halved when second email is sent in the short term
• Over one in five SME employees (22%) on average clicked a suspicious link in the first phishing email.
• There are indications of a short-term effect of a phishing email, but no indications of a medium or long-term effect. Employees who had received a phishing email roughly a month earlier were significantly less likely to click a second email than those who were yet to receive the first email (9.9% vs. 19.8%). No significant effect was measured where the time interval between emails was 2.5 or 3.5 months.
• People with a risk-taking attitude benefit most from a phishing test. In a questionnaire amongst the participating companies (303 respondents) at the end of the experiment, 72% of companies said they planned to take measures to enhance their resilience against cyberattacks. One year later (47 respondents), companies were asked whether they had actually taken these measures: 51% said they had implemented these measures, and 23% said they still planned to do so.

Impact: enhanced resilience against cyberattacks
This study had impact in three ways:
• It provided insights into the resilience of SMEs against cyberattacks. The results confirmed the urgency of enhancing cybersecurity.
• The results showed that an imitation phishing email can be effective in the short term in making employees less likely to click the next phishing email they receive.
• This study and the experience with the imitation phishing email and feedback have in themselves led to enhanced awareness about SME resilience against cyberattacks. This was also reflected in the follow-up study conducted a year later

Source: https://www.binnl.nl/home+-+en/knowledge/publications/bin+nl+publications/HandlerDownloadFiles.ashx?idnv=2719979