What a Security Awareness Program Should Measure: Report Rate, Time to Report and Fair Handling of Repeat Clickers

Counting what is easy versus what matters

Most programs report the same two numbers: the percentage of staff who finished the course and the percentage who clicked in the last simulation. Both are simple to collect. Neither says much. A person can finish a video while doing something else; a click rate can fall because the lure got easier or because staff learned that clicking is watched, not because they got better at spotting attacks.

The U.S. National Institute of Standards and Technology speaks to this in SP 800-50 Rev. 1, Building a Cybersecurity and Privacy Learning Program, published in September 2024. It says that in NIST's own research, participating organizations reported that despite good intentions they often used a limited number of metrics that gave an incomplete view of effectiveness. It suggests a broader set, including attendance, completion, engagement, cost per participant and behavior change.

The behavior metric NIST points to

For behavior change the guide's example is the percentage of participants who changed a behavior in the workplace, such as reporting a suspected phishing email. The measurement it describes is the number of participants who received a phishing test email and reported it appropriately. The analysis is the impact of training on recognizing phishing emails and how they are reported. Among the suggested program data are metrics on incident reporting that show whether employees can recognize and report potential events, simulated attack response tracking, and longitudinal data showing behavior change over time.

In other words, the guide treats reporting as the measure of success, with clicking as supporting information. The guide also notes that phishing exercises produce what are usually called click-through or reported measurements, that is, whether the user clicked or whether the user reported.

Why report rate beats click rate

• It measures the action you want. Reporting is what lets your security team contain a real attack early. A click is only a failure to do something.
• It rewards honesty. If reporting is rewarded and clicking is not punished, staff report even their own slips. You will not get that with a click rate that is tied to blame.
• It is hard to game. A click rate can be lowered by making simulations easy or rare. A report rate can only rise if people do the thing.
• It gives real value. Every report of a simulation is practice for the day a real attack arrives, and real reports go straight to your analysts.
• It captures people who saw the email and ignored it. Ignoring is better than clicking but worse than reporting, since the next person receives the same email.

Track both numbers, report the report rate first, and look at the gap. A healthy program shows the share of staff who reported rising over time, with those who clicked and did not report falling.

Time to report

A single early report can be worth more than a thousand late ones. Measure the time from the moment the simulated or real message was delivered to the first report, and the median across reporters. If the first report arrives in minutes, your security team has a chance to remove the message from every inbox before most people open it. If it takes hours, the attack has already spread.

Also measure how fast your own team responds: the time from a report to a verdict and a reply. A slow reply teaches staff that reporting is pointless. A short reply, even an automatic thank-you with a human follow-up, keeps the habit alive.

Click rate, handled with care

Click rate is still useful if you read it carefully. NIST's guide describes the NIST Phish Scale, which rates how hard a simulated email is by taking the employee's context into account. A click rate means little without the difficulty: 10 percent on a very hard lure may show better awareness than 5 percent on an obvious one. Use a range of lures, tag each one with its difficulty, and compare like with like. Also report results by group, since a team that receives vendor invoices all day faces different lures than one that does not.

Repeat clickers: measure them, but fairly

Some people click more than others, and you should know who needs help. How you treat them decides whether the program works. NIST's guide says phishing exercises should not be punitive and that no employee should be called out for a response. Keep individual results visible only to the small group that coaches, and agree this in a written policy with HR and legal counsel before you start.

• Define the group simply, for example people who clicked in three of the last six simulations, and keep the definition fixed.
• Offer private help: a short conversation, a targeted lesson, or a review of how the person reads mail on a phone.
• Look at the lure difficulty and the person's role before drawing a conclusion.
• Add technical protection: stronger mail filtering, safe link opening, and phishing-resistant sign-in such as security keys, so that one click cannot lead to a lost account.
• Re-test the group after coaching and tell them when they improve.

Pair behavior with outcomes

Behavior measures show what people do in a test. Outcomes show what happens in real life. Track the number of real suspicious messages reported, the share of real incidents that began with a user action, the time from first report to containment, and the number of accounts that needed a password reset because of a phishing event. Put these next to the simulation numbers. If simulations improve but real incidents do not, the simulations may not match real attacks.

A one-page dashboard

• Report rate on simulations and on real mail, month by month.
• Median time to first report, and median time for the security team to reply.
• Click rate by lure difficulty, shown by group, not by person.
• Repeat-clicker group size and its trend after coaching.
• Real incidents that began with a user action, and how long containment took.
• Participation and completion, kept as a minor line, since they are the least telling.

Review it quarterly with leadership, and change the training based on what it shows: a department that reports slowly gets a short session on how and where to report; a lure type that works every time gets a place in the next lesson.

A companion article covers how to run simulations that people do not resent, since the measures above only work if staff trust the program.

Sources

• NIST SP 800-50 Rev. 1, Building a Cybersecurity and Privacy Learning Program (NIST CSRC, September 2024)
• NIST SP 800-50 Rev. 1, full text (NIST)

This article is general guidance, not legal advice. Adapt it to your organization, and ask your counsel about employment, privacy and contract questions before you rely on it. Last reviewed November 9, 2026.

Next article

Next · SecurityPhishing Simulations Staff Don't Resent: Fair Design, Realistic Lures, HR and ConsentA phishing simulation can teach people what attacks look like, or it can teach them that security is a trap. The difference lies in design: how realistic the lures are, how staff are told, what happens to someone who clicks, and who in the company agreed to it first. This is a practical guide to simulations that raise the rate of reporting without raising resentment.7 min read

Related reading

• SecuritySecurity Awareness for Finance Teams and Executive Assistants: Payment Fraud and Deepfake Voice Requests7 min
• SecurityProtecting Unreleased Content: Access Control, Watermarking and Freelancer Accounts for Media Production8 min