USF study finds smarter way to train employees to thwart phishing scams

Media Contact:
John Dudley
(814) 490-3290 (cell)
jjdudley@usf.edu

Click here for images and a PDF of the journal article

Key takeaways

• Phishing “gotcha” tests may backfire: New USF-led research finds that on-the-spot phishing training can trigger defensiveness and limit learning — making employees less likely to absorb lessons that prevent future attacks.
• Broader feedback builds stronger defenses: Training everyone after a phishing simulation — not just those who clicked — helped participants stay alert and resist scams for months.
• Rethinking a cybersecurity ‘best practice’: The study challenges a cornerstone of the anti-phishing industry and offers a more effective, research-backed model already being adopted by companies like KnowBe4.

TAMPA, Fla. (Nov. 3, 2025) – Companies often send out simulated – or fake – phishing emails to employees to see who takes the bait and click. Those who fall for such scams typically receive an on-the-spot lesson meant to help them recognize suspicious messages the next time.

These phishing simulations — known as embedded training because once users fail, they are sent into training mode — are widely considered to be a “best practice” in the cybersecurity anti-phishing industry.

But new research co-led by University of South Florida’s Muma College of Business faculty finds that approach might not be the best way to help employees learn from their mistakes.

The findings publish in MIS Quarterly on Monday, Nov. 3, at 9 a.m. ET and are embargoed until that time. The paper is co-authored by Dezhi Yin and Matthew Mullarkey of USF’s Muma College of Business, Gert-Jan de Vreede of Stevens Institute of Technology, and Moez Limayem, president and professor at the University of North Florida, who was selected this month to become USF’s president-elect.

The researchers identify two shortcomings associated with embedded training:

• Instant feedback can be limited in reach. Only those who were duped received training, while those who passed may end up falling for a real phishing attack later, the research showed.

• Catching employees at the exact moment of failure – known as “just-in-time” training – can be counterproductive. Such on-the-spot training can lead to negative reactions in employees who feel exposed and may become defensive.

Instead, the researchers recommend taking a non-embedded approach. By providing feedback to everyone after the entire simulation ends, the exercise turns into a broader and more positive learning opportunity, they found.

The study employed three large-scale experiments using a real phishing simulation platform. Thousands of students received realistic, but simulated, phishing emails. Some provided immediate feedback after clicking, while others provided follow-up messages days later. The team then tracked how likely participants were to fall for future simulated scams over the next several weeks and months.

“Giving feedback only to the people who clicked the ‘fake’ phishing email misses a big opportunity,” Yin said. “We found that employees learn better when everyone — even those who didn’t fall for it — gets a follow-up message explaining the phishing test.”

Among the study’s key insights, researchers discovered:

• Sharing lessons with the entire group, not just those who got duped, helped people recognize scams more effectively and stay alert for months afterward.
• Training does not need to be delivered at the point of failure to be effective. A time-delayed but more inclusive approach ultimately builds a better defense against real attacks.

“Phishing training companies can directly make use of our key insights in designing more effective software tools, and we heard that KnowBe4 is already doing that,” Mullarkey said.

The project began with support from KnowBe4, a Clearwater, Florida-based cybersecurity company that donated software licenses for more than 12,000 users and provided technical expertise and research funding.

“This is an example of research that literally would not exist without the industry partnership,” Mullarkey said. “KnowBe4 gave us access to the platform, helped us understand how to launch phishing simulations and track user behavior, and even funded research stipends.”

The study’s findings could help companies strengthen their cybersecurity defenses as phishing scams grow more sophisticated and increasingly use artificial intelligence.

“Employees are widely considered the last line of defense in the anti-phishing training industry,” Yin said. “Non-embedded training provides a more effective alternative to fortify this last defense than the status quo.”

###

About the University of South Florida

The University of South Florida is a top-ranked research university serving approximately 50,000 students from across the globe at campuses in Tampa, St. Petersburg, Sarasota-Manatee and USF Health. In 2025, U.S. News & World Report recognized USF with its highest overall ranking in university history, as a top 50 public university for the seventh consecutive year and as one of the top 15 best values among all public universities in the nation.  U.S. News also ranks the USF Health Morsani College of Medicine as the No. 1 medical school in Florida and in the highest tier nationwide. USF is a member of the Association of American Universities (AAU), a group that includes only the top 3% of universities in the U.S. With an all-time high of $738 million in research funding in 2024 and as a top 20 public university for producing U.S. patents, USF uses innovation to transform lives and shape a better future. The university generates an annual economic impact of more than $6 billion.  USF’s Division I athletics teams compete in the American Conference. Learn more at www.usf.edu.