Paper Title

Devising and Detecting Phishing: large language models (GPT3, GPT4) vs. Smaller Human Models (V-Triad, generic emails)

Corp Blog Article

https://blog.knowbe4.com/ais-role-in-cybersecurity-black-hat-usa-2023-reveals-how-large-language-models-are-shaping-the-future-of-phishing-attacks-and-defense

Paper Abstract Summary

Phishing Attacks

• AI programs utilize large language models to automatically generate phishing emails with minimal user data. This is unlike manual phishing email design based on hackers’ experiences.
• The V-Triad rules allow manually crafting phishing emails based on cognitive biases.
• Study compared participant responses to GPT-4 auto-generated emails, V-triad manual emails, and their combination. A control group received generic phishing emails for comparison.
• Phishing emails were sent to 112 participants (Harvard students) offering Starbuck gift cards.
• Control group clicks: ~20% (19-28%)
• GPT-generated emails clicks: ~30% (30-44%)
• V-Triad-generated emails clicks: ~70% (69-79%)
• GPT-V-triad emails clicks: ~45% (43-81%)

Example GPT-V-triad Email

Phishing Defense

• Four popular large language models (GPT, Claude, PaLM, LLaMA) used to detect phishing email intent. AI detection was then compared to human detection.
• AI excelled in identifying malicious intent, even for non-obvious phishing emails.
• AI sometimes outperformed humans, though often with slightly lower accuracy.
• Claude’s results were highlighted for not only achieving high results in detection tests but also providing sound advice for users.

Paper Abstract

AI programs, built using large language models, make it possible to automatically create phishing emails based on a few data points about a user. They stand in contrast to traditional phishing emails that hackers manually design using general rules gleaned from experience. The V-Triad is an advanced set of rules for manually designing phishing emails that exploit our cognitive heuristics and biases. In this study, we compared how many participants pressed a link in emails created automatically by GPT-4 and created manually using the V-triad. We also combine GPT-4 with the V-triad to assess their combined potential. A fourth group, exposed to generic phishing emails, was our control group. We utilized a factorial approach, sending emails to 112 randomly selected participants recruited for the study. The control group emails received a click-through rate between 19-28%, the GPT-generated emails 30-44%, emails generated by the V-Triad 69-79%, and emails generated by GPT and the V-triad 43-81 %. Next, we used four of the most popular large language models (GPT, Claude, PaLM, LLaMA) to detect the intention of phishing emails and compared the results to human detection. In some cases, the AI programs are surprisingly good at detecting malicious intent, even for non-obvious phishing emails, sometimes surpassing human detection, although often being slightly less accurate than humans.