How To Spot AI-Written Emails and Phishing Attempts

Generative AI helps scammers scale their phishing efforts quickly and cheaply, which increases risk for ordinary people.

In un 2024 survey conducted by Harvard Business Review, 60% of participants fell victim to AI-automated phishing, boasting a larger success rate than human-written initiatives.

Immergiamoci in questa storia.

---

Punti di forza

• Generative AI makes it easier and cheaper for attackers to create vast amounts of legitimate-looking phishing emails.

• You can flag AI-written phishing emails by investigating writing styles, tone, detail inconsistencies, sending addresses, link domain mismatches, authentication results, and attachments. AI detection can simplify the investigation by analyzing writing styles faster. 

• Building an AI screening workflow ensures that your team can identify suspicious emails more consistently, prioritize higher-risk messages, and verify sensitive requests before taking action.

---

What Are AI-Written Phishing Emails? 

Phishing emails deceive recipients into giving attackers information, money, credentials, or access. An attacker may impersonate a colleague, vendor, executive, bank, or other trusted organization to make the request seem legitimate.

AI-written phishing emails use generative AI to create or improve that deception. An attacker might ask an AI tool to:

• Draft an email that impersonates an executive
• Rewrite a message in a more professional tone
• Translate a phishing email into another language
• Personalize a message for a particular employee or company
• Create several variations of the same scam
• Remove spelling and grammatical errors from an existing draft

While the technology does not change the basic goal of phishing, it makes the messages easier and faster to produce at scale. Harvard Business Review even claims that generative AI reduces the cost of producing phishing emails by 95%. 

How Generative AI Changed The Way Everyday Business Emails Get Written 

Generative AI has also changed legitimate business communication. Legitimate organizations use AI to write emails, summarize conversations, improve grammar, translate messages, and adjust their tone.

That creates a problem for anyone trying to identify AI-written phishing attempts. An email can sound unusually polished because a legitimate employee used an AI writing assistant. A scammer can use the same technology to produce an equally polished message.

AI also lets attackers create more convincing variations at scale. Instead of sending one generic message to hundreds of people, an attacker can generate different versions for different roles, companies, or situations.

AI-Written Emails vs. Old School Scams

Attackers initiating phishing scams often operate with limited time and resources. Before generative AI became widely accessible, they created emails using quickly produced templates and poor translations, which lead to grammatical errors, awkward phrasing, strange formatting, and other obvious flaws. 

However, generative AI makes these flaws easier to correct. Attackers can use generative AI to  produce error-free messages that match common business conventions. 

LLMs can also learn information and linguistic patterns faster and from a wider variety of sources, which enables personalization. They can scrape LinkedIn posts, company news, and organization charts to impersonate individuals or businesses more convincingly. 

AI-Written Phishing Emails vs. Legitimate AI-Assisted Emails

AI use does not automatically prove malicious intent. While scammers use AI to scale phishing initiatives, businesses also use the same tools to make ordinary communication more efficient.

AI use is just another signal reviewers should consider when evaluating suspicious emails.

To verify intent, combine the information with other elements, such as:

• Sender: Does the address belong to the person or organization it claims to represent?
• Contesto: Does the message follow an existing conversation?
• Knowledge: Does the sender mention details they could reasonably know?
• Request: Does the action match the sender’s role and normal process?
• Destination: Do links and attachments lead where you expect?
• Verifica: Can you confirm the request through a trusted channel?

These checks can help you distinguish AI-written emails created for fraudulent purposes and legitimate emails that involve AI assistance. 

Signs That an Email Was AI-Written

Generative AI produces responses by predicting the most statistically likely continuation of a unit of language. As a result, most AI-generated emails share recognizable stylistic patterns.

Though none of these patterns explicitly prove that an email was AI-written, two or three together may warrant justification. 

Uniform Sentence Length And Rhythm

Human writing usually uses variations in sentence length, pacing, and phrasing to convey tone or emotion. In contrast, AI-generated messages maintain consistent cadences or sentence structures. 

Evenly Distributed Connectors And Hedging

AI-generated writing may rely more heavily on explicit transition words, such as “moreover,” “furthermore,” and “additionally,” to connect ideas and maintain a clear flow.

While these connectors also appear in human writing, they become cause for suspicion when used very frequently, or when their usage differs from the sender’s typical style. 

Empty Summaries

LLMs generate text unit by unit and lack a reliable internal mechanism for judging when an email contains enough information. As a result, they may add unnecessary summaries, conclusions, or explanations.

Formatting Anomalies

Text pasted directly from an AI chat can retain formatting that does not fit normal email communication, including unexpected markdown artifacts, unusual spacing, or unnecessary headings.

Signs an Email Is a Phishing Scam

While phishing emails can deceive even careful recipients, knowing what to look for can help you identify suspicious messages before you act on them.

Most phishing emails contain common signals that can help you recognize a potential scam, such as unusual sender addresses, suspicious links, unexpected requests, and attempts to create urgency.

Generic Specificity

Phishing emails may reference vague current concerns, such as a “recent project,” “current workflow,” or “upcoming priority,” without providing details that the recipient can verify. This language creates the impression of personalization to deceive recipients. 

Urgency and Secrecy

Attackers often prey on their target’s sense of stakes or urgency. They pressure recipients with tight deadlines, threats, or requests to keep matters secret. 

Watch for:

• Same-day or immediate deadlines.
• Requests to keep the matter secret.
• Sudden payment-detail changes.
• Gift card requests.
• Threats involving account closure or missed payments.

Inconsistent Details

An email might include accurate but outdated information, or mention details that the supposed sender could not reasonably know.

Requests that Bypass Normal Processes

Attackers often make requests that ask recipients to ignore established processes. For example, an email might ask you to:

• Bypass procurement
• Skip a ticketing system
• Avoid a normal approval process
• Send payment without the usual documentation
• Transfer funds without two-person approval

Should you receive a sensitive or unusual request, verify the sender through a trusted channel rather than directly through the email or any links provided within the email.

For example, if a vendor emails you asking to change their payment details, call the vendor using a phone number from your existing records or contact your usual account representative through a trusted channel.

Sending Addresses

Attackers can spoof the display name of a trusted organization or person to make an email appear legitimate. They often use lookalike or unrelated domains for the sender address, although some attackers can spoof the visible From address.

Always check the full sending address for subtle differences from the legitimate domain. A familiar display name alone does not confirm that an email came from the person or organization it claims to represent.

Link Domain Mismatches

Clicking on a fraudulent email link can expose you to several risks, including credential theft, malware, and account takeover. 

Before clicking a link, hover over it on desktop or long-press it on mobile to preview the destination URL. In fraudulent emails, domain links do not match the website the sender claims to represent. Attackers might hide this mismatch through subtle changes, such as misspellings, extra words, or unusual subdomains. 

You should also be cautious when the link uses a URL shortener or redirects you to an unexpected domain. If you cannot verify where a link leads, do not click it. Instead, navigate to the organization’s official website directly or contact the sender through a trusted channel.

Authentication Results

Email headers can contain authentication results that provide useful evidence:

• SPF: Checks whether the sending server is authorized to send mail for the domain.
• DKIM: Uses a cryptographic signature to help verify that the message came through an authorized system and was not altered in transit.
• DMARC: Builds on SPF and DKIM and lets domain owners specify how receiving systems should handle authentication failures.

An authentication failure does not automatically mean that someone sent a phishing email, but it provides an important signal for investigation.

Unexpected Files, QR Codes, and Document Invitations

Attackers typically use attachments to lead recipients to malicious locations. Common examples include:

• Unexpected files, which attackers use to carry malware
• QR codes, which often lead to malicious websites
• Shared document invitation links, which typically lead to fake login pages

Be careful with emails that contain these attachments, especially when the sender the attacker intends to impersonate rarely uses these methods. Check other signals, such as the sending address and domain mismatches, before taking action.  

How to Use AI Detection for Phishing Emails

One way to flag phishing emails efficiently is to run them through an AI text detector. These tools use forensic analysis to scan text for signals commonly associated with generative AI. 

Based on these signals, they calculate the likelihood that the text was AI-generated or AI-edited. The resulting detection score can serve as a triage signal that triggers additional investigation. 

How To Build an AI Screening Workflow

An effective email screening process combines AI detection with human verification and established security controls. 

Step 1: Set a triage rule

The first step in building an email screening process is to determine which types of messages need additional screening. It’s best to target the types of emails that scammers often impersonate, such as:

• Finance and payment requests, such as invoices, wire transfers, or bank account changes
• Vendor communications, especially messages about payments, contracts, or account updates
• HR communications, such as requests for employee information or benefits documents
• Executive requests, particularly urgent requests for money, sensitive information, or unusual favors
• IT and security alerts, such as password resets, account suspensions, or software updates
• External first-contact emails, especially when the sender has no established relationship with the recipient
• Shared-document invitations, particularly unexpected requests to view or edit a file

Setting up a triage system helps your team focus efforts on higher-risk messages. It ensures employees know how to handle common cases, which improves efficiency. 

Step 2: Define the escalation path

The next step is determining how to handle flagged messages. This means establishing where employees should send suspicious emails and defining who can approve unusual requests. 

Step 3: Add out-of-band verification

If a flagged message asks for sensitive information or an important action, it’s best to verify the legitimacy of the sender before responding.

You can achieve this by requiring employees to confirm sensitive or unusual requests through a known phone number or internal communication channel, rather than directly through the email or any channels listed within the email.

For example, if you receive an email from a vendor asking you to change their payment details, you could verify the request by:

• Calling the vendor using a phone number you already have on file.
• Contacting your usual account representative through a separate email address or phone number.
• Checking with your procurement or finance team to confirm the change.
• Reviewing the vendor’s existing records to make sure the new details match a verified request.

This extra step stops you from acting on potentially fraudulent messages, reducing the risk of financial loss or data exposure. 

Step 4: Train with real examples

It is also essential to train employees on common cases. Schedule routine training programs that show employees flagged emails from the past. Explain what made each message suspicious and describe how the organization responded. 

This way, employees can build familiarity with real tactics in a risk-free environment. It prevents them from making errors when real money, information, or other assets are at stake. 

Step 5: Log the outcomes

To improve future workflows and training efforts, keep a log of past responses. Track which messages turned out to be legitimate and which represented actual threats. These outcomes can help you refine your rules and understand where your screening process produces false positives or false negatives.

Where an AI Detector Fits

AI detectors can serve as an additional screening layer within an email workflow. While detection scores do not prove that an email is malicious, they give human reviewers another signal to consider alongside the sender, message context, authentication results, links, attachments, and requested action.

Together, these signals provide a clearer basis for deciding how to handle the email.

Domande frequenti

Pensieri finali

Though AI makes phishing emails easier to create, organizations can still identify many of these attempts by establishing a clear AI screening workflow that combines AI detection technology with human judgment.

Use AI detectors on high-risk email types to flag them faster, then evaluate the detection score alongside factors like sending addresses, writing style, message context, links, and attachments. 

AI non rilevabile can help you flag AI-generated phishing attacks quickly. Our detection scores can serve as an additional screening signal, supporting your existing email security checks for informed decision-making.