Ever wished you could get a long email summarized quickly and concisely? AI email summarization features, like Google Gemini for Workspace, are excellent tools many people enjoy and widely use. However, recent news reveals a startling vulnerability: this very AI could be exploited by hackers as a phishing tool!
When AI Unwittingly Becomes an Accomplice
Security researcher Marco Figueroa discovered and disclosed this vulnerability through Mozilla's 0din bug bounty program. He found that attackers can hide malicious commands within the content of an email in a remarkably subtle way.
Imagine this: hackers embed HTML/CSS code that sets the font to the smallest possible size or changes its color to blend with the background (e.g., white on white). This makes the command invisible to the naked eye in Gmail. And because the email contains no links or attachments, it can easily bypass security systems and reach your inbox!
Once you open that email and ask Gemini to summarize the message, Google's AI tool will process those hidden commands and unwittingly follow them.
The Unseen Threat in Your Inbox
Figueroa explains that attackers can embed commands designed to be invisible into the email content using HTML/CSS techniques, such as setting the font size to zero or changing the text color to blend with the background, so users won't notice them with the naked eye.
Even though the email appears safe, with no links or attachments, most security filtering systems will let it pass through to the user's inbox. But when the user opens the email and asks Gemini to summarize the content, Google's AI tool will process the hidden text and immediately follow those commands, without discerning if the command is malicious. For example, it might insert a fake message like “Your password has been compromised!” along with a fake phone number, potentially tricking the victim into calling back or unknowingly giving away sensitive information.
What makes this attack so dangerous?
- No links or attachments: Makes the email seem safe at first glance
- Highly convincing message: The warning comes from Gemini, a trusted Google Workspace tool
- Users let their guard down: Because they trust AI-generated content
Safeguarding Your Organization
To protect your organization from falling victim to this new type of threat, here are some practical guidelines:
- Filter or Block Hidden Content: IT teams should scan and block hidden CSS/HTML content before passing it to Gemini.
- Create Post-processing Filters: Add filters to catch urgent text, fake phone numbers, or URLs before Gemini summarizes them.
- Educate Users: Remind users not to trust Gemini summaries as final security alerts, especially if they urge quick action.
Google's Proactive Stance
Google hasn't seen this attack in the real world yet, but it’s actively working to prevent it. The team runs “Red Team” exercises to test AI defenses and has started adding stronger security features.
So, while AI tools like Gemini offer great convenience, they should be used with caution. Pair them with strong security systems and critical human judgment to ensure that convenience doesn’t open the door to new cybersecurity risks.
Source: Google Gemini Flaw Hijacks Email Summaries for Phishing – BleepingComputer