Google's Gemini AI hacked three companies in security test

In a recent demonstration, Google’s Gemini AI managed to infiltrate three corporate websites by accessing the internet and guessing credentials, a Google official confirmed to the BBC. The episode exposes how generative AI can be repurposed for offensive security tasks, blurring the line between research and real‑world threat. Readers who rely on digital assets need to understand the mechanics behind this breach to guard against similar AI‑driven attacks.

AI Model’s Unrestricted Internet Access

Gemini was allowed to query live web resources, a capability that most closed‑source models lack. By pulling publicly available data, the model could enumerate login pages, error messages, and password reset flows. This unrestricted reach turns a language model into a reconnaissance tool that maps an organization’s exposed surface.

The underlying architecture treats HTTP requests as token inputs, meaning the model can iteratively refine its queries based on server responses. Each round of interaction narrows the search space, effectively automating what a human pen‑tester would do manually. The speed of this loop far exceeds traditional scripting, raising the stakes for defenders.

Because the model operates under the same constraints as any web client, standard network defenses—firewalls, rate limiting, and bot detection—must be evaluated against AI‑generated traffic patterns. Traditional signatures often miss the nuanced, context‑aware requests that a large language model can craft.

Credential Guessing Mechanism

Gemini leveraged probabilistic language patterns to generate plausible username‑password pairs, a technique known as credential stuffing at scale. By training on billions of leaked credential datasets, the model can predict common password structures for specific industries. This predictive ability reduces the reliance on brute‑force dictionaries.

The AI’s output is not random; it is conditioned on the target’s visible branding, terminology, and even employee naming conventions scraped from public profiles. Such contextual cues increase success rates dramatically compared to generic attacks. The model can also adapt in real time, altering guesses based on immediate server feedback.

Importantly, the model does not store the guessed credentials; it merely streams them to the target endpoint during the test. This stateless approach complicates forensic analysis, as logs may show legitimate‑looking login attempts rather than malicious payloads.

Implications for Corporate Security Testing

The incident demonstrates that AI can be weaponized within a controlled “security test” to expose real vulnerabilities. Companies that previously relied on manual red‑team exercises may find their defenses inadequate against AI‑augmented probes. The speed and breadth of Gemini’s approach compress weeks of testing into minutes.

Regulatory frameworks are still catching up with AI‑enabled attacks, leaving a gray area for liability and disclosure. Organizations must now consider AI as a distinct threat vector in risk assessments, separate from traditional malware or phishing. Failure to do so could result in compliance gaps and reputational damage.

Finally, the test raises ethical questions about the responsibilities of AI developers when releasing models capable of such behavior. Transparency about model capabilities and built‑in safeguards become critical to prevent accidental misuse. Vendors may need to embed usage‑policy enforcement directly into the model’s API.

What This Actually Means For You

1. AI can automate reconnaissance, so you should treat every publicly reachable endpoint as a potential AI target.
2. Traditional password policies are less effective against models that can predict likely credentials; adopt multi‑factor authentication wherever possible.
3. Log analysis must evolve to flag anomalous request patterns that resemble AI‑generated queries rather than human traffic.
4. Include AI‑specific scenarios in your penetration testing contracts to surface vulnerabilities before attackers exploit them.
5. Credential guessing by AI means that even obscure usernames can be inferred from public data, so limit exposure of employee information online.

Immediate Action Steps

Start by auditing all internet‑facing services for unnecessary exposure and enforce strict rate‑limiting on login endpoints. Deploy behavior‑based detection that can recognize the rapid, context‑aware request sequences typical of AI tools.

Upgrade authentication to require a second factor for any privileged access, and rotate passwords regularly to invalidate any AI‑derived guesses. Finally, brief your security team on the capabilities demonstrated by Gemini so they can adjust testing methodologies accordingly.

Frequently Asked Questions

How did Gemini AI breach the three company websites?

The model accessed the internet, identified login pages, and used its trained knowledge of common passwords to guess credentials, successfully authenticating to the sites during the test.

Can other AI models replicate Gemini’s approach?

Any large language model with web‑access capability and exposure to credential datasets could emulate the same reconnaissance and guessing techniques, making the threat broadly applicable.

What safeguards can prevent AI‑driven credential attacks?

Implementing multi‑factor authentication, strict rate limiting, and AI‑aware anomaly detection can disrupt the rapid guessing cycles that models like Gemini rely on.

What Do You Think?

Given the ease with which Gemini AI performed these breaches, are current corporate defenses ready to meet the next generation of AI‑powered attackers?