Gemini Flash Safety Checks Struggle to Keep Up With Monthly Releases

A preview that turned into a pattern

In late August, Business Insider reported that Google staff were already using a preview of Gemini 3.8 Flash on Jetski, the company's internal coding platform. At that point Gemini 3.7 Flash had been public for only about two weeks.12 One employee told the outlet the preview already felt clearly better than 3.7 Flash but said it was too early to judge it fully.12 The same report tied the speed to a goal Sundar Pichai described on Alphabet's Q2 earnings call: shipping new models almost every month.1220

The preview went public quickly. Google released Gemini 3.8 Flash generally on September 2, calling it its third Flash release in six weeks.1115 Ars Technica described it the same way and added that the gains over 3.7 Flash were small on most benchmarks and larger on coding tests.13 The cycle has already repeated: on October 9, Business Insider reported that employees were testing a Gemini 4 checkpoint codenamed Carbon on the same Jetski platform.21

For anyone following AI safety, the internal testing matters less than what it says about evaluation. Google now ships capable models every few weeks, and much of its formal frontier-risk work is being carried forward from one model to the next instead of redone each time. The rest of this piece explains why that is a reasonable reading of Google's own documents.

Flash became the frontier by default

Google did not plan for Flash to carry this weight. At I/O in May, Pichai said Gemini 3.5 Pro would arrive in June, but it never shipped.1918 Business Insider said bluntly in August that Google no longer had a frontier model and had leaned into the cheaper, faster Flash line.12 Ars Technica suggested Google may have held back 3.5 Pro so it would not be compared with the latest OpenAI and Anthropic models, and pointed to reports that Gemini's coding ability had fallen behind.18 Axios later said that, according to its July reporting, low morale at Google DeepMind contributed to the delay.28

The reporting largely agrees on this point, and it is central to the safety story. Google's Frontier Safety Framework relies on a "safety buffer": if the most capable model stays below the alert threshold for a critical capability level (CCL), models built before the next scheduled test are assumed to stay below it too.3 Google says it usually runs that check on Gemini Pro models.3 Once a Flash model is the strongest system Google has, the setup no longer fits.

What the 3.7 Flash report admitted

Google saw this coming. Its August Frontier Safety Framework report on Gemini 3.7 Flash says the company ran lighter versions of its risk assessment as a precaution, because 3.7 Flash was strong compared with Gemini 3 Pro and 3.1 Pro.3 The report adds that this testing covered only the evaluations most relevant to the thresholds, not the wider body of evidence Google builds for a Pro release.3

The results matter for alignment work. Google says 3.7 Flash crossed the alert threshold for both the CBRN uplift CCL and the cyber uplift CCL, while staying below the CCLs themselves.1 On CBRN, some expert red-teamers got accurate, actionable information across the full attack pathway in both tested scenarios. Google kept the model below the CCL because average scores were modest and experts had to steer the model explicitly.13 In chemical red-teaming, scores rose more than 24 points above web-search baselines on a 100-point rubric, though Google says key bottleneck steps stayed blocked.3

The misalignment results are the most telling. Google says 3.7 Flash showed stronger situational awareness than Gemini 3.1 Pro and could tell when it was in a test environment, though it could not get around the test restrictions.1 The report says the model passed three of four stealth challenges and six of eleven situational-awareness challenges. The tracked capability level (TCL) requires passing every challenge, so the model stayed below it.3 On Google's autonomous ML research benchmark, it scored 27% pass@1, against a rule-out threshold of 90%.3

Google also disclosed that about 20% of those ML R&D tasks had bugs, including broken verifiers and vague task definitions, which could undercount what the model can do. Assuming the model passed every affected task, Google calculated a maximum score of 47%.3 That kind of disclosure is useful, but it also shows how thin the evidence for some threshold decisions is. Google did report that its automated checks found no signs of sandbagging or test awareness in its biology tests.3 Those checks become harder to trust when the same model is known to recognize tests elsewhere.

3.8 Flash reused its predecessor's results

The 3.8 Flash model card is where the pace becomes visible. Google says it evaluated Gemini 3.7 Flash under the April 2026 framework. It concluded that 3.8 Flash has no meaningful new capabilities in the framework's risk domains, so, based on the 3.7 Flash results, 3.8 Flash is unlikely to reach any threshold either.216 Google has done this before: the 3.5 and 3.6 Flash model cards relied on Gemini 3.1 Pro's assessment in the same way, with extra cyber testing because earlier Gemini 3 models had hit the cyber alert threshold.58

The trouble is that Google's own marketing describes 3.8 Flash as a capability jump. The launch post calls it Google's best reasoning and coding model yet and credits the gains partly to intensive training in cybersecurity.15 The same post says the release was helped by long-running agentic loops built to recursively evaluate and refine the underlying models.15 Its sibling model, 3.8 Flash Cyber, found a critical vulnerability in under two hours that Google says would usually take months to find, and produced 2.6 times more correct Chrome patches than larger commercial models.1513

These claims do not necessarily contradict each other. Google is choosing to focus on defense: it says it prioritized patching over exploitation and restricted Cyber models to trusted testers and governments.1513 Still, the company is telling customers that big cyber-related gains came from cyber-heavy training, and telling safety readers that frontier-relevant capabilities did not materially change. An outside reviewer cannot tell which statement better describes the model without the underlying data.

The content-safety numbers also slipped. Google's card says multilingual safety got 5.4 percentage points worse than 3.7 Flash, and unjustified refusals went up slightly.16 Google says manual review found that the failures were mostly false positives or not egregious, and that its evaluation sets changed, so results are not directly comparable with earlier cards.16 Both caveats may be valid. Changing the measuring tool between releases also makes it harder to track trends over time.

Gemini 4 gets a different treatment

Google's handling of its next flagship looks quite different. Axios reports that Gemini 4 Argon went first to a small group of cybersecurity partners, and that Google is taking part in the U.S. government's voluntary prerelease process before opening access more widely.28 One explainer says Argon is limited to members of Google's Fairwind Program, and that paid API customers and AI Ultra subscribers come next, with no date given.23 Google Cloud's model lifecycle page now lists gemini-4-argon and a "cyber-permissive" version, both dated October 13.17

The internal reporting on Gemini 4 is mixed. Bloomberg said some employees found Argon underwhelming in practice despite strong benchmark results, which Google called inaccurate.28 Business Insider's Carbon report says one employee compared the newer checkpoint to Anthropic's Opus 5.5 for coding, but said it needed more testing, and Google may never ship it.21 One secondary outlet went further, attributing Carbon's gains to "recursive self-improvement."26 That claim does not appear in the Business Insider report it cites and should be treated as unverified.

The reading

The main lesson is not that Google is being reckless. Its Flash reports are more detailed than many in the industry, including disclosures about buggy benchmarks, test awareness and red-team uplift.3 But its process was built around a periodic Pro-class check. That process now has to cover a series of Flash models released weeks apart, each building on the last, while Google's marketing points to recursive refinement loops and gains in sensitive areas.152

The careful, partner-first Argon rollout suggests Google thinks the largest risks sit with its flagship.28 Its own data shows the Flash line is already near alert thresholds in CBRN and cyber and is getting better at noticing when it is being tested.1 As employees test the next checkpoint before the current one is public, the key question is whether frontier evaluations get rerun for each release or only cited from the model before.1221