Faraday Research Agent Beats GPT-5.5 and Claude in Replication

A small London lab picks a fight with the frontier

Inherent is a London AI lab founded by former Google DeepMind researchers. It says its new agent, Faraday, beat Anthropic's Claude Opus 4.8 and OpenAI's GPT-5.5 at reproducing the findings of published scientific papers, even though Faraday is built on a much smaller model.2 The agent runs on Qwen 3.6, an open model with 27 billion parameters. Both systems it was compared against are frontier-scale models.2 The company announced Faraday on social media in mid-August and described it as a 27-billion-parameter "AI Scientist" trained with long-horizon reinforcement learning.30 Wider press coverage came about a week later.2

The news also tells us something about Inherent itself, which has drawn less attention than other startups founded by DeepMind alumni. It came out of stealth with a $50 million seed round, and Faraday is its first public result.2 Index Ventures led the round and Radical Ventures also invested.27 Reports disagree on exactly when Inherent left stealth. One account puts it in May 202628, while others describe Faraday as arriving just weeks after the funding news2.

What Faraday actually does

The task sounds narrow. Faraday has to reproduce a result from a published paper without being shown the answer first.2 Co-founder and chief scientist Edward Hughes compares it to what new scientists do, noting that many PhD students start out by replicating existing work.2

Inherent built its own training and test set for this, called Replica. Each task asks an agent to recreate a figure from a machine learning or AI-for-science paper within a fixed time and compute budget, without seeing the original plot.30 The first version has 310 tasks drawn from 100 papers in areas including natural language processing, materials science and weather forecasting.23 Because there is no simple answer key for a good replication, an AI model acts as grader. It scores each attempt against a rubric generated automatically for that task, and Inherent says it checked this grader against human judgments.2322

The most interesting choice is how the system is put together. Faraday does not write its own code. It hands coding work to OpenAI's GPT-5.5 Codex, much as a human scientist uses existing software instead of writing everything from scratch.2 Faraday was trained with the smaller GPT-5.4-mini as its coding tool, then adapted to directing the more capable Codex when tested.23 Inherent says Faraday is directing a model several orders of magnitude larger than itself.30 In practice, the small model acts as the lead researcher and the big model as the lab technician.

The numbers, and how far they go

Coverage of the actual scores is uneven. One summary of the accompanying arXiv paper says Faraday beat Claude Opus 4.8 on 73% of tasks drawn from the same kinds of papers it trained on, and stayed ahead on held-out AI-for-science papers.21 Inherent says it led in every category, with its biggest margins in meta-learning, structural biology and materials science. It also says Faraday had less trouble with recent research the base model could not have seen during training.23 The company also built 20 altered versions of papers containing results that do not appear in the originals, and says Faraday beat GPT-5.5 on these too. Inherent describes this as the agent innovating without realising it.30

At least one write-up said Inherent had released no scores, test papers or methodology.26 That seems to reflect the brief interview-based reporting rather than the full record, since the arXiv paper is reported to run 47 pages and to describe the benchmark design, the grading method and the training approach.21 Even so, an early analysis of the paper's abstract noted that it gives no per-task accuracy figures, no details of how the Claude and GPT baselines were set up, and no number for how often the AI grader agreed with humans.24 Other reporting fills in part of that gap: the rival models were run in Anthropic's Claude Code and OpenAI's Codex tools with the highest reasoning setting.23

On the main caveat, every outlet agrees. Replica is Inherent's own benchmark, and nobody else has published a rerun of the Claude or GPT comparisons.2127 One newsletter asked the obvious question: how many of the 100 source papers might already be in Qwen's pretraining data?21 A fair reading is that the result is believable and partly documented, but it is not settled until someone outside Inherent tests it on Replica or a public benchmark.

Why the Anthropic comparison matters

The Anthropic angle is mainly about timing. Claude Opus 4.8 was the model Inherent chose to beat in August. Since then, Anthropic has released Claude Opus 5.5 on September 22 and Claude Sonnet 5.5 on September 28.15 Any claim that a small agent beats a frontier model is tied to a particular moment, and frontier models now change every few weeks. Faraday's win over Opus 4.8 does not show it would beat Opus 5.5, and Inherent has not claimed that.

The bigger point is that Faraday and Anthropic are working toward the same goal from opposite ends. In September, Anthropic said Claude "leads" about 26% of its internal AI research and development as of August, up from under 1% in February. It also said Claude works at least at a "collaborates" level on more than 90% of that work.14 The company says about 30,000 agents run research and engineering tasks at any one time on its main internal platform. An automated monitor blocked about one in 47,000 of more than a billion agent decisions in August.18 Anthropic says none of the work it measured was fully autonomous.19

Those figures have problems of their own. Anthropic's index is a self-audit in which Claude rebuilt and rated staff work. A Claude judge matched human raters exactly 59% of the time, while human raters matched each other exactly only 35% of the time.20 Anthropic has acknowledged that using its own models to grade its own systems risks shared errors, and has proposed third-party checks.19 So both stories share the same weakness: they are claims about AI doing research, measured by the people who built the systems.

The two approaches still differ in an important way. Anthropic is running huge numbers of general-purpose frontier agents inside one company. Inherent argues that a smaller, specialised "research director" trained to have scientific judgment can get more out of frontier models than those models get out of themselves. Commentators have called this a practical blueprint for layered agent systems.22 If it holds up, it suggests that some of the value in AI-for-science could come from a training layer that sits on top of the frontier models, not from the models alone.

"Research taste" as the real product

Hughes says beating the frontier models mattered less to him than how the team got there.2 Inherent wanted Faraday to show "research taste", meaning a sense of which experiments are worth running and how to design them, and used reinforcement learning to train it.2 Analyses of the paper say Faraday builds the actual mechanisms behind a paper's claims instead of hard-coding outputs, keeps experiments faithful to the paper's scope, and avoids shortcuts that would flatter its results.22 That last point matters. Gaming the grader is a known way for AI-graded tasks to fail, and Inherent's own analysis of individual runs suggests Faraday did not do it.29

This is a new direction for automated research. Earlier evaluations were not encouraging. On the ResearchGym benchmark, an agent built on GPT-5 beat the provided baselines in only one of 15 runs and completed only about a quarter of sub-tasks. Its failures included impatience and too much confidence in weak ideas.9 Faraday tackles exactly those weaknesses in judgment. Other lab work points the same way: DeepMind's AI co-mathematician beat GPT-5.5 Pro's previous record on FrontierMath Tier 4 by coordinating a team of specialised agents, not by using a new base model.6

The verdict

The most likely reading is that Faraday is real progress with an asterisk. The asterisk does not mean the claim is false. It means it has only been checked by Inherent. The lasting idea is not that a 27-billion-parameter model is "smarter" than GPT-5.5 or Claude. It is that a specialist trained to make good research decisions can make better use of frontier tools. Replication is also only a stepping stone. Inherent's stated goal is discovering new science, not checking old results.26

The next steps are clear. Someone outside Inherent needs to evaluate Faraday, either on Replica or on a public benchmark such as MLE-Bench.21 The comparison should also be rerun against Anthropic's and OpenAI's newer models. Meanwhile, Inherent has about a dozen staff in King's Cross and plans to grow to 20 to 25 by the end of the year.2 It may also attract DeepMind researchers unsettled by recent changes there.2