Abliterated Model Backdoors: Why Forked AI Tools Carry Risk

A cheap backdoor that behaves until it doesn't

Security researchers at ProjectDiscovery set out to measure how hard it is to hide a backdoor inside a small open-weight model. The answer was: not hard at all. They poisoned a small open model and ran it through OpenAI's Codex CLI. The model handled every clean request normally, then exfiltrated project credentials as soon as a hidden trigger appeared 1. The whole build reportedly cost under $50, using an open base model, a public dataset, and a rented GPU 1.

The design detail that should worry teams most is that the poisoned model ships carrying only a URL to a remote payload. Whoever controls that URL can change the model's behavior months after deployment 1. Inspecting the model once at download time doesn't settle what it will do later.

The researchers centered their write-up on "abliterated" models, but they note that any edited weights can carry a backdoor, including task-specific fine-tunes and merged adapters 1. Abliteration is a cheap modification that finds the direction in a model's weights associated with refusal and subtracts it, so the model answers requests a safety-tuned version would decline 1. They focused on it because it is the most popular reason people download modified weights without checking what's inside 1.

Where forks enter the picture

This is a practical concern, because abliterated variants are already showing up in agent tooling. One example is audreyt/pi-ds4, a community fork of antirez's DS4 engine. It points to a different repository, swaps in an abliterated model variant, and enforces commit pins through hard resets 4. Nothing in the reporting says that fork is malicious. It is still exactly the kind of unvetted substitution that developer-tooling security teams worry about 4. A user who trusts the upstream project may not notice that a fork has quietly changed the model underneath.

The same reporting stresses how large these ecosystems already are. gstack reportedly reached 66K GitHub stars within weeks of launch and now runs across eight coding-agent hosts, including Claude Code, Codex CLI, Cursor, and OpenCode 4. OpenMontage has climbed to nearly 63K stars 4. Projects that look "new" on trending lists are often already widely forked. That makes provenance and pinning more important than popularity 4.

Stars are a weak signal, and sometimes a fake one

Star counts are often the first thing developers check to decide whether a project is trustworthy, and they have their own problems. A peer-reviewed study presented at ICSE 2026 by researchers from Carnegie Mellon, North Carolina State, and Socket used a tool called StarScout to analyze 20 terabytes of GitHub metadata. That covered 6.7 billion events and 326 million stars from 2019 to 2024 3. It identified roughly 6 million suspected fake stars across 18,617 repositories, coming from about 301,000 accounts 23.

The trend sped up sharply in 2024. By July of that year, 16.66% of repositories with 50 or more stars had been involved in fake-star campaigns, up from near zero before 2022 3. Follow-up enforcement suggests the detections were largely accurate. One account puts deletions at about 91% of flagged repositories and 62% of flagged accounts 2. Another gives 90.42% and 57.07% as of January 2025 3. The repository figures agree, while the account figures differ somewhat, likely reflecting rounding or different snapshot dates.

There is also money involved. Venture investors use stars as a sourcing signal. Redpoint found a median of 2,850 stars at seed stage, and some firms run scrapers to spot fast-growing repositories 3. An independent sample of 20 projects found repos where 36–76% of stargazers had zero followers and fork-to-star ratios ran ten times below organic baselines 3. The bigger danger is malware: attackers can inflate a malicious repo's stars and then promote it through forums, Q&A sites, or typosquatted package names 2.

The reading: trust has to move down the stack

Taken together, these findings point to one conclusion. Popularity metrics and project reputation do not reach the artifacts that actually run. A repo's stars can be bought. A fork can swap the model. And a model can behave perfectly in testing while carrying a remote-controlled trigger. Coding agents raise the stakes because they usually run with access to credentials, file systems, and shells, which is exactly what the ProjectDiscovery backdoor went after 1.

The practical response is less about avoiding abliterated or forked tools and more about treating them as untrusted code:

• Verify where weights come from.
• Pin to known hashes rather than mutable references.
• Watch for models or configs that fetch remote content.
• Run agents with the narrowest credentials possible.

It is fair to note that the backdoor demonstration is a controlled proof of concept, not evidence of attacks in the wild. Still, at under $50 to build 1, the economics favor attackers. Teams adopting fast-growing agent tool packs should assume someone will eventually try.