Original title: Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain
Pengyin Shan · September 2026
AI coding assistants now pick and install software, but attackers exploit this by using fake package names, hacked accounts, and altered text. To protect users, developers provide machine-checkable trust signals like digital signatures, build proofs, and official software lists. Researchers wanted to know if assistants actually check these safety signals before installing research software. They set up tests using six projects, modifying them to have missing, valid, or fake signals. They then ran assistants across different settings and tracked what the tools actually did inside system logs rather than trusting their words. The assistants almost never checked the signals. In 9 of 1,920 registered trials, assistants opened a provenance signal, and not a single trial ran a verification command. Paying more did not help, as the most capable model verified nothing. Simply publishing trust signals is not enough; tools running the assistants must force these checks.
Imagine you order a package online. The delivery company says it will check the package for safety before handing it to you. But when the package arrives, the delivery person just drops it off without any inspection. In this paper, the AI coding assistant is like the delivery person, and the trust signals are the safety checks that are supposed to happen before installation. The researchers found that the assistant almost never performs those checks, even when the safety information is clearly attached to the package.
Developers rely on AI coding assistants to install software, but these assistants rarely check security signals like signed releases or build provenance. In a controlled study, assistants opened a trust signal in only 0.5% of trials, even when signals were present. This means attackers can still exploit the supply chain through invented packages or compromised accounts, and publishing security signals alone does not protect users.
The study only examined six research software projects (HPC and quantum computing) from a corpus of 87, so results may not generalize to all software domains. It tested three models under two operation modes, but real-world usage may vary. The study measured whether assistants opened or verified signals, not whether they could be trained to do so. The abstract does not specify the models tested or the exact conditions of the frontier model supplement.
Ask anything about this paper. Answers come from its abstract, and say so when the abstract doesn't cover your question.
Do AI Coding Assistants Check Before They Install? A Pre-Registered Demand-Side Audit of Trust Signals in the Research Software Supply Chain — Pengyin Shan
Abstract licence: CC0
Explanation generated by Synthesis from the abstract — it is not the original paper. This is a preprint, so it hasn't necessarily been peer reviewed.