Do You Blindly Trust AI? What Watching AI Install Things Taught Me About Supply Chain Risk

Why This Matters

I watch a lot of tutorials. YouTube, courses, documentation, all of it. One thing I keep noticing becoming the norm: someone, or the AI itself, finds a package that does what they need, and installs it. No hesitation. The assumption is that if it’s popular, if it has thousands of downloads, if lots of people are using it, it must be safe. Right?

I give most people the benefit of the doubt. I don’t distrust someone until they give me a reason to. AI is not a person, and it’s a different kind of concern. The more I work with AI tools, the more that concern has grown.

Here’s the problem. A package can have millions of legitimate historical downloads and still turn malicious after a maintainer account, a publishing credential, a build pipeline, or a dependency gets compromised. Popularity does not equal current safety. When AI installs packages for you, you’ve handed off a security decision without necessarily noticing what you actually did.

What I Watched AI Do

I was following a tutorial on Remotion. Part of the workflow needed to read a PDF. Claude can read PDFs natively, no extra tooling required. Instead, it wanted to run a Brew install for poppler. Why? It didn’t need it. But AI defaults to the easy route and to pleasing you. It doesn’t stop to check whether something is necessary or safe. It finds a tool that does the job and reaches for it, instead of trying what’s already built into the harness or the OS.

Being called a “thinking model” doesn’t make it one. It’s predicting tokens, not weighing consequences. It can check its output against what you asked for, but that’s verification, not judgment about whether an install is warranted, or whether an app pulled from a GitHub repo was compromised after the fact.

Then it got worse. The same tutorial had me install WhisperCpp, which I approved since I was evaluating it for this piece. The next step was to transcribe a demo video. Right after installing WhisperCpp, Claude wanted to install ffmpeg too. I asked why. That should have been the end of it.

It tried to install ffmpeg again. I had already told it Remotion was installed and ships with ffmpeg bundled in. It still needed to be walked back a second time before it found the version already sitting on my system. Two unnecessary install attempts for a dependency that was already there, on a task that never needed a new install to begin with.

That tutorial also opened by telling viewers to unblock Claude’s permissions so they wouldn’t have to approve anything. If I hadn’t been sitting there approving each step by hand, both of those installs would have gone through without a second thought. And if it tried installing something I didn’t catch this time, on a task I know less about, I wouldn’t have known to push back at all.

Where This Stops Being Theoretical

None of this requires a malicious tutorial or a careless AI to turn into a real incident. It’s already happened to people who were doing everything right.

tj-actions/changed-files: 23,000 repos, trust turned invalid without warning

The tj-actions/changed-files GitHub Action was used by more than 23,000 repositories. It was widely trusted and widely audited. In 2025, an attacker compromised the supply chain and inserted malicious code, then repointed existing Git tags so that trusted version references pointed at the malicious commit instead. Organizations referencing those tags ran different code without changing a single line of their own configuration. The malicious code exposed credentials and secrets from CI/CD environments.[1][2]

The scary part is that the people using it didn’t have to install some new, obviously malicious version. They had already decided they trusted tj-actions/changed-files and were using it in their GitHub workflows. It had worked fine before.

The attackers were able to change the existing version tags so they pointed to the malicious code. A workflow that was still configured to use the same version tag could now run different code the next time it executed. Nothing had to change in the victim’s own configuration.

GitHub recommends referencing a specific commit instead of relying on a version tag for this reason. A version tag can be changed to point somewhere else. A specific commit identifies the exact code you intended to run.[3] GitHub later added controls organizations can use to enforce this practice.[4]

Codecov: the download was legitimate, the payload wasn’t

Developers commonly downloaded and immediately ran Codecov’s Bash Uploader straight from Codecov’s own infrastructure. An attacker obtained credentials that let them modify the uploader where it lived in Google Cloud Storage. Users hitting the legitimate URL got a different script than the one that had been reviewed. The malicious version extracted environment variables and repository data out of CI environments.[5]

It was caught because one customer checked the SHA256 hash of the downloaded script against the version on GitHub and noticed they didn’t match.[5][6] The person who didn’t blindly trust the download is the reason the attack got exposed at all.

Shai-Hulud: millions of downloads didn’t protect anyone

The Shai-Hulud campaign compromised npm packages and stole developer credentials, then used those stolen tokens to compromise more packages maintained by the same developers, spreading through relationships that were already trusted. One affected library, @ctrl/tinycolor, was pulling millions of downloads a week.[7][8] Download count measures adoption. It says nothing about current security.

Red Hat npm compromise: valid provenance, compromised pipeline

In 2026, attackers compromised packages under Red Hat’s @redhat-cloud-services npm namespace by compromising an employee’s GitHub account and pushing malicious commits that bypassed normal code review. The packages still came out of the legitimate build pipeline.[9][10]

The part that matters most here is that the infected packages still passed checks showing they came from the expected build process. They even had valid SLSA provenance, which is basically a record showing where and how the software was built.[10][11] The problem was that the build process itself had been compromised.

So even though the package could prove it came from the right place, the right place was no longer safe.

Trust Is Not One Thing

Most people collapse several different questions into the single word “trusted.” They’re not the same question.

Reputation: this repo has 40,000 GitHub stars.
Popularity: this package has 10 million downloads.
Identity: I know who maintains this project.
Historical security: someone audited this six months ago.
Provenance: this artifact came from the expected build system.
Integrity: this is exactly the artifact I intended to run.
Current security: this specific artifact, its dependencies, its install scripts, and everything it reaches out to, is safe right now.

The first five don’t establish the last two. Not automatically, and not ever on their own.

This reminds me of the old days when email viruses spread like crazy. I trust this link, it came from my friend.

My Rule, and Why I Keep It

I have a strict rule: I approve every installation myself. No brew, no npm, no pip, no npx, nothing runs unless I explicitly say yes. When something asks, I investigate the command first. I look at download counts, but also when the package was created, when it was last updated, and who wrote it. Was it a company with something to lose, or some random developer’s weekend project that might not know how to secure a build pipeline? Not everyone who ships insecure code means to. Plenty of people just don’t know better yet.

I even built an ICM workflow where I can paste a command and it breaks it down for me and checks for any known or reported issues. That won’t always catch something, the examples above prove that, but it’s still a check I make.

Most people don’t do any of this. They see:

This requires package X.
Install package X? [Y/n]

And the whole decision collapses to: do I trust the AI’s recommendation?

The better questions to ask look like this:

Why is this dependency necessary? Who publishes it? What runs during installation? Does it have install or postinstall scripts? What dependencies does it pull in? Does it reach out for more executable content once it’s running? Are those resources pinned to something immutable? Are hashes or signatures verified? What privileges does it get? What credentials or files can it touch? Has this specific version actually been reviewed by anyone?

AI coding agents add a layer most people haven’t priced in. A human developer doing the install themselves might do some baseline evaluation. When AI proposes the package, it has effectively run the entire software-selection process on your behalf, and your decision shrinks down to approving what it already decided. The Codecov and tj-actions incidents are concrete evidence for why “I checked this once before” doesn’t cover it anymore.

The Bigger Pattern

This is turning into something like script-kiddie culture, except now it applies to everyone, not just people trying to hack something. Nobody thinks they need to know how anything works anymore. Download a template, click a button, done. It used to be that you learned by doing and earned the ability to think through a problem instead of just executing whatever you were told. You earned credibility with your experience and knowledge. That muscle wastes away fast when the AI is willing to do all of it without being asked twice.

And it doesn’t take a malicious maintainer to trigger this. A project can be compromised without its owner ever doing anything wrong, and the tj-actions incident is proof: more than 23,000 repositories were affected through a dependency that had nothing to do with who originally built it.[1] People who are genuinely well versed in security and run disciplined CI/CD pipelines still fell victim to code they had every reason to trust, because that code changed after the fact.

What You Can Actually Do

Stop treating an AI’s install recommendation as a security approval. Treat it as a suggestion that still needs evaluating.

For npm packages: check the install and postinstall scripts. Confirm what’s declared in package.json matches what actually runs. Check when the package last updated and who can publish to it. Run something like npm audit against the current version.

For Homebrew: check the formula for anything that runs on install, and look at where the source is actually coming from.

For pip: check PyPI for the last update date, look at the source repo, and check for a setup.py that could run arbitrary code during install.

For GitHub Actions: pin to the full commit SHA. Not a tag, not a branch. That’s the only way to be confident you’re running what you reviewed.

And when a tutorial tells you to bypass your AI tool’s permission prompts so you stop having to approve things: that instruction is the whole problem in one sentence. It removes the one moment where a human actually looks at what’s about to run.

One simple extra step: use a different AI, paste the command into it, and ask whether it’s safe and why it’s needed. Have it validate the first answer instead of trusting it outright.

Maybe once I feel my ICM workflow is reliable enough, I’ll post it on GitHub. If I do, check the hash and research it yourself before you trust mine either. LOL.

The Bottom Line

Popularity is not a security audit. Historical safety is not current safety. And convenience is very good at hiding the security decisions it’s making on your behalf. When AI suggests installing something, treat that as the start of an evaluation, not the end of one.

You don’t have to approve everything an AI recommends. That decision is still yours. The Remotion tutorial I was watching only needed WhisperCpp. It tried to hand me a second tool I never asked for, and the task never called for, twice, and would have gotten away with it if I hadn’t been watching. It just decided that was required to get the work done.

Sources

[1] Unit42 Palo Alto Networks. “GitHub Actions Supply Chain Attack: A Targeted Attack on Coinbase Expanded to the Widespread tj-actions/changed-files Incident.” https://unit42.paloaltonetworks.com/github-actions-supply-chain-attack/

[2] NVD. “CVE-2025-30066 Detail.” https://nvd.nist.gov/vuln/detail/CVE-2025-30066

[3] GitHub. “Secure Use Reference - GitHub Actions.” https://docs.github.com/en/actions/reference/security/secure-use

[4] GitHub Changelog. “GitHub Actions Policy Now Supports Blocking and SHA Pinning Actions.” https://github.blog/changelog/2025-08-15-github-actions-policy-now-supports-blocking-and-sha-pinning-actions/

[5] Codecov. “Post-Mortem and Root Cause Analysis (April 2021).” https://about.codecov.io/apr-2021-post-mortem/

[6] CISA / GovDelivery. “Codecov Releases New Detections for Supply Chain Compromise.” https://content.govdelivery.com/accounts/USDHSCISA/bulletins/2d712c1

[7] Semgrep. “Security Advisory: NPM Packages Using Secret Scanning Tools to Steal Credentials.” https://semgrep.dev/blog/2025/security-advisory-npm-packages-using-secret-scanning-tools-to-steal-credentials/

[8] Snyk. “Zero-day Extensive NPM Package Compromise - Shai Hulud Supply Chain Attack.” https://snyk.io/blog/embedded-malicious-code-in-tinycolor-and-ngx-bootstrap-releases-on-npm/

[9] Red Hat. “RHSB-2026-006 Supply Chain Compromise of @redhat-cloud-services npm Packages.” https://access.redhat.com/security/vulnerabilities/RHSB-2026-006

[10] Boostsecurity. “Trusted Publishing, Untrusted Branch: Inside the Red Hat npm Compromise.” https://labs.boostsecurity.io/articles/trusted-publishing-untrusted-branch-red-hat-npm/

[11] SLSA. “Mini Shai-Hulud: Where SLSA’s Boundaries Fall.” https://slsa.dev/blog/2026/05/mini-shai-hulud-what-slsa-can-and-cannot-do

[12] OpenSSF. “Concise Guide for Evaluating Open Source Software.” https://best.openssf.org/Concise-Guide-for-Evaluating-Open-Source-Software.html

[13] OpenSSF. “Package Analysis.” https://openssf.org/package-analysis/

Until next time, stay curious and build smarter, my friends.