Prompt injection, secret leaks & supply-chain risk

> qwen to do a large scale attack on random nerds running llms on their gaming gpu they would not do it on random nerds and I'm struggling to imagine what it could be used for practically, but generally speaking putting backdoors in tech and infra is not unheard of.

Muromec · Oct 13, 2026 · comment in “AI Models Can Provably Hide Arbitrary Capabilities”

The Jev-competitors only proved that you can get or beat the accuracy, nothing about the other properties. Especially the “zero hallucination” output and the (if it works how the documentation make it sound) prompt injection resistant architecture.

dkersten · Oct 9, 2026 · comment in “Typesafe AI raises $870M at $7.5B”

So as long as that's the case, the only moat left for you is customer access, customer loyalty, and pricing (though I think that last one is arguable) The challenge as I see it is that for new products in this space to be successful, the solo dev either needs to BE or be partner with someone who has deep intimate domain knowledge and large network for the pain point their product solves (e.g. a 15 year supply chain …

bmoathn · Oct 4, 2026 · comment in “Ask HN: Where can a solo dev still find a moat in LLM inference?”

So Claude is a supply chain risk in the same way that an employee is a supply chain risk. I'm struggling here because you're basing this determination entirely on things which the government never acused anthropic of, and haven't asked anthropic to address, and which the underlying law probably doesn't support.

joshuamorton · Sep 26, 2026 · comment in “U.S. appeals court upholds designation of Anthropic as supply chain risk”

Sorry, I never linked it! This is from the latest Anthropic safety report (of "Anthropic Houtis build missile" fame), and no, these cannot be hallucinated -- the leaked secrets were inputs, not ouputs. https://www.anthropic.com/threat-intelligence-report-september-2026

bbor · Sep 17, 2026 · comment in “How GLM built its own inference infrastructure”

I hate it, but my guess is if they don't play ball here, they risk losing government contracts or worse, being blacklisted as a "supply chain threat" as they tried to do with Anthropic.

m4ck_ · Aug 31, 2026 · comment in “Google Maps Showing 'Lake America' in U.S.”

I managed to get the sandbox working, and the option to break out of sandbox (with approval). The network sandboxing doesn't work yet (I don't want malicious script to be execute and exfiltrate the data), I created an issue here: https://github.com/erichll/pi-packages/issues/1 - might submit a PR once I get a reply.

ptgamr · Aug 17, 2026 · comment in “Pi's Minimalism Is Its Advantage”

Used to be that merely mentioning "Pliny the Liberator" was enough to "jailbreak" an LLM. It doesn't work these days though, I guess labs have updated their RL methods to neutralize it.

krackers · Jun 22, 2026 · comment in “Prompt Injection as Role Confusion”

Wow! I already knew from previous research shared here that hallucinations are a fundamental problem for LLMs and likely to be unfixable, just like prompt injection, but I didn't realize the hallucination rates were so bad!

solid_fuel · Jun 19, 2026 · comment in “GPT-5.5 hallucinates 3x more than MIT-licensed GLM-5.2”

I find the article extremely annoying for not having a clear timeline of when these malicious package versions were available.

AgentME · Jun 2, 2026 · comment in “Malicious npm packages detected across Red Hat Cloud Services”

Of course, crates.io has surely had some malicious packages. (I'd assume it isn't all that unlikely there could be some undiscovered right now; it's definitely large enough for something like that to slip under the radar, even if it is relatively small compared to say, NPM.) But, I think it really hasn't had its XZ backdoor moment, its left-pad, where you really get to see how well it does or doesn't handle a seriou…

jchw · May 11, 2026 · comment in “Incident Report: CVE-2024-YIKES”

I've been wanting to use something like OpenClaw for a while but couldn't get myself to give it access to anything important due to all the risks involved. Prompt injection is still a problem (even though some people seem to ignore it) and so are hallucinations and mishaps that cause agents to do things like delete production data [1].

yakkomajuri · Apr 29, 2026 · story in “Show HN: AgentPort – Open-source Security Gateway For Agents”

CI catches it after the fact. Given recent supply chain attacks, this feels like table stakes, and I'm really fucking annoyed at Github for trying to shove Copilot down my throat instead of helping me with basics like this.

CountVonGuetzli · Apr 29, 2026 · story in “Ask HN: How are you stopping supply chain attacks via compromised dev keys?”

I've been wanting to use something like OpenClaw for a while but couldn't get myself to give it access to anything important due to all the risks involved. Prompt injection is still a problem (even though some people seem to ignore it) and so are hallucinations and mishaps that cause agents to do things like delete production data [1].

yakkomajuri · Apr 28, 2026 · story in “Show HN: Integrations gateway for agents with 2FA for destructive ops (OSS)”

Is it about to exfiltrate PII? Is it stuck in a loop calling the same tool 500 times?

aegisproxy · Apr 20, 2026 · story in “Is anyone else bothered that AI agents can basically do what they want?”

The biggest problem with crev is it is (last I checked) entirely centralized on git/github making it not all that useful for early supply chain dependencies that often just live on random tar files on servers, or svn, or cvs, or mercurial.

lrvick · Apr 9, 2026 · comment in “Open source security at Astral”

For fine grained permission control, you can use Deno Workers. - I went with absolute minimal dependencies, I am so fed up with Dependabot alerts and npm supply chain hacks.

earthlingdavey · Apr 1, 2026 · comment in “EmDash – A spiritual successor to WordPress that solves plugin security”

Author here. I've been working on action-level governance for AI agents, the main problem is that safety frameworks today operate on intent and content, but supply chain attacks make the action itself indistinguishable from normal behavior. npm install axios is the same command whether the package is clean or compromised.

digitalegoai · Apr 1, 2026 · comment in “The Axios NPM compromise and the missing trust layer for AI coding agents”

Is this the first time you have ever thought about the idea of supply chain attacks? This is the first thought 90% of people have and it doesn't work.

dj_mc_merlin · Mar 31, 2026 · comment in “Axios compromised on NPM – Malicious versions drop remote access trojan”

This problem is inherently unsolvable because LLMS are prone to hallucinations and prompt injection attacks.

jesse_dot_id · Mar 22, 2026 · comment in “OpenClaw is a security nightmare dressed up as a daydream”

- 12 built-in eval rules across 4 categories: completeness (output length, coverage), relevance (keyword overlap, hallucination markers), safety (PII detection for SSN/credit card/phone/email, prompt injection patterns, blocklist), and cost (USD threshold, token efficiency)

iparent · Mar 14, 2026 · story in “Show HN: Iris – first MCP-native eval and observability tool for AI agents”

Behind the scenes, he's working with the DoW to sign a contract with them, to replace us the instant we are designated a supply chain risk. But he has to do this in a way that doesn't make it seem like he gave up on the red lines and sold out when we wouldn't.

layer8 · Mar 5, 2026 · comment in “Dario Amodei calls OpenAI’s messaging around military deal ‘straight up lies’”

Behind the scenes, he’s working with the DoW to sign a contract with them, to replace us the instant we are designated a supply chain risk. But he has to do this in a way that doesn’t make it seem like he gave up on the red lines and sold out when we wouldn’t.

GranPC · Mar 5, 2026 · comment in “Dario Amodei calls OpenAI’s messaging around military deal ‘straight up lies’”

An agent that writes and runs code can do anything an unsupervised developer can — one prompt injection from a malicious page and you have a real problem.

iranu · Mar 3, 2026 · comment in “Show HN: ScrapAI – We scrape 500 sites. AI runs once per site, not per page”

Show HN: BreakMyAgent – Open-source red-teaming sandbox for LLM system prompts As a developer, I got tired of manually testing my AI agents and chatbots against the same prompt injections and jailbreaks every time I tweaked a system prompt.

breakmyagent · Feb 26, 2026 · story in “Show HN: BreakMyAgent – Open-source red-teaming sandbox for LLM system prompts”

No LLM in the loop. Scans are reproducible, run in ~38ms, and avoid hallucination or prompt injection issues.

ComCat · Feb 24, 2026 · comment in “NPM install is stealing your passwords – I built a tool to catch it”

Claude Desktop runs MCP tools with raw user permissions. A hallucination or prompt injection can read your ~/.ssh keys.

goshtasb · Feb 23, 2026 · story in “Show HN: OmniGlass – An open-source, sandboxed Visual Action Engine”

The benchmark seemingly only checks whether the agent identifies which function is backdoored, not the specific nature of the backdoor. Since Claude guessed the right function in advance, it could hallucinate any backdoor and still pass.

comex · Feb 22, 2026 · comment in “We hid backdoors in ~40MB binaries and asked AI + Ghidra to find them”

The problem: AI agents are getting good enough to run shell commands, query databases, and manage infrastructure autonomously. But one hallucinated rm -rf / or a prompt injection that exfiltrates your .env can do real damage.

subho007 · Feb 14, 2026 · comment in “Show HN: Clawsec - Open-source plugin for OpenClaw that blocks dangerous actions”

An agent loop or prompt injection can burn through hundreds of dollars before anyone notices. Rate limiting doesn't help because it doesn't understand cost.

satgate · Feb 11, 2026 · story in “Show HN: SatGate – An economic firewall for AI agent traffic”

The problem: AI agents execute code on your machine. Prompt injections, hallucinations, or compromised tools can read ~/.ssh, exfiltrate credentials, or worse.

decodebytes · Feb 2, 2026 · story in “Show HN: Nono – Kernel-enforced sandboxing for AI agents”

The problem: AI agents execute code on your machine. Prompt injections, hallucinations, or compromised tools can read ~/.ssh, exfiltrate credentials, or worse.

decodebytes · Feb 1, 2026 · story in “Show HN: Nono – Kernel-enforced sandboxing for AI agents”

But how do we know that you did not hallucinate the claim that ChatGPT does not hallucinate its version number? We could try to exfiltrate the system prompt which probably contains the model name, but all extraction attempts could of course be hallucinations as well.

johndough · Jan 29, 2026 · comment in “Retiring GPT-4o, GPT-4.1, GPT-4.1 mini, and OpenAI o4-mini in ChatGPT”

I bet there's probably a jailbreak for all models to make them say slurs, certainly me asking for regex code to literally filter out slurs should be allowed right? Not according to Grok, GPT, I havent tried Claude, but I'm sure Google is just as annoying too.

giancarlostoro · Jan 21, 2026 · comment in “Claude's new constitution”

Cleaning the prompt and injecting it with common patterns like XML tags seems to help a bit but still feels extremely risky for some reason. Injecting it in the assistant or user also seems flaky and prone to prompt injection.

weli · Jan 15, 2026 · story in “Ask HN: How to overcome the limit of roles in LLM's”

> More than 130,000 new projects created Is there any way to prevent PyPI from becoming a morass of supply chain attacks like NPM etc.?

musicale · Jan 2, 2026 · comment in “PyPI in 2025: A Year in Review”

I hate everything about the Claude code plugin system. They saw GitHub Actions supply chain Fiasco and said: great let's add hallucinations on top.

arianvanp · Dec 23, 2025 · comment in “Claude Code gets native LSP support”

I struggle to understand what this specifically has to do with rails or global IDs. In ANY framework or query system, if you are asking an LLM to produce IDs which you are then passing to a database for lookup, you need to understand those identifiers could be hallucinated or incorrect in surprising or malicious ways, and can lead to data leaks or exfiltration.

yetanotherjosh · Dec 16, 2025 · comment in “Be Careful with GIDs in Rails”

But that freedom is dangerous. One hallucinated rm -rf / or a malicious package install, and your host system pays the price.

dorianzheng · Dec 13, 2025 · comment in “BoxLite Love AI agent – SQLite for VMs: embeddable AI agent sandboxing”

Adding one more library increases the risk of a supply-chain attack like this. As long as you're using npm or any npm-compatible runtime, then it remains to be an unsolved recurring issue in the npm ecosystem.

rvz · Nov 24, 2025 · comment in “Shai-Hulud Returns: Over 300 NPM Packages Infected”

> If the agent works on the user's data on the user's behalf (ex: vibe coding), then I don't consider jailbreaking to be a big problem. It could help write malware or things like that, but then again, it is not as if script kiddies couldn't work without AI.

calibas · Nov 20, 2025 · comment in “Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs”

If the agent works on the user's data on the user's behalf (ex: vibe coding), then I don't consider jailbreaking to be a big problem. It could help write malware or things like that, but then again, it is not as if script kiddies couldn't work without AI.

GuB-42 · Nov 20, 2025 · comment in “Adversarial poetry as a universal single-turn jailbreak mechanism in LLMs”

We all know how easy it is to generate emails etc at a minimum level to fool the elderly and doing that via ai is not really “jailbreaking” anyway.. Doing a study is only to get PR value and its frustrating to me why the AI world has this cargo cult world of studies and fake benchmarks with every advancement a new academic paper instead of doing normal engineering…

AIorNot · Nov 20, 2025 · comment in “Measuring the impact of AI scams on the elderly”

In Terraform/OpenTofu you just run into unreliable providers, 3rd party providers that make your supply chain a little questionable, or providers with half-broken APIs that weren’t ever intended to be called via terraform.

sshine · Nov 3, 2025 · comment in “Your Infra Isn't Special: Why Open Source Infrastructure as Code (IaC) Wins”