Non-deterministic, inconsistent outputs

It will confidently tell you it's done a thing, and then you ask, it admits can't actually do that but will happily try and fail again. Very frustrating, because I really like Colab as a platform for little reproducible experiments that may or may not require CUDA.

throwaway219450 · Sep 2, 2026 · comment in “Gemini 3.8 Flash and 3.8 Flash Cyber”

Correction: AI is not deterministic, the only realistic low-error solution is not a more complex use of non-deterministic AI, but deterministic transpilation. The problem is that this results in COBOL-in-Java which runs correctly but it is a nightmare to maintain.

dragonwriter · Aug 3, 2026 · comment in “AI migrated legacy COBOL programs to Java, bugs included”

There are dozens more. The innate non-determinism of the LLMs flips the dice sometimes and that leads to subtle bugs -- which is maddening, especially if the disciplines on how to write one thing or another are clearly spelled out in `AGENTS.md`.

pdimitar · Jul 26, 2026 · comment in “The new rules of context engineering for Claude 5 generation models”

The whole point is that it allows introducing controlled side effects as long as the output is reproducible. Bur you're extremely annoying to talk to so I'm not gonna continue engaging.

arianvanp · Jun 19, 2026 · comment in “Migrating from GNU Stow to Chezmoi”

It's easy enough (well, it's still addons whose versions you have to keep updated each on their own) once it is set up, but getting to the point where you have something reproducibly running for the first time is annoying as hell.

mschuster91 · Jun 15, 2026 · comment in “What job interviews taught me about Kubernetes”

There's another layer to the non-determinism of LLM agents: what are the execution params the provider is using today? I hate the feeling that a worn path that I've grown to trust will "do the right thing" over the last few months will suddenly start doing the wrong thing simply because an engineer at Anthropic or OpenAI found a way to save N million dollars by "optimizing" thinking token usage.

kurige · May 26, 2026 · comment in “The User Is Visibly Frustrated”

The plan/implement/test workflow is very basic and represents the most common agentic use case. But the state machine pattern applies to any multi-step work where agents are useful but susceptible to death spirals, hallucinations, or other non-deterministic quirkiness.

azurewraith · May 13, 2026 · comment in “Show HN: Statewright – Visual state machines that make AI agents reliable”

> I'm not sure why people struggle with the fact that an abstraction can be built on top of a non-deterministic and stochastic system.

rock_artist · May 4, 2026 · comment in “LLMs Are Not a Higher Level of Abstraction”

I'm not sure why people struggle with the fact that an abstraction can be built on top of a non-deterministic and stochastic system.

dimtion · May 3, 2026 · comment in “LLMs Are Not a Higher Level of Abstraction”

Task Inference, Task Evaluation, Task Adherence, tool calling, all do very well on my evals. I did however for the first time in a while end my Claude Max subscription because, after their post-mortem [0] I for the first time saw true, reproducible, incredibly frustrating regressions in model output when using Claude Code.

Topfi · May 3, 2026 · comment in “Show HN: State of the Art of Coding Models, According to Hacker News Commenters”

These new revs of the LLM products are just as stupid and nondeterministic as the old ones. They also still make the same classes of stupid mistakes, are pretty much as dangerously unreliable as they always have been [0], and -just like previous versions- have 'capability rot' that cannot be anticipated, but might be caused by inability to handle current demand, deliberate shifting of backend resources to serve newe…

simoncion · Apr 17, 2026 · comment in “The future of everything is lies, I guess: Where do we go from here?”

(In general it is mildly frustrating to me to see software developers treat floating point as some sort of magic and ascribe all sorts of non-deterministic qualities to it.

dwohnitmok · Mar 12, 2026 · comment in “Kotlin creator's new language: talk to LLMs in specs, not English”

If you ask an agent to "build a dashboard," you get a different result every time. If you write a massive, descriptive prompt detailing every padding and flex-box property, you waste time.

MegaByteMark · Mar 11, 2026 · comment in “Show HN: A Markdown DSL to stop AI agents from hallucinating UI code”

Repo: https://github.com/crasofuentes-hub/deterministic-agent-system Would love feedback from people working on reproducible agents, AI safety evals, or enterprise workflows where non-determinism is a real problem.

Agente2026 · Mar 9, 2026 · comment in “Building reproducible LLM agents with strict determinism guarantees”

Hi HN — I built Trajectly, a tool for deterministic regression testing of AI agents. Problem: agent “evals” are often flaky (network, time, tool nondeterminism, model drift), so it’s hard to tell if a change actually broke behavior.

ashmawy · Mar 5, 2026 · comment in “Trajectly – deterministic regression tests for AI agents”

I was able to fine tune the agent client-side and then it worked until the context cleared, but I didn’t want to hard code into context/agents.md how to access an API that will keep changing. I hate all this non-deterministic programming stuff but it's still too good to not do it :)

dervishcat · Mar 5, 2026 · story in “Show HN: Stop LLMs from brute forcing (guessing) APIs”

However LLMs are being pushed as the thing to replace much of the deterministic things that exist in the world - and anyone seen to be thinking otherwise gets punished e.g. in the stock market. This world of extremes is annoying for people who have the ability to think more broadly and see a world where deterministic systems and non-deterministic systems can work together, where it makes sense.

df2dd · Feb 25, 2026 · comment in “OpenAI resets spending expectations, from $1.4T to $600B”

No LLM in the loop. Scans are reproducible, run in ~38ms, and avoid hallucination or prompt injection issues.

ComCat · Feb 24, 2026 · comment in “NPM install is stealing your passwords – I built a tool to catch it”

Most of us want to run autonomous agents (like OpenClaw or LangChain) locally on our daily-driver machines so they can actually interact with our IDEs and real workflows. But giving a non-deterministic, hallucination-prone script raw access to your host OS is basically local remote code execution.

Kidiga · Feb 23, 2026 · story in “Show HN: Raypher–Running local AI agents (OpenClaw) on your own local computer”

Workflows that I use and are very reproducible start to flake and then fail. After a certain number of tokens per day, it becomes unusable.

jampa · Jan 29, 2026 · comment in “Claude Code daily benchmarks for degradation tracking”

Notes on Clawbot: agents are starting to look like OS runtimes design now I spent a few days poking at Clawbot and it clarified something I’d been hand-waving: the hardest part of “useful autonomous agents” is increasingly not model quality, but the runtime that keeps the whole thing coherent when tools/LLM calls are slow, flaky, or non-deterministic.

chenyusu · Jan 28, 2026 · story in “Notes on Clawbot: agents are starting to look like OS runtimes design now”

This is an interesting direction for “open” tooling. Combining containerization (Docker) with reproducible environments (Nix) addresses two of the biggest pain points in developer workflows: environment drift and opaque build/runtime assumptions.

jaynamburi · Jan 23, 2026 · comment in “OpenCode with superpowers. It can do everything in a container with Docker / Nix”

DataFlow addresses a real pain point in AI research and product development — reliable, reproducible, and scalable data pipelines powered by large language models.

Mey0320 · Dec 23, 2025 · comment in “DataFlow: Using LLMs to Build Reproducible, End-to-End Data Pipelines”

> it seems pretty random as to when it decides to drop that out of context Overcoming this kind of nondeterministic behavior around creating/following/modifying instructions is the biggest thing I wish I could solve with my LLM workflows.

zarp · Dec 22, 2025 · comment in “Scaling LLMs to Larger Codebases”

Now LLMs can somewhat bridge that gap due to how frequently we write about code. But it’s a non-deterministic process and hallucinations are by design.

agentultra · Dec 10, 2025 · comment in “If you're going to vibe code, why not do it in C?”

• MIT license + full docs + architecture diagrams If you work in ML, science, infra, or compliance, you already know how painful nondeterministic pipelines are.

MLoffshore · Dec 2, 2025 · story in “Show HN: Prima Veritas – Deterministic Analytics Engine for Reproducible ML”

I have no doubt that the oligpolists in charge of our tech landscape will at some point "infuse" this shit into the games or whatever the shitterm they use for coupling the text generators with perfectly fine and deterministic software in order to raise their DAU/MAU/WAU numbers and in the process make the outcomes of using the software less reliable, non-deterministic and absolutely frustrating.

hansmayer · Nov 15, 2025 · comment in “SIMA 2: An agent that plays, reasons, and learns with you in virtual 3D worlds”