Your eval library calls the judge once per test case and prints a number. The judge flips its verdict on 5-15% of reruns. That number is noise wearing a suit.
Every LLM Eval Library Has the Same Bug: Stochastic Judges Used as Deterministic Oracles
Related
Building AI Agents for Social Media with TypeScript and Hono.js
Everyone's talking about AI agents right now, but most tutorials stop at "call an LLM in a loop." If...
Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency 40%
Optimizing RAG at Scale: Chunking, Retrieval, and the Bayesian Search That Cut Latency...
This week in Cursor + .NET — 7 rules (week ending July 19, 2026)
A weekly digest from the Agentic Architect persistence kit: 7 senior C#/.NET rules for engineers keeping Cursor honest across sessions.