Weekly Review: The Burden of Proof
The week's curated set of relevant AI Tutorials, Tools, and News, on why the claim got cheap and the proof is where the work now lives
Thanks for opening Altered Craft’s weekly AI review for developers, and for making this a Monday habit. One thread runs through this week’s picks: the burden of proof has moved onto the output. An essay on AI confidence theater puts it plainly, sounding competent quietly replaced being competent, and the rest of the issue answers with evidence. Verify the fix, contain the code, gate the endpoint, score the prompt, prove the exploit. The claim is cheap now, so the proof is where the work lives.
TUTORIALS & CASE STUDIES
This week’s tutorials walk the burden of proof down the stack: verify the fix a human should still own, contain the code you cannot fully trust, gate the endpoint behind real authorization, then run it all on infrastructure that records what happened.
The Sorcerer, Not the Apprentice: Fixing a Parser Bug With AI
Estimated read time: 6 min
It starts with the fix you are tempted to accept. Walking through a real hyperscript parser regression, this essay shows AI excelling at root-cause investigation and tests while proposing progressively hackier fixes. The clean solution came from a knowledgeable human restraining complexity an agent alone would have shipped as technical debt.
The takeaway: Let AI investigate causes and grind out tests, but keep a human who understands the architecture deciding which fix actually ships.
Running Untrusted Agent Code Without a Sandbox
Estimated read time: 6 min
If you cannot fully trust the fix, trust the generated code even less. Complementing our coverage of AWS Lambda’s isolated MicroVMs[1] last week, LangChain takes the opposite tack: with prompt injection unsolved, its Code Interpreter for Deep Agents runs orchestration inside WebAssembly with QuickJS, so the runtime starts with nothing and bridges capabilities deliberately, even pausing for approval.
The principle: Constrain what agent code can touch instead of trusting it to behave. Start from zero capabilities and bridge in only what each workflow genuinely needs.
[1] AWS Lambda MicroVMs: Isolated Sandboxes for User and AI-Generated Code
Put Your LLM Endpoint Behind an OAuth Boundary
Estimated read time: 16 min
Beyond the code it runs, the endpoint itself needs a gate. This tutorial wires Azure API Management and Entra ID to add OAuth 2.0 authorization in front of a Foundry-hosted GPT model. The principle: don’t expose your model endpoint directly. Policies validate each token’s tenant and client.
Worth noting: Gate your LLM calls behind an API gateway that checks token issuer and client, so not every valid Entra ID token gets through to the model.
Mistral’s Workflows: Document Processing in 30 Minutes
Estimated read time: 3 min
With those checks in place, the pipeline still has to run provably. Mistral introduces Workflows, an orchestration platform for building, executing, and monitoring AI-driven pipelines. The walkthrough shows how to build a document processing workflow in 30 minutes, backed by durable, fault-tolerant execution on distributed infrastructure.
What this enables: If you are chaining AI steps into a pipeline, purpose-built orchestration with fault tolerance and monitoring beats hand-rolled glue code you cannot replay.
TOOLS
The tools split into the engines that do the work and the ones that prove it: an affordable frontier-class tier, a router that decides without a model, a full GLM stack, then two that insist on evidence, scored prompts and working exploits.
Claude Sonnet 5: Opus-Class Agentic Work at a Lower Price
Estimated read time: 5 min
The tools start with the engine. Anthropic’s latest mid-tier model narrows the gap with Opus-class performance at lower prices. Sonnet 5 plans, uses browsers and terminals, and sustains multi-step coding autonomously, and early partners report it finishes complex jobs earlier Sonnets abandoned.
Why this matters: Adjustable effort levels let you tune the cost-performance tradeoff, making Sonnet 5 a practical default for sustained agentic coding and tool-use work.
Wayfinder: A Model Router That Makes Its Routing Call Without a Model
Estimated read time: 9 min
You do not need that engine for every request. Wayfinder scores each prompt’s structure and wording, sending easy requests local and hard ones to your expensive tier. The standout is a deterministic, offline decision with no model call, no key, no network, and it speaks the OpenAI API.
The opportunity: Stop paying frontier prices for “fix my typo.” Route prompts by difficulty with a free, deterministic decision that drops in behind your existing OpenAI client.
ZCode and the GLM Coding Plan: An Agentic Coding Stack Built Around GLM-5.2
Estimated read time: 2 min
For a full stack around a cheaper engine, ZCode pairs a coding agent with tiered GLM plans from $16.20/month, tuning GLM-5.2 for reasoning and multi-agent work. The pitch centers on keeping every engineer on the frontier, with long-running Goals and bot control from WeChat, Feishu, or Telegram.
Try this: If you want an agentic coding setup that steers work from chat and handles long-running tasks, ZCode’s GLM-tuned tiers are worth a look.
Stop Hand-Tuning Prompts: How DSPy Automates the Work
Estimated read time: 7 min
Whatever model you land on, the prompt driving it needs proof it works. Prompt engineering is slow and unreliable, since eyeballing a few outputs rarely shows a prompt holds in production. This piece argues for treating prompts like ML models, using DSPy to define test data and a scoring function that grades candidates automatically.
The pattern: Bring ML-style evaluation to your prompts. Define a test set and a scoring function, then let DSPy generate and grade candidates instead of guessing.
Strix: Autonomous AI Agents That Pentest Like Real Hackers
Estimated read time: 6 min
Proof matters most where security is on the line. Strix is an open-source pentesting tool where AI agents validate vulnerabilities with working proof-of-concept exploits, not static-analysis noise. It ships a full offensive toolkit, multi-agent orchestration, and a CLI that drops into CI/CD to block insecure code before it merges.
The payoff: Wire Strix into your pull request workflow to catch and reproduce real exploits before insecure code reaches production.
NEWS & EDITORIALS
The editorials weigh substance against performance: the theater of sounding capable, the expertise that judges real output, the loop that keeps a system honest, AI proving flaws at scale, and the unglamorous fundamentals that outlast clever bets.
Please Stop the AI Confidence Theater
Estimated read time: 11 min
The editorials open on the gap the whole issue answers. A pointed critique of performative AI hype flooding social feeds and hiring, where sounding competent replaced being competent. The piece argues the gap between demo and daily reality erodes trust and buries the real work of tuning living AI systems.
The context: Be honest about what your AI workflows actually do. A prototype that saves 20 minutes beats a viral claim that collapses on inspection.
The Twilight of the Chatbots
Estimated read time: 7 min
If competence has to be shown, someone has to judge it. AI capability is climbing a better-than-exponential curve, with agents running autonomously for hours on work that once took teams weeks. The shift moves from chatbots to managing fleets of agents, where domain expertise predicts who gets the most useful output.
The shift: Treat agents like direct reports you manage rather than chatbots you collaborate with, because your domain expertise turns long-running AI into real output.
Autoresearch: When the Loop Becomes the Product
Estimated read time: 10 min
Building on our coverage of loop engineering[2] last week, Introspection’s Roland Gavrilescu unpacks “autoresearch”: an outer loop where agents maintain and improve the primary system via evals, judges, and human signals. The reframing is that the loop is the product, packaged into portable “agent recipes” teams can own.
Why now: Invest first in the signals you want agents to respond to, and control loop costs, before chasing full autonomy.
[2] From Prompting Agents to Loop Engineering
The CVE Spike Behind AI-Powered Vulnerability Discovery
Estimated read time: 3 min
Proof cuts the other way too. Epoch AI charts a sharp rise in severe cybersecurity disclosures after Anthropic revealed Claude Mythos Preview could autonomously find flaws. High- and critical-severity CVEs from notable organizations spiked more than 3.5x in June 2026, following hardening work by Anthropic and OpenAI.
The signal: Frontier models now discover vulnerabilities at scale, so expect a faster patch cadence and treat dependency updates as a security priority, not a chore.
The AI Blog Prize: Three Contrarian Takes on the Post-Automation World
Estimated read time: 14 min
We close on where durable advantage actually lives. Dwarkesh Patel announces three winning essays from 600 submissions on big AI questions, ranging from ending airborne disease with far-UVC infrastructure to a grounded case for boring, timeless economic policy over galaxy-brained schemes.
The lens: In a world reshaped by AI, the most durable advantages may come from unglamorous fundamentals like stable institutions and capital formation rather than clever bets.
That’s the week. The throughline: the claim got cheap, so the proof is the product now, whether that proof is a contained runtime, a scored prompt, or a working exploit. See you next Monday.