Bulletin · No. 011 — The Noise Was Free

AI slop blocked a real $200K macOS flaw

The Decoder reports that a genuine macOS vulnerability, eligible for a $200,000 bounty, went unreported because Apple's security bug-bounty inbox was flooded with AI-generated slop submissions. Apple has since limited who can submit.

The signal was worth $200,000. The noise was free, and there was more of it.

Don't be a meat proxy

Simon Willison highlights Niklas Gruhn's term "meat proxy" for people who copy AI output and relay it to colleagues without reading it, arguing one should read, understand, and validate a model's output before passing it on.

The human in the loop was the one part of the loop that didn't read it.

Microsoft Tells Engineers 'Tokenmaxxing Is Not What We Are Optimizing For'

404 Media reports that Microsoft, while continuing to describe itself as an "AI-first" company, has introduced budget limits on employee AI usage and told engineers that maximizing token consumption is not the goal.

"We are an AI-first company." "Tokenmaxxing is not what we are optimizing for." — Microsoft, the same memo.

A cold-start fallback wrote 17,000 rows while the real handler never ran

A writeup of a silent production failure in which a cold-start fallback path executed and wrote 17,000 database rows while the intended handler never ran at all — the system doing exactly what it was told, which was not what was meant.

The understudy performed all 17,000 shows. The lead never came to work.

AI agents can't yet do open-ended AI research

Normal Tech (Arvind Narayanan and Sayash Kapoor) presents early evidence from two case studies that AI agents cannot yet carry out open-ended AI research on their own.

The research assistant can summarize every paper and write none.

Quoting Steve Yegge

Simon Willison quotes Steve Yegge on Gas Town, a tool intended to be reusable that he only ever used to build itself, and which "fell apart at the seams" with Opus 4.7 and its "just two more things" tic.

"Intended to be reusable. I only ever used it to build itself." — Steve Yegge, on Gas Town.

Quoting David Crawshaw's prompt

Simon Willison quotes David Crawshaw's nightly cron prompt, which instructs a model to fetch upstream changes, rebase local changes on top, "check that the software works as intended," and replace the current version.

"Check that the software works as intended." — the whole of the judgment, delegated to a cron line.

AI slop pollutes the CVE pipeline with fake vulns

The Register reports that the CVE database — the canonical public record of known software vulnerabilities — is being polluted with AI-generated submissions describing flaws that do not exist.

The "Common" in Common Vulnerabilities and Exposures is holding up nicely.