Learning Machine

Letting Codex Agents Commit: Making .git Writable in the workspace-write Sandbox

2026-07-22T00:00:00-05:00

Codex CLI's workspace-write sandbox lets the agent edit anything in your repo -- except commit. Every git commit dies with:

fatal: Unable to create '/path/to/repo/.git/index.lock': Read-only file system

This is deliberate. On Linux, Codex's bubblewrap sandbox mounts your writable roots read-write, then re-applies .git (plus .codex and .agents) as read-only on top. The security rationale is sound: a writable .git/hooks would let the agent plant a hook that executes outside the sandbox the next time you run git yourself.

But if you're already running with --ask-for-approval never -- YOLO mode -- you've made that trust decision, and an agent that can't commit its own work is a constant irritation. There's no official toggle: no allow_git_writes option exists anywhere in the codebase, and the open issues asking for one (#15505, #12280, #14338) have no maintainer response.

The trick: name the `.git` path itself

The obvious fixes don't work. Adding the repo root to sandbox_workspace_write.writable_roots fails because the .git protection is applied per writable root -- a writable parent still gets a read-only .git carved out of it (#15505). Passing extra directories with --add-dir fails because the read-only mount is applied after the command-line write dirs, so it wins (#14338).

But reading the sandbox policy source (codex-rs/protocol/src/permissions.rs), the carveout is added by a function named append_default_read_only_path_if_no_explicit_rule. The name says it all: if an explicit rule already covers the exact .git path, the read-only protection is skipped.

So point writable_roots at the .git directory itself:

codex exec --sandbox workspace-write \
  -c 'sandbox_workspace_write.writable_roots=["/path/to/repo/.git"]' \
  "commit your work"

Verified on codex-cli 0.144.6: without the flag, the commit fails with the read-only error above; with it, the commit lands -- confirmed in git log, not just the agent's self-report.

Wiring it into a shell function

The writable path is per-repo, so compute it at invocation time. My yolo-codex wrapper now does:

# Make .git writable inside the workspace-write sandbox so the agent can
# commit. Codex mounts .git read-only unless an explicit writable_roots
# entry names the .git path itself. Includes the common gitdir so linked
# worktrees can commit too.
local -a git_writable=()
if git rev-parse --is-inside-work-tree >/dev/null 2>&1; then
  local git_dir git_common_dir
  git_dir=$(git rev-parse --absolute-git-dir)
  git_common_dir=$(git rev-parse --path-format=absolute --git-common-dir)
  if [[ "$git_dir" == "$git_common_dir" ]]; then
    git_writable=(-c "sandbox_workspace_write.writable_roots=[\"$git_dir\"]")
  else
    git_writable=(-c "sandbox_workspace_write.writable_roots=[\"$git_dir\",\"$git_common_dir\"]")
  fi
fi

codex --ask-for-approval never --sandbox workspace-write "${git_writable[@]}" "$@"

Two details worth the extra lines:

Linked worktrees. In a git worktree checkout, --absolute-git-dir points at the per-worktree gitdir, but commits also write objects and refs into the common gitdir back in the main repo. Both paths need to be writable. Tested: a commit from inside a linked worktree works with both entries, and a plain repo needs only the one.
The -c flag sets only writable_roots. Anything else you have under [sandbox_workspace_write] in ~/.codex/config.toml (like network_access = true) is untouched.

Outside a git repo, the array stays empty and the function behaves exactly as before.

Caveats

This is undocumented behavior. The explicit-rule exemption is an implementation detail of the sandbox policy, not a documented feature. A future Codex release could re-protect .git unconditionally -- if agent commits start failing after an upgrade, this is the first thing to re-check.
You're removing a real protection, not working around a bug. A writable .git means a writable .git/hooks, and hooks run unsandboxed when you next invoke git. If you're not already running approval-free, think twice.
The blunt alternative is --sandbox danger-full-access, which trades the entire filesystem sandbox to fix one directory. Carving out .git keeps everything else protected.

Tested on codex-cli 0.144.6, Linux (bubblewrap sandbox).

The Dark Arts of Web Automation

2026-07-16T00:00:00-05:00

Ominous, right?

Sounds like I'm about to teach you something you'll need a lawyer for.

…We'll come back to the lawyer in a bit.

Here's a little bit of background…

So…

I was getting ready for this talk -- and OpenAI threatened to ban my account ...

Just for the work that I was doing in preparing it…

I checked my email the other day, and I found this.

What a shocker this was!

So -- what does one have to do to earn the banhammer?

For cyber abuse?

With a web browser?

This. This is what I was doing.

Every one of these is being solved by an AI agent using a web browser

... and in 15 minutes or so, you'll understand exactly how.

I want my agents to be able to use the web the way that I do -- book the things, send the emails, fill in the forms -- so I don't have to.

But, the moment something that isn't a person starts clicking, many webpages fight back.

So -- this is a talk about winning that fight. And it starts with one slightly unconventional idea.

Here it is. The premise of the whole talk.

'A CDP browser is just like a meatbag with a mouse.'

...

At least as far as Google, Cloudflare, and the rest can tell

No joke -- if you have your agent drive a browser through the Chrome DevTools Protocol, your agent's clicks and keystrokes travel the exact same path inside Chrome that yours do.

That's the big idea. The rest of the talk is how you pull it off. And that comes down to three things.

A CLI -- not an MCP.

The Chrome DevTools Protocol -- which is where a tool I built, called chrome-agent comes in.

And a loop, run on a ladder.

So, let's start with the one that picks a fight.

First thing: give your agent a command line interface (shell-based tools), not an MCP server. And before some of you knuckle-up on this debate, there are specific reasons for this.

It's worth noting that capability is a wash -- CLI and MCP both got to the right answer about 83% of the time in a recent study by Arize AI.

HOWEVER, a CLI beats an MCP in reuse, speed and cost.

Reuse first. A CLI sequence can be programmed: write it once, run it a thousand times, without a model in the loop. Whereas MCP hits the model on every turn.

The CLI is FASTER for a similar reason -- because there's no model in the middle of every step. In that same study -- MCP took 71 round-trips and 8 minutes for the same task that was completed with just 7 calls and under a minute using the CLI. Hold onto this one -- speed comes back at the very end.

And token cost -- Anthropic reported that executing code instead of running MCP can be 75x cheaper in terms of token usage.

So -- what are we actually running on the command line?

We drive the browser with the Chrome DevTools Protocol -- the second thing your agent needs to appear human.

You already know of this protocol, even if you've never heard its name. That developer panel that opens in Chrome when you hit F12 drives the browser using it. Your agents can speak it, too, using the chrome-agent tool. Chrome Agent also makes it easy for your agent to write code to replay CDP interactions.

The surface area of CDP is enormous. 57 domains, hundreds of methods and events, covering everything the browser can do. I've grouped these domains into 8 buckets to make it easier to hold in your head.

But the good news is that you don't need all 57 domains. To interact with a page like a human, you usually need just a small subset of these.

It's easiest to think about the subset of CDP domains you'll frequently use in terms of the 'digital senses' they provide your agent.

You SEE the page -- read its structure from the DOM, its semantics from the accessibility tree, or just take a screenshot when you want the pixels.

You HEAR what the page reports about itself -- the network data, its console, and its logs.

And you OPERATE the page -- with clicks, keystrokes, and navigation.

The third and final thing you need for your agent to appear human when using the browser is what I call "a loop on a ladder."

So here's the loop:

Sense, act, verify -- and you repeat it until the page gives in.

Sense -- perceive the page through one or more channels -- the DOM, the accessibility tree, a screenshot.

Then Act -- do ONE thing -- click something, type something, select something.

And then Verify -- sense again, but through a different channel than the action. e.g. If you clicked something, don't ask the click whether it worked -- look at the screen, or the network instead.

Sense where you are. Make one move. Confirm it landed. Then iterate again.

And when the loop won't close -- so when you sense, act, verify, and the page still won't do what you want -- that's the page fighting back and telling you to climb the ladder.

The Meatbag Ladder, that is! This is a ladder of techniques that are increasingly more human as you climb it.

It has 3 rungs and you climb only as high as the page forces you. Said differently, you climb to the lowest rung that works.

On Rung 1 you don't act human at all. If you can just call the API exposed within the page, or fire a synthetic JavaScript click, do that -- it's easy, it's free, it's instant, and it's the right default.

Climb to Rung 2 when faking it stops working: e.g. when you need a real click using CDP's Input domain. This is agent input that the page cannot tell apart from your own.

Climb to Rung 3 when you need human input plus human behavior -- a real mouse path, a little dwell and some jitter, or vision to actually see and interpret things.

So you start cheap, and climb one rung at a time -- only when the page makes you. Climb the ladder, run the loop on each rung, then write down the path that worked.

And that's how you arrive at fully automated AI agent-driven browsing.

First you explore -- you run the loop by hand, climbing rungs, until the thing actually works.

And then you automate -- you write that solution down so you never have to discover it again.

And you write it down as CODE, as an AGENT SKILL or, quite often, as both.

So let me show you how this all comes together -- starting with a word from my attorney.

I told you we'd come back to the lawyer.

Everything you're about to see is real -- all of this comes from real agent browser use, in the wild.

HOWEVER, On the advice of counsel, everything you're about to see runs only on infrastructure and accounts I own.

NOW ...

Who's ready to see some agents getting BUSY with the browser?

Alright, so here's some simple, everyday use. Let's send a batch of personalized emails -- each one different -- from your Outlook web client.

We've got some lovely pseudocode here to make it easy to get the general sense of how this all works in code.

Outlook's compose box has nothing to defeat, so this is Rung 1 of the Meatbag Ladder -- you don't act human at all. A synthetic click opens compose, you fill it in programmatically, and a synthetic click sends it.

And then you just … let it rip!

The reason this is going so smoothly is that it's executing a program. You capture the sequence once then loop it -- 20 emails, or 200, from one command. Solve it once, run it forever.

The agent is riffing on the content to personalize it, whilst running a program for all the interactions.

You may ask "Why drive the web UI at all -- why not use the API?" Because in a corporate environment the official API needs an app registration and an admin's sign-off -- which, as an employee, you often can't get. HOWEVER, The logged-in web session needs nothing but the login you already have. The web UI becomes a universal API in this pattern.

That's Rung 1 ... But what happens when the page pushes back?

So -- let's say you're shopping at your favorite online megastore. Let's just call them … Demazon. They're a crafty bunch over there at Demazon, and their pages don't quite care for your bots. Run that exact same fake click that just worked on Outlook, point it at an 'Add to Cart' button, and … nothing. No error, no complaint. The page just ignores it.

Because the page is checking: did this click come from a real person, or from JavaScript? Chrome stamps every event with that answer -- whether an input is trusted, or untrusted. A click you fire from code is stamped untrusted, and, in this case, the page quietly drops that input.

BUT ... No worries -- you just climb the Meatbag Ladder! On Rung 2 you run the click through Chrome's Input domain which uses the same input path your actual mouse uses. Now it's stamped trusted, the page can't tell the difference, and the item drops into the cart.

Now, we've got quite the inside view of Demazon.com. Watch the bottom panel and let's imagine that's what the server's logs might look like. Every fake click is rejected, but the real ones are accepted.

As a heads-up -- when you see the mouse pointer in these screen recordings, that has been added programmatically to show what mouse inputs the agent is simulating using chrome-agent / CDP. The agent isn't actually using my mouse.

Rung 2 is where the real meatbag inputs begin BUT … that, alone, is not enough to let your agent replace you in the browser.

So we climb to the top of the Meatbag Ladder. Rung 3 -- this is the narrow frontier where pages are actively hunting for bots. There are a variety of techniques we deploy here, though, so let's talk through a few of them.

I'm sure this guy looks familiar. This is Cloudflare Turnstile and it appears deceptively simple, but it's the hardest target we've hit yet. You can't easily get a hold of that checkbox via typical web automation programming.

It's buried behind three nested boundaries -- a closed shadow root, then a cross-origin iframe, then another shadow root inside that. To every cheap trick, that checkbox simply does not exist. There is no element to grab.

SOOOO ... you stop trying to grab it! You ask the browser where the iframe sits on screen, do a bit of maths to calculate the position of the checkbox, and fire a trusted click at that point on the glass. And Chrome does the rest!

A real click lands right on the checkbox.

No human in the loop -- all agent. That's one level, cleared, and mostly the trick here was just figuring out how to interact with it. The next levels make you prove you can actually see.

So here's MTCaptcha -- remember these ones? These types are still around ...

The agent has to actually read this guy. So it simulates what you'd do -- it takes a screenshot of the challenge and looks at it, and its own vision capability picks the characters out of the noise.

Then it types the answer back using real, trusted keystrokes, routed into the challenge's cross-origin iframe, one character at a time. The same inputs your keyboard sends.

And the server agrees: text correct, token issued!

Now there's one more level before the final boss -- and this one is won or lost based on how you move like a meatbag.

So this one's by Lemin -- it's a little jigsaw puzzle where you have to spot where the piece belongs, and drag it into the gap -- and there's an entire class of CAPTCHAs like this.

This one's tricksy in different ways. There's no shadow root and no cross-origin iframe to pierce -- the piece is sitting right there in the page. The hard part is the drag itself.

When you drop the puzzle piece, these types of CAPTCHAs sample the mouse movement into a trail of points the whole way across, including jitter and changing speed and all. So it's not just solving the puzzle, but solving it with Moves like Jagger!

SOOOO … the agent drags the way a hand drags -- eased in, gently curved, a small overshoot and then settles. Just like a meatbag with a mouse! It uses Vision to see the gap and human-like motion to cross it.

So that's Turnstile. MTCaptcha. and Lemin -- three gates built to keep agents out, and we've just beat each of them. Which leaves only one boss standing ...

And here he is -- The final boss of The Internet. reCAPTCHA v2.

The little checkbox, the grid of blurry traffic lights and crosswalks -- everyone in this room has squinted at one.

BUT we've got the whole kit for how to beat this guy, too!

The digital senses … the loop to deploy them in … the Meatbag Ladder … everything you need to take down this cheeky bastard!

So let's go!

Here's the whole machine, and it comes in two halves.

On one side we have The Solver. Pure code -- no agent, no model. It does everything programmatically:

the trusted click on the checkbox,
piercing into the challenge iframe,
and then every round it screenshots the tile grid;
if a round expires, it just re-arms and goes again.

This bit is deterministic, fast, free.

There's one step in that loop the code can't do and that's look at a grid of fuzzy photos and know which ones have a bus in them.

That's vision and thinking, and it needs eyes and a brain.

So that's the only job the agent gets. And we call that The Operator. The solver taps the agent on the shoulder -- and the agent:

takes one look at the grid
picks the tiles with the thing in them
hands the answer back to the Solver
then stands by until the next lap.

This is really the entire talk, running as one system. Code does the deterministic driving; the agent does only the part that takes eyes and a brain.

Who wants to see it go?

Allllllright!

So while that's playing, by the way, some of these are really hard! The agent spotted bicycles in photos that I didn't!

But look at that -- it's solved and server-verified and IT'S FAST!

And fast is the whole point! This big bad boss is on the clock! Every round expires, and a full solve can be many rounds back to back. An agent that round-trips a model on every click and every look BURNS THAT CLOCK and loses -- the challenge resets before it ever finishes.

The only thing I've found that defeats this is what you're watching: deterministic code running at machine speed, with one quick AI look per round.

Remember when I told you to hold onto speed? This is why.

And this is why it had to be a CLI, on CDP -- never a model sitting in the middle of every single interaction step.

In case you're wondering, solving this wasn't a fluke …

Here it is, again and again ...

A reliable, repeatable solution, every time.

But HERE's what I actually want you to walk out of here with …

The big takeaway is the methodology enabling this. The CAPTCHAs were just a tricksy test of it.

This came down to careful, disciplined engineering -- it's the engineering that enabled an agent to do something it simply could not do on its own.

And the method is simple:

Give your agent a CLI, so you can program it.
Drive the whole browser through CDP -- using its digital senses.
Run it as a loop on the Meatbag Ladder -- climbing only as high as the page forces you.
Then Explore until you solve it and write the solution down.

That's what makes this durable and truly useful: Figure it out once. Do it forever.

Which brings us all the way back … to the crew at OpenAI and the impending banhammer.

After a quiet word, they kindly rescinded the threat.

So, I've still got access to Codex, which is nice!

So, you, too, can use Chrome Agent. It's installable in the Python ecosystem.

It's also open-source and available on GitHub: github.com/captivus/chrome-agent

That's the tool I wrote to do all of this with, and I use it all day, every day!

OR ... build your own! We live in the age of unbounded, personalized software.

Please do follow me on X and let's chat -- I want to hear how you're automating the web with AI.

Happy hacking!

How I Fixed Handy's Memory Leak Without Knowing Rust

2026-05-28T00:00:00-05:00

For the TikTok attention span ("TL;DR")

I applied The Framework to diagnose and fix a nasty memory leak in the wonderful Handy voice transcription app. This required some novel adaptations of The Framework that Software and AI Engineers alike will appreciate.

Developed and implemented an agent autonomy harness so that my AI agents could run Handy end-to-end without interrupting my work (this was massive)
Instrumented the Handy runtime for observability and agent sensory feedback (also massive)
Used this instrumentation to analyze and narrow systematically to isolate the source of the leak
Fixed the leak in 3 ways, and tested each bit of the fix on its own, and together, measuring how each contributed (one of the candidate fixes was dropped after seeing its minimal contribution)
Deployed adversarial agents to scrutinize findings
Surfaced a patched version of the app for me to run for days to prove, via additional telemetry, that the fix worked (the important meatbag-in-the-loop bit)

All of this was done in a completely reversible way, leaving no trace of the Rust development environment on my machine after successfully diagnosing and fixing the memory leak.

This is what AI-accelerated software engineering looks like when it's not "vibe coding" 🤮.

Have you tried Handy for voice transcription?

If not, get some! I really dig it. Voice transcription has been my primary input device (measured by characters input) for over a year now. I wrote my own for use in WSL, before I ditched Windoze forever. I've even deployed the most controversial keyboard in the world to charactermaxxx with it.

The problem was that Handy contained a devastating memory leak that rocked my beloved Linux box, blasting my productivity with a system-shattering OOM kill that disrupted my work severely. After recovering, I decided to be a good open source citizen and figure out WTF had caused this, file a bug report, and attempt to fix it.

I'm not a Rust programmer though ...

Handy is a Tauri app -- a kind of clever spin on the Electron concept of building web applications that run everywhere. It's a Rust backend with a React frontend and, thanks to Tauri, instead of shipping an entire browser engine (vis Electron + Chromium) it borrows the already-installed browser. Neat!

That said ... I known't Rust. Whatever shall I do?

I heard a strange, humming sound from my Ghostty terminal ...

An off-the-shelf installation of Claude Code was buzzing with the urge to vibe slop a purported solution!

That simply would not do, though. The Vibe Coding Hangover is, as we all know, quite real.

Courtesy of The Cure for the Vibe Coding Hangover

Plus, I couldn't responsibly vomit slop into someone else's repo, especially not for a tool that I use daily. This remained a software engineering problem.

Adapt and apply The Framework.

Okay, so what is this ✌️"The Framework"✌️?

I'm so relieved that you asked! The Framework is the engineering discipline I've built for using AI coding agents to ship real software -- production stuff, not throwaway demos. I first presented it as a talk at the AI Engineer Summit: The Cure for the Vibe Coding Hangover. The seminal essay is forthcoming -- I'm working through it now -- but the talk is the public version for the time being.

Three components, all interlocking:

Principles -- the philosophy. You are the architect, the agent is the implementer. Specify, don't prompt. Define done before building. Reduce until irreducible. Ten of these in total, each a pithy aphorism that makes "the gospel" of it easy to remember.
Process -- two phases that translate the philosophy into work. The Planning Phase is where you do the architectural thinking: vision, features, specifications, dependencies, plan. The Implementation Loop is where the agent takes those specifications and turns them into working code. Planning produces artifacts; the loop spins on them.
Tools -- a coding environment, version control, context engineering, and the Multi-Sensory Feedback System. The agent observes its own work through three "digital senses" -- Visual (what renders), Auditory (what the logs report), Tactile (how the thing responds when you actually use it). Tests give the agent a pass/fail verdict; the senses tell the agent why something is or isn't working. Done means tests pass AND all senses report clean. Both, not either.

🍸 Sound Smart at Cocktail Parties: "The Multi-Sensory Feedback System closes the diagnostic loop between formal verification and qualitative observation."

That's The Framework, and it works masterfully when building new software. It works just as well when applying it to existing software that wasn't developed using the methodology, with some adaptation.

Right, so what did that adaptation actually look like?

The Framework is built for writing new software from scratch -- but this wasn't that. I was parachuting into someone else's Rust codebase without a map, without knowledge of the language, and chasing a bug that only shows its face after many hours of use. This was classic hard-bug territory, and the agents couldn't just write some tests and check the senses -- there were no "memory leaks" tests, no observability, and you can't curl your way into a voice transcription cycle.

1. The autonomy harness

My AI agents had to be able to run the app. Handy is a desktop app triggered by a global keyboard shortcut -- without a way for the agents to spin it up, inject audio, trigger recording cycles, and measure what happened, all autonomously, every experiment would have needed me to sit there mashing keys for 30-minute stretches, doing nothing else. That's very costly digital babysitting. So I built the agents their own little sandbox:

a dedicated Xvfb display so they weren't fighting me for screen real estate
a PipeWire null sink piping deterministic test audio through paplay so they could actually speak into Handy's microphone (that's right -- my AI agents speak into my microphone ... do yours not?)
a SIGUSR2 trigger to start recordings without spawning a second Handy instance and polluting the trace

All of this plus full reversibility: snapshot the system, install the tooling, do the work, diff-based uninstall, verify the system's back to baseline. Tear the whole rig down with no trace afterward.

flowchart LR
    subgraph WS["My workstation"]
        Me["Me, working normally<br/>(unaffected)"]
    end

    Agent(("AI agent"))

    subgraph Sandbox["Agent's sandbox (isolated)"]
        direction TB
        Audio["test_audio.wav<br/>→ paplay<br/>→ PipeWire null sink"]
        Trigger["SIGUSR2 trigger"]
        Handy["Instrumented Handy<br /> (see below)"]
        Display["Xvfb virtual display"]
        Audio -->|mic input| Handy
        Trigger -->|start/stop recording| Handy
        Handy --> Display
    end

    Agent -->|drives| Sandbox

The autonomy harness: agent drives Handy end-to-end in an isolated sandbox; my workstation is untouched.

This was massive because without it, the Implementation Loop simply couldn't spin -- the agents had nowhere to put their hands on the app and actually use Handy.

2. The instrumentation

The agents also had to be able to see inside the app. A memory leak in a Tauri application is a multi-process, multi-runtime problem -- the Rust process, the WebKit subprocess, the IPC layer stitching them together -- and no single observation tool sees all of it. You can stare at the Rust files all day and learn nothing; the bug only exists at runtime, while the thing is running, across processes. So I wired Handy up to four observation tools, all recording simultaneously:

uftrace for every Rust function call
strace for every syscall and IPC payload
the WebKit Remote Inspector for the JavaScript heap
and a smaps_rollup sampler for memory usage per process, per memory category

flowchart TB
    subgraph Handy["Handy at runtime"]
        direction LR
        Rust["Rust process"]
        IPC["Tauri IPC layer"]
        WebKit["WebKit subprocess"]
        Memory["Process memory<br/>(by category)"]
    end

    Rust -.observed by.-> U["<b>uftrace</b><br/>every Rust function call"]
    IPC -.observed by.-> S["<b>strace</b><br/>syscalls + IPC payloads"]
    WebKit -.observed by.-> I["<b>WebKit Inspector</b><br/>JavaScript heap"]
    Memory -.observed by.-> SM["<b>smaps_rollup sampler</b><br/>memory by category"]

Four tools, one per layer of Handy's execution surface. None of them sees the whole picture alone.

Four tools, synchronized. Each one sees what the others can't; together, they cover the entire execution surface where allocations could come from. This is The Framework's Multi-Sensory Feedback System adapted for chasing memory leaks in a complex desktop app -- Visual, Auditory, and Tactile senses, instantiated for runtime memory analysis instead of UI rendering.

This was also massive: the Implementation Loop requires that AI agents have digital senses to successfully observe the target -- runtime memory footprint, in this case -- and close the feedback loop. As explained in The Framework, closing the loop is critical to success.

3. Narrowed systematically until only one candidate remained

Here's where the engineering discipline goes full on. Most "AI debugging" starts with a hypothesis and tries to confirm it -- which is just expensive vibe coding with extra steps. I started with the data.

We had eight candidate mechanisms that could explain what we were seeing; five iterative measurements later, we had one.
The smaps sampler localized growth to a specific WebKit subprocess (the recording overlay).
The WebKit heap snapshot diff ruled out JavaScript -- 1.2% of the growth, a rounding error.
The smaps category breakdown ruled out shared libraries -- 100% of growth in private-anonymous pages, i.e., C++ allocations.
uftrace and strace identified the upstream driver: emit_levels firing at 24 Hz, dual-broadcasting events to a hidden overlay subprocess the user never even sees.

This was a constraint-satisfaction problem reduced to one surviving candidate -- and by the time it was, the only intervention left was cutting the event source.

4. Fixed it three ways, tested each piece on its own and together

The fix had three candidate components, each wired to a runtime switch -- an environment variable I could flip on or off without rebuilding the app. Then I ran the same memory test against several configurations: all switches off (does the leak still reproduce? yes), each switch on by itself (what does that one piece contribute?), and all switches on (combined effect). The numbers per configuration told me which pieces were doing work and which weren't pulling their share. One of the three -- throttling the event rate from ~24 Hz to 20 Hz -- had no measurable effect and was decidedly cut from the PR. The other two carried the fix. This was impossible to know without the measurements.

"What were the actual fixes for the bloody memory leak?" he asked, brows furrowed with frustration and anticipation.

Handy has a cool voice visualizer feature (off by default) that bounces about to show microphone input activity. The memory leak lives in this, though it's not as straightforward as it sounds, as the path from mic to visualizer runs through the previously mentioned multi-runtime execution. The actual memory leak lives upstream of Handy in WebKit -- specifically in the recording overlay's WebKit subprocess -- not in Handy's codebase, which means we couldn't fix it directly so we had to be a bit clever in how we mitigated the leak in Handy itself by starving the upstream of the event traffic that drives the leak.

Don't send voice visualizer data when the overlay is turned off. This turned out to be the biggest contributor to the memory leak and, as the overlay is off by default, this fix helps most users. Even when it's off, there's a hidden WebKit subprocess that is still alive, still listening and still leaking (😭).

Cache the overlay-enabled state in an AtomicBool, check it at the top of emit_levels, return early if the overlay is off. For most Linux users, the event source just goes silent. The leak source is starved.

Narrow the way that voice visualizer data are sent when the overlay is turned on. The original implementation was 2 global broadcasts of microphone level events, sent to every webview, twice per audio callback. This was replaced with a targeted emission directly to the overlay which halved the WebKit dispatch work per callback.

It's a quirk of Tauri 2: both AppHandle::emit and WebviewWindow::emit are global broadcasts. Replace the dual call with a single app.emit_to("recording_overlay", "mic-level", levels)

Before the fix:

flowchart LR
    Mic["microphone input<br/>(audio callback ~24 Hz)"] --> EL["emit_levels"]
    EL --> AHE["app_handle.emit<br/>(global broadcast)"]
    EL --> OWE["overlay_window.emit<br/>(also a global broadcast)"]
    AHE --> Overlay["recording_overlay<br/>WebKit subprocess<br/>(hidden when overlay off,<br/>but still alive and listening)"]
    OWE --> Overlay
    AHE --> Others["other webviews"]
    OWE --> Others
    Overlay --> Leak["💥 WebKit C++ allocations<br/>(unbounded growth)"]

After the fix:

flowchart LR
    Mic["microphone input<br/>(audio callback ~24 Hz)"] --> EL["emit_levels"]
    EL --> Check{"AtomicBool check:<br/>is overlay enabled?"}
    Check -->|"no (most Linux users)"| Drop["return early<br/>(no emit, no leak)"]
    Check -->|"yes"| ET["app.emit_to<br/>(recording_overlay only)"]
    ET --> Overlay["recording_overlay<br/>WebKit subprocess"]
    Overlay --> Bounded["✅ WebKit C++ allocations<br/>(bounded)"]

Before: events fanned out to every webview, twice per audio callback, with the hidden overlay still receiving and leaking. After: events are gated on overlay state, and when they do fire, they go straight to the overlay only.

5. Three rounds of adversarial review

Even with all of the good engineering work done thus far, before a single claim went into the PR description, agents were explicitly tasked to try to break the analysis. Are you actually measuring what you think you're measuring? Is that a sustained leak or just warmup? Does the data distinguish your hypothesis from alternatives?

This three-stage adversarial review puts the artifact through:

a claim check (every assertion classified by evidence type: measured, verified, inferred, assumed),
then three reviewer subagents role-playing the audience (the Handy maintainer, a Tauri developer, a WebKit memory expert) with no investigation context,
then two cold reviewers with zero memory of prior drafts.

Agents in each round have a clear objective: tear it apart. Find anything wrong, misleading, or that wastes a maintainer's time. The PR doesn't bullshit because it's been stress-tested by independent agents whose job was to find the bullshit. This was particularly important as I didn't have language familiarity to fallback on as I reviewed the PR.

6. The meatbag-in-the-loop soak

Controlled captures are great for proving things in a lab. But the fix had to run on my machine, in production, with days of normal use, before I'd call it done. Memory leaks accumulate over hours and days, not minutes -- and the bug that initially took me down had occurred after 36 hours of continuous use. So I ran the patched build for myself, in actual daily work, for six days straight (well, precisely, it was 5 days, 18.5 hours) across seven separate launches, while a tiny shell script polled every 30 seconds and dumped per-process memory readings into a CSV. The data shows the fix worked. And honestly, the more important validation: I -- as a human, daily user of this tool -- am writing this very article using the patched build right now. If it had broken something, I'd have noticed, and I'd have the data to identify exactly what had gone wrong. Buuuuuuuuuuut it's working brilliantly, due to the application of the engineering discipline prescribed in The Framework!

What's next, then?

With all of this real software engineering bolstering it, I submitted the pull request to fix the memory leak. Now we play the waiting game ...

You, too, can be an AI agents ninja!

Are you struggling with bugs in daily driver open source tools that you love to use? Lift this approach. Fix them. Tell me how it went!

I literally had my AI agents write-up the methodology and instrumentation so that yours could learn and apply them. Give those to them, along with this article, and let them rip.

Made it this far? Well done, you. Now go and transcend vibe slop with all you've read!

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. A glowing microphone on a dark workbench, wired into a translucent holographic diagram of a multi-process application — three stacked subsystems (a Rust core, an IPC bridge, and a WebKit subprocess) rendered as luminous geometric containers. From the WebKit container, glowing memory blocks balloon outward in an unbounded cascade, depicting a runaway memory leak. Surrounding the diagram, four floating instrumentation panels emit thin neon beams of light into each subsystem, like sensory probes converging on the leak. In the foreground, a severed neon data stream shows the leak being starved at its source, with bounded, contained allocation blocks on the opposite side. The scene sits inside a darkened engineering workshop with circuit-trace patterns etched into the walls, volumetric haze, and reflective surfaces catching the neon glow.

Unlocking Codex GPT-5.4's Full 1M-Token Context Window

2026-05-25T00:00:00-05:00

Update (July 2026): as of Codex 0.144.4, this technique no longer works -- the model_context_window override is now ignored, and context ceilings are enforced server-side. See the full update at the end of this post.

GPT-5.4 supports a million-token context window but Codex launches it at 272,000.

Codex doesn't ask GPT-5.4 how many tokens it can handle. It looks up the answer in a model catalog -- a JSON file that ships with Codex listing every available model alongside its capabilities: name, context window, supported features. The default catalog sets GPT-5.4's context_window to 272,000. As far as Codex is concerned, that's the ceiling.

Codex profiles let you create named configurations -- a different model, different sandbox rules, different settings -- and switch between them with --profile name. Each profile lives in its own file at ~/.codex/<name>.config.toml, using the same top-level keys as ~/.codex/config.toml. Anything you set in a profile file overrides the matching value in config.toml for sessions launched under that profile.

That includes model_context_window.

The profile that unlocks 1M

Create ~/.codex/gpt54_1m.config.toml:

model = "gpt-5.4"
model_context_window = 1000000
model_auto_compact_token_limit = 900000

Three lines. model_context_window tells Codex to use a 1M-token window for sessions launched under this profile. model_auto_compact_token_limit triggers conversation compaction at 900K, leaving headroom before the effective ceiling.

Launch with:

codex --profile gpt54_1m

Verifying it works

Inside the Codex TUI, /status shows the context window size. For a definitive check, inspect the rollout JSON from your session:

latest_rollout=$(
  find ~/.codex/sessions -name 'rollout-*.jsonl' -printf '%T@ %p\n' \
    | sort --numeric-sort \
    | tail --lines=1 \
    | cut --delimiter=' ' --fields=2-
)

jq -r '
  select(.type == "event_msg" and .payload.type == "task_started")
  | .payload.model_context_window
' "$latest_rollout"

This returns 950,000. GPT-5.4 applies a 95% effective context window internally, so the runtime ceiling is 950,000 of the catalog's 1,000,000.

To prove the window is actually usable end-to-end -- not just declared in metadata -- pipe several hundred thousand tokens of input to a single-turn codex exec and check that the rollout records the full input acceptance:

codex exec --model gpt-5.4 -c 'model_context_window=1000000' \
  --skip-git-repo-check 'Reply with only the literal word RECEIVED.' \
  < big_input_file

On my Plus account, this accepted a single turn with 591,580 input tokens -- well above the 272,000 default ceiling.

Why this works now (and didn't before)

Until Codex 0.134.0, profiles lived inside ~/.codex/config.toml as [profiles.X] tables -- and that table parser silently dropped model_context_window (openai/codex#14456). Setting the key inside [profiles.X] did nothing. The 0.134.0 release moved profiles into their own files at ~/.codex/<name>.config.toml, using top-level keys where the override was never broken. The bug fix was incidental to the file-shape change.

What about GPT-5.5?

GPT-5.5 has the same shape of gap: OpenAI's API docs document a 1,050,000-token context window, but Codex's catalog still ships it at 272,000.

The same simple override does not work for GPT-5.5 on the Plus tier (internally labeled prolite in Codex's rate-limit telemetry). I verified this by stress-testing both models on the same account, back to back:

GPT-5.4 with model_context_window = 1000000: a single-turn session accepted 591,580 input tokens.
GPT-5.5 with an analogous setup: real working sessions hit "Codex ran out of room in the model's context window" three times in a row, each time when a single turn's input crossed ~270,000 tokens. Codex's /status reported a 998K window throughout; the OpenAI API rejected the requests anyway.

So the override-tells-Codex layer worked for both models, but only the GPT-5.4 API actually honored it for Plus. The catalog patch convinces Codex it has more headroom; the server still enforces the plan-tier ceiling for GPT-5.5.

I haven't tested this on Pro / Enterprise / API access, so I can't say whether the limit relaxes there. If you're on a higher tier and want to find out, the test is: stress-test a GPT-5.5 1M profile with several hundred thousand tokens of single-turn input and read the rollout's last_token_usage.input_tokens. If it succeeds, the gate is plan-tier; if it fails the same way, the cap is universal on Codex.

Update -- July 2026 (Codex 0.144.4): this technique no longer works

Since I wrote this, Codex changed how it resolves context windows, and the override above stopped doing anything. I re-tested the whole thing on the same subscription account against Codex 0.144.4, in an isolated config directory so nothing touched my live setup. Three things changed.

1. The `model_context_window` override is ignored

Whether you set it in a profile file (codex --profile gpt54_1m) or inline (codex exec -c 'model_context_window=1000000'), the key no longer changes anything. A session launched with the 1M profile still reports a 258,400-token window -- 95% of the 272,000 catalog default. The setting parses without error and has no effect.

2. The catalog now drives only the displayed window -- not the enforced one

Codex 0.144.4 reads the window from the model catalog (~/.codex/models_cache.json), and editing an entry's context_window does change the number Codex shows you. But that number is cosmetic. I patched GPT-5.5's catalog entry down to 100,000 -- Codex then displayed a 95,000-token window -- and a 132,606-token input still went through. I patched it up to 1,000,000 -- displayed 950,000 -- and inputs above ~272,000 tokens were still rejected client-side with "ran out of room in the model's context window."

The real limit is enforced server-side, per model and plan tier, and no client-side edit moves it in either direction. (Codex also re-fetches the catalog from the server on a timer, silently overwriting your edit unless you make the file read-only.)

3. There's now a hard 1 MiB cap on a single input message

Independent of tokens, Codex rejects any single message over 1,048,576 characters with input_too_large, before it ever checks the context window. The original end-to-end test in this post -- piping a 591,580-token file to one codex exec -- can't even run today: that much text is well over 1 MiB.

What the windows actually are now

Measured by feeding single-turn inputs of increasing size and reading the accepted input_tokens straight from each session's rollout JSON:

GPT-5.5 -- accepts 268,606 tokens; the next step up is rejected. Usable ceiling ~272,000.
GPT-5.6 (it ships as three variants -- sol, terra, luna) -- sol accepted 356,472 tokens and rejected around 392K. Its usable window is clearly larger than 5.5's, but still short of 400K.
GPT-5.4 -- accepted 291,221 tokens (I didn't push it to its ceiling). It's the only model whose catalog still carries max_context_window: 1000000.

Every one of these reported the same cosmetic 258,400-token window regardless of the model.

The practical upshot inverts the original premise. You can no longer unlock a bigger window by editing config -- the number you can change is the one that doesn't matter. But for the newer models you don't need to: GPT-5.6 already grants more usable context than GPT-5.5 by default. GPT-5.5 is the outlier, capped lowest of the three -- the same regression subscription users flagged in openai/codex#19464, where 5.5 gives less long-context headroom than 5.4 did.

As before, this is one ChatGPT-subscription account. On Pro / Enterprise / API the server-enforced ceilings may well be higher -- I haven't tested them.

Getting Handy to work in GNOME Wayland (Ubuntu 25.10+)

2026-05-15T00:00:00-05:00

Background & what we’re trying to achieve

Handy (CJ Pais's Tauri 2.x speech-to-text desktop app, v0.8.3 Linux AppImage) is the chosen dictation tool. The required workflow:

Press a global keyboard shortcut to start recording.
Speak.
Press the shortcut again (toggle mode) to stop. Handy transcribes locally via Whisper.
The transcribed text appears in the focused field of whichever app the cursor was in (terminal, GUI text editor, browser text field, Electron app -- universal).

On Ubuntu 25.10 / GNOME 49 / Wayland, neither of those steps works out of the box. The keyboard shortcut only fires when Handy's window itself is focused, and Handy's default text-delivery mechanism fails because Mutter doesn't implement the protocol it depends on. Both problems are solvable from outside Handy with no upstream patch. Here’s how.

The solution overview

The dictation workflow has two distinct failure points on GNOME Wayland, each requiring a "go-around" fix. Neither can be addressed by changing Handy's settings alone. Both fixes work by replacing an in-process mechanism that GNOME Wayland refuses to support with an external mechanism that the system already permits.

Failure	Why it fails on GNOME Wayland	The go-around
Global hotkey doesn't fire when Handy is unfocused	Tauri's `global-shortcut` plugin grabs keys via Wayland's keyboard listener, which by design only delivers to the focused surface. So Tauri's "global" shortcut is in fact window-local.	A GNOME custom keybinding at the WM level. GNOME's keybinding handler runs in `gnome-shell` and fires regardless of which client has focus. It invokes Handy's CLI which IPCs the running daemon.
Transcribed text isn't typed/pasted into the focused window	Handy's typing tool defaults route through `wtype`, which needs the `zwp_virtual_keyboard_v1` Wayland protocol. Mutter (GNOME’s window manager & compositor) intentionally does not implement it (it's a wlroots-only protocol). Every paste method that simulates a keystroke through wtype fails.	An External Script paste method invoking `ydotool type`, which synthesizes input via `/dev/uinput`. uinput is kernel-level -- the kernel forges input events that propagate through libinput to whichever app is focused, with no need for any Wayland protocol Mutter is missing.

See the appendix for why simpler fixes don’t work.

Handy's UI on this system shows only Auto (Recommended) and wtype in the Typing Tool dropdown. The v0.8.3 source code at src-tauri/src/settings.rs actually defines the full enum TypingTool { Auto, Wtype, Kwtype, Dotool, Ydotool, Xdotool }. Handy's UI hides options whose backing binary isn't on PATH. After apt install ydotool, Ydotool does appear in the dropdown -- but it expects 1.x semantics, so it doesn't actually work with the apt version. That's why we use External Script instead of flipping the dropdown.

Implementing the fixes

NB — you’ll want to search and replace “\<your username>” with your actual username.

Problem 1: global hotkey delivery

Step 1: Bind a GNOME custom keyboard shortcut (with absolute path)

GNOME Settings -> Keyboard -> Custom Shortcuts, or via gsettings:

# Inspect current custom-keybindings list first:
gsettings get org.gnome.settings-daemon.plugins.media-keys custom-keybindings
# Add an entry path if needed:
gsettings set org.gnome.settings-daemon.plugins.media-keys custom-keybindings \\
  "['/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/custom1/']"

gsettings set org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/custom1/ name 'Handy Transcription'
gsettings set org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/custom1/ command '/home/<your username>/.local/bin/handy --toggle-transcription'
gsettings set org.gnome.settings-daemon.plugins.media-keys.custom-keybinding:/org/gnome/settings-daemon/plugins/media-keys/custom-keybindings/custom1/ binding '<Control>space'

Critical detail: the Command field must be an absolute path. GNOME's keybinding handler runs with the session PATH set at graphical login from ~/.config/environment.d/, not from ~/.zshrc. On Ubuntu 25.10 that PATH is /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin -- no ~/.local/bin. A bare handy --toggle-transcription silently fails to resolve. (Long-term cleaner fix: add PATH=...:%h/.local/bin to ~/.config/environment.d/path.conf, which makes the keybinding handler and any future custom shortcuts find user-local binaries automatically. Optional.)

Step 2: Ensure Handy daemon is running

handy --toggle-transcription is a DBus IPC to the running Handy daemon. With no daemon running, the command exits 0 with no effect (a particularly silent failure mode). Handy creates an autostart entry at ~/.config/autostart/Handy.desktop automatically when autostart_enabled: true is set in its settings. Edit that file's Exec= line to add --start-hidden if you don't want Handy's window to pop visibly on every login:

Exec=/home/<your username>/.local/bin/handy --start-hidden

To start the daemon for the current session without rebooting:

nohup /home/<your username>/.local/bin/handy --start-hidden >/tmp/handy-startup.log 2>&1 &
disown

Problem 2: text delivery via ydotool

Step 3: Install ydotool and grant uinput access

sudo apt install -y ydotool
echo 'KERNEL=="uinput", GROUP="input", MODE="0660", OPTIONS+="static_node=uinput"' \\
  | sudo tee /etc/udev/rules.d/60-uinput.rules
echo uinput | sudo tee /etc/modules-load.d/uinput.conf
sudo modprobe uinput && sudo udevadm control --reload-rules && sudo udevadm trigger

What each piece does:

apt install ydotool -- installs ydotool 0.1.8-3build1 (the legacy monolithic version, no separate daemon; talks to /dev/uinput directly).
The udev rule -- when the kernel creates the uinput device, set its group to input and mode to 0660 so members of the input group can write to it without sudo. The OPTIONS+="static_node=uinput" clause makes systemd-udev pre-create the node with the right permissions even before any uevent fires (relevant because uinput is module-loaded, not auto-discovered).
/etc/modules-load.d/uinput.conf -- causes systemd-modules-load.service to modprobe uinput at boot. Without this, /dev/uinput won't exist after reboot until something else loads the module.
The modprobe + udevadm chain -- applies everything immediately without a reboot: load the module now, reload the rules, replay uevents so the new rule's permissions take effect on the just-created node.

Confirm the user is in the input group (groups should list it; was already true on this system as Ubuntu 25.10 default).

Step 4: Write the External Script

Save as /home/<your username>/.local/bin/handy-paste-wl-copy (name retained for historical reasons -- this script no longer uses wl-copy):

#!/usr/bin/env bash
# Handy External Script: type transcribed text directly into the focused window
# via ydotool's `type` subcommand. At --key-delay 0 this is faster than paste
# for typical dictation lengths and works universally (terminals, GUI editors,
# Notion, etc.) without depending on each app's paste-shortcut convention.

set -u

TEXT="${1:-}"
[ -z "${TEXT}" ] && exit 0

env -i \\
    HOME="${HOME}" \\
    USER="${USER:-<your username>}" \\
    LOGNAME="${LOGNAME:-${USER:-<your username>}}" \\
    PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \\
    LANG="${LANG:-en_US.UTF-8}" \\
    /usr/bin/ydotool type --key-delay 0 "${TEXT}" >/dev/null 2>&1

exit 0

Make it executable: chmod +x /home/<your username>/.local/bin/handy-paste-wl-copy.

Key design choices:

env -i sanitization. Handy's AppImage runtime prepends its own bundled lib/bin paths to LD_LIBRARY_PATH, PYTHONHOME, PYTHONPATH, GTK_PATH, etc. for the AppImage process and its children. Without scrubbing, the system /usr/bin/ydotool would attempt to dynamic-link against the AppImage's bundled libs, which can fail silently or behave erratically. env -i clears everything and we explicitly pass only what ydotool needs (PATH, HOME, USER, LOGNAME, LANG). Note that WAYLAND_DISPLAY and XDG_RUNTIME_DIR are not needed by ydotool because uinput is kernel-level, not a Wayland client.
ydotool type with -key-delay 0. Tested empirically: 225 chars completes in \~280ms with default -delay 100 initial wait. The default key-delay (12ms) would make this 2.8s -- noticeable. Zero delay is reliable on this system for typical dictation lengths.
No clipboard touch. Unlike a clipboard + paste-shortcut pattern, this leaves your clipboard contents alone. Earlier versions of the script wrote via wl-copy and dispatched Ctrl+Shift+V via ydotool; that approach worked in Ghostty but failed in GNOME Text Editor (which doesn't accept Ctrl+Shift+V as paste). Switching to ydotool type removed the app-by-app paste-shortcut compatibility problem entirely.

Step 5: Configure Handy

In Handy's settings (Advanced tab):

Paste Method: External Script
External Script Path: /home/<your username>/.local/bin/handy-paste-wl-copy
Clipboard Handling: Don't Modify (cosmetic; our script doesn't touch the clipboard, so this preference is honored implicitly).

These map to fields in ~/.local/share/com.pais.handy/settings_store.json:

"paste_method": "external_script",
"external_script_path": "/home/<your username>/.local/bin/handy-paste-wl-copy",
"clipboard_handling": "dont_modify",

Verification

Confirm each link in the chain works.

ydotool can drive /dev/uinput:

ls -la /dev/uinput
# Expected: crw-rw---- root input ... (group input, mode 660)
test -w /dev/uinput && echo "user can write" || echo "DENIED"
ydotool key shift 2>&1
# Should print "ydotoold backend unavailable" (informational, not an error) and exit 0.

ydotool type delivers fast enough:

LONG=$(printf 'the quick brown fox jumps over the lazy dog. %.0s' {1..5})
echo "Length: ${#LONG}"
time ydotool type --key-delay 0 "$LONG"
# Expected: ~250-350ms wall-clock for ~225 chars. Output should land in the focused window with no dropped characters.

Handy delegates to the script:

After triggering a transcription, the Handy log at ~/.local/share/com.pais.handy/logs/handy.log should show:

[handy_app_lib::clipboard][INFO] Using paste method: ExternalScript, delay: 60ms
[handy_app_lib::clipboard][INFO] Pasting via external script: /home/<your username>/.local/bin/handy-paste-wl-copy
[handy_app_lib::actions][DEBUG] Text pasted successfully in Nms

If the "Text pasted successfully" line shows a duration much greater than \~50ms, the script is hanging -- typically because a subprocess is daemonizing without closing inherited file descriptors. (Earlier wl-copy-based versions of the script hit this; ydotool type doesn't, but it's the canary to watch for.)

Troubleshooting / failure modes

Pressing the shortcut does nothing. Check gsettings get .../custom1/ command shows an absolute path. A bare handy --toggle-transcription won't resolve from gnome-shell's PATH.
Pressing the shortcut transcribes but no text appears. Tail ~/.local/share/com.pais.handy/logs/handy.log while testing. Look for the "Pasting via external script" line. If absent, Handy isn't invoking the script -- check Paste Method = External Script and the path field. If present but no text appears, run the script manually with a test argument to verify it works in isolation:
Should type "manual-test" into whatever's currently focused.

/home/<your username>/.local/bin/handy-paste-wl-copy "manual-test"

Wrong characters appear (e.g. "2442" instead of text). Means ydotool got the wrong syntax (likely a confusion between 0.1.8 key-name and 1.x keycode:state). Confirm the script uses ydotool type (not ydotool key 29:1 47:1 ...).
Garbled characters with non-US keyboard layout. ydotool 0.1.8's type assumes US QWERTY. Non-US layouts mistranslate. Realistic fixes: switch to dotool (layout-aware; build from sr.ht source -- not in apt) or pursue the RemoteDesktop portal path (Handy upstream PR #689).
Handy is ready notification but the window doesn't focus when you click Settings in the tray menu. Separate problem -- Handy's Tauri tray icon doesn't expose the standard .Activate / .ContextMenu D-Bus methods, only .Scroll and .SecondaryActivate. This is a Tauri/dbusmenu activation-token gap, not a GNOME limitation. Workarounds: Alt+Tab to Handy, pin to dock, or middle-click the tray icon (which fires .SecondaryActivate). Upstream fix would need to come from Tauri.

Appendix: Why simpler fixes don't work

Here’s what didn’t work, and why.

Path	Verdict	Reason
Just use Tauri's in-app shortcut	Half-dead	Only fires while Handy is focused -- defeats the purpose of dictation
GNOME custom shortcut `handy --toggle-transcription` (bare name)	Dead	`~/.local/bin` is NOT on `gnome-shell`'s session PATH; the bare command silently fails to resolve. Absolute path required.
GNOME custom shortcut without a running Handy daemon	Dead	`handy --toggle-transcription` is an IPC to the running daemon; without daemon, exits 0 with no effect. Daemon must be autostarted.
`Paste Method = Direct` (Handy default) on GNOME	Dead	Routes through wtype -> fails on Mutter
`Paste Method = Clipboard (Ctrl+V)`	Dead	Writes to clipboard then sends Ctrl+V via wtype -> same Mutter failure. Worse: Handy's error-path tears down its wl-copy child process, so the clipboard ends up empty anyway.
`QT_QPA_PLATFORM=xcb` to force XWayland	Dead	Flatpak/AppImage's bundled Qt runtime lacks `libxcb-cursor`; DISPLAY isn't forwarded into the sandbox
Built-in `Typing Tool = Ydotool` with apt's ydotool 0.1.8	Dead	Handy's dispatch expects ydotool 1.x (client + daemon, `keycode:state` syntax). Apt ships 0.1.8 (monolithic, no daemon, key-name syntax). Version mismatch -- the wrong syntax gets used and "2442" gets typed instead of Ctrl+V.
`ydotool key 29:1 47:1 47:0 29:0` (1.x syntax)	Dead	0.1.8 parses each token as an unknown key name, falls back to typing the first digit of each. "2442" is the literal evidence.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Depict a glowing soundwave entering from the left and transforming into a stream of luminous text characters flowing toward a focused input field on the right. Between them, two heavily fortified barrier walls labeled with subtle compositor/protocol motifs block the direct path -- each wall stamped with a glowing "access denied" hexagon. Around each wall, a bright neon conduit reroutes the signal in a smooth go-around arc: the first detour loops up through a system-level keyboard sigil etched into the architecture, the second tunnels down through a deep kernel-level channel that emerges past the barrier. The two reroute paths converge and deliver the glowing text into the waiting field, which lights up to confirm delivery. Render everything as sleek translucent circuitry and data conduits against a dark high-tech backdrop, with volumetric neon glow, fine grid lines, and a sense of signal flowing around obstacles rather than through them.

Is HTML "Strictly Better" Than Markdown for Claude Code?

2026-05-11T00:00:00-05:00

Thariq Shihipar of Claude Code team fame posted an article last week, "The Unreasonable Effectiveness of HTML", arguing that Claude should output HTML files by default for basically everything. PR reviews, postmortems, status reports, the lot. It's a good piece, racked up 8.3M views, and most of it I agree with. But in the replies, when somebody pushed back to say markdown's fine for text-heavy stuff, Thariq held the line: "idk I think HTML is strictly better for all of that too." So I ran it all. Three of his use cases, instrumented end to end on Opus 4.7. What follows is what I measured, what surprised me, and where I ended up agreeing and disagreeing. Short version: he's right about half of it, and that half is very right. The other half I don't think holds up, and the data here is why.

TL;DR

Thariq's piece is good. Read it before you read this.

Visual tasks (design exploration, prototypes, drag-and-drop widgets): HTML wins outright. Markdown can describe a design; HTML can show one. The 2-4x token cost is the price of admission, not a tradeoff.
Text-with-structure tasks (PR reviews, technical explainers): both formats produce a real artifact. HTML adds polish for about 1.4-1.5x the dollar cost. In one of my two text cases (the rate-limiter explainer) the markdown surfaced more gotchas (12 vs 8) at lower cost. Which cuts against "strictly better."
Does Claude read HTML better than markdown? (the question Thariq's argument quietly assumes a HTML-favorable answer to): two blinded judges scored Claude 7/7 on factual accuracy from both formats, identically. On explanatory specificity HTML got a ~7% edge, at 33% higher cost per call. Pick your poison.
The "1M context handles it" hand-wave: at N=50 artifacts re-ingested K=3 times with cache evictions, HTML's overhead eats roughly the whole 1M window. Not nothing, but depending on the work being done maybe worth it.

Sample is small (N=1 per condition for content quality, three instrumented cases) and the methodology I used actually biases against markdown. Even with the thumb on the scale, "strictly better" isn't what I saw.

What Thariq actually said

He walks through five buckets of tasks where he reckons HTML should be the default. Roughly:

Specs, planning, exploration (e.g. design exploration)
Code review & understanding (e.g. PR review with annotated diff)
Design & prototypes (e.g. an interactive checkout button)
Reports, research, learning (e.g. a rate limiter explainer)
Custom editing interfaces (e.g. drag-and-drop Linear ticket reorderer)

He flags three costs honestly: more tokens, slower generation, noisier diffs. He concludes the advantages outweigh them. Reasonable people can disagree on the conclusion.

What I want to look at is the reply to the comment. "Strictly better" is a specific, testable claim. That's the one I want to examine.

There are actually two questions here

The article treats format choice as a knob: pay a bit extra in tokens, get nicer output, decide if it's worth it. That framing works for half the use cases and quietly stops working for the other half. Here's the split.

Sometimes HTML can do something markdown structurally can't. Render a six-mockup grid of UI designs. Run a slider that updates a preview in real time. Drag a card across a kanban. Markdown isn't a worse version of these; there is no markdown version. Asking "is the extra cost worth it?" is like asking whether the airfare to Mars is good value when the alternative is staying home and looking at a globe.

Other times both formats can do the job, and HTML just costs more. PR reviews. Postmortems. Technical explainers. Implementation plans. Markdown handles all of them. HTML renders them more attractively. The substance lives in either.

The "use HTML for everything" framing treats both questions like the second one. The case for HTML gets stronger, not weaker, when you split them apart.

How I ran this

I picked three of Thariq's five use cases and instrumented them properly. Same prompt, both formats, capture every token. The other two are the categorical ones from the previous section; there's nothing fair to compare them against, so I just demo them at the end.

The three I instrumented: 1. Design exploration: onboarding screen, six approaches in a grid (his specs/planning bucket) 2. PR review of a real commit: toks commit c6d70f9, "Respect .gitignore when target has no .git directory" 3. Rate limiter explainer over the slowapi source: about 12K tokens of Python as substrate

The two I just demo: 4. Checkout button prototype (interactive sliders, animation, copy-as-prompt) 5. Linear ticket reorderer (drag-and-drop kanban with copy-as-markdown export)

For each instrumented case I ran three probes: generate the artifact, feed it back into a fresh agent session as context for a downstream task, and apply a semantic edit while capturing the diff. The first probe is the obvious one. The other two matter because that's where the cumulative token costs actually bite. Re-ingestion and editing happen constantly in real Claude Code workflows.

On the design exploration case I ran two methodologies side-by-side: Thariq's prompt unchanged versus a neutrally-phrased version of the same task. Why? Because his prompt literally says "create an HTML file." If you only do a "HTML" → "markdown" word-swap, you're asking Claude to make a markdown file in the shape of an HTML file, which is not the same as asking for markdown. Tracking? The difference between the two methodologies turned out to be larger than I expected: a 3.6x cost ratio under ablation, 2.1x under neutral phrasing. Worth being honest about.

What "artifact tokens" means below. I ran toks <file> --for claude on each generated artifact, using Anthropic's own tokenizer to count the file the way Claude would see it if fed back as context. That number is different from usage.output_tokens (what the API reported as the generation length) and from cache_creation_input_tokens (artifact + system prompt + user message, all of it). The first matters for re-ingestion cost. The other two matter for other things.

Where this is honestly a bit thin: - My reader-side reactions are N=1. One user (me), one session, I knew which file was which. No blinding, no randomization. Take them as gut reactions, not data. - Generation runs are also N=1 per condition. Content-quality findings like "markdown surfaced more gotchas" might be variance, not signal. K=5 per condition would tell us. I didn't run K=5. - Three instrumented cases is a sample, not a census. - The agent-as-reader test below covers shared-content questions only, i.e. facts present in both artifacts. I didn't test whether HTML's verbose-priming or callout-salience changes output quality (beyond binary accuracy) in ways that matter downstream. - I didn't test "colleagues actually read HTML more." That'd need users I don't have. - Methodology asymmetry is load-bearing, and I'm flagging it here so the reader knows the thumb is on the scale before the data tables start. I ran two methodologies only on the design exploration case. The PR review and rate-limiter cases used Thariq's verbatim prompt structure, which is HTML-affording ("create an HTML artifact..."). That biases against the markdown artifact compared to a neutrally-phrased prompt. The cost ratios I report for those cases (1.51x for PR review, 1.44x for rate limiter) would likely shrink under a neutral prompt; how much, I didn't measure.

Design exploration: this is where HTML earns its keep

Thariq's prompt (verbatim): "I'm not sure what direction to take the onboarding screen. Generate 6 distinctly different approaches, varying layout, tone, and density, and lay them out as a single HTML file in a grid so I can compare them side by side. Label each with the tradeoff it's making."

Cost data (verbatim methodology):

Format	Artifact tokens	Output tokens	Generation time	Cost
HTML	6,877	10,135	117 s	$0.51
MD	1,896	3,284	49 s	$0.27

Cost data (format-neutral methodology):

Format	Artifact tokens	Output tokens	Generation time	Cost
HTML	6,871	9,161	102 s	$0.48
MD	3,332	4,769	73 s	$0.34

Reading both artifacts side by side, my reaction was unambiguous:

"HTML is massively nicer to read in these cases versus the markdown. It's much better ... it's not speed that makes the entire difference. It's quality and detail of visual output. This question is fundamentally visual and HTML allows for near-perfect visual communication of the ideas vs text-only in markdown."

The HTML rendered six high-fidelity onboarding mockups in iframes inside a responsive grid. The markdown produced a six-section text document that described what each mockup would look like.

Onboarding Screen — 6 Approaches

Each tile is a self-contained mockup of the same product moment (first launch, account just created). They vary in layout, tone, and information density. The label on each names the tradeoff that approach is choosing.

01 Minimal — single field

Buys: near-zero friction, fast to value. Costs: no signal of product depth, no personalization data captured.

What should we call you?

One question. We will set up the rest as you go.

Press Enter to continue

02 Wizard. Multi-step

Buys: thorough personalization, sets expectations. Costs: highest abandonment risk; each step is an exit ramp.

SetupStep 2 of 4

What kind of work do you do?

We will tailor templates and integrations to match.

Engineering

Design

Product

Marketing

Operations

Something else

03 Skip-first. Load sample data

Buys: fastest path to seeing the product work. Costs: users may never come back to configure; sample data can mislead.

You are in.

Try it now with a sample workspace, or import your own data.

Skip and start from scratch →

04 Conversational. Chat intake

Buys: warm, human tone; lowers stakes of each ask. Costs: slow, sequential, hard to scan or skip ahead.

NovaSetup assistant

Hey! Glad you are here. Mind if I ask a couple things to set up your space?

Sure

What should I call you?

Sam

Nice to meet you, Sam. Is this for personal use or for a team?

Just me A small team A whole company

05 Dashboard-preview. Checklist overlay

Buys: shows the destination immediately; setup feels like progress, not a gate. Costs: dense and busy; can overwhelm on first impression.

Acme

Dashboard

Projects

Inbox

Reports

Settings

Welcome, Sam

0Projects

0Tasks

1Members

Get set up (1 of 4)

Two minutes. You can finish later.

Create account

Invite a teammate

Connect a tool

Start a project

06 Persona picker. Pick a role

Buys: instant personalization from one tap. Costs: forces a category choice users may not have ready; hard to fit edge cases.

Who are you, mostly?

Pick the closest match. We will preset your workspace.

⚙

Builder

Code, tickets, deploys.

✎

Maker

Drafts, files, calendars.

☎

Connector

People, threads, follow-ups.

☰

Operator

Lists, runbooks, status.

Not sure — show me everything

" height="700" width="100%" style="border:1px solid #d0d7de;border-radius:6px;margin:1.5em 0">

The HTML artifact, rendered above. Six approaches, side by side. This is the thing markdown structurally can't do.

Three things worth noting from this case:

Markdown leaks HTML, but only when the prompt corners it. When I ran Thariq's verbatim prompt (with its "single markdown file in a grid" phrasing), the model embedded 10 HTML tags into the "markdown" output (<table>, <tr>, <td>). Of course it did; there's no markdown way to lay out a grid. When I dropped "in a grid" from the prompt, the markdown came out clean. Zero HTML tags. So if you've ever wondered why your markdown agents sometimes spit out HTML, here's one answer: you asked them to do something markdown can't do.
Methodology shifts the cost ratio. Verbatim ablation: 3.63x artifact tokens; format-neutral: 2.06x. Single-number ratios reported without methodology disclosure should be read skeptically.
Generation time partially supports Thariq's "2-4x slower" admission. Measured 1.4-2.4x depending on methodology. The high end of his range is reachable, the low end is below what I measured.

The call: HTML wins outright. The token cost isn't a tradeoff, it's just what the capability costs.

PR review: a fair fight, and HTML costs about 40% more for the polish

Methodology note: this case used verbatim ablation only (the format-neutral methodology was only run on the design exploration). The verbatim prompt structure ("create an HTML artifact...") is HTML-affording and biases against markdown. The 1.51x cost ratio reported here would likely shrink under a neutrally-phrased prompt; I didn't measure by how much.

Thariq's prompt (lightly adapted for my substrate): "Help me review this PR by creating an HTML artifact that describes it. I'm not familiar with the gitignore/path-resolution logic so brace on that. Render the actual diff with inline margin annotations, color-code findings by severity..."

Substrate: toks commit c6d70f9, a real bug fix (~30 lines of code change plus a new test).

Cost data:

Format	Artifact tokens	Output tokens	Generation time	Cost
HTML	11,223	19,308	234 s	$0.82
MD	3,810	10,437	154 s	$0.54

Ratios: 2.95x artifact tokens, 1.85x output tokens, 1.52x time, 1.51x cost.

The markdown PR review is, frankly, a real PR review. TL;DR verdict at the top ("Approve with two non-blocking notes"), a severity legend in a markdown table, a structured walkthrough of the gitignore logic, line-by-line diff annotations, a test-coverage assessment. If a teammate sent me this in a Slack DM I'd be perfectly happy.

HTML adds a color-coded verdict bar, rendered diffs with green/red highlighting and line numbers, eight severity-tagged finding cards (Pass / Concern / Nit), and inline severity callouts on the diff. Prettier. More navigable. The substance is the same.

PR Review — toks c6d70f9: Respect .gitignore when target has no .git directory

PR Review — Respect `.gitignore` when target has no `.git` directory

Repository toks · commit c6d70f9 · author Corey Gallon · 2026-04-29

Approve with notes

The fix is small, correct, and targeted. It closes a real bug: .gitignore was previously a no-op for any directory that didn't sit under a .git root. The new test exercises the fixed path. Two pre-existing rough edges in the surrounding logic are surfaced below as concerns—not blockers—because the PR description's motivating example (.venv being scanned) intersects them and a future reader will want to know.

Pass — nothing to change Nit — style/clarity Concern — worth discussing Blocker — must fix

Background: how gitignore resolution actually works here

Two functions, two directions of walk

The scanner relies on two helpers in src/toks/scanner.py. To read this PR you have to hold both in your head at once:

find_git_root(start=target) walks upward from the target, looking for a .git directory in each parent. Returns that parent, or None if it reaches the filesystem root without finding one.
load_gitignore_specs(git_root, target) walks downward from git_root via os.walk(git_root), collecting every .gitignore it finds, and prefixing each pattern with the .gitignore's relative directory so a nested foo/.gitignore rule like build/ becomes foo/build/.

In the scan loop, each candidate file is reduced to a path relative to that same root via file_path.relative_to(git_root) before being matched against the spec. This is the critical invariant: the root used to build the spec must be the same root used to relativize the candidate, or every match silently fails (or, worse, raises ValueError from relative_to).

The bug being fixed: when find_git_root returned None, the old code skipped load_gitignore_specs entirely. So a project with a .gitignore but no git init got no filtering at all.

Before this PR (no `.git` upstream)

/projects/myproj/        &larr; target
  .gitignore         (says: .venv/)
  src/
    main.py          scanned
  .venv/
    lib/site-pkgs/
      django/...     scanned (bug!)
      numpy/...      scanned (bug!)

find_git_root returns None → gitignore_spec stays None → the .gitignore is silently ignored.

After this PR (no `.git` upstream)

/projects/myproj/        &larr; target = gitignore_root
  .gitignore         (says: .venv/)
  src/
    main.py          scanned
  .venv/
    lib/site-pkgs/
      django/...     ignored
      numpy/...      ignored

Fallback: gitignore_root = target. load_gitignore_specs walks the target subtree, builds a spec from the local .gitignore, and the per-file check now matches.

Annotated diff — `src/toks/scanner.py`

@@ -104,11 +104,11 @@ def scan_files(...):

104

raise ValueError(f"Not a directory: {target}")

105

106

gitignore_spec = None

107

git_root = None

107

gitignore_root = None

108

if not no_gitignore:

109

git_root = find_git_root(start=target)

110

if git_root:

111

gitignore_spec = load_gitignore_specs(git_root=git_root, target=target)

110

gitignore_root = git_root if git_root else target

111

gitignore_spec = load_gitignore_specs(git_root=gitignore_root, target=target)

112

113

results: list[tuple[Path, str, int]] = []

line 107 — rename

Pass — better name

Renaming git_root → gitignore_root is the right call. After the fix the variable can hold a path that has nothing to do with git (it can just be the target). The new name reflects what it's used for rather than where it came from.

lines 110–111 — the fix

Pass — correct fallback

git_root if git_root else target establishes a non-null root in every branch where no_gitignore is false. The same value is then threaded into load_gitignore_specs as git_root=…. The keyword name in the callee now reads as a slight misnomer (it's not necessarily a git root anymore), but that's a follow-up rename, not a bug.

line 109 — pre-existing

Concern — upstream .git can hijack the root

find_git_root walks all the way up to /. If the user has any unrelated .git upstream of the target (e.g. a dotfiles repo at ~/.git, or a parent monorepo), it becomes the root. load_gitignore_specs then os.walks the entire ancestor tree to harvest every .gitignore under it. This was true before the PR too — the PR doesn't make it worse — but it's worth surfacing because the new fallback only kicks in when no .git is found, and the more common surprising case is finding the wrong one.

line 111 — small style

Nit — consider renaming the parameter too

load_gitignore_specs(git_root=…) still takes a parameter named git_root. After this change, callers pass either a true git root or the target. Renaming the parameter to root would close the loop on the rename you started in this PR. Optional — do it now or in a follow-up.

@@ -128,8 +128,8 @@ def scan_files(...):

128

if file_path.is_symlink() and file_path.is_dir():

129

continue

130

131

if gitignore_spec and git_root:

132

rel = file_path.relative_to(git_root)

131

if gitignore_spec and gitignore_root:

132

rel = file_path.relative_to(gitignore_root)

133

if gitignore_spec.match_file(str(rel)):

134

continue

line 132 — invariant preserved

Pass — root used to build = root used to relativize

The critical invariant from the brace above is intact: the spec is built with gitignore_root in load_gitignore_specs, and candidate files are relativized to the same gitignore_root here. If a future change splits these, that's where bugs would creep in.

line 131 — defensive check

Nit — and gitignore_root is now redundant

After the fix, whenever gitignore_spec is non-None, gitignore_root is also set (the only path that produces a spec assigns the root first). The and gitignore_root guard is harmless but no longer carries information. You could drop it, or keep it as defensive code — not worth blocking on.

whole hunk — pre-existing perf

Concern — gitignore filtering is per-file, not per-directory

The walk in scan_files only consults the spec for files. Directory pruning is hard-coded to .git and symlinks. So even after this fix, if .gitignore contains .venv/, os.walk still descends into .venv/, stats every file, and then drops them one-by-one via match_file. The output is correct; the walk is slow on a directory full of site-packages. The PR description names .venv specifically, which is exactly the case where users will notice the cost. Worth a follow-up: prune dirnames against the spec before recursing.

Annotated diff — `tests/test_scanner.py`

@@ -95,3 +95,16 @@ class TestScanFiles:

empty.mkdir()

results = scan_files(target=empty)

assert results == []

def test_gitignore_respected_without_git_dir(self, tmp_path):

100

(tmp_path / ".gitignore").write_text("ignored/\n*.log\n")

101

(tmp_path / "keep.py").write_text("print('hi')\n")

102

(tmp_path / "debug.log").write_text("noise\n")

103

(tmp_path / "ignored").mkdir()

104

(tmp_path / "ignored" / "junk.py").write_text("x = 1\n")

105

106

results = scan_files(target=tmp_path)

107

names = {r[0].name for r in results}

108

assert "keep.py" in names

109

assert "debug.log" not in names

110

assert "junk.py" not in names

test as a whole

Pass — exercises the fixed path

tmp_path is a fresh directory with no .git anywhere upstream (pytest's tmp lives outside any project repo by default). It covers both a glob (*.log) and a directory (ignored/), and asserts both positive (keep.py kept) and negative (two paths excluded). This test would have failed against the pre-fix code.

coverage gap

Nit — consider one more case

load_gitignore_specs supports nested .gitignore files (it walks the whole subtree and prefixes patterns with the relative directory). The new test only places a .gitignore at the root of tmp_path. A second test with tmp_path / "sub" / ".gitignore" and a file inside sub/ would lock in the nested-spec behavior under the new fallback path — that's the part most likely to regress in a future refactor.

latent assumption

Concern — test would silently fail if pytest tmp ever lived under a .git

The test depends on pytest's tmp_path not having a .git ancestor. That's true today on every CI runner I'm aware of, and almost always true locally, but it's an implicit assumption. Adding an explicit assert find_git_root(start=tmp_path) is None at the top of the test would make the assumption visible and produce a clearer failure if it ever broke.

Findings summary

Pass Fix is correct and minimally scoped

scanner.py:107–111, 131–132

The same root value flows through spec construction and per-file relativization. No new code paths added; behavior with a real git root is byte-for-byte unchanged.

Pass Variable rename improves accuracy

scanner.py:107

git_root → gitignore_root reflects the broadened semantics. Reading the new code, the intent is clearer than the old code's intent was.

Pass New test exercises the bug

test_scanner.py:99–110

Test fails on pre-fix code, passes on post-fix code, covers both a glob pattern and a directory pattern.

Concern Pre-existing: an unrelated upstream .git hijacks the root

scanner.py:109 (and find_git_root)

find_git_root walks to /. With ~/.git (dotfiles), running toks on ~/projects/whatever picks up the dotfiles repo and triggers os.walk(~) in load_gitignore_specs. Not introduced by this PR. Worth a follow-up to either bound the upward walk (e.g. stop at the user's home or filesystem boundary) or treat that case the way you treat “no git root” here.

Concern Pre-existing: gitignored directories are still walked into

scanner.py:scan_files loop

The PR description specifically calls out .venv. With this fix, .venv's files are correctly excluded from results, but os.walk still descends through every site-packages file and stats it. On a fresh venv that's tens of thousands of stats. Pruning dirnames against gitignore_spec before recursing would address it. Out of scope for this PR; flag as follow-up.

Nit load_gitignore_specs parameter still named git_root

scanner.py:73

You renamed the variable in the caller; the parameter in the callee is now slightly misleading (callers may pass a non-git path). Trivial follow-up rename.

Nit and gitignore_root guard is redundant

scanner.py:131

After the fix, the two flags are coupled. Harmless but no longer informative.

Nit One more test would lock in nested-.gitignore behavior on the fallback path

test_scanner.py

Add a test where tmp_path/sub/.gitignore defines a pattern and a file inside sub/ matches it — verifies load_gitignore_specs's prefix logic still works when the root is the target rather than a real git root.

Behavioral coverage matrix

Cases the implementation can encounter and where each is covered:

Scenario	Pre-PR behavior	Post-PR behavior	Test coverage
Target is inside a git repo, has `.gitignore`	Honored	Honored (unchanged)	Yes — existing fixture-based tests
Target has `.gitignore`, no `.git` upstream	Silently ignored (the bug)	Honored via target-as-root fallback	Yes — new test
Target has `.gitignore` at root and a nested `sub/.gitignore`, no `.git` upstream	Both ignored (bug)	Both should be honored (relies on `load_gitignore_specs` walking from `target`)	Not directly tested
Target with no `.gitignore` and no `.git`	No filtering	No filtering (`load_gitignore_specs` returns `None`)	Implicit via `test_empty_dir`
Target with `no_gitignore=True`	Skipped	Skipped (unchanged)	Not directly tested (pre-existing gap)
Target inside an unrelated upstream `.git` (e.g. `~/.git`)	Walks entire ancestor tree	Same — not addressed	Not tested

Recommendation

Ship this PR as-is. The fix is correct, well-named, and has a test that would catch a regression. The two pre-existing concerns (upstream-.git hijacking and per-file rather than per-directory filtering) are worth filing as follow-ups but should not block this merge — widening scope would dilute a clean, easy-to-review change.

" height="700" width="100%" style="border:1px solid #d0d7de;border-radius:6px;margin:1.5em 0">

The HTML PR review. Verdict bar, severity-tagged findings, annotated diff. Polished. But the markdown version below has the same substance.

Rendered markdown

PR Review — `toks` c6d70f9

Title: Respect .gitignore when target has no .git directory
Author: Corey Gallon
Date: 2026-04-29
Files touched: src/toks/scanner.py (+4 / -4), tests/test_scanner.py (+13 / -0)

TL;DR

Verdict: Approve with two non-blocking notes.

The fix is minimal, correct for the bug it targets, and well-tested. The rename from git_root to gitignore_root is the right framing — the variable is now "the root we're matching gitignore patterns relative to," and that name finally tells the truth.

The two notes worth flagging before merge:

The fallback intentionally narrows discovery: .gitignore files above the target are no longer consulted when no .git is present (they never were, but the fallback formalizes that). Worth a one-line docstring note so future-you doesn't re-litigate it.
The new test is mildly fragile to ancestor-.git pollution from tmp_path. In practice fine; a single no_gitignore=False is implicit and may want a sibling assertion to lock the contract.

Neither blocks the merge.

Severity legend

Tag	Meaning
`[CRITICAL]`	Blocks merge. Correctness, security, data loss.
`[HIGH]`	Should fix before merge. Likely to bite users.
`[MEDIUM]`	Address before merge if cheap, otherwise track.
`[LOW]`	Nit / polish. Optional.
`[POSITIVE]`	Worth calling out — got this right.

This review found: 0 CRITICAL · 0 HIGH · 2 MEDIUM · 3 LOW · 2 POSITIVE.

Bracing the gitignore / path-resolution logic

I want to walk this carefully because the bug is exactly the kind of thing that hides in path semantics. Here's the model after the change:

target = the directory the user pointed toks at
git_root = nearest ancestor containing .git (or None)
gitignore_root = git_root if git_root else target   # NEW

gitignore_root is then used for two things:

As the walk root for .gitignore discovery — load_gitignore_specs(git_root=...) does os.walk(git_root) and concatenates patterns from every .gitignore it finds, prefixed by their relative path under that root.
As the basis for relativization at match time — file_path.relative_to(gitignore_root) produces the path string passed to pathspec.match_file.

These two uses must use the same root for matching to be correct. The PR keeps them in lockstep — that's the load-bearing invariant, and it's preserved.

Walking the four cases

Case	git found?	`git_root`	`gitignore_root` (after)	Discovery walks	Relativizes against	Correct?
Target inside a git repo	yes	`/repo`	`/repo`	`/repo`	`/repo`	yes (unchanged)
Target IS the git root	yes	`target`	`target`	`target`	`target`	yes (unchanged)
Target has its own `.gitignore`, no `.git` anywhere	no	`None`	`target`	`target`	`target`	yes — newly fixed
Target has no `.gitignore`, no `.git`	no	`None`	`target`	`target` (yields no patterns)	`target`	yes — `load_gitignore_specs` returns `None`, so the guard `if gitignore_spec and gitignore_root:` short-circuits

The fourth row is the one I'd want a reviewer to verify by reading. load_gitignore_specs returns None when patterns is empty, the assignment becomes gitignore_spec = None, and the per-file guard skips matching. No regression.

What the fallback does NOT do

There's one behavior it would be easy to assume but isn't true: the fallback does not search upward for .gitignore files when there's no .git. If the layout is

/parent/
  .gitignore        # contains &quot;*.log&quot;
  child/
    debug.log

…and the user runs toks /parent/child, debug.log will be scanned. find_git_root returns None (no ancestor has .git), gitignore_root falls back to child, and os.walk(child) never sees /parent/.gitignore.

This is consistent with the commit message ("fall back to the target directory itself") and consistent with how git itself behaves (no .git → no project boundary → no upward .gitignore chain). I'd just like one line in the docstring saying so, because the next person to think about this will think about it again otherwise.

The diff, annotated

Below: each hunk, followed by per-line notes. Annotations cite line numbers from the new file.

Hunk 1 — `scanner.py` lines 104-112

   target = target.resolve()
   if not target.is_dir():
       raise ValueError(f&quot;Not a directory: {target}&quot;)

   gitignore_spec = None
-  git_root = None
+  gitignore_root = None                                                       # ← (A)
   if not no_gitignore:
       git_root = find_git_root(start=target)
-      if git_root:
-          gitignore_spec = load_gitignore_specs(git_root=git_root, target=target)
+      gitignore_root = git_root if git_root else target                       # ← (B)
+      gitignore_spec = load_gitignore_specs(git_root=gitignore_root, target=target)  # ← (C)

Mark	Annotation	Severity
A	Rename is the right call — this variable now means "the root we relativize against," not "the git repo root." Name follows semantics.	`[POSITIVE]`
B	Ternary is readable. Could equivalently be `gitignore_root = git_root or target`, which is shorter and idiomatic Python. Style preference; either is fine.	`[LOW]`
C	The `target=target` keyword arg is now unused inside `load_gitignore_specs` — `target_resolved = target.resolve()` is computed and never read. Pre-existing dead code, but this PR is the natural moment to either delete the parameter or use it. See finding F-3 below.	`[MEDIUM]`

Hunk 2 — `scanner.py` lines 128-135

           if file_path.is_symlink() and file_path.is_dir():
               continue

-          if gitignore_spec and git_root:
-              rel = file_path.relative_to(git_root)
+          if gitignore_spec and gitignore_root:                               # ← (D)
+              rel = file_path.relative_to(gitignore_root)                     # ← (E)
               if gitignore_spec.match_file(str(rel)):
                   continue

Mark	Annotation	Severity
D	Guard updated in lockstep with the rename. The two checks (`gitignore_spec`, `gitignore_root`) are now both truthy iff we successfully built a spec — `gitignore_spec` alone is sufficient (since spec is only built when root is set), but the redundant guard is defensive and harmless.	`[POSITIVE]`
E	`relative_to(gitignore_root)` is safe: `file_path` comes from `os.walk(target)`, and `gitignore_root` is either `target` or an ancestor of it, so `file_path` is always under `gitignore_root`. No `ValueError` risk.	`[POSITIVE]`

Hunk 3 — `tests/test_scanner.py` lines 99-110

+  def test_gitignore_respected_without_git_dir(self, tmp_path):
+      (tmp_path / &quot;.gitignore&quot;).write_text(&quot;ignored/\n*.log\n&quot;)
+      (tmp_path / &quot;keep.py&quot;).write_text(&quot;print('hi')\n&quot;)
+      (tmp_path / &quot;debug.log&quot;).write_text(&quot;noise\n&quot;)
+      (tmp_path / &quot;ignored&quot;).mkdir()
+      (tmp_path / &quot;ignored&quot; / &quot;junk.py&quot;).write_text(&quot;x = 1\n&quot;)
+
+      results = scan_files(target=tmp_path)                                   # ← (F)
+      names = {r[0].name for r in results}
+      assert &quot;keep.py&quot; in names
+      assert &quot;debug.log&quot; not in names                                         # ← (G)
+      assert &quot;junk.py&quot; not in names                                           # ← (H)

Mark	Annotation	Severity
F	Test exercises the regression directly: `tmp_path` is system tmp on Linux/macOS and typically has no ancestor `.git`. Fragility note in F-1 below.	—
G	Covers the file-glob case (`*.log`).	`[POSITIVE]`
H	Covers the directory case (`ignored/`). Both major gitignore pattern shapes get coverage.	`[POSITIVE]`

The test asserts the positive (keep.py in) and the negatives (debug.log, junk.py not in). That's the right shape — a test that only asserted exclusions could pass with scan_files returning [].

Findings

`[MEDIUM]` F-1 — Test is silently dependent on `tmp_path` having no ancestor `.git`

find_git_root walks upward from target.resolve() until it hits the filesystem root, looking for any .git. If a developer's test environment puts tmp_path somewhere under a git checkout (rare but possible — custom tmpdir configs, certain CI sandboxes, network-mounted home dirs with stray .git symlinks), the test would skip the new code path entirely. It would still likely pass because of how the patterns happen to be structured, but it would no longer be testing what its name claims.

Suggested fix: Either (a) explicitly verify no ancestor has .git at the start of the test, or (b) make the intent unambiguous by also asserting via a second call with no_gitignore=True that the included set differs:

results_no_ignore = scan_files(target=tmp_path, no_gitignore=True)
no_ignore_names = {r[0].name for r in results_no_ignore}
assert &quot;debug.log&quot; in no_ignore_names  # confirms gitignore did the filtering

That second assertion locks the contract: "filtering happened because of gitignore handling," not "filtering happened, somehow."

`[MEDIUM]` F-2 — Behavior of fallback should be documented

The docstring of scan_files doesn't mention gitignore handling at all today. With this change, the rule "we'll honor a .gitignore in target even without .git" is now part of the contract. One line is enough:

&quot;&quot;&quot;Scan a directory for files, returning (path, mime_type, file_size) tuples.

…

When no_gitignore is False (default), .gitignore files are honored. The
gitignore root is the nearest ancestor containing .git, or the target
itself if no such ancestor exists. .gitignore files above the gitignore
root are not consulted.
&quot;&quot;&quot;

`[LOW]` F-3 — Unused parameter in `load_gitignore_specs`

def load_gitignore_specs(*, git_root: Path, target: Path) -&gt; pathspec.PathSpec | None:
    patterns: list[str] = []
    target_resolved = target.resolve()  # ← never read
    ...

target is no longer used inside this function. Pre-existing, not introduced by this PR — but the PR is the natural pass-by for it. Either remove the parameter and the dead line, or use it (e.g., to skip .gitignore files outside the target subtree if you wanted to make scoping stricter — though I'd argue you don't, because git itself doesn't).

If removing: also rename git_root → root while you're there, since the parameter is no longer git-specific in concept.

`[LOW]` F-4 — Style: `git_root or target` over the ternary

gitignore_root = git_root if git_root else target
# vs
gitignore_root = git_root or target

Path instances are always truthy, and find_git_root returns None or a Path, so the short form is both safe and idiomatic. Pure preference.

`[LOW]` F-5 — `find_git_root` accepts `.git` as either file or directory; commit message says "directory"

find_git_root checks (current / ".git").exists(), which is true for both directories and files. Worktrees use a .git file (containing gitdir: ...). The commit message says "no .git directory," but the code does the right thing for worktrees too. This is a wording nit on the commit message, not a code issue. The change correctly does not regress worktree handling.

`[POSITIVE]` F-6 — Minimal diff

Four-line change in production code, two of them pure renames, plus a focused test. Doesn't touch unrelated logic. Doesn't introduce new abstractions. The kind of fix that ages well.

`[POSITIVE]` F-7 — Test assertions cover both pattern shapes

*.log (file glob) and ignored/ (directory) are the two pattern syntaxes most users care about, and both are exercised. A .gitignore parser regression in either would fail this test.

Suggested commit-message tweak (optional)

Respect .gitignore when target has no .git directory or worktree marker

Tiny edit; keeps the message accurate for the worktree-file case the code already handles.

Pre-merge checklist

[ ] Add the docstring note from F-2 (10 seconds).
[ ] Optional: tighten the test per F-1 (one extra no_gitignore=True call).
[ ] Optional: address F-3 in a follow-up cleanup commit.
[ ] No security implications. No performance regression: when git_root is None and target has no .gitignore, load_gitignore_specs walks the target tree once and returns None. That walk is bounded by the same tree the main os.walk traverses anyway, so worst case is one extra traversal of a directory the user already chose to scan.

Ready to merge after F-2.

" height="700" width="100%" style="border:1px solid #d0d7de;border-radius:6px;margin:1.5em 0">

The same review, in markdown. Same content. Roughly 30% cheaper. The cost of "looks nicer" is a real number, and that's it.

I also asked Claude to make a semantic edit to each artifact (add a "Test coverage assessment" section) and looked at the resulting diffs. HTML's was 5,658 bytes / 93 lines; markdown's was 4,618 bytes / 55 lines. HTML's structural overhead (1.2x more bytes than markdown for the same logical edit) is real but modest, because both formats already had a fair bit of structure to begin with.

So: markdown does the job. HTML is prettier and costs about 50% more. Worth it for a review template you'll reuse across the team and reference repeatedly; probably not for a one-off you'll glance at and close.

Rate limiter explainer: where the markdown actually won on substance

Methodology note: same as above, verbatim ablation only. The 1.44x cost ratio is from a HTML-affording prompt and would likely be smaller under a neutral one.

Thariq's prompt (lightly adapted): "I don't understand how slowapi's rate limiter actually works. Read the relevant code and produce a single HTML explainer page: a diagram of the rate-limiting flow, the 3-4 key code snippets annotated, and a 'gotchas' section..."

Substrate: slowapi core (extension.py, middleware.py, wrappers.py), about 12K tokens, ~1,200 lines of Python.

Cost data:

Format	Artifact tokens	Output tokens	Generation time	Cost
HTML	8,700	14,713	191 s	$0.79
MD	3,629	7,915	118 s	$0.55

Ratios: 2.40x artifact tokens, 1.86x output tokens, 1.62x time, 1.44x cost.

Both artifacts are real explainers. The bit I wasn't expecting: both used Mermaid for the flow diagram. The markdown one wraps it in a ```mermaid fence; the HTML one loads the Mermaid CDN and embeds the same chart definition. GitHub renders the markdown version as a real diagram. So does VS Code. The "HTML can show diagrams and markdown can't" intuition that does some quiet work in Thariq's argument is mostly gone for technical writing in 2026.

How slowapi's rate limiter actually works

A one-shot tour of extension.py, middleware.py, and wrappers.py.

The shape in one paragraph. slowapi is a thin orchestration layer over the limits library. It collects rate-limit declarations from three places (decorators, app-level application limits, app-level default limits), and on each request it builds a list of which of those apply, calls limits.hit() on each in turn against a configured backend (memory / Redis / etc.), and either lets the request through or raises RateLimitExceeded. The check can be triggered from a decorator wrapper or from a Starlette middleware -- they are two entry points into the same core function, _check_request_limit.

Rate-limiting flow

flowchart TD Req([Incoming HTTP request]) Req --> Entry{"Entry point"} Entry -->|"Middleware path
(SlowAPIMiddleware /
SlowAPIASGIMiddleware)"| MW[Find route handler
via app.routes] Entry -->|"Decorator path
(@limiter.limit / @limiter.shared_limit)"| Dec[Decorator wrapper
extracts request from args/kwargs] MW --> Exempt{"_should_exempt?
handler missing OR
name in _exempt_routes OR
name in _route_limits"} Exempt -->|yes| Pass[Pass through, no check] Exempt -->|no| CallCore["_check_request_limit
(in_middleware=True)"] Dec --> Flag{"request.state._rate_limiting_complete
already True?"} Flag -->|yes| RunHandler1[Run handler] Flag -->|no| CallCoreD["_check_request_limit
(in_middleware=False)"] CallCoreD --> SetFlag[Set _rate_limiting_complete = True] SetFlag --> RunHandler1 CallCore --> Build CallCoreD --> Build Build["Build all_limits list:
• application_limits (only if in_middleware)
• route_limits + dynamic_route_limits (only if NOT in_middleware)
• default_limits (unless route has override_defaults=True)"] Build --> StorageDead{"_storage_dead
AND fallback_limiter?"} StorageDead -->|yes| Fallback[Use _in_memory_fallback limits
via _fallback_limiter] StorageDead -->|no| Eval Fallback --> Eval Eval["__evaluate_limits:
for each Limit in all_limits"] Eval --> ForEach{"per-limit checks"} ForEach -->|"is_exempt(request)
or method mismatch"| Skip[Skip this limit] ForEach -->|"otherwise"| Hit["self.limiter.hit(
limit, key_func(request), scope, cost=...)"] Skip --> Eval Hit --> Allowed{"hit returns True?"} Allowed -->|yes, track smallest as
limit_for_header| Eval Allowed -->|no| Fail["Set request.state.view_rate_limit
raise RateLimitExceeded"] Eval -->|"all passed"| Done["request.state.view_rate_limit
= smallest limit seen"] Done --> RunHandler2[Run handler / call_next] RunHandler1 --> Inject[_inject_headers / _inject_asgi_headers:
X-RateLimit-Limit / Remaining / Reset / Retry-After] RunHandler2 --> Inject Fail --> ExcHandler["_rate_limit_exceeded_handler
builds 429 JSON response"] ExcHandler --> Inject Inject --> Resp([Response to client]) Pass --> Resp

The four code paths that matter

1. The core check: _check_request_limit extension.py

Both entry points funnel here. The job of this function is to assemble the list of limits that apply to this request, then hand them to __evaluate_limits. The in_middleware flag is the pivot: it controls which buckets of limits are pulled in, so middleware and decorator don't double-count or miss each other.

def _check_request_limit(self, request, endpoint_func, in_middleware=True):
    endpoint_url     = request["path"] or ""
    endpoint_name    = f"{endpoint_func.__module__}.{endpoint_func.__name__}" if endpoint_func else ""
    _endpoint_key    = endpoint_url if self._key_style == "url" else endpoint_name

    # Bail-outs: disabled, exempt, or a request_filter says skip.
    if (not _endpoint_key or not self.enabled
        or endpoint_name in self._exempt_routes
        or any(fn() for fn in self._request_filters)):
        return

    limits, dynamic_limits = [], []
    if not in_middleware:                              # decorator path only
        limits         = self._route_limits.get(endpoint_name, [])
        dynamic_limits = [l for lg in self._dynamic_route_limits.get(endpoint_name, [])
                            for l in lg.with_request(request)]

    route_limits      = limits + dynamic_limits
    all_limits        = list(itertools.chain(*self._application_limits)) if in_middleware else []
    all_limits       += route_limits

    # Defaults apply unless THIS route's limits all set override_defaults=True.
    combined_defaults = all(not l.override_defaults for l in route_limits)
    if (not route_limits or combined_defaults):
        all_limits   += list(itertools.chain(*self._default_limits))

    self.__evaluate_limits(request, _endpoint_key, all_limits)

2. The actual hit-or-miss: __evaluate_limits extension.py

This is the hot loop. For each Limit it computes the bucket key (key_func(request) + scope), tracks the smallest limit seen so far for header reporting, and calls self.limiter.hit(...) -- this is the limits library's RateLimiter, the thing that actually increments the counter in Redis/memory. hit returning False means "you're over"; we capture the failed limit and break out so subsequent limits aren't decremented.

def __evaluate_limits(self, request, endpoint, limits):
    failed_limit = None
    limit_for_header = None
    for lim in limits:
        limit_scope = lim.scope or endpoint
        if lim.is_exempt(request): continue
        if lim.methods is not None and request.method.lower() not in lim.methods: continue
        if lim.per_method:
            limit_scope += ":%s" % request.method

        limit_key = lim.key_func(request) if "request" in inspect.signature(lim.key_func).parameters else lim.key_func()
        args = [limit_key, limit_scope]
        if all(args):
            if self._key_prefix: args = [self._key_prefix] + args
            # Track the SMALLEST limit -- this is what gets reported in headers.
            if not limit_for_header or lim.limit &lt; limit_for_header[0]:
                limit_for_header = (lim.limit, args)

            cost = lim.cost(request) if callable(lim.cost) else lim.cost
            if not self.limiter.hit(lim.limit, *args, cost=cost):  # &lt;-- the actual check
                failed_limit = lim
                limit_for_header = (lim.limit, args)
                break                                          # stop -- don't decrement remaining limits

    request.state.view_rate_limit = limit_for_header        # picked up by header injection later
    if failed_limit:
        raise RateLimitExceeded(failed_limit)

3. Decorator entry point: __limit_decorator wrapper extension.py

The @limiter.limit("5/minute") decorator registers the limit into _route_limits / _dynamic_route_limits at decoration time, then returns a wrapper. The wrapper finds the Request in args/kwargs, runs the check (passing in_middleware=False), runs the handler, injects headers. Note the _rate_limiting_complete flag -- this is the only guard against double-checking when middleware also fires.

async def async_wrapper(*args, **kwargs):
    if self.enabled:
        request = kwargs.get("request", args[idx] if args else None)
        if not isinstance(request, Request):
            raise Exception("parameter `request` must be an instance of starlette.requests.Request")

        if self._auto_check and not getattr(request.state, "_rate_limiting_complete", False):
            self._check_request_limit(request, func, False)        # in_middleware=False
            request.state._rate_limiting_complete = True          # &lt;-- only the decorator sets this

    response = await func(*args, **kwargs)

    if self.enabled:
        if not isinstance(response, Response):
            self._inject_headers(kwargs.get("response"), request.state.view_rate_limit)
        else:
            self._inject_headers(response, request.state.view_rate_limit)
    return response

4. Middleware entry point middleware.py

The middleware path is what handles application limits and default limits for routes that don't have their own decorator. _should_exempt deliberately skips routes that already have a static decorator-defined limit -- "the decorator handles it." After the handler returns, the middleware injects headers from request.state.view_rate_limit (set by __evaluate_limits).

class SlowAPIMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request, call_next):
        app, limiter = request.app, request.app.state.limiter
        if not limiter.enabled:
            return await call_next(request)

        handler = _find_route_handler(app.routes, request.scope)
        if _should_exempt(limiter, handler):                # handler missing,
            return await call_next(request)               # in _exempt_routes,
                                                            # OR in _route_limits (decorator owns it)
        error_response, should_inject_headers = sync_check_limits(limiter, request, handler, app)
        if error_response is not None:
            return error_response                          # 429 path

        response = await call_next(request)
        if should_inject_headers:
            response = limiter._inject_headers(response, request.state.view_rate_limit)
        return response

def _should_exempt(limiter, handler):
    if handler is None: return True
    name = _get_route_name(handler)
    if name in limiter._exempt_routes: return True
    if name in limiter._route_limits: return True     # static decorator limits only -- NOT dynamic ones
    return False

Where the three buckets of limits come from

Bucket	Set by	Applied when	Stored in
application_limits	`Limiter(application_limits=[...])` or `RATELIMIT_APPLICATION`	Middleware path only (`in_middleware=True`). One shared bucket across the whole app, scope=`"global"`.	`_application_limits`
route limits	`@limiter.limit(...)` / `@limiter.shared_limit(...)`	Decorator path only (`in_middleware=False`). Static strings live in `_route_limits`; callables live in `_dynamic_route_limits`.	both above
default_limits	`Limiter(default_limits=[...])` or `RATELIMIT_DEFAULT`	Either path -- but only if the route has no decorator-applied limits, OR every route limit was declared with `override_defaults=False`.	`_default_limits`

Gotchas

Gotcha 1 -- The flag that prevents double-checking is set by only one side

The decorator wrapper sets request.state._rate_limiting_complete = True after running its check. The middleware reads that flag (in _check_limits) but never sets it. In practice this is fine because _should_exempt tells the middleware to skip routes whose name is in _route_limits -- but that check uses only static decorator limits. If your route has a dynamic limit (callable passed to @limiter.limit), the middleware will not exempt it, and you'll evaluate application/default limits in the middleware and the dynamic limit in the decorator on the same request.

Gotcha 2 -- override_defaults behaves the opposite of how it reads

It defaults to True, meaning "if I have a route limit, skip the defaults." The merge logic is:

combined_defaults = all(not l.override_defaults for l in route_limits)
if not route_limits or combined_defaults:
    all_limits += default_limits

So defaults are added only when every route limit explicitly opts in by setting override_defaults=False. A single override_defaults=True limit on the route silences all defaults, even other route limits' opt-ins.

Gotcha 3 -- Headers report the smallest limit, not the one you might expect

If a route is decorated with "5/minute;100/hour", __evaluate_limits tracks limit_for_header as the limit with the smallest amount seen so far. X-RateLimit-Limit / Remaining / Reset always reflect that limit. Clients reading those headers won't see the hourly bucket at all unless it becomes the binding constraint at the moment the response is built. Also: when a limit fails, the failing limit overwrites limit_for_header regardless of size, so the headers on a 429 response always describe the limit that just rejected you.

Gotcha 4 -- hit() short-circuits on the first failure

The loop in __evaluate_limits calls self.limiter.hit(...) for each limit in order, and breaks on the first False. hit increments the counter as a side effect, so a 429 response means earlier limits in the list were incremented but later ones were not. If you depend on multiple counters being kept in lock-step (e.g., to derive analytics from Remaining), they will drift the moment any single bucket overflows. Order matters: limits are iterated in the order they were registered.

Gotcha 5 -- SlowAPIMiddleware downgrades async exception handlers

The classic middleware uses sync_check_limits, which contains:

if inspect.iscoroutinefunction(exception_handler):
    exception_handler = _rate_limit_exceeded_handler

If you registered a custom async handler for RateLimitExceeded on the app, the middleware silently swaps it out for the default sync handler. Custom 429 responses, audit hooks, etc. won't run for requests that hit through this path. Use SlowAPIASGIMiddleware (which uses async_check_limits) if you need the async handler to fire.

Gotcha 6 -- The "fallback to in-memory" is best-effort and self-flips on any header error

_inject_headers wraps the window_stats call in a bare except:. Any exception there -- not just storage timeouts -- causes self._storage_dead = True and a recursive call against the in-memory fallback. Until the next storage probe (__should_check_backend, exponentially backed off), every request goes to the local memory store. In a multi-worker deployment this means each worker silently gets its own private counter, and your effective rate limit is multiplied by worker count. The probe will eventually re-test the real backend and recover -- but the window of inconsistency depends on MAX_BACKEND_CHECKS (5) and 2**check_count seconds.

Gotcha 7 -- Two latent bugs in the codebase

slowapi_startup references a bare name app with no enclosing scope (see the # type: ignores). Calling it raises NameError. The intended wiring is app.state.limiter = limiter done by the user.
Limit.scope calls self.__scope(request.endpoint) when the scope is callable -- but request is undefined in that property and Request objects don't have .endpoint in Starlette anyway. The in-source FIXME confirms this path is broken; callable scopes will raise.

Gotcha 8 -- The key_style choice silently changes which routes share buckets

With key_style="url" (the default) the bucket scope is the literal request path, so /users/1 and /users/2 have separate rate-limit buckets even though they hit the same handler. With key_style="endpoint", both share a bucket. If you have parameterized routes and you set up a per-endpoint quota expecting "10 calls per minute against this handler," the default URL keying gives you "10 per minute per distinct path you happen to call" instead. Pick deliberately.

" height="700" width="100%" style="border:1px solid #d0d7de;border-radius:6px;margin:1.5em 0">

The HTML rate-limiter explainer. Mermaid flowchart via CDN, callout boxes for the 8 gotchas, syntax-highlighted code via CSS classes. Looks the part.

Rendered markdown

slowapi rate limiter, explained

slowapi rate-limits requests to a Starlette/FastAPI app. It is a thin coordinator on top of the limits library: limits owns the storage (memory/Redis/...) and the strategy (fixed-window/...). slowapi owns which limits apply to this request, what key identifies the caller, and what headers go on the response.

There are two ways limits get registered, and two ways they get checked. Keeping those four straight is most of understanding the code.

How limits get registered

Source	Registered into	Applied by
`Limiter(default_limits=[...])`	`_default_limits`	both paths, when route has no overriding limit
`Limiter(application_limits=[...])`	`_application_limits`	middleware path only
`@limiter.limit("5/minute")` (static)	`_route_limits[name]`	decorator wrapper
`@limiter.limit(callable)` (dynamic)	`_dynamic_route_limits[name]`	decorator wrapper
`@limiter.exempt`	`_exempt_routes`	both paths skip

A "limit" is parsed by limits.parse_many("5/minute;100/hour") into one or more RateLimitItem objects, each wrapped in a Limit (wrappers.py). LimitGroup is iterable and yields one Limit per parsed item.

How a request is checked

flowchart TD
    A[Request arrives] --&gt; B{Middleware&lt;br/&gt;installed?}
    B --&gt;|Yes| C[SlowAPIMiddleware.dispatch]
    B --&gt;|No| K

    C --&gt; D{_should_exempt?&lt;br/&gt;handler in&lt;br/&gt;_exempt_routes or&lt;br/&gt;_route_limits}
    D --&gt;|Yes, skip&lt;br/&gt;middleware checks| K[call route handler]
    D --&gt;|No| E[_check_request_limit&lt;br/&gt;in_middleware=True]
    E --&gt; F[Builds: application_limits&lt;br/&gt;+ defaults if applicable]
    F --&gt; G[__evaluate_limits]
    G --&gt;|hit OK| H[continue to handler]
    G --&gt;|hit fails| X[raise RateLimitExceeded&lt;br/&gt;→ 429 JSON response]
    H --&gt; K

    K --&gt; L{Route has&lt;br/&gt;@limiter.limit?}
    L --&gt;|No| Z[response]
    L --&gt;|Yes| M[decorator's sync/async_wrapper]
    M --&gt; N{_rate_limiting_complete&lt;br/&gt;already True?}
    N --&gt;|Yes, already checked| O[run handler]
    N --&gt;|No| P[_check_request_limit&lt;br/&gt;in_middleware=False]
    P --&gt; Q[Builds: route_limits&lt;br/&gt;+ dynamic_limits&lt;br/&gt;+ defaults if applicable]
    Q --&gt; R[__evaluate_limits]
    R --&gt;|hit OK| S[set _rate_limiting_complete=True]
    R --&gt;|hit fails| X
    S --&gt; O
    O --&gt; T[_inject_headers&lt;br/&gt;X-RateLimit-* + Retry-After]
    T --&gt; Z

    style X fill:#fdd
    style G fill:#ffd
    style R fill:#ffd

The decorator and middleware paths are independent. The middleware checks if a route has a registered @limiter.limit — if so, it skips its own check and lets the decorator handle it (_should_exempt returns True for routes in _route_limits). Application-wide limits therefore only fire on routes that do not carry a decorator, unless you also wrap them with the decorator separately. This is the most surprising design choice in the library.

Key code, annotated

1. The decorator wrapper — where rate-limiting actually attaches to a route

extension.py, inside __limit_decorator:

@functools.wraps(func)
async def async_wrapper(*args, **kwargs):
    if self.enabled:
        request = kwargs.get(&quot;request&quot;, args[idx] if args else None)
        # Idempotency guard: if the middleware already checked, don't double-hit storage.
        if self._auto_check and not getattr(
            request.state, &quot;_rate_limiting_complete&quot;, False
        ):
            self._check_request_limit(request, func, False)  # in_middleware=False
            request.state._rate_limiting_complete = True
    response = await func(*args, **kwargs)
    if self.enabled:
        # Even if the check was skipped (already complete), headers still get injected
        # using request.state.view_rate_limit set during evaluation.
        self._inject_headers(response, request.state.view_rate_limit)
    return response

The _rate_limiting_complete flag is only ever set by the decorator wrapper — never by the middleware. So middleware → decorator double-checking is prevented by the middleware's _should_exempt returning early for decorated routes, not by the flag. The flag exists for the rarer case of nested decorators / repeated dispatch.

2. Building the limit list — when do defaults apply?

extension.py, inside _check_request_limit:

route_limits: List[Limit] = limits + dynamic_limits
all_limits = (
    list(itertools.chain(*self._application_limits))
    if in_middleware
    else []
)
all_limits += route_limits
combined_defaults = all(
    not limit.override_defaults for limit in route_limits
)
if (
    not route_limits
    and not (in_middleware and endpoint_func_name in self.__marked_for_limiting)
    or combined_defaults
):
    all_limits += list(itertools.chain(*self._default_limits))

Read the boolean carefully — Python parses it as (A and not B) or C:

A: this route has no route-level limits, AND
B (negated): we're not in the middleware-skipping-a-decorated-route case, OR
C: every route-level limit was registered with override_defaults=False.

So defaults stack with route limits only when you opt out of overriding via override_defaults=False (note: the decorator default is True, i.e. route limits replace defaults by default). This is the second-most-surprising design choice.

3. The hit loop — where requests are actually counted

extension.py, __evaluate_limits:

def __evaluate_limits(self, request, endpoint, limits):
    failed_limit = None
    limit_for_header = None
    for lim in limits:
        limit_scope = lim.scope or endpoint
        if lim.is_exempt(request): continue
        if lim.methods is not None and request.method.lower() not in lim.methods: continue
        if lim.per_method:
            limit_scope += &quot;:%s&quot; % request.method

        # key_func may or may not accept `request` — sniffed via inspect.signature
        if &quot;request&quot; in inspect.signature(lim.key_func).parameters.keys():
            limit_key = lim.key_func(request)
        else:
            limit_key = lim.key_func()

        args = [limit_key, limit_scope]
        if all(args):  # silently skip if either is empty/falsy
            if self._key_prefix:
                args = [self._key_prefix] + args
            # Track the SMALLEST limit (most restrictive) for headers.
            if not limit_for_header or lim.limit &lt; limit_for_header[0]:
                limit_for_header = (lim.limit, args)

            cost = lim.cost(request) if callable(lim.cost) else lim.cost
            if not self.limiter.hit(lim.limit, *args, cost=cost):
                # First failure wins — break, do NOT hit subsequent limits.
                failed_limit = lim
                limit_for_header = (lim.limit, args)
                break
        else:
            self.logger.error(&quot;Skipping limit: %s. Empty value found in parameters.&quot;, lim.limit)
            continue
    request.state.view_rate_limit = limit_for_header
    if failed_limit:
        raise RateLimitExceeded(failed_limit)

Two points worth burning in:

The header-target is the smallest limit, by RateLimitItem.__lt__ (smaller window/amount = more restrictive). This is what the client sees in X-RateLimit-*.
On failure, the loop breaks — slowapi reports the first limit that failed and never increments counters for limits later in the list. This matters when a route has multiple stacked limits ("5/sec;100/min;1000/hr"); only the smallest-window-that-fails counts the request.

4. Header injection — the contract with the client

extension.py, _inject_headers (asgi variant is identical, just operating on MutableHeaders):

window_stats = self.limiter.get_window_stats(current_limit[0], *current_limit[1])
reset_in = 1 + window_stats[0]   # window_stats[0] is the unix ts when the window resets
response.headers.append(self._header_mapping[HEADERS.LIMIT], str(current_limit[0].amount))
response.headers.append(self._header_mapping[HEADERS.REMAINING], str(window_stats[1]))
response.headers.append(self._header_mapping[HEADERS.RESET], str(reset_in))

existing_retry_after_header = response.headers.get(&quot;Retry-After&quot;)
if existing_retry_after_header is not None:
    reset_in = max(self._determine_retry_time(existing_retry_after_header), reset_in)
response.headers[self._header_mapping[HEADERS.RETRY_AFTER]] = (
    formatdate(reset_in) if self._retry_after == &quot;http-date&quot;
    else str(int(reset_in - time.time()))
)

Retry-After defaults to delta-seconds (reset_in - now); set retry_after="http-date" to emit RFC2616 dates instead. Headers are only injected if headers_enabled=True and a limit was evaluated for this request — otherwise the response is silently un-annotated.

Gotchas

Application limits only fire on un-decorated routes. SlowAPIMiddleware short-circuits via _should_exempt for any route present in _route_limits. If you set application_limits=["1000/hour"] and decorate every endpoint, the application limit never runs. Either keep the application limit's routes un-decorated, or stack the limit explicitly.
override_defaults=True is the decorator default. Adding @limiter.limit("5/minute") replaces your default_limits, it does not stack. Pass override_defaults=False to stack. The variable combined_defaults in _check_request_limit reads as if it computes a union — it does not; it just decides whether to append defaults.
Empty key_func() or scope silently skips a limit. The if all(args): guard means a buggy key function that returns "" or None (e.g., user not authenticated, header missing) results in a request that is not rate-limited at all, with only an error log. This is a common bypass vector — always make key_func return a sentinel like "anonymous" rather than an empty string.
Multiple stacked limits: only the first-failing one counts the hit. A request that would breach 5/sec and 100/min simultaneously increments the per-second counter and stops; the per-minute counter is untouched for that request. So aggregate "minute" usage will under-report whenever the second-window limit is being hit.
The headers report the smallest limit, not the first. If you stack 1000/hr;5/sec, X-RateLimit-Limit will be 5. Clients implementing backoff against a single header pair will see the tightest limit, which is usually what you want — but be aware your "1000/hr" budget is invisible to them.
Limit.scope is broken for callable scopes (wrappers.py). The property body references an undefined request.endpoint (a Flask-ism leftover, with a FIXME in the source). Passing a callable as scope= to shared_limit will raise NameError at evaluation time. Use string scopes only.
scope= on @limiter.limit() is a no-op. __limit_decorator does _scope = scope if shared else None, and limit() calls it without shared=True. Only shared_limit() actually uses scope.
In-memory fallback hides backend outages. When _storage_dead = True and _in_memory_fallback_enabled, slowapi uses local memory until the backend recovers. In a multi-process deployment each worker maintains its own counter while the backend is down, so the effective limit during outages is N_workers × configured_limit. Health is rechecked with exponential backoff (pow(2, count)), so recovery can take up to ~32 seconds after the backend returns.
slowapi_startup is dead code. It references a free variable app and would NameError if called. Don't wire it up; set app.state.limiter = limiter and register the RateLimitExceeded exception handler manually (or use the middleware, which reads app.state.limiter).
SlowAPIMiddleware (BaseHTTPMiddleware) only handles sync exception handlers; if your custom RateLimitExceeded handler is async it silently falls back to the default _rate_limit_exceeded_handler. Use SlowAPIASGIMiddleware if you need async exception handlers.
key_func signature is sniffed, not specified. __evaluate_limits introspects key_func.parameters and calls key_func(request) if "request" is in the signature, else key_func(). A decorator like @functools.wraps that hides parameters, or a lambda *a: ..., will land on the wrong branch.
request.state.view_rate_limit is the contract between check and header injection. If you wrap the response in custom middleware after slowapi without preserving request.state, headers won't be injected. The exception handler _rate_limit_exceeded_handler also reads from it — clearing it breaks 429 headers.

" height="700" width="100%" style="border:1px solid #d0d7de;border-radius:6px;margin:1.5em 0">

The same explainer, in markdown. Twelve gotchas instead of eight. Mermaid flowchart in a fenced block, fenced code blocks letting the viewer handle syntax highlighting, no CSS chrome. Cheaper. And, to my eye, the better artifact.

The HTML version's distinctive features are CSS chrome: a stylesheet for syntax-highlighting tokens (<span class="kw">, <span class="str">), background panels with border-radius for code blocks, color-coded callout boxes for gotchas. The markdown version uses fenced code blocks which most modern viewers syntax-highlight automatically.

I counted the gotchas in both files. HTML had 8. Markdown had 12. The HTML compresses two findings into one "Two latent bugs" entry, so charitably call it 9-10. Even charitably, the markdown surfaces more distinct gotchas the HTML doesn't reach: the empty-key_func bypass, the inspect.parameters signature sniffing, the request.state.view_rate_limit contract between checking and header injection. The HTML caught one the markdown missed (the key_style="url" vs "endpoint" distinction), but it isn't a wash. Markdown was the more substantive artifact.

That surprised me. Going in, I'd assumed the verbose-priming effect (HTML's tag-heavy output register leading Claude into more verbose responses) would push HTML toward more comprehensive answers. It didn't here. It pushed the other way.

Think of it like two technical writers given the same source code. One hands you a clean PDF with a cover page, syntax-highlighted snippets, and tasteful colored boxes around the warnings. The other gives you a longer markdown file with no styling, but he caught four more gotchas. Which one do you want before you ship to prod?

The edit probe threw me a small surprise. I asked Claude to add a "Custom storage backends" section to each artifact and looked at the diffs. HTML's diff was 2,471 bytes over 19 lines; markdown's was bigger at 2,987 bytes over the same 19 lines. Same line count, but markdown's paragraph-style prose has longer lines than HTML's tag-broken structure. The "HTML diffs are noisier" claim turns out not to generalize cleanly. When the content is mostly prose, markdown isn't a saving on diff size.

Verdict, with the obvious caveat that this is one sample on one substrate: the markdown was the better artifact. More comprehensive (12 gotchas vs effectively 9-10 once you count the HTML's "two latent bugs" as separate items), cheaper, with a real Mermaid diagram in the fenced block. The intuition that "HTML wins because richer information" doesn't hold up in this case. Whether it'd hold up at K=5 generations I genuinely don't know, and I think that's the right answer to give. One sample is one sample. But the direction of the surprise is striking. I expected HTML's verbose-priming effect to push HTML toward more thorough answers; it pushed the other way here.

"The 1M context window handles it." Does it though?

Thariq's FAQ says: "With the 1M context window in Opus 4.7, the increased token usage is not really noticeable in the context window."

Fair point for one-shot generation. Not so fair for re-ingestion, which is what Claude Code actually does. Feed yesterday's artifact back in as context for today's follow-up turn, over and over. Let me show what that costs.

Cache_creation tokens (artifact-as-input-context) per re-ingestion across the three cases:

Use case	HTML	MD	Delta
Design exploration	27,309	21,657	+5,652
PR review	33,892	25,532	+8,360
Rate limiter expl.	32,280	26,379	+5,901

Run the math. Project produces N artifacts. Each gets re-fed into the agent K times. The cache evicts between feeds (i.e. calls more than ~5 minutes apart, where you pay the cache_creation cost fresh). The HTML-over-markdown overhead is roughly N × K × 6,000 tokens. At N=50, K=3 that's 900,000 tokens of cache_creation, basically the whole 1M context window, spent on markup you wouldn't have if you'd used markdown. For tight call chains where the cache stays warm, subsequent reads hit cache_read_input_tokens (much cheaper, \$1.50/M vs \$15/M for fresh input), and the overhead amortizes to roughly the artifact's first-ingest cost.

The dollar cost differential is muted by cache pricing for tight chains, so individual calls feel cheap. The context budget is where the cost most reliably shows, because that's measured in tokens regardless of cache state.

This matters more if you're not on an enterprise plan with effectively unlimited tokens. Several commenters in Thariq's thread raised exactly that point: the "context budget isn't a concern" framing reads differently from inside Anthropic than it does for a developer paying retail. The math behind that pushback is real.

Does Claude read HTML better than markdown? (Spoiler: not really.)

Several commenters on Thariq's piece pushed back on the agent side of his argument. HTML "wastes the model's cognitive space on tags, nesting, closure, styles" (@AlexMares). It has "low signal-to-noise ratio" and risks hallucination (@AkhilDevelops). Markdown is just the right format for agents to read (@AiAGiAI). And Thariq's whole pitch quietly assumes the opposite: that the agent processes HTML at least as well as it processes markdown. Otherwise his "use HTML for everything, Claude reads it back later" workflow doesn't really hold up, does it?

So I tested it. Seven specific factual questions about slowapi's rate limiter, each answerable from both the HTML and the markdown version of the same explainer artifact (shared content, different format). I asked Claude (fresh session, Opus 4.7) to answer all seven questions from each format, then scored the answers against ground truth derived from the slowapi source.

Here's what came back:

Metric	HTML source	MD source	Ratio HTML/MD
Correct answers	7 / 7	7 / 7	equal
Input cache_creation tokens	30,213	24,312	1.24x
Output tokens generated	1,560	785	1.99x
Cost per call	$0.246	$0.185	1.33x

Both formats produced equally accurate answers to all seven questions, confirmed by two independent judges scoring blinded (neither judge knew which set came from which format). Both judges gave both sets 7/7 on factual accuracy.

On a separate axis (explanatory specificity, the "clarity and learning value" of the answer) both judges scored HTML marginally higher. Judge 1 gave markdown 6.5/7 and HTML 7.0/7. Judge 2 gave markdown 6.0/7 and HTML 6.5/7. The HTML-sourced answers consistently cited more specific code constructs (e.g. all(not l.override_defaults for l in route_limits), MAX_BACKEND_CHECKS=5, inspect.iscoroutinefunction). The markdown-sourced answers were slightly tighter prose. Both judges independently described the same pattern.

So the strong commenter worry, that HTML "wastes the model's cognitive space" or "lowers signal-to-noise to the point of hallucination," didn't show up. If anything, the mild opposite did: HTML-sourced answers came back a hair more specific. Both blinded judges said so. But the price-quality math is bad either way: you pay 33% more per call for answers that score ~7% better on specificity and identically on accuracy. The verbose-priming story holds up at the qualitative level too. HTML doesn't spend its extra tokens on factual errors; it spends them on more specific code citations.

A few things to keep in mind before reading too much into this:

N=1 per condition. Single shot per format. Could be variance.
Tested only on shared content in a single artifact. I didn't try cases where the two formats actually differ in what they covered.
Didn't test whether answer quality beyond factual accuracy matters for downstream agent reasoning. Two answers can both be correct and still be differently useful for the agent's next turn.

What the test does is rule out the strong version of the agent-side worry. HTML doesn't degrade accuracy on shared content. It does cost more on both directions of the interaction for the same outcome.

The two I didn't bother instrumenting (because there's no markdown alternative)

Checkout button prototype. Thariq's prompt: "...Create a HTML file with several sliders and options for me to try different options or actions. Eventually, give me a copy button..." This is a working interactive widget. Markdown has no equivalent. The cost question is moot; there is no markdown alternative to compare against.

Linear ticket reorderer. Thariq's prompt: "Make me an HTML file with each ticket as a draggable card across New / Next / Later / Cut columns..." Drag-and-drop is an HTML+JavaScript capability. Markdown cannot do this.

Both of these are real use cases. They demonstrate the categorical-win bucket. The token cost is the price of admission, and there is no marginal-cost-vs-benefit question because there is no marginal alternative.

Going through Thariq's claims, one by one

Claim from the article	My verdict
HTML has higher information density than markdown	True for visual content; false for text-only content. Markdown handles prose, tables, code blocks well.
HTML is more readable past 100 lines	True for visual artifacts (confirmed by user reading observation); equivocal for text-with-structure (markdown's structure works well at length).
HTML is easier to share than markdown	Partly true (raw browsers don't render `.md` files), but understated: GitHub renders markdown natively (and renders Mermaid as real diagrams since 2022); VS Code's built-in preview renders markdown; Slack's own "mrkdwn" subset handles a useful chunk. The "you have to attach as email" framing in the article ignores that uploading to GitHub/Gist and sharing a link is a one-step workaround. Untestable here at engagement-level.
HTML enables two-way interaction	True categorically. This is the strongest argument in the article (categorical-win bucket).
"Token overhead is not really noticeable" with 1M context	False. Per-artifact overhead is 1.4-1.9x in dollars, 2-4x in tokens. Cumulative overhead approaches the 1M context limit for moderate-sized projects.
"2-4x slower" generation	Partially supported: measured 1.4-2.4x across the three cases. The high end of his range is reachable; the low end is below what I measured.
HTML diffs are noisy (acknowledged downside)	Confirmed for verbatim methodology (1.2-2.5x diff bytes). Less clear for text-heavy content where markdown's prose lines have their own size.
HTML is more joyful to work with	Subjective. Skipped.
"HTML is strictly better even for text-focused content" (his hardline reply)	Not what I saw in the data. Both text cases produced competent markdown that did the job, at 1.4-1.5x less cost than the HTML. One of them produced more substance than the HTML. N=1 per condition isn't enough to refute the hardline definitively, but it's plenty to say the strong version of the claim isn't warranted by what I measured.
Implied claim: HTML reads better as agent context downstream	Mixed. On shared-content factual retrieval (7-question test on slowapi gotchas), two blinded judges scored both formats 7/7 on accuracy. On explanatory specificity, both judges gave HTML a ~7% edge (6.5-7.0 vs 6.0-6.5). HTML costs 1.33x more per call (1.24x input + 1.99x output). HTML-sourced answers are marginally more specific (more code-citation detail) at significantly higher cost; the quality-per-dollar trade favors markdown.

So what should you actually do?

HTML if the task is fundamentally visual or interactive: design exploration, prototypes, mockups, dashboards, drag-and-drop widgets, slider-tuned UIs, throwaway editor surfaces. Or you specifically need to send it to someone who won't render markdown themselves. The token cost is what the capability costs. Pay it without thinking.

Markdown if the task is text with structure: PR reviews, postmortems, design docs, implementation plans, status reports, meeting notes, technical explainers. The substance lives in the words. HTML's polish costs about 40-50% more per generation, and in my one rate-limiter sample (take it for what one sample's worth) that polish came at the expense of substance, not on top of it.

Maybe HTML on a text task if the artifact is going to be the centerpiece of a meeting, or you need visual hierarchy to handle dense reference material (tables of tables, comparison matrices), or the reader can't comfortably read raw markdown. Otherwise, save the tokens.

The "HTML for everything" stance treats two different questions as one. Half of it is true and worth paying for. The other half is paying extra for things the markdown was already going to give you. Pick the right tool for the right job.

If you want to try this yourself

You don't need anything fancy. Opus 4.7 (the model Thariq is pitching for) via Claude Code's non-interactive mode does the whole thing: claude --print --model claude-opus-4-7 -- "<the prompt>". The prompts are scattered through this post in blockquotes; use them verbatim for the HTML-affording methodology, or rephrase to drop the format-specific cues if you want the neutral version.

For substrate, pick something real. A non-trivial recent commit from a codebase you know for the PR review case. Any reasonably-sized library you don't fully understand for the explainer case (I used slowapi). Anything plausible and visually rich for the design exploration.

For token counts, I used toks, a small Anthropic-tokenizer wrapper that reports an exact token count for whichever provider you're targeting. Without it I would have been squinting at character counts and guessing.

One thing I'd do differently if I were starting over: run K=5 generations per condition. Mine were all K=1, which is why a lot of the findings (especially "markdown was more substantive on the rate-limiter") come with the N=1 caveat. If your finding is going to be load-bearing, run it five times.

What this cost me

The whole thing. Three instrumented cases, two methodologies on one of them, three probes each, the agent-as-reader experiment, plus the two demo cases. Ran me $9.42 across 26 API calls on Opus 4.7. About 28 minutes of API time, less in wall clock once you parallelize. Cheaper than the lunch I had while it ran. Tracking?

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Depict two colossal floating monolithic slabs facing each other in a rain-slicked neon city alley, suspended in mid-air. The left slab is constructed of intricate, ornate glowing HTML tag scaffolding -- angle brackets, nested div structures, and CSS class hieroglyphs etched in luminous magenta and electric pink, dense and baroque, with cables and wires spilling from its edges. The right slab is sparse and elegant, made of clean horizontal markdown rules, hashtag headers, and asterisk emphasis carved in cool cyan and deep purple light, minimalist and crystalline. Between them hovers a glowing translucent balance scale made of holographic wireframe, tipping almost imperceptibly, with streams of tokenized data particles flowing from both slabs into the central fulcrum. The background is a vertical canyon of dark chrome skyscrapers with flickering neon signage in Japanese and binary code, volumetric fog rolling at the base, puddles reflecting the pink and blue glow. Wet asphalt, lens flares, chromatic aberration, cinematic depth of field, ultra-detailed, atmospheric haze, Blade Runner meets technical schematic.

The Web Is Now a Two-Way Street for AI

2026-04-12T00:00:00-05:00

Yohan Lasorsa, Developer Advocate at Microsoft, and Olivier Leplus, Developer Advocate at AWS, opened their joint talk with a premise most web developers already accept: AI helps you build websites. Their argument is that the reverse is now equally true -- the web is becoming AI's runtime, its data source, and its interface layer. Browsers ship local models. Web pages expose tools for agents. The relationship, they argue, is no longer one-directional.

"It's 2026, it's no longer the question of can I code my web app with AI, but rather how to get the best results out of AI coding agents."

Skills as Composable Agent Plugins

Yohan started with "skills" -- lightweight text-based plugins stored in .agent/skills/ folders that follow an open specification supported by most coding agents. Each skill has a name, a description the agent uses to decide when to load it, and instructional content.

He demonstrated chaining several together: a GitHub CLI skill pulls an issue, Playwright records a video of the implemented feature, a tunnel skill creates a public URL for mobile testing, and a Telegram skill sends that URL to his phone. An agents.md file orchestrates this into a repeatable workflow -- implement, record, tunnel, notify, then wait for human confirmation before closing the issue.

The practical upshot: instead of improving your prompting, you improve your skills library. The wordplay was intentional -- Lasorsa joked that getting better results from coding agents "is mainly a matter of skills -- but don't get me wrong, it's the one that you install and use with your favorite code agent."

Debugging Without Opening DevTools

Olivier demonstrated the Chrome DevTools MCP server -- an open-source project that exposes Chrome DevTools capabilities as tools callable by coding agents. Click, fill forms, read console messages, run Lighthouse audits, take screenshots, capture performance traces -- all available as MCP tools without manual DevTools interaction.

In the demo, an agent autonomously launched Chrome, navigated to the app, and ran performance traces under three network conditions -- no throttling, fast 3G, and slow 2G. It produced a report with LCP, CLS, critical path latency, and render-blocking resource analysis. The agent did the work a developer would normally do by hand across multiple DevTools panels.

Leplus also showed Chrome's built-in AI features in DevTools: an "explain with AI" button on console errors, a "debug with AI" option on failing network requests, and an "Ask AI" button on performance traces. These are baked into Chrome itself, not extensions.

A 4GB Model in Your Browser

The most technically surprising section covered the Web AI APIs -- currently in W3C draft status, implemented in Chrome behind flags, with Opera also adding support. These are browser-native APIs that run a local model on the client machine.

Lasorsa and Leplus demonstrated three APIs:

Summarizer: ai.summarizer.create() with options for output type (TLDR, teaser, key points, headline), length, and language. They summarized product reviews live.
Proofreader: Returns corrected text with start and end indices for each correction.
Prompt API: ai.languageModel.create() supporting multimodal input -- text, images, audio -- with structured JSON output via schema constraints. They uploaded a photo of headphones and got a generated product review back.

The model download is roughly four gigabytes, according to Leplus, downloaded once and shared across all websites. Chrome's chrome://on-device-internals page provides debugging for model status and token usage. The speakers emphasized these APIs are highly experimental -- the language specification requirement was added in the week before the talk.

Making Websites Agent-Readable

The second half of the symbiosis argument: if AI builds the web, the web should also feed AI. Lasorsa and Leplus covered two approaches.

llms.txt is a markdown file served at a domain's root that acts as a map for AI agents to discover documentation pages -- a hybrid of robots.txt and sitemaps. There's also an llm-full.txt variant that consolidates all site content into a single file, useful for feeding coding agents up-to-date framework docs. Yohan pointed to Angular.dev as an example already shipping this.

WebMCP is far more ambitious and, as Leplus put it, "very, very highly experimental." The proposal lets websites register MCP tools directly on web pages. Leplus showed two approaches: a JavaScript API where you define a tool with name, description, schema, and execute function, then register it via the browser's navigator object; and a declarative HTML approach using attributes like tool-name, tool-description, and tool-auto-submit on existing form elements. The browser derives the tool schema from form inputs and their labels automatically.

He demonstrated calling these registered tools from both a Chrome extension and from a coding agent IDE, with the tool executing directly on the web page.

The Responsive Design Analogy

Olivier closed with an analogy that frames the stakes clearly.

"It's like responsive design. At some point, you had to adapt your website for mobile. And if you didn't do it, then the competition did it. And then people wouldn't go to your website on the mobile."

His argument is that agent-readiness is the next version of that transition. Websites that don't expose structured interfaces for AI agents -- via llms.txt, WebMCP, or whatever standards emerge -- will be at a disadvantage as agentic browsers become mainstream. Whether the timeline is as compressed as responsive design's was is an open question, but the direction Lasorsa and Leplus describe is concrete enough to act on today.

Yohan Lasorsa and Olivier Leplus spoke at AI Engineer Europe 2026. Lasorsa is a Developer Advocate at Microsoft; Leplus is a Developer Advocate at AWS.

Watch the full talk

Your Digital Exhaust Is the Most Underused Dataset You Own

2026-04-12T00:00:00-05:00

Šimon Podhajský (LinkedIn, X), Head of AI at Waypoint AI, opened with a premise that cuts against the grain of most personal AI demos: what if the most valuable thing an AI can do with your data is simply read it?

Every other personal AI project seems to be racing toward agents that send emails and manage calendars on your behalf. Podhajský built the opposite -- a read-only system that queries six personal data sources (email, journal, tasks, CRM, browser sessions, notes) but can never write back to any of them. He argues this isn't a limitation to overcome. It's the point.

"It's my term for the digital activity that is a byproduct of your cognition, like exhaust fumes for a car engine. Individually, it's just waste, but if you analyze the exhaust, you can diagnose the engine."

Cross-Source Signal Is the Product

The core of Šimon's argument is that individual tools are blind to each other. Your email client doesn't know what you journaled. Your task manager doesn't know what you're browsing. The value isn't in any single data source -- it's in the patterns that emerge when you read across all of them.

He demonstrated three use cases, all requiring cross-source analysis:

Intention-action gaps: Comparing what you said you'd do (tasks, journal entries) against what you actually did (email, browser activity).
Attention drift: Detecting when your browsing patterns diverge from your stated priorities.
Relationship decay: Surfacing contacts you've been neglecting by cross-referencing CRM data against communication patterns.

Three Zones, No Write Path

The architecture is deliberately simple. Šimon described three zones:

Sources -- the six data sources, all read-only. The AI never writes back.
Workspace -- where analysis happens, using structured prompts and Python scripts.
Outputs -- results land in a separate destination (he uses an Obsidian vault), never back into the source systems.

The separation between sources and outputs is load-bearing. Podhajský argues that the moment an AI writes back to your data sources, the exhaust is contaminated -- you can no longer tell which patterns are yours and which are the AI's.

"The moment your AI writes to your data sources, the exhaust fumes are contaminated. You're no longer observing your cognition. You're observing a human-AI hybrid, and you can't tell which patterns are yours."

The Weekly Mirror

In a live demo, Šimon ran a weekly reflection -- a David Allen-style review generated by pulling data from all six sources and synthesizing it into a markdown document. The output covers themes of the week, tensions, commitments, relationship gaps (notable for what's missing, not just what's present), and reflection questions.

He described the output as "occasionally brutal." The system told him he'd been avoiding his most important project for two weeks -- something no single tool would have flagged.

A second demo showed cross-source reading recommendations: given what he's currently reading (pulled from his browser's local SQLite database), who in his network should he discuss it with? The system matched articles to contacts by interest. He noted the CRM integration was the slowest part and the whole thing is token-heavy -- best run in a clean session.

Read Errors Are Free, Write Errors Are Not

Podhajský's risk analysis is where the talk gets sharp. He frames the choice between read-only and agentic AI as fundamentally asymmetric:

"The downside of a read-only error is zero. I just ignore it. The downside of a write error is unbounded."

Or, more bluntly: "I'd rather miss out on automated emails than have a misfire nuke my life."

He also acknowledged the risks that remain. Cross-referencing personal data creates what he calls the mosaic effect -- the same capability that makes the system useful makes it a devastating target if compromised.

He referenced Simon Willison's "lethal trifecta" (private data, untrusted content, external communications) and initially thought his read-only design broke it, but conceded it doesn't fully -- shell access still provides external communication channels.

"I'm not claiming the system is secure. I'm claiming that I've thought about where it isn't and I've decided which risks I'm willing to carry."

A Mirror, Not a Broken Butler

The sharpest reframe in the talk is Šimon's insistence that read-only AI isn't a stepping stone toward "real" agentic AI. It's a different product category entirely.

"A mirror isn't a broken butler."

The industry, he argues, frames read-only as a limitation you graduate from. He thinks that's wrong. The observer -- the system that shows you what you're actually doing versus what you think you're doing -- produces more value per interaction than the agent, in his experience. The race to build personal AI agents that act on your behalf skips over something more fundamental: most people don't have a clear picture of their own cognitive patterns. He published an open-source template for others to try the approach.

"Your digital exhaust is the most underused dataset you own."

Šimon Podhajský spoke at AI Engineer Europe 2026. Head of AI at Waypoint AI.

Watch the full talk | Slides | Personal Intelligence Kit (GitHub) | LinkedIn | X

Your Multi-Agent System Isn't Failing Because of the AI

2026-04-12T00:00:00-05:00

Sandipan Bhaumik (LinkedIn), Data & AI Tech Lead at Databricks, opened with an anecdote that set the tone for the whole talk. A single credit-scoring agent ran for two weeks in production without issues. The team added four more agents. Within days, 20% of risk ratings were wrong -- not because the LLM was hallucinating, but because a caching layer between agents wasn't invalidating correctly. A classic distributed systems race condition on stale data.

"They think adding more agents is just like adding more features. It's not. It's building a distributed system."

His argument: when multi-agent systems break, teams blame the model or the prompts. Almost every time, Bhaumik says, it's the architecture.

The Coordination Complexity Problem

Sandipan points out that going from one agent to five doesn't create five times the complexity. Five agents have at least ten potential coordination points -- each one a failure surface. The math is straightforward (pairwise connections), but teams consistently underestimate it because adding an agent feels like adding a feature.

The fix, he argues, isn't better AI. It's applying decades of distributed systems engineering to a problem space that's pretending those lessons don't exist.

"This is no longer an AI problem. This is a distributed system problem."

Choreography vs. Orchestration

Sandipan breaks agent coordination into two patterns, and argues most teams pick one instinctively and regret it.

Choreography is event-driven and decentralized. Agents publish events to a message bus when they finish work; downstream agents subscribe to the event types they care about. It scales well and makes adding new agents easy. The downside: debugging is brutal without strong observability. You can't trace which agent failed to publish, whether events were consumed, or whether they were consumed twice.

Orchestration is centralized. A workflow orchestrator calls each agent directly, manages parallelism, tracks the full execution graph, handles retries, and logs every step. Agents are deliberately simple -- they take input, do work, return output. He says financial services uses orchestration almost exclusively because rollback capability and auditability matter more than agent autonomy.

His decision framework maps workflow complexity against autonomy requirements across four quadrants.

Simple workflow with high autonomy needs points to choreography. Complex workflow with low autonomy tolerance points to orchestration. Complex workflow with high autonomy needs points to hybrid patterns -- choreography with saga patterns for compensation.

"I've seen teams choose choreography because it feels more agentic, more autonomous. Then they spend months firefighting because they can't debug distributed event flows."

Immutable State Over Shared Mutable State

The anti-pattern Sandipan flags most often: shared mutable state where multiple agents read and write the same database records concurrently. Even with modern database protections, teams use default isolation levels, skip explicit locks, and ship race conditions to production.

His recommended pattern is immutable state snapshots with versioning. Each agent produces a sealed, immutable state version -- append-only inserts, never updates. At each handoff, the receiving agent validates the schema against a data contract before processing. If an agent fails, you roll back to the previous version. For debugging, you replay state evolution from version 1 through version N.

Data contracts enforce that one agent's output schema matches the next agent's expected input. If a research agent outputs data with a confidence score below a threshold, the contract rejects the handoff at the boundary rather than letting bad data propagate three agents downstream.

Circuit Breakers and Compensation

He covers two failure recovery patterns he considers essential for production multi-agent systems.

Circuit breakers wrap every agent call. After a configurable number of consecutive failures, the circuit opens and the system fails fast instead of waiting for timeouts. After a cooldown period, it goes half-open and tests with a single request. This prevents one failing agent from cascading into a full system outage.

"Circuit breakers are the single most important failure recovery pattern for multi-agent systems."

The saga/compensation pattern gives transactional semantics across distributed agents. Every agent implements two methods: execute and compensate. If an agent fails mid-workflow, the orchestrator walks backward through previously successful agents, calling compensate on each to undo their work.

Sandipan acknowledges it's not glamorous work. But it's how production systems handle partial failures without human intervention at 2 a.m.

The Unsexy Work That Keeps Systems Running

Bhaumik's closing is blunt. Demos are easy -- anyone can use an LLM to show something cool. The hard part is everything he covered: choreography versus orchestration decisions, immutable state, circuit breakers. All of it is infrastructure work that won't get applause.

"You won't get applause for implementing a circuit breaker, but you make your systems more reliable. They don't fail at 2 a.m. in the night."

His core argument is that the teams succeeding with multi-agent systems in production aren't the ones with the best prompts or the most capable models. They're the ones treating agent coordination as what it is -- a distributed systems problem -- and applying the patterns that have solved those problems for decades.

Sandipan Bhaumik spoke at AI Engineer Europe 2026. Data & AI Tech Lead at Databricks.

Watch the full talk | Slides | LinkedIn

Your LLM Evaluator Is Probably Lying to You

2026-04-12T00:00:00-05:00

Mahmoud Mabrouk (X, LinkedIn), co-founder and CEO of Agenta AI, opened his AI Engineer Europe workshop with a scenario most teams will recognize: your LLM agent is in production, your observability dashboard looks clean, but customers keep saying the thing doesn't work. The culprit, he argues, isn't the agent -- it's the evaluator. A miscalibrated LLM-as-a-judge gives you false confidence while producing no useful signal.

"You'll find a prompt not very far from this one: 'You'll be given an LLM output, write whether it's a hallucination, make no mistakes.' Now obviously, how the hell would the agent know whether it's a hallucination? If it could, then your app would have worked from day one."

Evaluation as a Machine Learning Problem

Mabrouk's central reframe is that building an LLM-as-a-judge is itself a machine learning problem, not a prompt engineering exercise. The evaluator prompt is a learned artifact that should be optimized against labeled data -- the same way you'd train any model. The quality ceiling of your evaluator, he argues, is determined by the quality of your annotation data and your optimization process, not by how clever your starting prompt is.

This leads to what Mahmoud calls the "holy grail of AI engineering" -- a data flywheel where you optimize your evaluation harness, observe traces in production, add new evals based on edge cases, and repeat.

"The speed in which you move to production or add features is actually the speed in which you can complete this loop."

Four Steps to a Calibrated Judge

Mahmoud's workflow has four stages:

Metric design -- Define evaluation axes from your specific use case, not from generic libraries. He identified four error types through manual analysis of traces: policy adherence, response style, information delivery, and incorrect tool calls. Each gets its own binary evaluator (pass/fail, not a 1-5 scale). He credits Hamel Husain for articulating this error analysis workflow.

Data annotation -- Subject matter experts annotate conversation traces with binary verdicts plus reasoning. Mabrouk stresses the reasoning field is critical -- without it, the optimization algorithm has to independently discover why something failed, which is extremely difficult for complex policy evaluation.
Optimization with GEPA -- GEPA is a prompt optimization algorithm that works like a genetic algorithm. Each iteration samples new candidate prompts (through mutation and merging), evaluates them against mini-batches of training data, then selects the best candidates using a Pareto frontier rather than simple averaging. The Pareto approach maintains diversity -- it picks the best candidate per task before merging into a final prompt.

Validation -- Evaluate the optimized judge on a held-out set to check for generalization.

"Although we went very quickly through the first step and the second step, these are actually the hardest part of the problem. Like in reality, as every data scientist knows, getting your data is the hardest thing."

The Counterintuitive Seed Prompt

One of the more surprising findings Mahmoud shared: the seed prompt that excluded the agent's policy outperformed the one that included it. His hypothesis is that including the full policy from the start traps the optimizer in a local minimum. Starting naive -- "if I don't have any rules, it should say everything is all right" -- with good annotations lets the algorithm explore the prompt space more effectively.

This inverts the common intuition that more information in the starting prompt is always better. In optimization terms, a good starting point isn't necessarily one that's close to the answer -- it's one that gives the optimizer room to move.

Results on a Real Benchmark

Mabrouk tested this on TauBench, a benchmark by Sierra containing 599 conversation traces from a simulated airline support agent. He split the data into 480 training and 112 validation traces, with roughly 62% compliant and 38% non-compliant conversations.

The naive seed judge scored 61% accuracy on validation, according to Mabrouk, with a 98% bias toward predicting "compliant" -- meaning it was essentially rubber-stamping everything. After GEPA optimization, he reports accuracy rose to 74% on the validation set, with the compliant prediction rate dropping to 64%, much closer to the actual distribution.

The optimized rubric had learned parts of the airline policy on its own -- cancellation rules, flight modification procedures, communication requirements -- all discovered through the optimization process rather than hand-written.

Practical Lessons from the Optimization Loop

Mahmoud shared several hard-won lessons from iterating on this approach:

Smaller or older models failed as both judge and reflector for this complexity level. His best results came from mixing models -- Gemini for reflection, Grok for the judge -- though GPT-4.1 Mini for both also worked.
He wrote a custom reflection template rather than using the default, embedding domain-specific priors about how to discover policy rules.
Start with small iterations, visualize the generated candidates and reasoning, and overfit to training data first before scaling up.
The Pareto frontier reached 100% on training data -- meaning for every training task, some candidate prompt solved it -- but consolidating all that knowledge into a single merged prompt remained the bottleneck.

"It's not an algorithm that you just take and it works from day one, unless for kind of toy examples."

Treat It Like ML, Not Prompt Engineering

Mabrouk's takeaway is straightforward: stop treating LLM evaluation as a prompt writing exercise and start treating it as a machine learning problem. Get labeled data from domain experts, optimize your evaluator systematically, and validate on held-out examples. According to his experiments, the optimization runs cost a few hundred dollars in tokens and take about an hour -- a modest investment for evaluators you can actually trust.

Mahmoud Mabrouk spoke at AI Engineer Europe 2026. Co-founder and CEO at Agenta AI.

Watch the full talk | Agenta on GitHub | LinkedIn | X

When Every Team Builds Its Own AI Agent, You Need a Registry

2026-04-12T00:00:00-05:00

Sonny Merla, Mauro Luchetti, and Mattia Redaelli (Quantyca) opened with a question that any large organization experimenting with AI agents will recognize: what happens when dozens of teams across multiple continents are all building agents independently?

"What happens when you have dozens of teams across three continents, all building AI agents, each one wiring up their own connections, reinventing their own security model, deploying their own infrastructures? You get chaos."

Their answer, built for a global enterprise operating across 26 countries, is a registry-based architecture that makes governance a side effect of deployment rather than a gate before it.

The Problem With Letting Teams Solve It Themselves

Sonny described the familiar pattern: teams build what they need, wire up their own security, deploy their own way. Multiply that across an enterprise and you get no central visibility into what AI tooling exists, no standardized deployment process, and no way to trace which business use cases depend on which models and services.

The challenges, as he framed them, fall into three buckets: maintenance and operations (keeping everything running), governance and compliance (knowing what's deployed and who owns it), and enterprise scaling (making it work across the whole organization without slowing anyone down).

Governance as a Byproduct of Shipping

The core architectural idea is straightforward. Rather than creating an approval process that sits between developers and deployment, they built a system where deploying an MCP server or an A2A agent automatically publishes its metadata to a central registry through CI/CD.

Mauro described it as making agent development "self-documenting." Tag a branch, and a GitHub Action publishes both the Docker image and the metadata -- an agent card for A2A agents, a server.json for MCP servers -- to the registry catalog.

"We want to make easy the life of developers to focus on the business logic inside the use cases, avoiding to reinvent the wheel every time we need to take care about the security, but also the deployment and maintenance of the use cases."

The result is that governance doesn't require extra work from developers. It happens because they shipped.

Three Registries, One Graph

The architecture centers on three interconnected registries.

The MCP Registry is a private extension of the open-source MCP registry specification. It contains both custom internal servers and a curated subset of approved public ones. Each entry carries enterprise metadata: ownership, environment, authentication model, and cost attribution.

The A2A Registry is based on agent cards from the Agent-to-Agent protocol. When an agent deploys, its card is automatically published. Other agents and developers can discover it immediately.

The Use Case Registry ties the other two to business context. It maps agents and tools to specific use cases and tracks which AI models each depends on.

Mauro argued this metadata isn't optional decoration:

"These are not simply metadata that are nice to have. This is something that really brings out the impact analysis functionality."

The registries together enable lineage analysis -- tracing the full dependency graph from a use case down through agents, MCP servers, and models. If a model goes down, you can see what breaks.

The Platform Layer

Sitting above the registries is a unified AI gateway that handles authentication, per-use-case budgeting, and central auditing of all LLM requests. Sonny described monthly and weekly cost caps that erode as tokens are consumed, giving each use case its own budget envelope.

Below the registries, two template repositories -- one for MCP servers, one for A2A agents -- give developers a starting point with authentication, cost tracking, and observability pre-configured. Mattia noted the A2A template is framework-agnostic, using interfaces so teams can implement with whatever agent framework they prefer.

The runtime discovery flow works through an API gateway with proxies that look up backend URLs from the registry catalog. Agents authenticate via a separate header to the actual backend.

Where This Stands

The speakers were transparent that the platform is not yet in production. Mauro noted it is "still in progress," and the demo used sample data. The architectural patterns -- self-registering services, metadata-rich catalogs, lineage graphs -- are well-established in data engineering. The contribution here is applying them specifically to the MCP and A2A ecosystem.

The bet is that as internal MCP servers and A2A agents multiply, the discovery and governance problem will hit every large organization. Building the registry into the deployment pipeline, rather than bolting it on after, is how they propose to keep it from becoming another compliance burden that developers route around.

Visibility First, Control Second

Sonny, Mauro, and Mattia are making a case that the scaling problem for enterprise AI agents isn't technical capability -- it's visibility. If you can't see what's deployed, who owns it, and what depends on what, you can't govern it. And if governance requires extra work, developers won't do it. The registry approach makes the right thing the default thing.

Sonny Merla, Mauro Luchetti, and Mattia Redaelli spoke at AI Engineer Europe 2026. Merla is Global Data Science and AI Manager at Amplifon; Luchetti is AI Center of Excellence Manager and Redaelli is AI Engineer at Quantyca.

Watch the full talk | Slides

RAG Isn't Dead, You Just Need a Better Starting Point

2026-04-12T00:00:00-05:00

Phil Nash (X, LinkedIn), a developer relations engineer at IBM, opened his AI Engineer Europe talk by taking aim at the "RAG is dead" discourse. His counter is simple: if every business had less than a million tokens of data, maybe. But they don't, and not everyone wants to pay for a million input tokens on every query. RAG isn't going anywhere -- it's just harder than people think.

"It turns out that RAG is actually hard, and it's hard for different reasons for different projects."

The Problem Isn't the Loop

Phil argues that the basic retrieve-and-generate pattern isn't where teams struggle. The difficulty is everything around it: parsing messy documents (especially PDFs), choosing embedding models, tuning chunking strategies, and adapting search to specific data and user patterns. Every organization's documents and users are different, so there's no universal solution.

His reframe: what's missing isn't a better algorithm but a better starting point. An opinionated-but-flexible baseline built from open-source components -- sensible defaults with every layer exposed for customization. He presented a stack combining three open-source projects (Docling for document processing, OpenSearch for search, and Langflow for orchestration) as one such baseline, currently at version 0.4.0.

Parsing Is the First Hard Problem

The document processing layer uses Docling, an open-source library from IBM Research Zurich. Phil walks through why parsing matters: RAG applications ingest HTML, Markdown, Word docs, slides, spreadsheets, audio, video, and PDFs. Each format has different challenges.

Docling runs multiple processing pipelines depending on file type:

Simple pipeline -- text extraction and hierarchy for straightforward formats like Markdown and HTML
ASR pipeline -- automatic speech recognition for audio and video
Standard PDF pipeline -- a collection of small focused models for layout analysis, table extraction, and image extraction, with optional OCR for scanned documents
VLM pipeline -- a 258-million-parameter vision model (Granite Docling) that extracts everything in a single pass

All of these produce an intermediate representation -- an XML-like format called DocTags -- that can be converted to Markdown, HTML, or JSON. The chunker then uses the document's parsed structure rather than arbitrary character counts, which Nash presents as a meaningful improvement over naive chunking strategies. The whole thing runs offline, which matters for air-gapped environments.

Search Beyond Vectors

The retrieval layer uses OpenSearch, the open-source Elasticsearch fork. Nash's point here is that vector search alone isn't enough -- hybrid search combining vectors and keyword matching gives better results out of the box.

A few features he highlights:

Multiple embedding models simultaneously -- useful when migrating between embedding models, though he notes it slows search down
Configurable filtering and aggregation -- exposed to end users as "knowledge filters" so they can scope queries to specific document sets
JVector -- a disk-ANN-based vector index plugin that replaces the default HNSW/IVF options. The key property: indexes don't need to fit entirely in memory, and it supports live indexing

Making Retrieval Agentic

The most interesting architectural choice Phil describes is replacing the traditional retrieval pipeline with an agent. Instead of the standard embed-query, top-K, stuff-into-prompt flow, the user's query goes to an agent equipped with search tools. The agent decides what searches to run and how many.

In his demo, the agent has access to an OpenSearch search tool, a calculator, and an MCP-based URL ingester. He demos the generation side running Granite 4 3B locally through Ollama, with Qwen3 Embedding 0.6B handling the embeddings -- also local.

"They're language models, not math models. So a calculator is always useful."

The orchestration layer (Langflow) provides a drag-and-drop visual editor for these flows. Nash demos adding guardrails to the agent pipeline by dragging in components -- the point being that customization doesn't require rewriting code.

Everything Is Swappable

Phil is explicit that the stack isn't prescriptive about model providers. It supports OpenAI, Anthropic, WatsonX AI, and Ollama for both embedding and generation. The demo uses fully local models, but you can swap in hosted APIs without changing the pipeline.

The project also includes cloud connectors for Google Drive, SharePoint, and OneDrive for document syncing, a chat UI with suggested follow-up prompts, and API keys for using the search and agent capabilities in external applications. There's also an MCP server available for handing the stack off to other agents.

The Takeaway

Nash's closing argument is that whether RAG is "solved" depends entirely on your data and your users. The answer isn't a single algorithm -- it's a stack that gives you a strong baseline and lets you tune every layer against your own evaluation criteria.

"Well, that's kind of up to you, to your data and to your users."

Phil Nash spoke at AI Engineer Europe 2026. Developer relations engineer at IBM.

AI Agents Can't Walk Upstairs and Ask for Help

2026-04-12T00:00:00-05:00

Juan Herreros Elorza (LinkedIn, GitHub), Team Lead on the Cloud Native Technology team at Banking Circle, makes a deceptively simple argument: the platform engineering practices that have always been "best practices" are now prerequisites. Not because they've changed, but because AI coding agents have become first-class users of internal developer platforms -- and agents can't compensate for the gaps that humans have been working around for years.

"If this situation was tricky for a developer, this situation is essentially impossible for a machine, because the machine is not going to go and try the pipeline and then go up to the second floor and talk to the person in that other team."

The New Developer Who Can't Improvise

Juan opens with a story anyone in a large engineering org will recognize. A new developer joins, writes their application, and hits the deployment wall. They copy a CI pipeline from a teammate. They chase down someone on the infrastructure team for a database. They wait days. Eventually, through Slack messages, hallway conversations, and borrowed tribal knowledge, they get their service running.

Humans muddle through this. Agents cannot. An agent can't wander over to the infrastructure team's desk. It can't read the room to figure out which Slack channel has the person who knows the answer. Every place where your platform relies on implicit knowledge or human intervention is a place where an agent hits a dead end.

He frames this not as doom but as opportunity. Organizations that have struggled to get buy-in for platform improvements now have executive attention on AI. That attention can fund the work that platform teams have been advocating for all along.

Six Principles for Agent-Ready Platforms

Drawing from his experience building an internal developer platform at a fintech, Herreros Elorza lays out six principles. None of them are new -- that's his point.

1. Self-service. Remove humans from all provisioning and deployment paths. If a developer or agent needs a resource, they should get it without filing a request or waiting on another team. Juan is specific about what doesn't count:

"If it is technically self-service, but it requires fetching some building blocks from five different places and putting them together and then triggering a flow somewhere else, then it's not really self-service."

2. API-based interfaces. Everything exposed through well-defined APIs with schema validation. Agents are good at calling structured APIs and discovering what's available. CLIs, MCP servers, or other wrappers on top are fine, but the API is the foundation. Schema validation means agents naturally construct valid requests.

3. Local-first. Since agents typically run on the developer's machine, make it possible to validate everything locally. Don't force an agent to push to version control and wait for a remote CI pipeline to fail minutes later. Local validation means tight iteration loops.

4. Documentation. Two strategies depending on scale -- docs next to the code for smaller repos, centralized docs for platform-wide concerns. Serve documentation via API so agents get structured content rather than parsing HTML. Use agent-specific files like agents.md or claude.md to describe build, test, deploy, and verification conventions.

5. Encourage contributions. Agents lower the barrier to contributing to platform code, so platform teams should expect more pull requests from product teams. But -- and Herreros Elorza emphasizes this -- the platform team still owns maintenance. Combine hard guardrails (security policies, compliance checks) with soft guidance (instruction files that steer agent behavior toward good patterns).

6. Measure outcomes. Use metrics to verify that platform changes actually helped. He references DORA metrics for delivery performance, reliability metrics for operational health, and platform-specific metrics like support request volume as a proxy for self-service effectiveness.

Rethinking Observability for Agents

One point Juan singles out deserves its own attention. Humans verify deployments by looking at dashboards -- graphs, charts, color-coded status panels. Agents can't do that.

"You also need to think: how does observability look like if the prime user is going to be an AI agent?"

Logs, metrics, and traces need to be available through APIs, CLIs, or MCP servers so agents can programmatically verify that their work succeeded. This is a subtle but important shift: observability systems were built for human visual consumption. Making them machine-readable is a prerequisite for agents that can autonomously deploy, verify, and iterate.

Use the Momentum

Juan's closing point is practical. AI agents don't require a new set of platform engineering principles -- they make the existing ones non-negotiable. And right now, there's organizational willpower to fund the work.

"Take advantage. Everyone from the executive level to the individual contributors are looking at AI now. It is a very hot topic. So you can use AI as the excuse to implement some best practices that, again, were always best practices if you didn't have the chance to do it until now."

Juan Herreros Elorza spoke at AI Engineer Europe 2026. Team Lead, Cloud Native Technology at Banking Circle.

Watch the full talk | Slides | LinkedIn | GitHub

Fitting the Model Isn't the Same as Running It Well

2026-04-12T00:00:00-05:00

Mozhgan Kabiri Chimeh (LinkedIn), a developer relations manager at NVIDIA, opened her AI Engineer Europe talk with the pain point that drives most AI developers to the cloud: you either run out of memory or you don't have the right software stack. The result is that development iteration speed depends on shared infrastructure, where your work gets scheduled against everyone else's compute jobs.

Her talk walks through benchmarking open-source models from 1.5 billion to 14 billion parameters on a local workstation, with a focus on the trade-offs between throughput, latency, and quantization format. It's a data-driven argument for when local inference makes sense -- and what actually determines whether it's viable.

"This isn't a theoretical talk, it's a data-driven journey through the trade-offs of modern AI infrastructure."

Memory Capacity Is Not Memory Bandwidth

The central insight Kabiri Chimeh presents is a distinction that's easy to overlook. A workstation with 128 GB of unified memory can fit models up to roughly 200 billion parameters. But fitting a model into memory is not the same as running it at useful speeds.

Throughput is governed by how efficiently the system moves data through memory, not just how much it can hold. She argues that this is where most local inference setups fall short -- developers load a model, confirm it runs, and then discover the tokens-per-second rate makes interactive use impractical.

"Memory capacity is not the same as memory bandwidth."

Quantization as the Decisive Lever

This is where Mozhgan's benchmarks get interesting. She tested the Qwen model family at different sizes and precision formats, and the results show that quantization format choice matters as much as the hardware itself.

The headline numbers for a 14 billion parameter model:

Base (unquantized): 8.40 tokens/second
4-bit quantized (NVFP4): 20.19 tokens/second

That's a 2.4x improvement from quantization alone -- on the same hardware, with the same model. For context, she notes that 20 tokens per second exceeds average human reading speed, which puts it in the range of viable interactive use.

At the smaller end, a 1.5 billion parameter instruct model hit 61.73 tokens per second. The pattern is clear: model size sets the ceiling, but quantization determines whether you're anywhere near it.

"On Blackwell hardware, the choice of quantization format is just as important as the hardware itself."

She describes 4-bit floating point quantization as effectively increasing "intelligence per byte" -- allowing a 14 billion parameter model to feel as responsive as a much smaller one.

Benchmarking That's Worth Reproducing

Mozhgan doesn't just present numbers -- she walks through the methodology in detail, which is arguably the most transferable part of the talk. Her benchmarking harness follows a strict protocol:

Environment isolation via Docker containers
Three mandatory warm-up runs before any measurement
Background GPU metrics logging at one-second intervals
Each run generates a unique directory with timestamp and sanitized model ID
Full capture of model endpoint response and metrics
Versioned artifacts containing metadata and benchmark results

She measures two key metrics: completion tokens per second (raw throughput) and time to first token (TTFT), which captures user-perceived responsiveness. The TTFT measurement uses explicit streaming response handling, timestamping the first chunk from the model server.

The TTFT results reinforce the quantization story: the 4-bit quantized 14B model is 3.4x faster to first token than the unquantized version.

When Local Compute Is the Right Choice

Mozhgan frames local inference not as a replacement for the cloud but as a complement. She identifies three use cases where it makes the most sense:

Steady-state workloads -- predictable inference demand that doesn't need elastic scaling
Privacy-sensitive data -- when data governance means nothing leaves the building
Rapid prototyping -- fast iteration cycles without waiting for shared infrastructure

"The key idea here is not replacing the cloud, but bringing powerful AI development closer to the developer."

The software stack she demonstrates uses the same serving framework (vLLM) and containerized environment that runs in data center deployments. Her point is that workflows developed locally can move to larger infrastructure without rearchitecting -- the iteration happens close to the developer, and the scaling happens later.

The Takeaway

Kabiri Chimeh's argument comes down to a practical framework: if your model fits in memory, the next question isn't whether it runs -- it's how fast. Quantization format is the lever that determines whether local inference is a viable development workflow or a frustrating bottleneck. Match your quantization to your use case, benchmark rigorously, and scale out only when the workload demands it.

"Run locally, iterate quickly, and when ready, scale to data center or cloud."

Mozhgan Kabiri Chimeh spoke at AI Engineer Europe 2026. Developer relations manager at NVIDIA.

Watch the full talk | build.nvidia.com/spark | LinkedIn

The Real Problem Is the Six Minutes After the Call

2026-04-12T00:00:00-05:00

Dippu Kumar Singh (LinkedIn), Leader of Emerging Technologies at Fujitsu North America, presents a talk that starts where most AI discussions stop. Most generative AI demos assume clean text input. In a contact center, the data starts as messy, overlapping, emotionally charged audio -- and the engineering challenge isn't transcription. It's what happens after the call ends.

"We cannot hire more people, we have to fundamentally engineer the stress out of the workflow."

The After-Call Work Problem

Singh's central reframe is about where the ROI actually lives. He shares internal baseline data showing the average contact center call lasts about 6.5 minutes -- and the post-call administrative work takes another 6.3 minutes. Operators spend nearly half their working hours on data entry, not talking to customers.

That near 1:1 ratio between talk time and paperwork creates a stress-turnover spiral. Operators burn out. Attrition rises. And as Singh argues, hiring more people doesn't fix a structural problem.

The pipeline he describes isn't designed to replace the operator. It's designed to auto-populate structured output that the operator validates with a quick visual check and a confirm click -- turning a 6.3-minute writing task into a seconds-long review task.

Four Stages of the Pipeline

The system Dippu describes is a four-stage architecture deployed in a high-volume contact center.

Voice Capture handles real-time audio intake with noise filtering and level normalization. The critical detail here is stereo channel separation -- isolating the agent on one channel and the customer on the other.

"If you mix them into a single mono track, overlapping with each other, the AI will struggle to figure out who said what and thereby ruining the entire downstream summary."

PII masking happens at this stage too. Credit card numbers and passwords are stripped from the audio buffer before anything reaches the LLM.

Speech-to-Text converts the cleaned audio. Singh says the STT accuracy needs to exceed 90% for the downstream AI to function. Domain-specific dictionaries help -- distinguishing "term life" from "turn right" in an insurance context, for example. Post-processing handles inverse text normalization, so spoken "five thousand dollars" becomes "$5,000" before entity extraction.

Generative AI Core is structured in three layers. An orchestration layer uses few-shot prompt libraries rather than open-ended "summarize this call" instructions. A reasoning layer classifies the call against a predefined list of categories (cancellation, new application, claim status) and outputs its reasoning. A trust layer handles token optimization for latency and automated hallucination checks.

Customer Data Sync maps the LLM's JSON output to CRM fields via an API gateway. The operator sees the AI-generated summary auto-populated on screen, makes any corrections, and confirms.

Why "Just Summarize" Doesn't Work

One of the more concrete points Dippu makes is about prompt design. He argues that asking an LLM to summarize a call produces a messy narrative paragraph -- not something you can feed into a CRM.

The alternative: structured few-shot prompts that instruct the LLM to output separate bullet-point lists -- one for customer inquiry, one for operator actions. The model receives a predefined list of call reasons and must classify against them while showing its work.

"This strict formatting is what turns an unstructured conversation into a database-ready asset."

This is where the audio quality decisions upstream pay off. Speaker separation means the LLM can distinguish customer intent from operator chit-chat. Without it, the structured extraction falls apart.

The Results

Singh reports that after-call work dropped to 3.1 minutes -- roughly a 50% reduction. Across what he describes as a 500-seat operation handling thousands of calls per day, that translates to what he characterizes as the equivalent of reclaiming dozens of full-time headcounts in productivity.

Structured data from the pipeline also flows into BI dashboards that aggregate voice-of-customer patterns and auto-flag candidates for new FAQ entries -- a secondary benefit that turns call data into a strategic asset.

What Comes Next

Dippu closes with the argument that this kind of pipeline transforms contact centers from high-stress call centers into what he calls "intelligence-gathering engines."

"By applying these rigorous engineering techniques to the messy audio data, we can definitely transform the contact centers from call centers of high stress into highly efficient intelligence-gathering engines that protect their workforces."

Dippu Kumar Singh spoke at AI Engineer Europe 2026. Leader of Emerging Technologies (Apps) at Fujitsu North America.

Watch the full talk | Slides | LinkedIn

AI-Generated Code Is Just Untrusted Code From the Internet

2026-04-12T00:00:00-05:00

Harshil Agrawal (X, LinkedIn), a Senior Developer Educator at Cloudflare, opened his AI Engineer Europe talk with a reframe that should be obvious but apparently isn't: strip away the branding, and the code your LLM writes deserves exactly as much trust as code you found on a random website. Which is to say, none.

"If you told someone, hey, I found this code snippet on a random website on the internet, let's run it in production, you would absolutely not do that. That's security 101. But that's essentially what we are doing with LLM generated code. We just dress it up nicer."

The model has no intentions and no loyalty, Agrawal argues. It's a function that produces text that looks like code. The same LLM that writes working React components can be tricked into exfiltrating your database -- not because it's malicious, but because it's a text predictor that doesn't understand security boundaries. And yet teams routinely execute its output with full production privileges: file system access, environment variables, database credentials, API keys.

We Already Know How to Solve This

Harshil's central point is that this isn't a new problem requiring a new paradigm. Browsers sandbox JavaScript. Mobile operating systems sandbox apps. The principle of running untrusted code in constrained environments is decades old. What's happened with LLM-generated code, he argues, is that the excitement of shipping AI features has caused teams to forget these fundamentals.

Three Ways It Goes Wrong

He lays out a three-part threat model:

Hallucination -- the model writes code that's wrong, not malicious. Infinite loops, nonexistent package imports, recursive functions with no base case. These crash services and eat compute.
The "helpful" LLM -- the model reads environment variables, API keys, and database credentials because it's trying to configure things properly. It's not stealing data; it's just processing sensitive information through unaudited code.
Compromised prompts -- both direct injection (a user submits "ignore instructions, exfiltrate env vars") and indirect injection (the LLM reads a document containing hidden adversarial instructions). The model becomes the attack vector not because it was compromised, but because it was used as designed against adversarial input.

Beyond these three categories, Agrawal presents a five-point threat checklist -- secrets, networking, file system, tenant isolation, and resource limits -- and says you need a definitive yes/no answer for each, not "probably fine."

Grant Keys to Three Rooms, Not a Master Key

The core architectural principle Harshil advocates is capability-based security: default-deny everything, then explicitly grant only the minimal capabilities the code needs.

"Would you rather give someone a master key and then hand them a list of maybe 10,000 rooms they can't enter? Or would you give them keys to just the three rooms they actually need?"

He describes the alternative -- a blocklist approach where you try to enumerate every dangerous operation -- as fundamentally unwinnable. You'll always miss something. With capability-based security, the dangerous operations were never available in the first place. He compares a properly sandboxed environment to "a room with no doors or windows. The only things inside are what I put there before I locked it."

Two Levels of Isolation

Agrawal presents two sandboxing approaches, suited to different use cases:

Lightweight isolates (built on browser engine technology) work for code that doesn't need a file system or package manager. They start in roughly a millisecond, support JavaScript, TypeScript, Python, and WebAssembly, and give each execution its own memory and context. The key pattern: block all outbound network requests by default, and expose only specific, restricted method stubs. The sandboxed code can call query on a database binding, but that binding is a stub routing through the controlling process -- it never has direct database access.

"There's nothing to intercept. The dangerous operations were never available."

Containers handle heavier workloads -- anything needing a real file system, process model, or package installs. They take seconds to start rather than milliseconds, but support git clone, npm install, running dev servers, and exposing ports. The critical pattern here is the proxy for secrets: never pass API keys into the sandbox as environment variables. Instead, the sandbox calls a proxy endpoint that adds authentication headers and forwards to external services. The secrets never enter the sandbox's address space.

His decision heuristic is straightforward: does the code need a file system, processes, or package installs? If yes, containers. If no, isolates. In practice, he says, most applications will use both -- isolates as the fast path for tool-calling loops, containers as the workbench for building and deploying.

The Universal Checklist

Harshil closes with an eight-item checklist he says applies regardless of sandboxing approach:

Default deny network access
Grant explicit capabilities, not broad access
Isolate per user -- one user, one sandbox
Set resource limits (timeouts, memory caps, CPU limits)
Keep secrets outside the sandbox (proxy pattern)
Destroy sandboxes when done (try/finally, max lifetimes)
Log everything (what code ran, when, who triggered it, what it did)
Validate input before it hits the sandbox (length limits, syntax validation, dangerous pattern detection)

Old Principles, New Context

Sandboxing AI-generated code isn't a new security paradigm, Harshil argues -- it's an old one applied to a new context. The tools and principles already exist. The gap is that teams aren't using them because the AI framing makes the risk feel different when it isn't.

"The cost of an extra sandbox is always less than the cost of a data leak."

Harshil Agrawal spoke at AI Engineer Europe 2026. Senior Developer Educator at Cloudflare.

Watch the full talk | Slides | LinkedIn | X

The Security Cliff Between Local and Production MCP

2026-04-12T00:00:00-05:00

Tun Shwe (LinkedIn) and Jeremy Frenay (LinkedIn), both AI Engineers at Lenses.io, gave a joint talk at AI Engineer Europe 2026 on what happens when MCP servers leave the safety of a developer's laptop. Their central claim: most MCP servers are built for single-player local development and collapse the moment you try to run them in production. There's no gradual on-ramp -- you go from zero security surface to needing OAuth, token management, CORS, TLS, and rate limiting all at once.

"If you get the design wrong, no amount of OAuth will save you."

Agents Are Not Humans with API Keys

Shwe frames the problem through three dimensions where agents differ from humans -- a framework he attributes to Jeremiah Lowen, creator of Fast MCP. Each dimension casts what he calls a "security shadow."

Discovery. A human reads API docs once and picks the endpoints they need. An agent enumerates every tool and reads every description on each connection. Every tool description becomes a surface for tool poisoning -- hidden instructions invisible in the UI but followed by the model.

Iteration. A human reruns a script in a second. An agent sends the full conversation history with each retry. Each round trip is a data leakage opportunity, including sensitive data from previous tool calls.

Context. Humans bring decades of intuition. An agent has a fixed context window. Unfiltered data in that window hands PII and credentials to a model that can be tricked into exfiltrating them.

Five Principles for Secure Agentic Design

Shwe's argument is that poor MCP design and poor MCP security are the same problem. He lays out five principles that reduce security exposure before you write a single line of authentication code:

Shrink the attack surface by design. Consolidate fine-grained operations into single coarse-grained, outcome-oriented tools. One permission check, one audit log entry, one authorization enforcement point. "Fewer doors, fewer locks to manage."
Constrain inputs at the schema level. Accept top-level primitives and enums. Use validation libraries like Pydantic. Reject free-form nested payloads -- an unconstrained string argument passed to a shell or query engine is a command injection waiting to happen.
Treat documentation as a defensive layer. Tool poisoning works by embedding malicious instructions in tool descriptions. Complete, unambiguous documentation crowds out the space a poisoned neighboring server would try to fill.
Return only what the agent needs. Oversized responses dump potential PII into a context window that can be tricked into exfiltrating it. Strip payloads to the minimum.
Minimize the blast radius. Scope permissions at the tool and resource level, not the session level. Use read-only annotations. Every tool removed is an attack vector eliminated.

"A badly designed MCP server is also a badly secured one."

The Cliff from Local to Remote

Standard IO mode -- a local process, single user, no network exposure -- works fine for individual developer productivity. But Shwe argues there's no halfway house between that and production.

"You can't do a little bit of production. You're either behind the wall or you're standing out in the open."

Moving to production means streamable HTTP transport, remote deployment, multiple clients, horizontal scaling, and centralized governance. The jump is abrupt: OAuth, token management, CORS, TLS, SSRF protection, rate limiting, and audit logging all land at once.

He cites load tests attributed to StackLock showing that standard IO transport buckled under even modest concurrency -- according to the cited results, 20 out of 22 requests failed with just 20 simultaneous connections.

OAuth's Client Identity Problem

Frenay takes over for the authentication deep-dive. The first approach most teams reach for -- long-lived API keys -- scales poorly. Keys are rarely rotated, not scoped to specific actions, often shared across systems, and stored in config files. In a remote deployment, the MCP server may simply pass the key through to an upstream API, creating what Frenay describes as a confused deputy vulnerability.

Dynamic Client Registration (DCR) was the initial answer. A client self-registers against the authorization server and gets a new client ID. But registrations aren't portable across devices, the registration endpoint is vulnerable to phishing, and the server blindly trusts self-asserted client metadata.

The approach Frenay presents as the current direction is Client ID Metadata Document (CIMD). Instead of self-registering, a client exposes its identity metadata on a public HTTPS URL. The authorization server fetches this metadata during authorization. Identity is proven by domain control rather than self-assertion, redirect URIs are explicitly bound in the metadata document, and the authorization server can selectively allow or deny clients.

Beyond Authentication

Frenay closes by arguing that OAuth alone isn't enough for enterprise deployment. He lists four additional requirements: role-based access control scoped at the individual tool level, data masking for PII fields before the agent sees them, audit logging that captures which agent called which tool with what parameters and what data came back, and end-to-end observability across the full request lifecycle.

"If you cannot trace what an agent did end to end, you cannot govern it."

The Takeaway

Shwe and Frenay's core argument is that MCP security isn't a layer you bolt on after building your tools -- it's a consequence of good tool design itself. Get the five design principles right and you've already shrunk your attack surface before touching OAuth. Get them wrong, and no amount of authentication infrastructure will save you.

Tun Shwe and Jeremy Frenay spoke at AI Engineer Europe 2026. AI Engineers at Lenses.io.

Watch the full talk | Slides | Lenses MCP Server (GitHub) | Tun Shwe LinkedIn | Jeremy Frenay LinkedIn

Your AI Agent Is a Junior Developer. Manage It Like One.

2026-04-11T00:00:00-05:00

Brendan O'Leary (LinkedIn, X), a Developer Relations Engineer at Kilo Code, opened his AI Engineer Europe talk with a observation that's easy to nod along with and hard to act on: most engineers have used AI tools by now, but almost none of them can articulate how they actually work with them. What do they hand off? What do they keep? How do they decide? That gap -- between using AI and working with it -- is where O'Leary spent his 27 minutes.

"Tools are things that you pick up and put down. You use a hammer. You don't work with a hammer."

The Junior Developer Mental Model

O'Leary's central framing is blunt and practical:

"You kind of have to think about your AI agent as an energetic, enthusiastic, extremely well-read, often confidently wrong junior developer."

This isn't just a throwaway analogy. He uses it as an actual management framework. The skills that make someone a good engineering manager of junior developers -- giving clear context, scoping work tightly, reviewing output carefully -- are exactly the skills that transfer to working with coding agents. If you've ever handed a vague ticket to an intern and gotten back something technically functional but completely wrong-headed, you already understand the failure mode.

Context Is the Bottleneck

The bulk of Brendan's technical content centers on what he calls context engineering -- the idea that the quality of an agent's output is bounded by the quality of context you give it. He claims context quality starts to degrade once the context window is roughly half full, and that stale or irrelevant context can actively poison output.

His four practices for managing context:

Persist information outside the context window -- scratch pads, memory files, and project-level instruction files so the agent can access knowledge without bloating the live session.
Be selective -- only pull in what's relevant for the current step. He specifically warns against leaving unnecessary tool integrations enabled, since each one adds tokens to the system prompt on every interaction.
Summarize and compress -- after a long debugging session, distill the context down to just the problem and solution before moving to implementation.
Isolate -- split work across parallel agents or sessions to prevent context accumulation. He points to the rise of parallel agent workflows over the past several months as a direct response to this problem.

The selectivity point lands hard. O'Leary describes a scenario where a database integration tool is left enabled during frontend work -- it wastes tokens and can mislead the agent into touching things it shouldn't. The principle is the same one that applies to human developers: don't leave irrelevant tools and docs scattered across someone's desk and expect focused work.

Research First, Code Last

O'Leary's recommended workflow inverts the assumption that AI's value is in code generation speed. He argues the real leverage comes before any code gets written.

He quotes Dex Horthy: "A bad line of research can potentially be hundreds of lines of bad code."

The workflow breaks into three phases:

Research: Use a restricted mode where the agent can read files and discuss but cannot write code. The goal is to understand the system, identify relevant files, map data flow, and brainstorm edge cases. The output is a research document the engineer reviews.
Plan: Outline specific files to create or change, define verification steps and test strategies, explicitly scope what's in and out. The output is a plan file with step-by-step instructions.
Implement: Start a fresh session with only the plan as context. This keeps context lean, enables careful per-change review, and -- because the hard thinking is already done -- can even use smaller, cheaper models.

The fresh-session trick is worth highlighting. By discarding the research and planning context and starting implementation with just the plan document, you sidestep the context degradation problem entirely. The plan becomes the distillation -- all the thinking, none of the noise.

Configuring Agent Behavior

Brendan breaks agent configuration into three layers:

Modes -- role-based behavioral configurations that constrain what the agent can do (research-only, planning, coding).
Project-level instruction files -- always-on rules covering conventions, build commands, and pre-commit requirements. O'Leary describes these as becoming a "de facto standard" across tools.
Skills -- on-demand reusable playbooks for specific workflows, like generating changelogs or creating assets from templates.

He also touches on connecting agents to internal APIs, listing four approaches for enterprises: use existing OpenAPI specs, convert API docs to markdown stored in the repo, provide a reference URL the agent pulls fresh each time, or build a custom integration server for complex multi-system workflows.

Thinking Is the Job

O'Leary's talk keeps circling back to one idea: the agent amplifies whatever you bring to it. Good preparation produces good code. Sloppy preparation produces confident-sounding garbage.

He quotes Dex Horthy again to make the point stick:

"AI can't replace thinking. It can only amplify the thinking you've done or the lack of thinking you haven't done."

The actionable takeaway from Brendan's talk is that the highest-leverage skill for working with coding agents isn't prompt engineering or tool configuration -- it's the unglamorous work of researching a problem thoroughly, writing a clear plan, and scoping the implementation tightly before letting the agent touch a single line of code. The same things that make code reviews go smoothly with human developers make agent output worth keeping.

Brendan O'Leary spoke at AI Engineer Europe 2026. Developer Relations Engineer at Kilo Code.

Watch the full talk | path.kilo.ai | LinkedIn | X

MCP Tools Are Raw Material, Not Finished Products

2026-04-11T00:00:00-05:00

Nimrod Hauser (LinkedIn, X), a founding engineer at Baz, opened his talk at AI Engineer Europe with a deceptively simple observation: public MCP servers ship tools designed for everyone, which means they're optimized for no one. When you plug generic tools into a production agent, the agent hallucinates URLs, saves files in the wrong places, and picks the wrong tool for the job.

"Agents are already non-deterministic, unpredictable things. You give them tools and you get unpredictability at scale."

His framing is that MCP tools are "glorified integration code written by a third party." The descriptions attached to those tools are the primary mechanism by which agents decide what to do -- and generic descriptions produce generic, often wrong, agent behavior. The fix isn't better prompts or a smarter model. It's reshaping the tool layer itself.

The Setup: A Spec Reviewer That Doesn't Work

Hauser demonstrated with a concrete example: an agent that reads a ticket and a Figma design, opens a browser via Playwright's MCP server, navigates to the deployed application, and checks whether the implementation matches the spec -- producing a pass/fail verdict with screenshot evidence.

The baseline version loaded all 21 Playwright MCP tools unmodified. The result: the agent hallucinated a URL, navigated to a 404, and returned a false negative. The tool descriptions were shallow and generic -- things like "Press a key on the keyboard" and "Close the page." Hauser doesn't blame the Playwright team for this. As he put it, "the people at Playwright don't know what our specific use case is."

Five Techniques to Reshape the Tool Layer

Rather than treating MCP integration as all-or-nothing, Hauser presented five iterative techniques, applied one at a time to the broken baseline.

1. Curate. Remove tools the agent doesn't need. Hauser cut 5-6 irrelevant tools (browser resize, drag, execute JavaScript, etc.), dropping from 21 to 16. Fewer tools means a smaller decision space for the agent.

2. Wrap with enhanced descriptions. Create wrapper tools that call the original functions but carry descriptions tailored to the use case. The Playwright "snapshot" tool -- actually an accessibility tree dump, not a visual screenshot -- got a directive description steering the agent to use it as a first step before clicking or hovering.

3. Add deterministic guardrails. Intercept tool calls with validation logic before they execute. His example: the screenshot tool accepts a file path, so a check validates that the path falls within the designated directory. If not, it returns a structured error message explaining the constraint. The agent self-corrects on the next attempt. Hauser stressed returning friendly error messages rather than raising exceptions, so the agentic loop keeps running.

4. Compose new tools from existing ones. He created an "evidence screenshot" tool wrapping the existing screenshot tool but with a separate description scoped to evidence-taking. The description instructs the agent to include the ticket number in the filename. This lets the agent distinguish between casual navigation screenshots and formal evidence captures.

5. Pull critical operations out of the agent loop entirely. Some steps are both critical and invariant -- the agent adds no value in deciding how to do them. Hauser's example: logging in by injecting JWT tokens into browser local storage via Playwright tools, called as plain functions before the agent starts. The agent receives a logged-in session and skips the error-prone login step entirely.

The Leverage Is in the Descriptions

The second technique -- wrapping tools with better descriptions -- is where Hauser sees the most underappreciated leverage. Descriptions are the interface between your agent and its tools. When you write "always prefer this over taking an actual snapshot," you're not prompting the agent -- you're reshaping its decision environment at the tool level.

This is distinct from prompt engineering. The descriptions travel with the tools themselves, which means they work regardless of how you structure your system prompt. Hauser groups techniques 1, 2, 4, and 5 under "context engineering" -- the practice of shaping what the agent sees and has access to, rather than what you tell it to do.

When to Take the Agent Out of the Loop

"There are aspects of your tasks that are just too sensitive to leave at the hands of the agents."

Hauser argues that the line between agentic and deterministic should be drawn per-operation, not per-system. His login example is instructive: authentication involves sensitive tokens, must happen exactly one way, and fails catastrophically if done wrong. Making it deterministic costs nothing -- the agent was never going to improve on a hardcoded sequence.

"Agents are non-deterministic things and sometimes they will just ignore you."

The broader point is that a production agent system is a mix of deterministic and non-deterministic steps. The wrapper layer between the MCP server and the agent is where you make those choices, and you still get the benefit of upstream MCP server updates because the underlying tool calls remain unchanged.

The Takeaway

Hauser's argument is that the gap between a working MCP demo and a production MCP integration is a tool-engineering problem, not a prompt-engineering problem. The five techniques -- curate, wrap, guardrail, compose, and de-agentify -- give you a repeatable framework for closing that gap. As he put it:

"There's no one size fits all. It's mainly a question of how do I kind of mold the tools to best fit my use case. Sometimes they'll be deterministic, sometimes they'll be flexible. It depends, and you're gonna need to tinker with it."

Nimrod Hauser spoke at AI Engineer Europe 2026. Founding Software Engineer at Baz.

Watch the full talk | Slides | LinkedIn | X

Build the Gym, Not the Dataset

2026-04-11T00:00:00-05:00

Stefano Fiorucci (X, LinkedIn, GitHub) is an AI/Software Engineer at deepset, where he contributes to the open-source LLM framework Haystack. At AI Engineer Europe 2026, he made a case that the next leap for open-source language models isn't better datasets -- it's better environments. The kind where models can act, fail, and learn from the outcome.

"This is exciting because the model is no longer limited by the quality of human examples. Through trial and error it can discover more efficient reasoning strategies."

From Imitation to Interaction

Fiorucci frames the shift by contrasting two training paradigms. Supervised fine-tuning is statistical imitation -- you show the model curated examples and it learns to mimic them. Reinforcement learning with verifiable rewards (RLVR), the approach used in DeepSeek R1, is different. The model generates reasoning traces, its answers get checked against ground truth, and rewards drive the next update. No curated dataset required -- just a clear reward signal and room to explore.

He paraphrases Andrej Karpathy: giving an LLM the opportunity to interact, take actions, and see outcomes means "you can hope to do a lot better than statistical expert imitation."

The mapping is straightforward. The language model is the agent. The environment encompasses the task, data, analysis, and scoring rules. The reward is the verification signal. What's less straightforward, Fiorucci argues, is the software engineering problem of making these environments reusable.

Environments as Software Artifacts

"Too often environments are locked into specific training stacks, making them difficult to reuse. And as a market for closed-source environments emerges, these open initiatives ensure we have a robust alternative."

This is where Stefano's core reframing comes in. RL environments for LLMs should be installable, distributable Python packages -- not one-off training scripts welded to a specific framework. He walks through an open-source library called Verifiers that implements this idea with a hierarchy of environment types:

Single-turn -- one model interaction (e.g., reverse a text string)
Multi-turn -- multiple exchanges with state tracking and stopping conditions
Tool environments -- models call Python functions during rollouts
MCP environments -- auto-connect to Model Context Protocol servers
Stateful tool environments -- per-rollout persisted state like database connections

The library abstracts model serving behind OpenAI-compatible API endpoints and integrates with multiple training frameworks. There's also a community hub for sharing environments -- the point being that if environments can be packaged and shared, the open-source ecosystem doesn't fall behind closed models just because it lacks training infrastructure.

"We don't want open-source models to lag behind just because they lack the right playground training."

Teaching a Small Model to Play Tic-Tac-Toe

The demo is where theory meets practice. Fiorucci takes LFM2, a small open model by Liquid AI, and tries to make it a competent tic-tac-toe player through a multi-stage pipeline.

The baseline is bleak. GPT-5 mini plays well and follows the expected format. LFM2 struggles with format, makes invalid moves, and loses constantly.

Stage 1: SFT warm-up. Generate 200 synthetic games from GPT-5 mini, filter out losses, fine-tune LFM2. This teaches format and valid move syntax -- nothing more. Training takes minutes on a single 96GB GPU.

Stage 2: RL training (round 1). Using CISPO (an improvement on DeepSeek's GRPO algorithm), the model plays against opponents with configurable skill levels -- random move probability ranging from 20% to 70%. Several design choices matter here:

Stratified sampling ensures each training batch contains a balanced mix of opponent difficulties
Deterministic seeding per example and per turn, so reward differences reflect model skill rather than opponent randomness
Invalid move tolerance with a small penalty (-0.1) rather than immediate game-over, preserving learning signal for small models
Minimum batch size of 256 -- smaller values caused training instability and collapse

Stage 3: RL training (round 2). Tougher opponents (0-25% random moves), higher temperature to encourage exploration. An initial performance dip -- interpreted as an exploration phase -- followed by recovery and improvement.

The Student Surpasses the Teacher

The result: after round 1, the small model dominates random opponents and draws roughly 85% of the time against optimal play. After round 2, it outperforms GPT-5 mini against an optimal opponent.

This is the non-obvious punchline. Fiorucci used GPT-5 mini to generate the initial training data, then used RL to push a small model past the model that taught it. His argument: if you can define a clear reward signal, the ceiling for a small specialized model isn't set by its teacher.

Lessons from Failed Experiments

Fiorucci is candid about what went wrong along the way:

Batch size too small leads to instability and model collapse -- the model learns from too few games and opponent types at once.
Hidden environment biases can sabotage training. His Minimax opponent implementation always selected the first valid position when moves had equal scores, so the model memorized one opponent's behavior instead of learning general play.
Starting from a reasoning model with long thinking traces on limited GPU meant forced truncation, which wastes compute budget and risks damaging the model's capabilities. Better to start from an instruct model.
Very small models may simply lack capacity for a given task. Evaluate base models first and look for promising behaviors before committing to training.

"Reinforcement learning is slow and takes time to see progress. If you continually monitor it, you risk the temptation to stop it and tweak something prematurely... So start training and go for a walk."

The Takeaway

Fiorucci's argument is practical: the barrier to training capable small models isn't algorithmic sophistication -- it's the lack of reusable, shareable environments where models can learn through interaction. Build the environment, define the reward signal, and a small model can beat a large one on a specific task at a fraction of the cost.

"You can do this at home too. If you can define a clear reward signal, you can build an environment and train a small specialized model to beat a large closed model on a specific task at a fraction of the cost."

Stefano Fiorucci spoke at AI Engineer Europe 2026. AI/Software Engineer at deepset.

Watch the full talk | LLM RL Environments Lil Course | Slides | LinkedIn | X

Subagent Modes in Claude Code

2026-04-01T00:00:00-05:00

What Claude Code's Source Code Actually Says About Subagents

A post making the rounds claims that Claude Code subagents share a prompt cache, making parallelism "basically free." It says you can spin up five agents and pay barely more than one. It lists three execution models — fork, teammate, and worktree — and says they all share the cache. Analysis of the source code reveals that some of this is true, but most of it isn't — or at least, isn't true in the way the post suggests.

h/t to @swyx for sharing this post as part of his excellent coverage of the Claude Code source leak in yesterday's AI News.

Fork mode exists, but you probably don't have it

The cache-sharing behavior the post describes is real. It lives in a subagent execution model called "fork," where a child agent inherits the parent's full conversation context and system prompt. The code goes to considerable lengths to keep the API request prefix identical across fork children so they hit the Anthropic API's prompt cache. The parent's rendered system prompt is threaded directly to forks rather than recomputed, specifically to prevent cache busting from feature flag state changes between turns.¹ Fork children receive the parent's exact tool array rather than building their own.²

The problem is that fork mode is gated behind feature('FORK_SUBAGENT'), a compile-time flag resolved by the Bun bundler.³ When the flag is off, the code is eliminated from the build entirely. There is no user-facing setting, environment variable, or runtime toggle to enable it. If your build doesn't have it compiled in, it doesn't exist.

When fork is disabled, omitting subagent_type on the Agent tool falls back to a general-purpose agent with no parent context and no cache sharing.⁴ This is the default subagent experience in Claude Code. (In case you're wondering, the /fork slash command doesn't register; its alias gets claimed by /branch instead.⁵)

The cost math doesn't add up

Even when fork mode is active, "5 agents cost barely more than 1" overstates things.

The cost model tracks input tokens, output tokens, cache read tokens, and cache write tokens separately.⁶ Cache reads cost about 10% of regular input, but the first fork pays a 25% write premium to populate the cache, and every fork pays full price for its own output tokens, its unique task directive, and all the tool calls and results it generates while doing its work.

The savings are proportional to how large the shared prefix is relative to total token usage. For agents doing real work — reading files, running commands, writing code — output and tool interaction tokens add up quickly. The shared prefix helps, but it doesn't make parallelism free.

This may not matter to Claude Max subscribers, but it certainly matters if you have "extra usage" enabled.

There aren't three execution models. There are four, and worktree isn't one of them.

The post lists fork, teammate, and worktree as three execution models. The routing logic in the source tells a different story.

Teammate is an independent worker spawned with a team_name and name. Teammates do not inherit the parent's conversation. The parent's messages are explicitly zeroed out at spawn time.⁷ They build their own history from scratch.

Fresh specialized is triggered by setting subagent_type. The agent gets its own system prompt, its own tool pool, and no parent context.

Fork is triggered by omitting subagent_type when the fork gate is enabled. It inherits parent context and is cache-optimized, as described above.

General-purpose is the fallback when subagent_type is omitted and fork is disabled. It behaves like a fresh specialized agent using a default agent definition.

Worktree is not an execution model. It's an isolation modifier that can be combined with any of the four models above.¹⁰ It creates an isolated git worktree so the agent's file operations don't touch the parent's working copy. A fork agent with worktree isolation still inherits context. A fresh specialized agent with worktree isolation still doesn't. Worktree changes where file operations land, not how context is constructed.

The post's central claim is that all subagent types share the prompt cache. This is wrong.

Fresh specialized agents build a different system prompt, assemble a different tool pool, and carry no parent conversation history. Different prefix, no cache sharing. Teammates have their messages explicitly emptied.⁷ They share nothing with the parent.

Fork children can't fork further, either. The code detects a boilerplate tag in conversation history and rejects recursive fork attempts.⁸ Cache sharing is one level deep.

Teammates don't all use file-based mailboxes

The post says teammates communicate via file-based mailbox. This depends on which backend spawns them.

In-process teammates use an in-memory Mailbox class with a queue-and-waiters async pattern.⁹ No files involved. Teammates spawned in separate tmux or iTerm panes do use file-based mechanisms for initial instruction delivery, but that's a different execution path.

What the source code actually tells us

The interesting story here isn't the one the post tells. The source reveals a team thinking carefully about API-level cache mechanics, building explicit cost tracking around them, and keeping the feature gated while they validate it. The CacheSafeParams type, the byte-exact system prompt threading, the identical placeholder tool results, the analytics tracking cache hit rates: this is deliberate, measured engineering work.

But it's behind an experiment gate. Most users interact with fresh specialized or general-purpose agents that share nothing with each other or the parent. That's worth knowing before you reorganize your workflow around a capability you may not have.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. A glowing central node pulses at the center of a dark circuit-board cityscape, splitting into three distinct branching pathways made of light. The first branch is a perfect mirror-copy of the central node, trailing an identical stream of data behind it — a fork. The second branch launches a fresh, smaller node with its own clean trajectory and no trailing data — a specialized agent. The third branch creates an independent floating terminal window connected back to the hub only by a thin mailbox-style data link — a teammate. Each pathway illuminates a different sector of the city below. A translucent hexagonal grid overlays everything, representing the shared prompt cache, but only the fork branch glows where it intersects the grid. Tiny flowing particles of data stream along the branches. In the background, isolated floating platforms with their own miniature cityscapes represent worktree isolation. The overall composition suggests parallel orchestration — multiple autonomous systems radiating from a single point of origin.

References

All references are to the Claude Code source as leaked March 31, 2026. Someone who isn't me obtained it and performed this analysis.

forkSubagent.ts:54-58 — Comment: "Reconstructing by re-calling getSystemPrompt() can diverge (GrowthBook cold→warm) and bust the prompt cache; threading the rendered bytes is byte-exact." ↩
AgentTool.tsx:627 — availableTools: isForkPath ? toolUseContext.options.tools : workerTools ↩
forkSubagent.ts:32-39 — isForkSubagentEnabled() checks feature('FORK_SUBAGENT'), then excludes coordinator mode and non-interactive sessions. ↩
AgentTool.tsx:322 — const effectiveType = subagent_type ?? (isForkSubagentEnabled() ? undefined : GENERAL_PURPOSE_AGENT.agentType) ↩
commands/branch/index.ts:8 — aliases: feature('FORK_SUBAGENT') ? [] : ['fork'] ↩
modelCost.ts:131-138 — tokensToUSDCost() sums input_tokens, output_tokens, cache_read_input_tokens, and cache_creation_input_tokens at different rates. ↩
spawnMultiAgent.ts:927-931 — Comment: "Strip messages: the teammate never reads toolUseContext.messages." Code: toolUseContext: { ...context, messages: [] } ↩↩
forkSubagent.ts:73-87 — isInForkChild() scans conversation history for the fork boilerplate tag and rejects recursive fork attempts. ↩
mailbox.ts:19-73 — In-memory Mailbox class with send(), poll(), and receive() methods using a queue-and-waiters pattern. ↩
AgentTool.tsx:431 — const effectiveIsolation = isolation ?? selectedAgent.isolation — resolved independently of execution model routing. ↩

Remapping the Logitech R500s on Ubuntu 24.04 with keyd

2026-03-25T00:00:00-05:00

Remapping the Logitech R500s on Ubuntu 24.04 with keyd

Here’s my latest keyboard:

That’s right — I said it — keyboard! This is so very nice when you want to just sit back in your chair and have a long working session with an AI. You just use the big button to talk, and the little button to send what you’ve typed with speech! This effectively replaces your keyboard with this lovely, ergonomic remote. In order to do this, though, you need to do a little bit of config, but it’s dead easy.

Here’s a complete guide to remapping presenter buttons to arbitrary keys or macros, including a technical explanation of how the Linux input stack works and why this approach works the way it does.

The Hardware: Logitech R500s

The R500s is a three-button Bluetooth/USB presenter clicker. Despite having no air mouse or cursor capability, Linux exposes it as two input nodes when connected:

/dev/input/event24 — Logi R500 Keyboard (where button presses register)
/dev/input/event25 — Logi R500 Mouse (exposed due to HID descriptor, largely inactive)

The device vendor/product ID is 046d:b505 (Logitech vendor 046d, R500s product b505).

How the Linux Input Stack Works

Understanding why keyd works requires understanding the path from physical button press to application event.

flowchart TD
    A[Physical button press] --> B[HID report over Bluetooth or USB]
    B --> C[Linux kernel\nhid-generic driver]
    C --> D[evdev event written to\n/dev/input/eventN]
    D --> E[Applications\ndesktop · keyd · evtest · etc.]

When you press the advance button, the R500s sends a HID (Human Interface Device) report over Bluetooth to the kernel. The kernel's hid-generic driver reads it and translates it into a standardized evdev event — a struct that says: device event24, event type EV_KEY, code KEY_RIGHT, value 1 (pressed). That event gets written to /dev/input/event24, which is just a character device file that any program can open and read.

evtest is simply a program that opens that file and prints what it sees. That's its entire job — it's a transparent window into the raw event stream, which makes it the right first tool when you need to know what a device actually sends.

Discovering What Your Device Sends

Before remapping anything, always verify the actual keycodes with evtest. Never assume based on documentation or what worked on a similar device.

sudo evtest /dev/input/event24

Press each physical button and observe the output. For the R500s:

Physical Button	Keycode Sent
Advance (big front button)	`KEY_RIGHT` (code 106)
Back button	verify with evtest
Laser button	verify with evtest

The evtest output also shows the MSC_SCAN value, which is the raw hardware scancode (value 7004f for the advance button). This is the scancode at the HID protocol level, before the kernel translates it to a named keycode.

How keyd Works

keyd is a key remapping daemon — a background service that intercepts input events and rewrites them before they reach your desktop.

When keyd starts, it reads your config files and grabs the matching devices. "Grabbing" means it opens /dev/input/event24 with an exclusive lock. Once grabbed, no other program can read from that file. The events go only to keyd.

keyd then reads those events, applies your remapping rules, and re-emits the translated events through a virtual input device it creates via /dev/uinput. Your desktop sees this virtual device and receives the remapped key. It never knows the R500s was involved.

flowchart TD
    A[Physical button press] --> B["/dev/input/event24\nKEY_RIGHT"]
    B --> C["keyd grabs it exclusively\nnothing else sees it"]
    C --> D["keyd applies rule\nright → pause"]
    D --> E["keyd emits KEY_PAUSE\nvia /dev/uinput virtual device"]
    E --> F[Desktop / applications\nsee KEY_PAUSE]

Because keyd runs as a system service (not a user session process), it starts early in the boot sequence before your desktop loads. This means remaps are active from the moment you log in, regardless of which application is focused.

Installation

keyd is not in the Ubuntu 24.04 default repositories. Build from source:

# Install build dependencies
sudo apt install git make gcc

# Clone the repository
git clone https://github.com/rvaiya/keyd
cd keyd

# Build and install
make
sudo make install

# Enable and start the service
sudo systemctl enable --now keyd

Verify it's running:

sudo systemctl status keyd

The Config File

Config files live in /etc/keyd/ and can be named anything with a .conf extension. keyd loads all files in that directory on startup.

Full config for the R500s

# Logitech R500s Presenter Remote Control
[ids]
046d:b505

# Button mappings
[main]
# Big advance button (front) → Pause key (triggers Handy speech-to-text)
right = pause
# Back button → Enter
left = enter

Critical gotcha: no inline comments. keyd only supports comments on their own line starting with #. An inline comment like 046d:b505 # my device will break the config — keyd will try to match the entire string including the comment text as the device ID and silently fail to grab anything.

How each part works

[ids] — tells keyd which physical devices this config applies to. The value 046d:b505 is the USB vendor ID and product ID of the R500s. keyd matches this against every input device on the system. Only the R500s gets grabbed and remapped. Every other device is unaffected, including your keyboard's own right arrow key, which has a different product ID.

[main] — the default layer, meaning "rules that are always active." keyd supports multiple layers (shift layers, function layers, tap-hold behaviors, etc.), but [main] is the base that's always on.

right = pause — the remap rule. Left side is the incoming keycode (KEY_RIGHT, written without the KEY_ prefix, lowercase). Right side is what keyd emits instead (KEY_PAUSE). keyd swallows the original event and injects the replacement.

left = macro(C-a delete) — a macro rule. Instead of emitting a single key, keyd fires a sequence: Ctrl+A followed by Delete. This selects all text then deletes it. C- is keyd's syntax for Ctrl, M- for Alt, S- for Shift.

Key Syntax Reference

Syntax	Meaning
`pause`	Pause key
`enter`	Return/Enter
`space`	Spacebar
`esc`	Escape
`f5`	F5
`C-a`	Ctrl+A
`C-z`	Ctrl+Z (undo)
`C-c`	Ctrl+C (copy)
`M-f`	Alt+F
`S-tab`	Shift+Tab
`macro(C-a delete)`	Ctrl+A then Delete
`macro(C-a C-c)`	Select all then copy
`macro(h e l l o)`	Types the word "hello"

Applying Config Changes

After editing the config file, restart keyd to pick up the changes:

sudo systemctl restart keyd

To verify the R500s was matched:

sudo journalctl -u keyd --no-pager | grep -i r500

You should see lines like:

CONFIG: parsing /etc/keyd/r500s.conf
DEVICE: match    046d:b505:a33bddee  /etc/keyd/r500s.conf  (Logi R500 Keyboard)
DEVICE: match    046d:b505:0ef6d98d  /etc/keyd/r500s.conf  (Logi R500 Mouse)

Both nodes being matched is correct and expected.

Why Device ID Scoping Matters

The [ids] section is what makes keyd safe to use. Because every rule is scoped to a specific device by vendor/product ID, you can write:

[ids]
046d:b505

[main]
right = pause   # R500s advance button → Pause

And your keyboard's right arrow key continues to work normally because your keyboard has a different product ID. keyd tracks the source of every event.

This is fundamentally different from older tools like xmodmap, which remapped keycodes globally system-wide regardless of which device sent them.

The systemd Integration

sudo systemctl enable --now keyd

enable tells systemd to start keyd automatically on every boot by creating a symlink in /etc/systemd/system/. --now also starts it immediately without requiring a reboot. The service file installed by make install is at /usr/local/lib/systemd/system/keyd.service.

Troubleshooting

Device not being matched: The R500s may not have been connected when keyd started. Toggle it off and on, or restart keyd:

sudo systemctl restart keyd

Remap not working after config edit: Config changes require a restart:

sudo systemctl restart keyd

Check full keyd logs:

sudo journalctl -u keyd --no-pager

Verify device is connected and recognized by the kernel:

ls /dev/input/by-id/ | grep -i logi

The Cure for the Vibe Coding Hangover

2025-09-01T00:00:00-05:00

A practical framework for building software with AI agents.

This was ultimately presented as a talk at the AI Engineer Code Summit in November 2025.

Inspiration strikes. You've got an idea, and you know exactly how you're going to build it: let's vibe code, baby! You fire up your favorite AI coding agent, you jam in those prompts, and you hand it over. The app works. This is what 10x engineering really feels like. You're a genius -- a rebel in the AI revolution.

Then Monday rolls around. You want to add a feature, or change the way something works, and you realize that you don't understand it, you can't maintain it, and you have to throw most or all of it away.

That's the hangover. Vibe coding is the low-spec, zero-planning approach to AI-accelerated development that feels productive but results in brittle, unmaintainable demo-ware. The hangover is the despair that follows when you try to build maintainable, understandable software that way.

There's a cure, though, and it's a framework for building with AI coding agents. That's what this is about.

Vibe coding feels like 10x engineering right up until the moment you have to live with what it built.

Who This Is For

You'll dig this if you value programming as a daily learning experience. If you want to understand and own the software you write with AI coding agents, just as you own all the other software you write. If you want to be the boss of the coding agents, not their confused intern. If working with agents lately makes you feel like a prompt jockey and no longer an AI engineer. If you're sick of throwing away code, burning time and tokens. Or if you want to use coding agents to build production applications that do real work.

On the other hand ...

This isn't for you if programming is a job and not a craft you're refining -- and that works for you. If you're satisfied having AI just do it for you without needing to understand how or why. Or if vibe coding gets you what you need and that's good enough. None of that is a judgment. It's just a very different path than the one I'm taking here.

The Framework in Overview

The Framework has three pillars.

Principles are the philosophy underpinning all of it.
Process is the workflow for actually getting software built using AI.
Tools are the accelerators and enablers of the process -- which also reflect the principles.

So what can you build with it? Really, anything. The Framework is adaptive to all types of software.

Here are a few examples of working software in the wild right now, built with this approach:

specialized litigation support applications for law firms;
real-time appliance monitoring packages for smart cooking devices;
digital publishing systems for dynamic content replatforming;
a code execution environment for secure, isolated development workflows;
a context-management suite that preserves conversational data across AI sessions;
a digital media production system for studios

... and on and on. Plus a litany of smaller tools and utilities used daily to automate routine tasks.

The point is that these aren't toys. They're real software applications that do real work every day, and they're evolved and maintained at breakneck pace by AI engineers who apply this Framework.

The Principles

There are ten principles. They map across three groups:

general principles that apply overarchingly,
principles that skew toward the planning phase, and
principles that skew toward implementation.

For each one, it helps to understand the problem it came from, the idea that answers it, and the one line I use to remember it.

1. AI Engineering Is Accelerated Learning

The problem this came from: treating AI coding agents as pure productivity tools, just to crank out code faster. Using AI to generate software and learning nothing from the process. Six months later, being no better an engineer -- plateaued. Or worse, becoming dependent on AI for debugging, modifications, architectural decisions. That's not AI augmentation. That's AI dependency.

The Framework isn't just about building faster -- it's a learning system. Every step creates specific learning opportunities, so you're not just shipping software, you're building yourself. The software is valuable, but the engineer you become is exponentially more valuable.

Always. Be. Learning. A-always, B-be, L-learning.

2. You Are the Architect, the Agent Is the Implementer

The problem: treating AI agents as replacements for architectural thinking, rather than implementers of your decisions once those decisions are well-specified.

Keep the architect/implementer boundary crystal clear. You own the thinking -- architecture and interfaces, the intent of the system, its structure, the design decisions and their trade-offs. The agent handles the doing -- implementation, typing code, following patterns, implementing the tests you specify, banging out boilerplate.

Delegate the doing, not the thinking.

3. Slow Down and Iterate to Go Fast

It's a little counterintuitive. The problem is the starting-over cycle: without deliberate iteration on validated work, you repeatedly start from scratch. Three months in, you've got multiple abandoned attempts instead of one consistently improving system.

Deliberate iteration enables compounding returns on both understanding and productivity. Week one feels slow. Week two builds momentum. Week three is dramatically faster.

Compound progress, accelerate velocity.

4. Specification > Prompt Engineering

The problem: prompt engineering treats AI interaction as an optimization problem rather than a communication problem -- hunting for magic words that produce the right output, instead of clearly defining what "right" means.

Specifications are different from prompts. A specification is a structured, precise definition of requirements, behavior, interfaces, and acceptance criteria. Writing one forces architectural thinking: you have to understand the problem completely, define interfaces precisely, and anticipate edge cases. In turn, the specification gives the agent clear, unambiguous direction -- it implements what you specified, not what it interpreted from a conversational prompt.

Write the blueprint, not the prompt.

5. Define "Done" Before Implementing

The problem: starting implementation without executable tests and observable success criteria means the agent has no clear completion criteria and no immediate feedback. It can't self-validate, can't self-correct, and doesn't know when it's done -- at least not in a way consistent with your specifications.

Defining "done" up front keeps you thinking deeply about requirements, and it lets the agent work autonomously. Tests defined upfront give the agent clear stop conditions and immediate feedback during implementation. And there's more than tests: the multi-sensory validation we'll get to lets agents observe through visual senses (what renders), auditory senses (what they hear through logs and errors), and tactile senses (how they interact with the system). Tests verify the correctness of the implementation; the senses reveal the actual behavior of the software as it's being built. A feature is done when the tests pass and the senses come back clean.

Specify success, then build.

6. Feature Atomicity

The problem: writing non-atomic features leaves the decomposition work for implementation time, which forces the agent to make architectural decisions on the fly.

Feature atomicity forces you to completely decompose each feature during specification, which then lets the agent implement within a manageable scope. Features become implementation work units -- atomic, irreducible tasks ready for an agent to execute completely. Keep them as small as possible to make agent implementation as successful as possible.

Reduce until irreducible.

7. Dependency-Driven Development

The problem: implementing without explicit dependency analysis treats all features as independent, when we know they actually form an interconnected graph.

Dependency-driven development forces you to understand how features relate and integrate -- and it ensures the agent never implements a feature that depends on incomplete work.

Schedule implementation by dependencies.

8. Implement One Atomic Feature at a Time

Now the implementation-related principles. The problem: working on multiple features treats implementation as parallel streams that can be context-switched freely. But implementation quality depends on sustained focus, complete context, and very tight feedback loops. Jumping between features fragments your focus.

So the agent implements a single, atomically-defined feature. You study it and understand it. You validate that it works. You commit it as a checkpoint. Then you move to the next. That rhythm builds both momentum and deepening understanding -- working software and engineering knowledge at the same time.

Complete one, commit one, continue.

9. Context Engineering and Management

The problem: treating context as something that just happens automatically, rather than something you actively engineer. You let conversation history passively accumulate instead of curating what actually matters. And if you don't build context resilience, state eventually fails to persist and you lose continuity.

So don't rely on conversational state persisting. Capture architectural decisions in persistent documents -- specifications, plans, design documents -- and build context from those artifacts, not from yours or your agents' memory.

Curate context, don't accumulate it.

10. Make It Work, Make It Right, Make It Fast

This one is borrowed from the annals of software engineering. The problem is treating all three phases as equally important from the start, or trying to achieve them all at once.

The Framework focuses on getting to make it work -- working software you can ship and use. Only after real usage reveals what matters do you selectively invest in make it right and make it fast. So stop pursuing elegance and performance upfront. Direct the agent explicitly to make it work -- a simple, functional implementation that passes tests and ships quickly -- and let real usage reveal what deserves further investment.

Build, ship, learn, improve.

So there they are -- ten principles, the philosophy that makes The Framework work. Mind sufficiently blown? Stick with me; it gets better.

The Process

The process is where we put the principles to work -- principles in action. It has two distinct phases:

the planning phase, where you do all the architectural thinking to define what to build, and
the implementation phase, where the agent executes your specifications with your oversight and validation.

Planning produces the artifacts that enable autonomous agent implementation. Implementation then uses those artifacts to build working software, feature by feature.

The Planning Phase

Planning is where you complete your architectural thinking. You transform a vague project idea into atomic, sequenced, fully-specified features ready for implementation. This is purely your work -- the architectural decisions, the decomposition, the specification writing, the dependency analysis. The agent can assist as a thinking partner, but you make every decision.

The five planning steps are sequential and build on each other: Vision → Features → Specification → Dependencies → Plan. It's a highly iterative process of extracting and refining your thinking into tangible artifacts. The input and output of each step are a template and a completed template, respectively -- a lot of work has gone into well-structured templates that both guide the thinking and capture the results.

Planning Step 1: Vision Capture

The purpose is to transform your vague project idea into a complete, structured Master Project Specification that articulates the problem, the users, the functionality, the scope, and the workflows.

The problem it solves: your initial idea exists only in your head, and it's incomplete. You have a general sense of the problem and an approach, but the details are fuzzy, the implicit assumptions are unexamined, and critical aspects are unformed. Without structured exploration you can't communicate your thinking, articulate requirements clearly, or create a shared foundation the agent can build on.

So you think out loud with an agent -- optional, but strongly recommended -- to refine and capture your vision across five sections:

Project purpose -- the problem you're solving, who experiences it, and the core value your software delivers.
Essential functionality -- the three to five fundamental workflows that solve the problem.
Scope boundaries -- explicit now / not / next decisions: now (must-have for the make-it-work version), not (out of scope), next (future enhancements).
Technical context -- where it runs, how users interact, what systems it connects to.
Workflow details -- for each core workflow, the goal, the high-level steps, and the expected outcome.

You iterate until the vision is clear and complete. The agent is great here for surfacing gaps, suggesting edge cases, and probing assumptions -- but you make every decision. The output is the Master Project Specification, the foundation for extracting features in step two.

Planning Step 2: Feature Identification and Categorization

The purpose is to systematically extract every unit of functionality from your Master Project Specification and organize them into a categorized Feature Inventory.

You don't jump straight from high-level vision to detailed feature specs -- that's too big a leap. So you work through the specification section by section with targeted extraction questions: what foundational capabilities does the system need; what discrete capabilities does each workflow require; what infrastructure makes the make-it-work version work now; what platform, integration, and interface features are needed; for each workflow, what handles input, processing, output, errors, and feedback; and across everything, what security, logging, configuration, and testing features span the system. You document each feature's source for traceability.

You build the raw feature list first -- capturing every capability, not organizing yet -- and challenge completeness ("what handles errors? what validates input? what provides feedback?"). Then you analyze for natural groupings, settling on three to seven categories that reflect how your specific software is structured, assign each feature to its best-fit category with a unique ID (like CORE-001 or API-101), and give each an initial complexity estimate of easy, medium, or hard. The categories emerge from your actual features, not from a predetermined template. The output is the Feature Inventory.

Planning Step 3: Iterative Specification Development

This is the critical step. The purpose is to transform each feature from the inventory into a complete, atomic, implementation-ready specification that defines exactly what to build, how it will be validated, and what it depends on.

You collaborate with an agent to refine each feature through a three-level pattern. First, a user story -- "as a [user type], I want to [action] so that I can [benefit]" -- capturing who needs this, what they're doing, and why. Then the implementation contracts, in three levels of increasing precision:

Level 1, plain English -- what the feature does in natural language: what it receives, what it does, what it produces.
Level 2, logic flow -- structured pseudocode with clear INPUT, step-by-step LOGIC, and defined OUTPUT.
Level 3, formal interfaces -- exact signatures, data structures, and API specs: precise input types, return types, and errors.

Then the validation contracts, also in three levels:

Level 1, plain-English scenarios -- every situation that needs validation: happy path, error cases, edge cases, security properties.
Level 2, test logic -- each scenario as GIVEN/WHEN/THEN, with setup, trigger, and expected outcomes.
Level 3, formal test definitions -- exact test interfaces with setup, inputs, precise assertions, and teardown.

Then you validate atomicity: can this be implemented in a single focused session? If the spec feels scattered or describes multiple capabilities, you split it and repeat. Finally you identify dependencies -- which other features must exist before this one can be implemented -- documented as explicit, binary dependencies: it either depends on another feature or it doesn't. The output is a complete Atomic Feature Specification for every feature in the inventory.

Planning Step 4: Dependency Analysis

The purpose is to transform your complete set of feature specifications into a validated dependency matrix that defines the exact order features can be implemented in -- eliminating circular dependencies and revealing natural phases.

Your specs contain accurate dependency declarations, but they're scattered across individual documents. You have the local picture (feature X depends on feature Y) but not the global one. Without a synthesized view you can't see the complete graph, detect cycles that span multiple features, identify the natural implementation phases, or know what has to be built first.

So you extract the matrix -- every feature is a row and a column, and you mark an X where a row feature depends on a column feature. You generate a graph from it (Graphviz, Mermaid, your pick) with features as nodes and dependencies as edges, which makes cycles immediately visible as closed loops and reveals the layered structure. You validate and clean with the binary dependency test on every mark: does the row feature actually require the column feature's specific output, configuration, or functionality to work? If yes, keep it; if no -- if it's only coordination or tool-sharing -- remove it. You regenerate the graph, then detect cycles, and where you find them you apply resolution strategies in order: first dependency elimination (re-examine with the binary test), then revised specification (rethink interfaces so features don't need each other's output), then feature splitting (maybe it wasn't atomic), and only as a last resort, consolidation. You iterate -- update matrix, regenerate graph, recheck -- until zero cycles remain. The outputs are the validated Dependency Matrix and the Dependency Graph.

Planning Step 5: Implementation Plan Development

The final planning step transforms the validated dependency matrix into a comprehensive, phase-organized implementation roadmap -- sequencing features into dependency layers, defining phase completion criteria, and establishing the validation strategies that enable the implementation loop.

Without this, you face implementation chaos: even with complete specs and a validated matrix, you can't say which features come first and in what order, or when it's safe to start a feature that depends on earlier work. So you organize phases via a topological sort: features with no dependencies are phase one; features depending only on phase one are phase two; and so on, each phase depending only on previous phases. You verify that no two features within a phase depend on each other, and you identify the critical path -- the longest dependency chain. Then you do validation strategy planning: for each phase, binary success criteria -- what tests must pass, what integration points must work, how you'll verify features work together -- and feedback loops that enable autonomous refinement and binary progress tracking (a feature is implemented or it isn't; there's no 20%-done). Finally, implementation sequencing: phase gates, guidance for how the agent selects the next feature, a blocker-management process, and progress-tracking mechanisms at the feature level, the phase level, and along the critical path. The output is the Implementation Plan.

The Implementation Loop

Implementation is where your planning artifacts guide the transformation of specifications into working, tested software. Unlike planning, which is linear and proceeds through five distinct steps, implementation is a tight, rapid loop run repeatedly for each atomic feature.

The Multi-Sensory Feedback Loop

This is a really key idea. The agent implements code, then executes it while gathering feedback through three digital senses:

a visual sense (what renders),
an auditory sense (what the system reports), and
a tactile sense (how interactions respond).

This sensory feedback provides rich diagnostic information about what's actually happening in the application. The agent also runs formal tests against the acceptance criteria -- but by correlating sensory feedback with test results, it understands both what failed, from the tests, and why it failed, from the senses. The loop continues until all acceptance criteria pass and all senses report clean execution.

Implementation Step 1: Context Assembly

The purpose is to transform planning artifacts into a curated context package that enables autonomous feature implementation within a single coding session.

You have atomic features fully specified and sequenced, but you can't just throw everything at the agent and hope. Dumping entire planning documents into a session wastes context window on irrelevant information, leads the agent to make decisions without critical context (or to stop and ask), and turns what should be autonomous implementation into constant back-and-forth.

So you curate exactly what this one feature needs, in four steps. Feature specification assembly: include the complete spec -- user story, technical contracts, acceptance criteria -- and @-reference its dependencies; this is the primary blueprint. Dependency context gathering: follow those @ references, each pointing to a dependency's specification and its actual implemented code (per The Framework, all dependencies are implemented previously), so the agent knows exactly what to integrate with. Implementation guidance extraction: pull only the relevant sections of the Implementation Plan -- what phase this is, the phase completion criteria, the validation strategy. Sensory capability enablement: read the acceptance criteria to identify which senses are required -- visual language like "sees / displays / renders" needs the visual tools, logging and error language needs the auditory tools, interaction language like "clicks / submits / completes" needs the tactile tools -- and @-reference the appropriate tool-usage guides (written once per tool, reusable across all features). The output is the Curated Context Package.

Implementation Step 2: The Implementation Loop

This is the only step in the entire Framework where the AI writes code. The purpose is to transform an atomic feature specification into working, tested code.

Without structure here, agents either write all the code before testing anything -- so problems accumulate undetected and you debug many interconnected issues at once -- or they write and test ad hoc, missing the problems tests don't catch: tests pass but the UI doesn't render, the workflow completes but errors fill the logs, the feature "works" but the interactions are broken. Because features are atomic, the complete implementation fits in one context window, so the agent keeps full understanding from start to finish -- no context loss, no reconstruction, no degraded fidelity.

The loop runs like this.

Write code following the specification's technical contracts, translating all three levels into working code so interfaces, inputs, outputs, and error handling match the spec exactly.
Execute and sense: run it immediately and gather comprehensive feedback through the three senses based on the acceptance criteria.
Test and validate: run all the test scenarios from the validation contracts for binary pass/fail signals.
Analyze feedback: correlate signals across all active senses and the test results -- multiple senses reporting the same issue confirm a diagnosis, conflicting signals reveal hidden complexity -- and this integrated view shows both what failed and why, enabling
Targeted refinement.

You loop until the feature is complete: all tests pass and all senses report clean execution -- no errors in logs, no rendering issues, interactions working. Then the agent makes an atomic git commit containing only this feature's changes, with a structured message including the feature ID, a specification summary, validation confirmation, and implementation notes. The output is a fully working, tested feature, validated by all three senses and ready for integration.

We've even blown the tiger's mind, now! Let's round the corner into tools.

The Tools

The Framework requires four foundational capabilities that enable the work done through the process. The Framework isn't prescriptive about which tools -- only about the capabilities those tools should possess.

Coding Environment

A complete development workspace that supports two fundamentally different kinds of work happening at once: your architectural thinking and planning, and the agent's autonomous implementation and testing. Four core components:

An AI coding agent -- obviously.
An execution sandbox -- a safe, isolated environment where all agent-written code executes and tests run. Autonomous implementation needs the freedom to experiment, iterate, and occasionally break things; the sandbox gives complete development capability in a disposable, risk-free space with easy reset and no consequences for failure.
An IDE (or a text editor, if you must) -- pick the one that suits you; I'll avoid the holy wars.
Voice input -- a rapid-capture system that converts speech to text at thinking speed. Planning means externalizing architectural thinking, which is often incomplete, exploratory, and iterative; voice removes the typing bottleneck and lets you think out loud far faster than you can type. I can't oversell how massively impactful a good voice-input tool is in applying The Framework.

Multi-Sensory Feedback System

A comprehensive validation infrastructure that gives agents the ability to observe their implementations through three complementary digital senses, enabling autonomous refinement through the same breadth of observation humans use during development:

Visual sense tools -- direct observation of what was produced: UI rendering (screenshots, layout, styling), system state (database contents, configuration, session data), and code structure. Visual observation catches what logs and tests miss -- broken rendering, incorrect state, structural issues.
Auditory sense tools -- what the system reports: logs (the system narrating its operations), errors and warnings, API responses, and stack traces. This explains why things fail, not just that they failed.
Tactile sense tools -- active interaction testing: simulating and executing user workflows end-to-end, API request/response cycles, performance validation, security checks, and integration testing. These reveal whether the software behaves correctly under actual use, not just in isolated tests.
An orchestration layer (MCP or equivalent) -- the protocol layer that surfaces these tools and coordinates them into one integrated feedback system, delivering structured sensory data so the agent understands both what failed and why.

Context Engineering and Assembly

A systematic approach to assembling focused, complete context packages for agents:

An @ cross-reference system -- declarative linking that lets documents explicitly reference other documents, code files, or sections, enabling automatic context assembly by following the dependency chains. These aren't hyperlinks for human navigation; they're actionable declarations that automation can follow.
Slash commands (or your environment's equivalent) -- process automation that triggers multi-step framework workflows from a single invocation, especially for context assembly, template instantiation, and implementation-session initialization.
A template system -- structured templates for every framework artifact (Master Project Specification, Feature Specification, Dependency Matrix, Implementation Plan, Implementation Record) that ensure consistent format and completeness. Without templates you reinvent the structure every time, and starting from scratch is exhausting.
Markdown documentation format -- agents are so literate in it that it's vital to have tooling that converts inputs to markdown rapidly. Anything you want to communicate to an agent should be instantly convertible to markdown in your toolchain.

Version Control and Progress Tracking

A dual-mechanism system for provenance and current-state visibility:

A version control system -- I'll just say it: Git, or the equivalent. Implementation history through atomic feature commits. It's like saving progress in a video game: obvious and essential.
The Implementation Plan with progress tracking -- the same planning artifact from step five, doing double duty. Git shows what changed and when, but not the project's state; the Implementation Plan fills that gap, tracking which features are complete, which are blocked, and what's next. This is the simplest form of project tracking -- it can get more complex and more integrated -- but it's enough.

For the curious, that's my own tool stack.

The Morning After

That's The Framework: the principles, the process, and the tools -- and the way they reinforce one another, with each process step and each tool capability tracing back to the principles. Build this way and the Monday after isn't a hangover. You understand the software, you own it, you can maintain and extend it -- and you're a better engineer than you were the feature before.

The software is valuable. The engineer you become is exponentially more valuable.

If you enjoyed this and want the slides, the soundtrack, and the other resources, I built a site just for the talk: vibecodinghangover.com. Happy hacking.

WTF is BM25? The Intuition Behind the Algorithm

2025-03-25T00:00:00-05:00

❓ Why Should You Give a Toss About BM25?

BM25 is an unsung hero powering heaps of stuff you use every day – think search bars, recommendation systems, or even those fancy AI chatbots that somehow dig up the right info from a mountain of text (we're looking at you, Retrieval-Augmented Generation).

If you’re building anything that needs to sift through data – like a search tool for your blog, a customer support bot, or a system to find the juiciest research papers – BM25 is part of your secret sauce. It’s fast, it’s clever, and it’s been battle-tested for decades, quietly outranking simpler methods while the modern AI models somehow get all the Instagram likes. Understanding it means you can make sense of why some results pop to the top and others sink, whether you’re coding it yourself or just trying to impress your boss with some “I know how this works” swagger.

Plus, in a world drowning in info, knowing how to rank what matters is important. BM25 isn’t just maths – it’s the art of cutting through the noise. So, whether you’re a dev, a data geek, or just someone who hates wading through irrelevant Google results, stick around. This algo’s got lessons for us all – and it might just save your sanity when the AI Apocalypse hits and we’re all searching for “how to reboot the robot overlords.”

We will build the intuition for how BM25 works, step by step, and unravel this:

$$ \text{Score}(D, Q) = \sum_{i=1}^{n} \text{IDF}(q_i) \cdot \frac{\text{TF}(q_i, D) \cdot (k_1 + 1)}{\text{TF}(q_i, D) + k_1 \cdot (1 - b + b \cdot \frac{|D|}{\text{avgdl}})} $$

into this:

$Score = Sum\ for\ Each\ Term(\ Adjusted\ Count\ of\ Term \times Rarity\ of\ Term\ )$

🚀 Ground Control to Major Tom

Imagine you're searching the Web for "Mars exploration" among this tiny library of 5 books (we'll use the term "document" interchangeably with "book"). How on Earth (or Mars) does a search engine perform keyword search to know which documents are most relevant to your search?

📱Mobile Readers Note: If you're reading this on mobile, and any of the tables or formulas overflow the page, they will scroll if you swipe left on them like you're ducking a rando on Tinder!

Here are our books:

Book	Title	Length (pages)	"Mars" count	"exploration" count
A	Mars Exploration Guide	50	8	6
B	Space Encyclopedia	200	10	4
C	Planetary Science	100	0	0
D	Exploration Techniques	75	0	12
E	Solar System	150	0	0

Left to your own considerable cleverness, how would you devise an algorithm to execute this search?

🔢 Step 1: Tonight We're Counting Words Like It's 1995 (Term Frequency)

If we were to try to solve this problem in the simplest possible way, perhaps the most basic way to determine relevance is to simply count how many times each search term appears in a document (book). This is how early attempts at search worked, by the way, and isn't too dissimilar to how search capabilities on the Web may have worked in 1995.

(Yes, I know the lyrics are "1999" ... Google was doing more sophisticated things than this by then ... just stay with me in 1995.)

Let's score each document as follows:

$Score = \text{Count of "Mars"} + \text{Count of "exploration"}$

So, the book with the highest score is the highest ranked, and so on.

Using this naive approach:

Book A: 8 + 6 = 14 points
Book B: 10 + 4 = 14 points
Book C: 0 + 0 = 0 points
Book D: 0 + 12 = 12 points
Book E: 0 + 0 = 0 points

Good enough? Nay, I say!

🔍 The Problem: Shortcomings of Simple Term Counting

Books A and B tie with 14 points – but, mate, surely the "Mars Exploration Guide" (Book A) is more focused on our search than some general "Space Encyclopedia" (Book B)! This approach just won't do. Let's fix it!

💡 Key Insight: Simple word counting treats every word equally, which doesn't quite work well.

❗🟰 Step 2: Not All Words Are Created Equal (Inverse Document Frequency)

In our simple counting approach, we treated each occurrence of "Mars" and "exploration" as equally valuable ... and it kind of worked, until it didn't. "Mars Exploration Guide" and "Space Encyclopedia" tied, despite one clearly being more about our search query. This happened because we treated all words equally. But in searches, some words are better at targeting relevant results. "Mars" is a laser beam. "Exploration" is a flashlight. "The"? A fog machine.

This suggests that the less frequent -- i.e. more rare -- a word is, the more useful it is as a search term. That makes intuitive sense, also. Imagine flipping through a phone book (no, really -- people actually did this at a distant point in human history) looking for someone's phone number. If your search is for a fellow named "Smith", you're going to have a much harder time than if you're looking for a professionally good-looking fellow called "Zoolander"! The lesson? The rarer a search term is, the faster it narrows or filters your search.

How about we weight the counts of terms by their "rarity"? Let’s adjust our scoring:

$Score = (\text{Count of Term}) \times (\text{Rarity of Term}) + (\text{Count of Term}) \times (\text{Rarity of Term})$

This immediately fixes a big part of our problem: words that appear everywhere ("exploration") get less credit than words that are rare ("Mars"), even if they appear just as often in a document.

💡 Key Insight: The rarer a term is in the whole collection, the more useful it is for identifying relevant documents.

So how do we calculate that rarity score?

✨ Simply Rare: It's Hardly There

Again, barely taxing our considerable intellect, we could calculate the rarity score simply by expressing the rarity of a term as the percentage of documents that the term appears in. Yeah?

$$ Rarity\ Score = \frac{1}{\text{\# of documents the term appears in}} $$

We could, then, simply weight the counts of each term by the rarity of each term. Tracking?

$Score = \text{Count of "Mars"} \times \text{Rarity of "Mars"} + \text{Count of "exploration"} \times \text{Rarity of "exploration"}$

So, then, we get the following scores for "Mars exploration" across each of the books in our library:

Book	Title	Mars Count	Exploration Count	Mars Rarity	Exploration Rarity	Score
A	Mars Exploration Guide	8	6	0.5	0.333	6.0
B	Space Encyclopedia	10	4	0.5	0.333	6.333
C	Planetary Science	0	0	0.5	0.333	0.0
D	Exploration Techniques	0	12	0.5	0.333	4.0
E	Solar System	0	0	0.5	0.333	0.0

Whilst this, again, kind of works, it's got some fairly significant shortcomings.

🔍 The Problem: Shortcomings of the Simplified Rarity

Overvalues Rare Terms: Simplified rarity over-rewards extremely rare words, treating a term in 1 document as 10× more valuable than one in 10 docs. A word in 1 document gets a rarity score of 1, while a word in 10 documents gets a rarity score of 0.10. Similarly, a word in just 2 documents gets a rarity score of 0.5. This says that the word is half as valuable in our search, but is that really true? Is a term that appears in 2 documents really only half as useful as one that appears in 1?

Drops Off Too Quickly: As term frequency increases significantly, simplified rarity becomes very small. In large collections, simplified rarity for frequent (common) terms becomes vanishingly small and meaningless.

This simplified calculation of rarity also misses a key fact about what rarity means in terms of filtering power for search terms.

💪 Rarity = Power: Why Uncommon Words Matter More

This is the core idea behind filtering power (fancily called "discriminative power") – the ability of a term to eliminate irrelevant documents. The more irrelevant documents it can eliminate in a single step, the more power the term has.

Let’s take an example:

A word in 50% of documents gives you 1 "unit" of filtering power. (Think about asking 1 yes/no question that perfectly eliminates half of the documents in the collection.)
A word in 25% gives you 2 units. (Think of asking 2 yes/no questions that, each, perfectly eliminate half of the remaining documents.)
A word in 12.5% gives you 3. (3 yes/no questions ...)
A word in 1% gives you 6.6 units. (Woah!)

These units "filtering power" are like asking the # of units of yes/no questions all at once in order to reduce the size of the search area. How the hell did we know that the last one was 6.6 units?

Each unit of filtering power halves your remaining search space. And this halving pattern is captured perfectly by our old mate from maths: 💥the logarithm.💥 This means that a word that's 10× rarer (i.e. less frequent) isn't 10× more useful -- it's more like 2× more useful. Why? Because filtering power is logarithmic!

$$Term\ Rarity\ \text{or}\ Term\ Filtering\ Power = \log_2\left(\frac{\text{Total # of Docs}}{\text{# of Docs Containing the Term}}\right)$$

This gives us a clean measure of a term’s power to shrink the search haystack. So, we were able to calculate the filtering power of a word that only appears in 1% of the documents collection as follows:

$$Term\ Filtering\ Power = \log_2\left(\frac{1}{0.01}\right) = 6.6$$

BM25 calls this, more formally, "Inverse Document Frequency (IDF)". One way to think about IDF that you may find helpful is as follows:

$$ \text{Term Frequency} = \frac{\text{# of Docs Containing the Term}}{\text{Total # of Docs}} $$

Soooo ...

$$ \text{Inverse of Term Frequency} = \frac{\text{Total # of Docs}}{\text{# of Docs Containing the Term}} $$

Sooooooooo ... dressing this up in BM25-specific evening attire ...

$$Inverse\ Document\ Frequency\ (IDF) = \log\left(\frac{\text{Total # of Docs}}{\text{# of Docs Containing the Term}}\right)$$

Bam!

💡 Key Insight: The less frequently a word appears in the document collection, the more filtering (discriminative) power it has – and that power grows logarithmically.

What a veritable buffet of intellectual satisfaction this is proving!

Let’s apply it to our lil' library:

"Mars" appears in 2 of 5 books → IDF = log₂(5 / 2) ≈ 1.32
"Exploration" appears in 3 of 5 books → IDF = log₂(5 / 3) ≈ 0.74

We can rewrite our intuitive formula with some $10 words, so that:

$\text{Score} = (\text{Count of Term}_1) \times (\text{Rarity of Term}_1) + (\text{Count of Term}_2) \times (\text{Rarity of Term}_2)$

Becomes:

$Score = TF("Mars") \times IDF("Mars") + TF("exploration") \times IDF("exploration")$

🧮 It's Time for the Calculator!

(Only the real ones from Chicago get that ...)

Let's calc it down now:

Book	Title	Count of "Mars"	Rarity of "Mars"	Count of "exploration"	Rarity of "exploration"	Score
A	Mars Exploration Guide	8	1.32	6	0.74	15.00
B	Space Encyclopedia	10	1.32	4	0.74	16.17
C	Planetary Science	0	1.32	0	0.74	0.00
D	Exploration Techniques	0	1.32	12	0.74	8.84
E	Solar System	0	1.32	0	0.74	0.00

🔍 The Problem: Say Less, Baby!

Our scoring is still not quite right. Book B edges out Book A because it says “Mars” more, even though it’s not focused on it. (Like so many "AI influencers" on social media ...) And Book D scores surprisingly high despite not mentioning “Mars” at all!

So even with IDF (weighting by the rarity of the term), we’re missing something crucial – document focus. Or maybe… document length?

Let’s fix that next!

🤵 Sound Smart at Dinner Parties

When "filtering power" puts on a tuxedo, it's known as "discriminative power"

🤦‍♂️ Step 3: The "Enough, Already – We Get It!" Approach (Diminishing Returns)

At the end of Step 2, we fixed a huge flaw: we now value rarer words more (thanks to the homie IDF). But two problems remain:

Book B still scores higher than Book A, despite A being more focused on Mars exploration.
Book D scores surprisingly well despite never mentioning "Mars" at all.

Recall that we've got the following expressions of BM25 thus far. In simple terms:

$Score = (Count\ of\ Term) \times (Rarity\ of\ Term) + (Count\ of\ Term) \times (Rarity\ of\ Term)$

Fine, here's the more formal version:

$Score = TF("Mars") \times IDF("Mars") + TF("exploration") \times IDF("exploration")$

Let’s dig into problem #1 first. We need to find a way to adjust how we calculate "Count of Term" -- i.e. Term Frequency (TF) -- to address this problem.

❗ The 10th “Mars” ≠ The 1st “Mars”

Book B mentions "Mars" 10 times. Book A mentions it 8 times. Our current scoring says more mentions = better score. But think about it:

First mention of "Mars": "Okay, this is about Mars – right on!"
Second mention: "Yup, still on about Mars. Cool."
Tenth mention: "Enough, already – we GET IT! You're about Mars!."

💡 Key Insight: After a few mentions, additional occurrences of a term provide less and less new information about relevance.

This is what's called diminishing returns. How can we reflect the diminishing returns of additional term frequency?

✨ Keeping It Simple: The “One-And-Done” Approach

Let’s say we score a term as 1 if it shows up at all, and 0 if it doesn’t. Simple!

If the term appears at least once → score = 1
If not → score = 0

This implies that our simple Score formula becomes:

$Score = (1\ if\ Term\ found,\ else\ 0) \times (Rarity\ of\ Term) + (1\ if\ Term\ found,\ else\ 0) \times (Rarity\ of\ Term)$

Calc it out, now!

Book	Title	"Mars" Present	"Exploration" Present	Rarity of "Mars"	Rarity of "exploration"	Score
A	Mars Exploration Guide	1	1	1.32	0.74	2.06
B	Space Encyclopedia	1	1	1.32	0.74	2.06
C	Planetary Science	0	0	1.32	0.74	0.00
D	Exploration Techniques	0	1	1.32	0.74	0.74
E	Solar System	0	0	1.32	0.74	0.00

🔍 The Problem: Too Hamfisted

This flattens everything too much – Book A and B tie again, and there's no reward for mentioning a term more than once.

We need something better: a curve that rises at first mention, but levels off with more and more mentions. This is consistent with the key insight above.

🤓 BM25's Solution: Smoothing Term Frequency with Saturation

What we want, is to revise our Score like this:

$Score = (\mathbf{Smoothed}\ Count\ of\ Term) \times (Rarity\ of\ Term) + (\mathbf{Smoothed}\ Count\ of\ Term) \times (Rarity\ of\ Term)$

BM25 uses a clever formula to model diminishing returns. It smooths the impact of each extra mention, and calls it "Saturated Term Frequency" like this:

$$ Saturated\ TF = \frac{Term\ Frequency \times (k + 1)}{Term\ Frequency + k} $$

Where k controls how fast the saturation kicks in. (Think of it like a knob for “how quickly we get tired of hearing the word.”) Oh, you're a smooth operator, BM25!

Let’s set k = 1.2, which is common in practice. We'll compute "Saturated" Term Frequency for Book A (8 mentions) and Book B (10 mentions):

Book A:
(8 × 2.2) / (8 + 1.2) = 17.6 / 9.2 ≈ 1.91
Book B:
(10 × 2.2) / (10 + 1.2) = 22 / 11.2 ≈ 1.96

Only a tiny bump in score for Book B, even with 2 more mentions. Exactly what we want!

Now we recalculate our score as follows:

$Score = Saturated\ TF(term) \times IDF(term) + Saturated\ TF(term) \times IDF(term)$

🧮 It's Time for the Calculator!

Book	Title	"Mars" Count	"Exploration" Count	Saturated TF ("Mars")	Saturated TF ("exploration")	Score
A	Mars Exploration Guide	8	6	1.91	1.83	3.88
B	Space Encyclopedia	10	4	1.96	1.69	3.84
C	Planetary Science	0	0	0.00	0.00	0.00
D	Exploration Techniques	0	12	0.00	2.00	1.47
E	Solar System	0	0	0.00	0.00	0.00

Victory! Book A now edges out Book B – just like it should. The repeated mentions in Book B still count, but no longer dominate.

Book D also ranks lower than both, which makes sense – it never even mentions Mars. Now we’re cooking!

🍸 Sound Smart and Be Sexy at Cocktail Parties:
"BM25 models diminishing returns using a saturation function."

📏 Step 4: Size Really Does Matter -- But Not How You Think (Document Length Normalization)

So, we’ve handled word rarity. We’ve handled diminishing returns. But there’s still a problem lurking in our scoring.

Imagine we add Book F:

Book	Title	Length (pages)	"Mars" count	"exploration" count
F	The Everything Book	1,000	15	12

This book is long. Like, Tolstoy long. It has 15 mentions of “Mars”! 12 mentions of “exploration”! No big deal -- we'll just sort it out in smooooothing, right?

Slow your roll.

A long document might mention "Mars" a bunch, even if it’s not the focus. A shorter document with fewer mentions might still be way more relevant if it’s concentrated on the topic.

💡 Key Insight: Longer documents mention everything more – not because they’re more relevant, but because they’re longer.

That means we need to adjust our scores based on document length.

🤔 The Intuition: Proportional Relevance

Take Book A and newly-added Book F (which we shall immediately drop and never consider again):

A 1-page document that mentions “Mars” 5 times
A 1,000-page book that mentions “Mars” 15 times

Which is more “about” Mars? The short one, obviously. It’s basically banging on about Mars every few lines.

So we want to reward concentration of terms, not just raw frequency. How could we do this?

✨ Simple, Yo!

We could just take our current scoring function, and adjust the frequency by the length of the document, right? That checks out.

Recall our simply worded score function:

$Score = (Smoothed\ Count\ of\ Term) \times (Rarity\ of\ Term) + (Smoothed\ Count\ of\ Term) \times (Rarity\ of\ Term)$

Which, when it graduates from university and gets an MBA from Wharton, becomes:

$Score = Saturated\ TF(term) \times IDF(term) + Saturated\ TF(term) \times IDF(term)$

What if we just adjusted our Smoothed Count by the length of the document thusly:

$$ Score = \frac{(Smoothed\ Count\ of\ Term)}{Document\ Length} \times (Rarity\ of\ Term) + \frac{(Smoothed\ Count\ of\ Term)}{Document\ Length} \times (Rarity\ of\ Term) $$

That would do just nicely! Let's run those calcs:

Book	Smoothed "Mars" Count	Smoothed "Exploration" Count	Document Length	Rarity of "Mars"	Rarity of "exploration"	Score
A	1.9130	1.8333	50	1.32	0.74	0.0776
B	1.9643	1.6923	200	1.32	0.74	0.0192
C	0.0000	0.0000	100	1.32	0.74	0.0000
D	0.0000	2.0000	75	1.32	0.74	0.0197
E	0.0000	0.0000	150	1.32	0.74	0.0000

🔍 The Problem: Where Did It Go?

Our simple, adjusted score gives short documents an edge (good!) and prevents long ones from coasting on raw term count (also good!). This works (-ish) but notice that our score is now reeeaaaalllyyy small. Look, in particular, at Books A and B.

Book A is short and focused – but its score is tiny.
Book B is long – and it scores basically nothing, even though it’s somewhat relevant to our search.

💡 Key Problem: We’re dividing by the raw length of the document, and it’s punishing longer documents a little too aggressively.

Long documents aren't necessarily bad – they just need a fair chance to prove their focus. And short documents shouldn't get an automatic VIP pass just because they’re compact.

⚖️📏 Size Does Matter, But It's Relative

Okay, dividing by raw length was a start, but it’s a bit too harsh–short docs get VIP passes, long ones get squashed. The real trick? Make it relative to the average document length, so everyone gets a fair shake.

Here’s how BM25 rolls: instead of just slapping a length penalty on after smoothing, we bake it right into the count adjustment. Think of it as tweaking how much repetition matters based on how chatty the document is compared to the norm. We call this the "Length Tweak":

$$ Length\ Tweak = Base\ Penalty + (Length\ Factor \times \frac{Document\ Length}{Average\ Document\ Length}) $$

- Base Penalty = A number like 1.2 (same as our k from smoothing–caps those extra mentions). - Length Factor = A dial, say 0.9, to decide how much length should nudge things (it’s Base Penalty × 0.75, but don’t sweat the math yet).

See how the Length Tweak, overall, calibrates the document being compared to the average document length? We then take this Length Tweak and scale the term count up or down, based on the length of the document it was found in. So, instead of:

$$ Score = \frac{(Smoothed\ Count)}{Length} \times Rarity $$

We mix it all together like this:

$$ Adjusted\ Count = \frac{(Count\ of\ Term \times Boost)}{(Count\ of\ Term + Length\ Tweak)} $$

For now, Boost = 2.2 – it's a little kick to Count of Term which we’ll explain soon.

Note that our Adjusted Count is scaled up if the document was relatively short -- i.e. we divide Count of Term by our Length Tweak which is between 0 and 1. The Adjusted Count is scaled down if the document was relatively long -- i.e. we divide Ccount of Term by our Length Tweak, which is >= 1.

Then, what we're left with, now, is our simplified Score of:

$Score = (Adjusted\ Count\ of\ Term) \times (Rarity\ of\ Term) + (Adjusted\ Count\ of\ Term) \times (Rarity\ of\ Term)$

💡 Key Insight: Long docs don’t get slammed just for being long – they only take a hit if they’re longer than average and still blathering.

See? Size does matter, but it’s all relative! 😜 We'll bring it all together in the next section.

💡 The BM25 Reveal: Why This Works 💡

Taking it all in, then, from Steps 1 - 4, BM25 ranks documents based on search terms by calculating:

$Score = \sum_{each\ Term} [ Adjusted\ Count\ of\ Term \times Rarity\ of\ Term ]$

Where:

Adjusted Count of Term =

$$ \frac{Term\ Frequency \times (k + 1)}{Term\ Frequency + k \times ((1 - b) + b \times \frac{Document\ Length}{Average\ Document\ Length})} $$

This takes the term’s count and gives it a little boost – called Boost and often set to 2.2 – then tones it down based on how often it repeats and how long the document is compared to the average. Why 2.2 for the Boost? Back in Step 3, we smoothed counts with a knob called k – set to 1.2 – to chill out extra mentions. The Boost is just k + 1, or 1.2 + 1 = 2.2, like the high-five between Maverick and Goose on the runway, before we calm things down with diminishing returns (saturation) and length normalization, all in one go.

Rarity of Term = Inverse Document Frequency =

$$ \log\left(\frac{Total\ Docs}{Docs\ containing\ the\ term}\right) $$

This measures how rare the term is across all documents–rarer terms get a bigger score because they’re more special.

OR, said in words that Mum will understand over Sunday dinner, for each search term in the query:

Count how often it pops up in the document (but don’t go overboard – we give it a boost then smooth it out so a few mentions are enough).
Multiply by how rare it is in the whole pile of books (rare words are gold).
Tweak it based on the document’s length (short and snappy gets a lift, long and wordy gets a gentle nudge down).
Add up the scores for all the terms you’re searching for.

🍷 Summarize BM25 at Gala Affairs

“Each term gets credit for how often it appears, how rare it is, and how focused the document is on it – with soft penalties for verbosity and overuse.”

This addresses the problems we saw along the way:

Raw counts aren't enough – a term appearing more often doesn't always mean the document is more relevant. (See: Book B beating Book A early on)
Some words matter more than others – rare terms provide sharper clues about relevance. (IDF to the rescue)
Extra mentions don’t add linear value – the 10th “Mars” isn’t as informative as the 1st. (Saturation fixes that)
Longer docs talk more – but that doesn’t mean they’re better matches. (Normalization keeps verbosity in check)

✅ BM25 Ranking of "Mars exploration" for our Baby Library

It's the last time for The Calculator! 😢 Here's how the books in our wee library rank:

Book	Title	Length	Adjusted Count (Mars)	Adjusted Count (exploration)	BM25 Score
A	Mars Exploration Guide	50	1.83	1.74	3.71
B	Space Encyclopedia	200	1.72	1.30	3.24
C	Planetary Science	100	0.00	0.00	0.00
D	Exploration Techniques	75	0.00	1.93	1.42
E	Solar System	150	0.00	0.00	0.00

So, all's well that ends well. Book A -- Mars Exploration Guide -- ranked highest in our search and provided us with all of the knowhow we need to put boots on the ground on Mars when Elon gets us there.

BM25 FTW! And best of all? It’s fast. It’s robust. And it works – in RAG pipelines, search engines, recommendation systems ... you name it!

🟰 Mathing All the BM25 Maths

Here's BM25 formally expressed, so that your mates don't bully you at your next gathering.

$$ \text{Score}(D, Q) = \sum_{i=1}^{n} \text{IDF}(q_i) \cdot \frac{\text{TF}(q_i, D) \cdot (k_1 + 1)}{\text{TF}(q_i, D) + k_1 \cdot (1 - b + b \cdot \frac{|D|}{\text{avgdl}})} $$

Where:

$D$ is the document,
$Q$ is the query with terms $q_i$.
$\text{TF}(q_i, D)$ is the term frequency of query term $q_i$ in document $D$.
$k_1$ is a parameter controlling term frequency saturation (commonly 1.2).
$b$ is a parameter controlling length normalization (commonly 0.75).
$|D|$ is the document length, $\text{avgdl}$ is the average document length.
$\text{IDF}(q_i) = \log\left(\frac{N - n(q_i) + 0.5}{n(q_i) + 0.5}\right)$ or a simpler variant like $\log\left(\frac{N}{n(q_i)}\right)$, where $N$ is total documents and $n(q_i)$ is documents containing $q_i$.

Be gone, maths bullies!

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a futuristic holographic interface floating in a dense digital matrix: abstract, glowing documents and cascading streams of neon equations that represent a sophisticated search ranking algorithm. Scattered holograms and digital grids display intricate formulas and logarithmic graphs, evoking the essence of BM25’s balancing of term frequency with filtering power. The scene is awash in swirling neon lights, pulsating data streams, and a high-tech, otherworldly ambiance that highlights the interplay between mathematical precision and futuristic innovation.

⚡️ Latent Space - The new OpenAI Agents Platform

2025-03-20T00:00:00-05:00

OpenAI have dropped their new Agents Platform. Swyx and Alessio chat with Nikunj Handa and Romain Huet of OpenAI to discuss the release.

Episode Show Notes

While everyone is now repeating that 2025 is the “Year of the Agent”, OpenAI is heads down building towards it. In the first 2 months of the year they released Operator and Deep Research (arguably the most successful agent archetype so far), and today they are bringing a lot of those capabilities to the API:

Responses API

Web Search Tool

Computer Use Tool

File Search Tool

A new open source Agents SDK with integrated Observability Tools

We cover all this and more in today’s lightning pod on YouTube !

More details here:

Responses API

In our Michelle Pokrass episode we talked about the Assistants API needing a redesign. Today OpenAI is launching the Responses API, “a more flexible foundation for developers building agentic applications”. It’s a superset of the chat completion API, and the suggested starting point for developers working with OpenAI models.

One of the big upgrades is the new set of built-in tools for the responses API: Web Search, Computer Use, and Files.

Web Search Tool

We previously had Exa AI on the podcast to talk about web search for AI. OpenAI is also now joining the race; the Web Search API is actually a new “model” that exposes two 4o fine-tunes: gpt-4o-search-preview and gpt-4o-mini-search-preview. These are the same models that power ChatGPT Search, and are priced at $30/1000 queries and $25/1000 queries respectively.

The killer feature is inline citations: you do not only get a link to a page, but also a deep link to exactly where your query was answered in the result page.

Computer Use Tool

The model that powers Operator, called Computer-Using-Agent (CUA), is also now available in the API. The computer-use-preview model is SOTA on most benchmarks, achieving 38.1% success on OSWorld for full computer use tasks, 58.1% on WebArena, and 87% on WebVoyager for web-based interactions.

As you will notice in the docs, computer-use-preview is both a model and a tool through which you can specify the environment.

Usage is priced at $3/1M input tokens and $12/1M output tokens, and it’s currently only available to users in tiers 3-5.

File Search Tool

File Search was also available in the Assistants API, and it’s now coming to Responses too. OpenAI is bringing search + RAG all under one umbrella, and we’ll definitely see more people trying to find new ways to build all-in-one apps on OpenAI.

Usage is priced at $2.50 per thousand queries and file storage at $0.10/GB/day, with the first GB free.

Agent SDK: Swarms++!

https://github.com/openai/openai-agents-python

To bring it all together, after the viral reception to Swarm , OpenAI is releasing an officially supported agents framework (which was previewed at our AI Engineer Summit ) with 4 core pieces:

Agents : Easily configurable LLMs with clear instructions and built-in tools.

Handoﬀs : Intelligently transfer control between agents.

Guardrails : Configurable safety checks for input and output validation.

Tracing utm_campaign=CTA_4">www.latent.space/subscribe

New OpenAI Agent Tools & APIs

🎧 Play snip - 2min (00:40 - 02:21)

OpenAI is launching three new built-in tools: a web search tool (ChatGPT for search in the API), a file search tool (upload, parse, chunk, embed, and search your data), and a computer use tool (powering the Operator product in ChatGPT).
They are also launching a new Responses API to support these tools, replacing the older ChatCompletions API and designed for future agentic products.
Finally, they are releasing an upgraded Agents SDK (formerly Swarm) with built-in tracing in the OpenAI dashboard for multi-agent orchestration.

Responses API: A Unified and Flexible Approach

🎧 Play snip - 4min (02:41 - 06:16)

The Responses API is a new, more flexible API from OpenAI, superseding the Chat Completions and Assistants APIs.
It's designed for agentic workflows, supporting longer, multi-turn tasks and tool use.
While Chat Completions remains available, Responses API offers a unified endpoint with broader capabilities.
It incorporates features from the Assistants API, like convenient tool access, while simplifying integration.
Responses API also offers stateless mode for compatibility with Chat Completions use cases.

GPT-4 with Search Preview Performance

🎧 Play snip - 9sec (08:09 - 08:19)

OpenAI's web search API uses a GPT-4 model fine-tuned specifically for search, named GPT-4.0 Search Preview.
GPT-4.0 with search has significantly better performance than the base model.
Simple QA accuracy jumps from 38% with the base GPT-4 to 90% with the search-tuned model.

Computer Use Tool

🎧 Play snip - 2min (18:16 - 20:08)

OpenAI's Computer Use tool empowers developers to build agents that can complete tasks using a computer or browser.
This tool utilizes a custom model optimized for computer use, enabling agents to interact with the screen by clicking, scrolling, typing, and reporting back.
By wrapping this functionality as a tool within the Responses API, developers can automate tasks and create multi-turn interactions where agents can execute complex actions over time.

DIY vs. OpenAI for Vector Search

🎧 Play snip - 1min (17:03 - 18:33)

If starting from scratch, you'll have more control over chunking and retrieval strategies.
OpenAI's tool provides an out-of-the-box managed service with customization options.
Start with OpenAI's solution and see if it meets your needs.
They plan to add more customization features over time.
Consider hand-rolling with other solutions if you require complete control.

OpenAI Agent SDK

🎧 Play snip - 3min (21:34 - 24:28)

OpenAI's new Agent SDK simplifies building and managing agents in production.
It supports type checking, guardrails for safer execution, and tracing for monitoring agent behavior.
The SDK is flexible, allowing integration with various chat completion APIs and tracing providers.
It leverages the 'handoff' technique popularized by Swarm, enabling complex agent workflows.
Combined with new tracing UIs in the OpenAI dashboard, developers can effectively troubleshoot and optimize their agent interactions.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a futuristic control center filled with holographic digital interfaces and abstract data streams that represent advanced AI and agentic workflows. The scene showcases glowing icons and symbols illustrating computerized search tools, file systems, and interactive APIs, all interwoven with sleek circuit patterns and dynamic grids. The overall atmosphere is busy yet orderly, evoking the cutting-edge innovations of OpenAI’s new Agents Platform and the rapid evolution of agent-based technology in a neon-lit digital landscape.

The Python Lambda Tutorial You Never Asked For

2025-03-19T00:00:00-05:00

Let's be real, nobody wakes up in the morning thinking, "You know what I need more of in my life? Python lambda functions." But here we are. Buckle up, because this is the lambda tutorial you didn't ask for -- but secretly need.

Lambda Functions: Anonymous, Lazy One-Liners

Lambda functions, also known as anonymous functions (because naming is overrated), let you squeeze a simple function into a single, no-nonsense line of code:

lambda arguments: expression

Arguments: Stuff you feed the function.
Expression: What the function spits out. And only one -- don't get greedy.

For instance, incrementing a number by 1, because who has time for def?

lambda x: x + 1

This is equivalent to:

def increment(x):
    return x + 1

It's basically the Netflix of Python functions: fast, easy, and minimal commitment.

When Lambdas Actually Matter

Lambda functions are perfect when:

You want a quick-and-dirty function without cluttering your namespace.
Passing functions as arguments -- especially with pandas' apply().

Here's an actual example from real-world usage:

df['vector'] = df['content'].apply(lambda x: model.encode(x, normalize_embeddings=True))

Translation:

Take each content item from your DataFrame.
Turn it into a vector (because vectors are cool).
Do it without polluting your script with needless function names.

It's like hiring an intern for a five-minute task -- without having to learn their name.

Lambda Goodness

Concise: One-liner code beauty.
Functional Programming Friendly: Easily passed around like a hot potato.
Cleaner: Prevents namespace pollution, aka fewer things to regret later.

Lambda Badness

Single Expression Limitation: Complex logic? Look elsewhere.
Debugging Nightmare: Anonymous means nameless means frustration.
Overuse Syndrome: Just because you can doesn't mean you should. (Looking at you, regex.)

Quick-and-Dirty Lambda Examples

Sorting a List

Here's where lambdas actually shine - complex sorting that would be a pain in the ass otherwise:

points = [(1, 2), (3, 1), (5, 0)]
sorted_points = sorted(points, key=lambda point: point[1])
print(sorted_points)  # Output: [(5, 0), (3, 1), (1, 2)]

What's happening here? We're telling Python "sort these tuples, but ignore the first number and sort by the second one instead." The lambda grabs each tuple, extracts the second value (point[1]), and sorted() does the rest. Without lambdas, you'd need a whole separate function for this trivial operation. Bloody waste of code.

Filtering a List

When you need to separate the wheat from the chaff:

numbers = [1, 2, 3, 4, 5]
even_numbers = list(filter(lambda x: x % 2 == 0, numbers))
print(even_numbers)  # Output: [2, 4]

The filter() function is expecting another function that returns True or False. Our lambda takes each number, divides by 2, and checks if there's a remainder. No remainder? It's even, keep it. Otherwise, chuck it. The result is only the even numbers making it through the filter. It's like a bouncer at a club where odd numbers aren't cool enough to get in.

Transforming a List with Map

Need to apply the same operation to every item in a list? map() + lambda is your friend:

numbers = [1, 2, 3, 4]
squared_numbers = list(map(lambda x: x ** 2, numbers))
print(squared_numbers)  # Output: [1, 4, 9, 16]

Here, map() takes our lambda and applies it to each number in the list. The lambda just squares whatever it gets. So simple it's almost boring, but that's the point. You don't need a whole function definition ceremony for something this straightforward. It's the difference between sending a text and writing a formal letter – sometimes you just need to get the message across without the fluff.

Wrap-Up: Lambdas in a Nutshell

Lambda functions are your go-to for simple, anonymous tasks that aren't worth the full-blown function definition. Sure, they're limited. Yeah, debugging them is annoying. But when used wisely, they're elegant as hell.

So next time you need a function that's here for a good time -- not a long time -- reach for lambda.

And that's it. Tutorial over. Go forth and lambda responsibly (or don't -- I'm not your boss).

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Create a futuristic digital scene featuring a sleek, neon-glowing computer interface that showcases snippets of Python lambda function code floating in holographic displays. Integrate abstract data streams, dynamic geometric grids, and circuit-like patterns that pulse with vibrant neon pink, blue, and purple hues to evoke the minimalist yet powerful nature of lambda functions in a high-tech, cyberpunk environment.

Founders #383: Todd Graves' $10B Chicken Finger Dream

2025-03-18T00:00:00-05:00

I listened to Senra's great episode on Todd Graves of Raising Cane's fame this morning. Here are my snips from the episode.

Episode Show Notes

Todd Graves is one of my favorite living entrepreneurs. He's a great example of Charlie Munger's maxim: Find a simple idea and take it seriously. Todd wanted to create a quick service restaurant that only focused on quality chicken finger meals and nothing else. Everyone told him that couldn't possibly work. The college paper that described the idea that would turn into Raising Canes got the lowest grade in the class. Banks wouldn't loan him any money —but nothing could stop Todd from living out his "chicken finger dream." He worked 95 hour weeks as a boilermaker, risked his life on a commercial fishing boat off the coast of Alaska, and scrounged up startup money from his bookie and a guy named Wild Bill. Todd made every mistake in the book, over leveraged himself, almost lost everything and yet he refused to give up or sell out. Today he has over 800 locations, 50,000 employees, and owns 90% of a business that's worth at least $10 billion. Todd's maxim is "Do one thing and do it better than anyone else."

Winning Is Personal

🎧 Play snip - 1min (03:39 - 05:06)

Approach your work with a personal stake, treating every win as deeply significant and every loss as a personal setback.
Internalize the drive to win and never let anyone take away what you've worked hard to achieve.
Cultivate this mindset from the beginning of your endeavors to fuel continuous hard work and determination.
Take your business or work seriously and be deeply invested in your success, similar to Todd Graves of Raising Cane's and legendary coach Pat Riley.

Maximize One or a Few Variables

🎧 Play snip - 5min (05:17 - 10:12)

Focus on maximizing one or a few key variables in your business.
Todd Graves built a successful chain by focusing on simple menu and speed.
Rockefeller's obsession with efficiency, like minimizing solder drops, saved vast amounts over time.
Small efficiency gains at scale can compound into large savings.
Consider how even minor changes in your system can multiply across your operations.

Passion and Perseverance

🎧 Play snip - 2min (16:03 - 17:54)

Choose a career you are passionate about because work can be a grind.
If you love what you do, you will be happier and more likely to stick with it.
This long-term commitment allows for compounding success over time, like with Raising Cane's.
Steve Jobs believed passion is essential for enduring the hardships of work.
Todd Graves echoed this sentiment, emphasizing that giving up or selling out was never an option for him.

Hands-On Management

🎧 Play snip - 2min (21:06 - 23:00)

Prioritize quality and refuse to compromise.
Maintain a hands-on management style, overseeing all aspects of the business.
Treat employees exceptionally well to cultivate loyalty and long-term retention.
Be relentless in your work ethic, setting an example for your team.

Todd Graves' Ownership

🎧 Play snip - 1min (28:10 - 29:28)

When asked about funding, Todd Graves revealed he prefers using his own money.
He owns over 90% of his multi-billion dollar business, Raising Cane's Chicken Fingers.
This contrasts with many entrepreneurs who get diluted by private equity and eventually sell out.
Graves' approach allows him to maintain control and passion for his brand.
The host emphasizes the rarity of such high ownership in large businesses, particularly in the restaurant industry.

Commitment to the Long Term

🎧 Play snip - 1min (30:04 - 30:51)

Your exit strategy should be death, meaning dedicate your life to your business.
Work on your best ideas and love what you do, like Steve Jobs, Charlie Munger, and Coco Chanel who worked until they died.
Have a high-risk tolerance as an entrepreneur and aim to retain as much equity as possible because the business is yours.
Build a multi-generational business that you wouldn't sell.

Todd Graves' and Daniel Ludwig's Creative Financing

🎧 Play snip - 5min (33:03 - 37:35)

Todd Graves expanded Raising Cane's by acquiring failing double drive-through burger joints and converting them cheaply.
He financed this by securing subordinated debt notes from angel investors with a guaranteed 15% return, using these notes as collateral for bank loans.
Daniel Ludwig, once the richest American, started with a 'two-name paper arrangement'.
He secured long-term charter agreements with oil companies, then used these as collateral for bank loans to build or renovate ships.
Oil company payments went directly to the bank, covering the loan and depositing the rest into Ludwig's account; this maximized leverage and minimized personal investment.

Todd Graves on Katrina and Pandemic as Opportunities

🎧 Play snip - 2min (39:06 - 40:54)

Todd Graves almost lost Raising Cane's due to over-leveraging when Hurricane Katrina hit, shutting down most of his restaurants.
However, being the first to reopen turned into an opportunity, gaining new loyal customers in a period with no competition.
Similarly, during the pandemic, Raising Cane's drive-thru model thrived as an essential business, boosting revenue from $1.5B to $4.5B in three years.
Graves emphasizes that his sole ownership allowed for quick decision-making during crises, focusing on crew and customers without board or shareholder constraints.
He highlights the importance of adaptability and decisive action during challenging times.

Raising Cane's 'Love' Department

🎧 Play snip - 3min (42:20 - 45:24)

Todd Graves prioritizes positive reinforcement and appreciation for his employees, creating a 'Cane's Love' department.
This department focuses on respecting, rewarding, and recognizing crew members, even for small acts of service.
Graves emphasizes that good quality work comes from positive motivational management, visiting kitchens and offering praise to employees.
He believes in caring for his employees, which in turn motivates them to work harder, leading to a positive work environment and efficient service.

Founder-Led Advantage

🎧 Play snip - 1min (47:42 - 48:27)

Todd Graves prioritizes company-owned Raising Cane's stores over franchises for greater control and quality.
He believes founder-led businesses have a competitive edge.
Large corporations often make impersonal financial decisions, while founders have a personal stake.
Graves emphasizes that Raising Cane's is personal to him, which drives his commitment to excellence.

Missionary Founder

🎧 Play snip - 1min (48:25 - 49:30)

Todd Graves believes his talent for chicken fingers is divinely inspired to help others.
He encourages aspiring entrepreneurs to pursue their unique concepts, fostering diversity and innovation.
Despite billion-dollar offers, he remains committed to his mission, prioritizing purpose over profit.
Graves believes that God makes everyone good at something to ultimately help others.

Leverage Your Assets

🎧 Play snip - 1min (49:56 - 50:42)

Consider what non-financial assets you possess that you aren't fully utilizing.
When starting out, you might lack money or experience, but you have assets that older, wealthier individuals don't.
Younger people often have abundant energy and determination, which can be leveraged to outwork others.
Todd Graves, the founder of Raising Cane's, emphasizes the importance of youthful energy and determination in his entrepreneurial journey.

Last Business

🎧 Play snip - 1min (50:51 - 51:32)

David Senra expresses more interest in a founder's final business than their initial ventures.
He believes understanding oneself and accumulating experience often leads to starting multiple businesses.
However, some founders, like Todd Graves and Sam Walton, find their ultimate business early on and dedicate themselves to it fully.
This dedicated focus allows for growth in unforeseen ways over time.

Extreme Patience and Intolerance for Slowness

🎧 Play snip - 1min (53:42 - 54:55)

Todd Graves took 25 years to build 600 Raising Cane's locations, then added 200 more in just two to three years.
Sam Walton, in his first five years of retail, only opened one store.
Three decades later, he launched Sam's Club and reached $1 billion in sales in three years.
In 7 years, he expanded from zero to 105 stores.
This exemplifies "extreme patience coupled with an extreme intolerance for slowness."

Trust Your Gut

🎧 Play snip - 1min (57:13 - 57:55)

Don't listen to so-called 'experts'.
Listen to your gut and intuition.
Stay true to your vision and what you do well.
Avoid trying to be all things to all people.
Know who you are and stick to it.

Henry Ford on Experts

🎧 Play snip - 1min (59:18 - 01:00:14)

Henry Ford didn't believe experts could see the potential of the internal combustion engine.
They were too focused on its limitations compared to steam.
Ford believed that experts could kill opposition with too much advice and little work.
He valued trusting his gut and staying true to his vision.

Focus on Details

🎧 Play snip - 15sec (01:00:37 - 01:00:53)

Limit the number of things you try to perfect.
Then, make every detail of those things perfect.
This allows for laser focus on making each item great.
Bring in people smarter than you in their respective areas but don't stop being detail-oriented yourself.

Stay in the Details

🎧 Play snip - 2min (01:00:56 - 01:02:48)

Work with great leaders, but stay involved in the details of your business.
Don't just delegate; understand the specifics of what's happening.
Even as your business grows, maintain awareness of seemingly small expenses, as they reflect broader spending habits.
Todd Graves emphasizes this, citing Edison Chouest, a successful shipbuilder, as an example of someone who paid attention to even the cost of bottled water.

Do What You Love

🎧 Play snip - 1min (01:02:39 - 01:04:01)

The most successful people stay engaged in the details of their work.
This is easier when you're passionate about what you do.
Don't be afraid to do things yourself, even if others advise against it.
Working on something you love doesn't feel like work; it's enjoyable.
“You don't work your entire life to get, to, to be able to do what you love, to not do it.”

Passion as a Competitive Advantage

🎧 Play snip - 1min (01:03:46 - 01:05:08)

Loving what you do makes hard work easier, as exemplified by Todd Graves's dedication to his "chicken finger dream."
True passion fuels long hours and hands-on involvement in every aspect of the business.
Competing with someone driven by such passion is difficult because their work ethic becomes a major competitive advantage.
Graves welcomes competition but warns potential rivals to be prepared for the intense dedication required to challenge a business built on personal commitment and family values.

Fear of Purpose

🎧 Play snip - 1min (01:05:40 - 01:07:09)

Todd Graves advises entrepreneurs to honestly confront their life's purpose.
He acknowledges that accepting this purpose can be daunting, as it requires dedicating oneself entirely to it.
This resonates with Kobe Bryant's message about facing our greatest fear: ourselves and our dreams.
Bryant emphasizes the fear of fully committing to a dream and potentially failing.
He encourages pursuing dreams fearlessly, not for external validation, but for self-fulfillment.

Todd Graves' Advice to Entrepreneurs

🎧 Play snip - 29sec (01:07:15 - 01:07:44)

Never give up on your vision, even when faced with challenges like securing financing or finding a suitable location.
Be fanatical about your pursuit.
True success comes from relentlessly pursuing your vision with unwavering passion.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Envision a dazzling futuristic metropolis at night where holographic skyscrapers and digital billboards rise like monoliths from a rain-soaked cityscape. In the heart of the scene, a sleek, high-tech restaurant stands out with a luminous, abstract neon chicken finger emblem that pulses with electric energy. Intricate, circuit-like patterns and reflective surfaces weave through the urban tapestry, evoking the raw determination, innovative spirit, and relentless ambition of a visionary billion-dollar dream.

How to Add Custom HTML Pages to a Pelican Site

2025-02-27T00:00:00-06:00

Introduction

Pelican is a powerful static site generator written in Python, primarily designed for blogs. However, sometimes you need to include custom HTML pages that don't fit into Pelican's standard content model. This guide explains how to add standalone HTML pages to your Pelican site while preserving their exact formatting and functionality.

Method: Using Static Files

The most reliable method for adding custom HTML pages to a Pelican site is to use the static files approach. This method bypasses Pelican's content processing system and treats your HTML file as a static asset.

Step 1: Create a Static Directory

First, create a static directory inside your Pelican project's content folder:

mkdir -p content/static

Step 2: Add Your HTML File

Copy your custom HTML file to the static directory:

cp your-custom-page.html content/static/

Step 3: Configure Pelican

Edit your pelicanconf.py file to include the following settings:

# Include static directory in static paths
STATIC_PATHS = ['images', 'static']  # Add other paths as needed

# Exclude static directory from content processing
ARTICLE_EXCLUDES = ['static']
PAGE_EXCLUDES = ['static']

# Optional: Configure a specific URL for your page
EXTRA_PATH_METADATA = {
    'static/your-custom-page.html': {'path': 'custom-page/index.html'},
}

This configuration does three important things:

STATIC_PATHS tells Pelican to copy files from the static directory to the output
ARTICLE_EXCLUDES and PAGE_EXCLUDES prevent Pelican from trying to process the files as content
EXTRA_PATH_METADATA allows you to specify a custom output path for your HTML file

Step 4: Build Your Site

Now when you build your Pelican site, your custom HTML page will be copied to the output directory:

pelican content

Your page will be accessible at yourblog.com/custom-page/.

Why This Works

This approach works because:

Bypasses Content Processing: By excluding the static directory from content processing, Pelican won't try to parse your HTML file for metadata or apply templates to it.
Preserves Original HTML: Your HTML file is copied as-is to the output directory, preserving all styling, scripts, and functionality.
Custom URL Structure: Using EXTRA_PATH_METADATA allows you to place the file at any URL path you want, creating clean URLs without the .html extension.

Common Issues and Solutions

Issue: Pelican tries to process HTML as content
Solution: Make sure you've added the static directory to both ARTICLE_EXCLUDES and PAGE_EXCLUDES.

Issue: CSS or JavaScript paths are broken
Solution: If your HTML file references relative paths for CSS or JavaScript, you may need to adjust them to work with your site's URL structure.

Issue: HTML file not showing up in output
Solution: Verify that you've added the static directory to STATIC_PATHS and that the file exists in the correct location.

Advanced: Linking to Your Custom Page

To link to your custom page from your Pelican site's navigation menu, you'll need to modify your theme's templates. Most themes use a variable like MENUITEMS in the pelicanconf.py file:

MENUITEMS = [
    ('Home', '/'),
    ('Blog', '/blog/'),
    ('Custom Page', '/custom-page/'),
]

After adding your custom HTML page to your Pelican site, you'll likely want to make it accessible through your site's navigation menu. There are several ways to accomplish this, depending on your theme and navigation preferences.

The simplest approach is to use Pelican's built-in MENUITEMS configuration variable, which most themes support. In your pelicanconf.py file, add or modify the MENUITEMS list:

MENUITEMS = [
    ('Home', '/'),
    ('Blog', '/blog/'),
    ('Custom Page', '/custom-page/'),  # This links to your custom HTML page
]

Each tuple contains two elements: 1. The display name that will appear in your menu 2. The URL path to the page (which should match what you configured in EXTRA_PATH_METADATA)

Balancing MENUITEMS with Dynamic Category Links

If your theme automatically displays categories in the navigation menu, you'll need to consider how static menu items and dynamic category links will interact. Here are some common configurations:

If you prefer to manually control all menu items, you can disable automatic category links:

# Disable automatic category links in menu
DISPLAY_CATEGORIES_ON_MENU = False

# Define your custom menu structure
MENUITEMS = [
    ('Home', '/'),
    ('Blog', '/blog/'),
    ('Categories', '/categories/'),  # Link to category index if available
    ('Custom Page', '/custom-page/'),
]

Option 2: Keep Both Systems

Many themes will gracefully handle both static menu items and dynamic category links, placing MENUITEMS first, followed by category links:

# Keep automatic category links enabled (default behavior)
# DISPLAY_CATEGORIES_ON_MENU = True  # This is usually the default

# Define static menu items
MENUITEMS = [
    ('Home', '/'),
    ('Blog', '/blog/'),
    ('Custom Page', '/custom-page/'),
]

Some themes implement custom navigation systems that don't rely solely on MENUITEMS. In these cases, consult your theme's documentation or examine its template files to understand how to integrate custom pages.

For example, the popular theme "Flex" uses a variable called MAIN_MENU with a more complex structure:

MAIN_MENU = True
MENUITEMS = (
    ('Home', '/'),
    ('Custom Page', '/custom-page/'),
    ('Categories', (
        ('Category 1', '/category/category-1/'),
        ('Category 2', '/category/category-2/'),
    )),
)

Whenever you modify your menu configuration, test your site by regenerating it:

pelican content

Then check the navigation links in your local development server to ensure they work as expected.

Troubleshooting Common Issues

Issue: Menu items don't appear
Check if your theme supports MENUITEMS or requires a different variable. Some themes require MAIN_MENU = True to be set as well.

Issue: Links don't work correctly
Verify that your URL paths match the structure defined in EXTRA_PATH_METADATA. Remember that if you're using RELATIVE_URLS = True for development, your paths will be relative.

Issue: Unexpected menu order
If categories are appearing between your static menu items, you may need to override your theme's template files to control the precise ordering.

Conclusion

Using the static files approach is the most straightforward way to add custom HTML pages to your Pelican site. It preserves the original HTML without modification, allows for custom URL structures, and integrates seamlessly with the rest of your site.

This method is particularly useful for: - Interactive pages with complex JavaScript - HTML generated by other tools - Pages with custom styling that shouldn't be affected by your site's theme - Embedding third-party applications or widgets

By following these steps, you can extend your Pelican site beyond its standard blog functionality while maintaining full control over your custom HTML content.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a futuristic digital workspace where sleek holographic panels display lines of glowing HTML code and static web assets, symbolizing custom HTML pages for a Pelican site. The background features a neon-lit cityscape with pulsating grids and cybernetic data streams flowing through an atmospheric digital environment. The scene exudes high-tech energy and the seamless integration of traditional web development with a modern, cyberpunk twist.

AI Engineer Summit 2025: Agents at Work

2025-02-22T00:00:00-06:00

Executive Summary

The 2025 AI Engineer Summit marked a pivotal transition from theoretical AI capabilities to practical implementation reality. We've had unprecedented advances in foundation models, reasoning capabilities, and infrastructure creating what some called "the perfect storm for AI agents," but let's be real – there's a massive gap between what these models can theoretically do and what they reliably deliver in production. The most successful implementations are focused on specific domains with clear business value rather than general-purpose solutions, with consistent reports that even advanced reasoning models achieve only 70-80% accuracy on complex real-world tasks. That sounds impressive until you realize it means every fifth request is completely wrong. The industry's entering a consolidation phase where implementation expertise, reliability engineering, and domain knowledge are becoming more valuable than raw model capabilities. We've got the rocket, and it (mostly) doesn't blow up, but to get to Mars, there are a hundred other things we need to figure out.

Key Takeaways

This is supposedly "the year of agents" (yeah, I know, we're all tired of hearing it already). Things still don't actually work reliably yet. That said, it's only February and we have 10 fast market months of AI development ahead of us!

There's a serious reality gap where models performing well in controlled settings regularly fail in complex real-world environments, with error rates of 15-20% being common – when did we abandon basic machine learning principles?

Evals matter more than ever – you need multi-dimensional evaluation frameworks that consider accuracy, cost, latency, and reliability rather than relying on benchmark performance that rarely translates to the real world.

Domain-specific models consistently outperform general-purpose solutions for enterprise use cases, crossing the threshold of sufficient reliability where general models still fall short.

Enterprise adoption requires addressing security, governance, and compliance concerns through multi-layered frameworks that most AI demos conveniently ignore.

The most future-proof systems are designed to scale with intelligence, improving automatically as underlying models get better rather than requiring constant reengineering.

Voice and multimodal agents represent the next frontier but come with massive hurdles for latency, reliability, and user experience that aren't worth chasing until the tech catches up.

Effective human-AI collaboration remains central to successful implementations, with the best systems amplifying human expertise rather than replacing it.

Speaker Landscape

The conference featured an exceptionally diverse and high-quality set of perspectives spanning the complete AI implementation ecosystem – all the real ones were there. Enterprise practitioners included leaders from financial services (Bloomberg, Jane Street, Ramp, Method Financial), pharmaceuticals (Pfizer), media (Thomson Reuters), technology (LinkedIn, Datadog), and travel (Booking.com). Major AI labs were represented by speakers from OpenAI, Anthropic and Google Gemini, providing insights into frontier model capabilities. The venture capital perspective came from Grace Isford (Lux Capital) and Heath Black (SignalFire), while infrastructure specialists included Paul Gilbert (Arista Networks) and Don Bosco Durai (Privacera). Emerging startups showcasing specialized tools included Windsurf, BrightWave, Sierra, OpenPipe, Writer and Contextual AI. Academic and research perspectives came from Will Brown (Morgan Stanley), Sayash Kapoor (AI Snake Oil), and Stefania Druga (Google). This blend created a comprehensive view of the state of AI implementation across different domains and organizational contexts.

Thematic Analysis

From Perfect Storm to Practical Implementation: The Reality Gap

Everyone's talking about 2025 as "the perfect storm for AI agents" with converging advancements in reasoning models, test-time compute, engineering optimizations, hardware costs, and infrastructure investments. Yet as Grace Isford from Lux Capital put it, "we're seeing a lot of thunder, a lot of great momentum, but we haven't seen that lightning strike." This tension between theoretical capabilities and practical implementation challenges dominated every track.

The gap is real and it's massive. Sayash Kapoor (AI Snake Oil) highlighted how many headline-grabbing agent capabilities fail spectacularly in real-world scenarios. Waseem Alshikh (Writer) revealed that even leading reasoning models achieve only 81% combined accuracy on real-world financial scenarios, meaning "every hundred requests, 20 of them are just completely wrong." Anju Kambadur (Bloomberg) emphasized that in finance, "precision, comprehensiveness, speed, throughput, and availability" are non-negotiable requirements that current agent technologies don't consistently deliver. This pattern revealed a consistent truth across presentations: the gap between AI's capabilities in controlled settings and its performance in complex real-world environments remains substantial.

Think of it in an analogy of getting to Mars – the LLM is the rocket, but to get to the Mars, there are a hundred other things we need to figure out. We've got a rocket that (mostly) doesn't blow up, but that's it. Space is unforgiving, and so (to a lesser extent) are production environments.

The path forward emphasized systematic evaluation and incremental improvement. Kyle Corbitt (OpenPipe) demonstrated how Method Financial scaled to 500 million agent deployments by carefully measuring error rates, latency, and cost across different models before fine-tuning a small, reliable model optimized for their specific use case. Barry Zhang (Anthropic) advised a simple architecture focused on three components—environment, tools, and system prompt—that could be iteratively improved rather than trying to build complex systems from the start. Multiple speakers recommended starting with single-purpose agents before attempting multi-agent systems, focusing on specific high-value domains rather than attempting to build general-purpose solutions.

Domain Specialization and Data as the Competitive Moat

A clear consensus emerged that domain-specific approaches consistently outperform general-purpose solutions in enterprise settings. Douwe Kiela (Contextual AI) advocated explicitly for "specialization over AGI," arguing that commercial applications require focus on specific domains where AI can deliver measurable value. Jonathan Lowe demonstrated how Pfizer's GraphRAG approach incorporates domain-specific relationships in pharmaceutical data, outperforming standard approaches by understanding connections between entities like compounds, proteins, and diseases.

Multiple speakers emphasized that proprietary data and domain knowledge provide the true competitive advantage in an era of rapidly commoditizing foundation models. Grace Isford dropped one of the conference bangers when she noted that "foundation models are the fastest depreciating asset class on the market right now," suggesting that organizations should focus less on model selection and more on developing proprietary data advantages. John Crepezzi described how Jane Street built custom models for working with OCaml code because off-the-shelf models were insufficient for their specialized domain.

I don't fully buy the "data is your moat" claim, but there's something to be said for domain-specific applications crossing the threshold of sufficiency where general models still fall short. Kyle Corbitt shared how Method Financial discovered that fine-tuned 8B parameter models outperformed much larger models for their specific use case while dramatically reducing costs and latency. Bruno Passos explained how Booking.com's AI coding tools leverage the context of their codebase to generate more relevant solutions than general-purpose tools. The bitter lesson still applies – over time, bigger general models with more compute will likely win, but in the near term, specialized models deliver the reliability enterprises need.

The Trust Framework: Evaluation, Security, and Reliability

When did we all abandon basic machine learning principles? You can't even train a model without a test set, but somehow we collectively forgot that evals matter when it comes to generative AI, let a lone AI agents! A consistent theme throughout both conference tracks was the critical importance of robust evaluation frameworks, security measures, and reliability engineering for production AI systems.

Aparna Dhinkaran (Arize) presented a comprehensive framework for evaluating AI agents at multiple levels, emphasizing that "evals aren't just at one layer of your trace" and that rigorous testing across all components is essential for production-ready systems. Sayash Kapoor criticized over-reliance on static benchmarks, noting that impressive benchmark performance "very rarely translates into the real world." He advocated for multi-dimensional metrics that consider not just accuracy but also cost and reliability, showing how these metrics revealed that Claude 3.5 performed as well as GPT-4o on some tasks at 1/10th the cost.

Security concerns received significant attention, with Don Bosco Durai outlining a multi-layered approach to AI agent security addressing the unique vulnerabilities created by agents running in a single process. His framework consisted of pre-deployment evaluation, runtime enforcement, and continuous monitoring, addressing different aspects of security, safety, and compliance at each layer. Anju Kambadur explained that Bloomberg's agents undergo rigorous testing with "remediation workflows and circuit breakers" to catch errors before they impact financial data.

Most speakers agreed that while 100% accuracy is unattainable, organizations need robust observability, attribution, and audit trails to handle cases where things go wrong. Mike Conover described how BrightWave's research agent includes detailed citation tracking and "receipts" so users can validate information sources. Diamond Bishop explained how Datadog's AI agents generate postmortems and maintain full visibility into their decision-making processes, making them accountable even when operating autonomously.

Engineering for Intelligence: Building Systems That Scale with Smarter Models

A forward-looking theme emerged around designing systems that automatically improve as underlying models get better. Rahul Sengottuvelu (Ramp) presented a framework distinguishing "classical compute" from "fuzzy compute" (neural networks), arguing that systems should maximize the latter since "if you did nothing, absolutely nothing... the big labs are still working, spending billions of dollars making those models better." He demonstrated an experimental email client where the LLM itself acted as the backend, rendering UI and handling user interactions without traditional software engineering.

This is where you need to catch the wave. That wave is barely a swell right now – it's way out there. It's not worth chasing yet. You've got to wait till that wave starts to crest, and then you catch it. But when you do, you're riding the momentum of exponential improvement in foundation models.

Will Brown (Morgan Stanley) discussed how reinforcement learning is becoming crucial for agent development, describing how models like DeepSeek's R1 demonstrate that "the long chain of thought... actually emerges as a byproduct" of training with appropriate reinforcement signals. The results of reinforcement learning are astounding – it's a completely intelligible methodology, you can totally get it, and that it works is borderline miraculous!

John Crepezzi described Jane Street's "AID" (AI Developer) framework which provides a unified backend for multiple editor integrations, allowing them to swap in new models or context-building strategies without changing frontends. Similarly, Kevin Hou (Windsurf) explained how their agent architecture was designed to "scale with intelligence" so that "if the models get better, our product gets better," including removing chat interfaces in favor of pure agent interactions. These approaches collectively point toward a future where AI systems continuously improve through reinforcement learning rather than requiring constant human reengineering.

Voice and Multimodal Agents: The Next Frontier

Voice AI emerged as an important frontier for agent development across multiple presentations, but let's be real – the latency issues with frontier models totally kill the voice agent experience. They're going to be great when run at full power, full bandwidth and locally, but the practical applications of remotely deployed voice models are still struggling.

Nick Karyotakis (SuperDial) outlined the challenges of building reliable voice agents, noting that modern approaches have shifted from "prescriptive to descriptive development." He described how SuperDial builds phone agents that handle insurance verification calls by traversing phone trees, extracting information, and escalating to humans when needed.

Zack Reneau-Wedeen (Sierra) shared how Sierra's voice assistants handle customer service for brands by creating a "responsive design" approach where "it's the same agent code" operating across different channels and modalities. He emphasized that for voice agents, "latency all of a sudden matters so much more" than with text interfaces, creating new engineering challenges.

Multimodal capabilities also received significant attention. Mukund Sridhar and Aarush Selvan (Google) presented Gemini Deep Research as a multimodal research agent that "can browse the web as much as it needs" to produce comprehensive answers. Karina Nguyen (OpenAI) discussed how multimodal capabilities in Canvas enable more natural collaboration between humans and AI on creative tasks, suggesting that "the kind of interface to AGI is blank canvas that kind of self-morphs into your intent."

This is a great example of the wrong end of the spectrum to chase right now. Yes, it's the next frontier, but there are big hurdles that make it impractical. Don't waste time and effort here – the tech has to bridge the gap further before we get there. Focus where the gap is narrow and closest to sufficiency – that's where you should experiment.

Human-AI Partnership and Education

Despite the focus on automation, speakers consistently emphasized the importance of thoughtful human-AI collaboration. Colin Flaherty described how their AI coding agent has written over 90% of its own codebase, but with human supervision at critical junctures. Diamond Bishop explained that Datadog's AI agents help human engineers by running investigations automatically but present results in ways that build trust and facilitate learning.

Karina Nguyen outlined an evolution from "models trained on RL and chain of thought using real-world tools" toward "co-innovators" that collaborate with humans on creative tasks. Kevin Hou demonstrated how Windsurf tracks a "unified timeline" of both human and AI actions, allowing their agent to understand what developers are doing and continue work seamlessly.

The conference closed with perspectives on democratizing AI engineering. Stefania Druga presented Cognimates, a platform that teaches children about AI by allowing them to train their own models and build applications. She emphasized that "kids are actually like little scientists" who can formulate and test hypotheses about how AI works, and that early exposure helps "demystify the intelligence" of AI systems. Multiple speakers touched on making agent development more accessible, with Will Brown sharing a simple reinforcement learning implementation that went viral because "it was one file of code... really simple... and it invited modification."

Notable Quotables

Foundation models are the fastest depreciating asset class on the market right now.

— Grace Isford, Lux Capital

In reality, you're saying every hundred requests, 20 of them are just completely wrong.

— Waseem Alshikh, Writer

Don't build agents for everything. Keep it as simple for as long as possible. And finally, as you iterate, try to think like your agent, gain their perspective, and help them do their job.

— Barry Zhang, Anthropic

Pilots are very easy. Production is incredibly hard.

— Douwe Kiela, Contextual AI

We're seeing a lot of thunder, a lot of great momentum, but we haven't seen that lightning strike.

— Grace Isford, Lux Capital

Language models are already capable of very many things. But if you trick yourself into believing this means a reliable experience for the end user, that's when products in the real world go wrong.

— Sayash Kapoor, AI Snake Oil

Benchmark performance very rarely translates into the real world.

— Sayash Kapoor, AI Snake Oil

We want you to spend time on things that you are good at, right? The things that make us all excited, which is shipping products, building great features, and generally just shipping code.

— Kevin Hou, Windsurf (on delegating tedious engineering tasks to AI agents)

The bitter lesson is just so powerful, and exponential trends are so powerful that you can just hitch the ride.

— Rahul Sengottuvelu, Ramp

If the models get better, our product gets better.

— Kevin Hou, Windsurf

Winning systems will perform end-to-end RL over tool use calls.

— Mike Conover, BrightWave

Specialization over AGI.

— Douwe Kiela, Contextual AI

At enterprise scale, data is your moat.

— Douwe Kiela, Contextual AI

You need to get your human wetware chatbot speaking the right language at the right level.

— Jonathan Lowe, Pfizer

The way I'm thinking about it is the kind of interface to AGI is blank canvas that kind of self-morphs into your intent.

— Karina Nguyen, OpenAI

Gartner hates us. Gartner thinks we've hit the peak. So it's only downhill from here, guys. Sorry to inform you that AI engineering is over.

— Swyx, AI Engineer Foundation (joking about the Gartner hype cycle)

Emerging Questions

How do we design evaluation frameworks that measure agent performance in real-world conditions rather than controlled environments?
What is the optimal balance between model capabilities and engineering scaffolding in production agent systems?
How can organizations quantify the business value of AI initiatives to justify continued investment in implementation expertise?
What new security threats emerge when autonomous agents are given access to sensitive systems and data?
How will the economics of agent development evolve as compute costs decrease but expectations for reliability increase?
Will domain-specific agents continue to dominate, or will general-purpose agents eventually become viable for enterprise use?
What new user interfaces and interaction paradigms will emerge to support effective human-AI collaboration?
How should education systems evolve to prepare both current professionals and future generations for an agent-driven world?
What organizational structures best support AI implementation across enterprise silos?
How can voice and multimodal agents overcome the increased latency and reliability challenges compared to text-only systems?

Action Items

For Technical Leaders

Implement multi-dimensional evaluation frameworks that consider accuracy, cost, latency, and reliability
Design systems that improve automatically as models get better
Start with single-purpose agents before attempting complex multi-agent systems

For Business Leaders

Focus on domain-specific applications with clear ROI
Invest in proprietary data advantages
Align AI initiatives with core business objectives

For Security Teams

Deploy multi-layered frameworks addressing pre-deployment evaluation, runtime enforcement, and continuous monitoring
Develop specialized approaches for agent-specific vulnerabilities

For Product Managers

Design for human-AI collaboration rather than full autonomy
Prioritize UX that complements AI capabilities
Focus on specific high-value use cases rather than general-purpose solutions

For Organizations

Invest in data infrastructure, cross-functional communication, and educational initiatives
Develop appropriate governance frameworks for agents with increasing autonomy

For AI Engineers

Learn reinforcement learning techniques for agent development
Focus on building systems that scale with model improvements
Develop both technical and communication skills

For Educators

Integrate AI literacy into curricula from childhood
Develop tools that help students understand AI concepts through hands-on experience

Resources

Development Platforms and Tools

Hugging Face - GitHub for machine learning (mentioned by Isford)

Together AI - Open source AI cloud infrastructure (mentioned by Isford)

OpenPipe - Platform for building, training, and deploying fine-tuned open source models (presented by Corbitt)

Arize - AI observability platform for agent evaluation (presented by Dhinkaran)

WindSurf - Agentic editor with background understanding of development context (presented by Hou)

PipeCat - Open source voice AI orchestration framework by Daily (mentioned by Karyotakis)

LangFuse - Self-hostable observability platform for LLM applications (mentioned by Karyotakis)

DSPy - Framework for optimizing prompts through automation (mentioned by Brown)

Tensor Zero - Structured and typed LLM endpoints for production (mentioned by Karyotakis)

Cursor - Named the #1 AI tool that engineers love (from Frontier Feud)

Security & Governance

Page.ai (PAIG) - Privacera's open-sourced solution for AI safety and security (presented by Durai)

Apache Ranger - Open-source data governance project for big data (mentioned by Durai)

FailSafe - Writer's evaluation framework for testing models (presented by Alshikh)

Models and Research

DeepSeek-R1 - DeepSeek's open source reasoning model inspired by OpenAI's o1 (discussed by Brown)

Llama 3.1 - Meta's open source language model (mentioned by multiple speakers)

GRPO Algorithm - Simple RL algorithm for fine-tuning LLMs (presented by Brown)

"Introducing Contextual Retrieval" - Post by Anthropic (mentioned by Bricken)

"Towards Monosemanticity" and "Scaling Monosemanticity" - Anthropic interpretability papers (mentioned by Bricken)

Rubric Engineering - Will Brown's approach to designing reward functions for LLM fine-tuning

gr.inc - Ross Taylor's project addressing reasoning gaps in open models (mentioned by Soumith Chintala)

Enterprise Solutions

Bits AI - Datadog's AI assistant for DevOps (presented by Bishop)

Contextual AI - Specialized RAG agents for enterprise use cases (presented by Kiela)

Writer - Domain-specific LLMs for enterprises (presented by Alshikh)

BrightWave - Knowledge agent for due diligence and financial research (presented by Conover)

Sierra - Voice AI platform for customer service and conversation automation (presented by Reneau-Wedeen)

Augment Code - Building AI coding agents (presented by Flaherty)

Infrastructure & Implementation

Arista Networks - Specialized networking infrastructure for AI data centers (presented by Gilbert)

Ultra Ethernet Consortium - Next-generation ethernet standard for AI workloads (mentioned by Gilbert)

Model Context Protocol - Anthropic's open source protocol for language models to interact with data sources

Educational Resources

Cognimates - Platform for teaching children about AI through visual programming (presented by Druga)

AI Literacy - EU AI Act now requires AI literacy as part of regulation (mentioned by Druga)

Scratch - Visual programming environment for children with 100+ million users (mentioned by Druga)

Beacon - SignalFire's AI ML platform tracking 650M+ employees and 80M+ companies (presented by Black)

Using the BytesIO Class in Python

2025-02-14T00:00:00-06:00

The io.BytesIO class in Python is an in-memory stream for binary data. It provides a file-like interface that lets you read and write bytes just like you would with a file, but all the data is kept in memory rather than on disk. This can be extremely useful when you need a temporary buffer, when you're processing data that's generated on the fly, or when you want to simulate a file without touching the filesystem.

Key Concepts

1. Binary Data

Binary Data vs. Text Data: Binary data is any data that is not necessarily human-readable (e.g., images, audio files, compiled programs). It consists of bytes, whereas text data is typically encoded in formats like UTF-8.
Why Binary?: When dealing with non-textual content (like images or executables), you need to handle data at the byte level to preserve its exact structure.

2. Files in Binary Mode

Opening Files in Binary Mode: When reading or writing binary data to a file, you typically open the file in binary mode ('rb' for reading and 'wb' for writing). This ensures that no encoding/decoding happens automatically, preserving the raw bytes.

```python # Writing binary data to a file: with open('output.bin', 'wb') as f: f.write(b'\x00\x01\x02')

# Reading binary data from a file: with open('output.bin', 'rb') as f: data = f.read() print(data) # Output: b'\x00\x01\x02' ```

3. `io.BytesIO` in Practice

In-memory File-like Object: BytesIO acts like a file that exists in memory. This is particularly handy for testing, manipulating binary data without writing to disk, or when performance matters (reducing I/O overhead).
Interface Similarity: It supports many of the same methods as regular file objects (like .read(), .write(), .seek(), etc.).

Applied Examples

Example 1: Manipulating Image Data

Imagine you’re working with an image processing library that expects a file-like object, but your image is coming from a web request as bytes. You can wrap the bytes in a BytesIO to provide that interface:

import io
from PIL import Image  # Pillow library for image processing

# Simulated image bytes (normally you'd get this from a request)
image_bytes = b'...'  # Replace with actual image bytes

# Wrap the bytes in a BytesIO object
image_stream = io.BytesIO(image_bytes)

# Open the image using PIL, which expects a file-like object
image = Image.open(image_stream)
image.show()  # Display the image

Example 2: Temporary Data Buffer

If you need a temporary buffer to collect binary data before writing it to disk, BytesIO is ideal:

import io

# Create an in-memory binary stream
buffer = io.BytesIO()

# Write some binary data to it
buffer.write(b'Hello, ')
buffer.write(b'world!')

# Move to the beginning of the buffer to read the data
buffer.seek(0)
data = buffer.read()
print(data)  # Output: b'Hello, world!'

Example 3: Testing Without File I/O

When writing tests, you might want to avoid creating actual files. BytesIO allows you to simulate file operations entirely in memory:

import io

def process_binary_data(file_obj):
    # Example function that reads binary data and processes it
    data = file_obj.read()
    return data[::-1]  # Return reversed data

# Create a BytesIO object with some binary data
fake_file = io.BytesIO(b'abcdef')

# Pass the BytesIO object to your function
result = process_binary_data(fake_file)
print(result)  # Output: b'fedcba'

When to Use `io.BytesIO`

Unit Testing: Simulate file objects without creating real files.
Web Applications: Handle file uploads or downloads in memory.
Data Processing Pipelines: Process streams of binary data (e.g., for image manipulation, compression, encryption) without intermediate files.
Performance Sensitive Applications: Reduce I/O overhead by using memory-based buffers.

Summary

io.BytesIO is an in-memory binary stream that mimics file operations.
It's useful when you need a temporary file-like object for binary data.
Common use cases include image processing, temporary buffers, and testing.
Understanding binary data is crucial when working with non-text files to avoid encoding issues.

This flexibility makes io.BytesIO a powerful tool in Python for efficiently handling binary data without the need for disk I/O.

Linking to Posts in Python Pelican

2024-12-16T00:00:00-06:00

Pelican provides a syntax for linking to another post by its filename.

Pelican provides a special syntax to simplify linking between content files. Use |filename| to automatically resolve the correct URL. For example:

Link to another [TIL](|filename|__main__.py file in a project.md) on this

Pelican converts this into the correct URL during site generation. This is particularly useful for linking without worrying about the final directory structure.

Copying Files in Linux With a Progress Bar

2024-11-28T00:00:00-06:00

I’ve long used rsync for high fidelity, detailed copies of large swathes of files in Linux. For example:

rsync -av --progress /source/directory /destination/directory

This gives you really nice, file-by-file progress indications — which is especially useful when moving large files across the network between machines or even locally across physical devices.

Today I was moving data from old hard drives and just wanted overall progress on the total job to be done. rsync still FTW!

rsync -a --info=progress2 /source/directory /destination/directory/

NB — do not use a trailling / in the source directory if you want the copied files to be retained inside of that directory name in the destination!

This provides lovely summary output like this:

ubuntu@ubuntu:/media/ubuntu$ rsync -a --info=progress2 /source/directory /destination/directory
  2,675,525,167  71%   13.49MB/s    0:03:09 (xfr#1281, to-chk=0/1712)

A Better, More Robust Way to Do This

Running this command gets some extra good stuffs.

rsync -a --info=progress2 --checksum --partial /source/directory /destination/directory/

Explanation of the Additions:

-checksum:
- Ensures that files are compared using checksums rather than just timestamps and file sizes.
- Guarantees that files are transferred only if their content differs, even if metadata like modification times are identical.
-partial:
- Retains partially transferred files if the transfer is interrupted, allowing resumption without starting over.

Why An 8TB Drive Isn’t 8 Usable TB

2024-11-28T00:00:00-06:00

I bought an 8TB external SSD to use as a live boot Linux device. I knew that this issue was due to the way that bits are aggregated into terabytes, but — thanks to our mate ChatGPT — here’s a nice explanation of why my 8TB SSD only shows 7,452.02 GB (7.452TB) of usable space.

The discrepancy between the advertised capacity of your SSD (8 TB) and the actual usable capacity (7,452.02 GB) is due to differences in how storage manufacturers and operating systems calculate disk space. Here’s a detailed explanation:

1. Manufacturer vs. Operating System Measurements

Manufacturer Calculation (Decimal - Base 10):
- Storage manufacturers use the decimal system, where:
  - 1 TB = 1,000,000,000,000 bytes.
  - 8 TB = 8,000,000,000,000 bytes.
Operating System Calculation (Binary - Base 2):
- Operating systems (like Windows, Linux, macOS) use the binary system, where:
  - 1 GiB = 1,073,741,824 bytes.
  - 1 TiB = 1,099,511,627,776 bytes.

When your OS displays capacity:

8,000,000,000,000 bytes ÷ 1,073,741,824 bytes per GiB = ~7,452 GiB.
Similarly:
- 8,000,000,000,000 bytes ÷ 1,099,511,627,776 bytes per TiB = ~7.28 TiB.

2. Why This Happens

Marketing: Manufacturers prefer the larger-looking decimal numbers.
Reality Check: Computers deal in binary, so the OS displays the actual capacity based on base-2 calculations.

3. Example of Conversion

If you calculate the conversion:

Advertised (Decimal): 8 TB = 8,000,000,000,000 bytes.
Actual (Binary): 8,000,000,000,000 bytes ÷ 1,073,741,824 (GiB) = ~7,452 GiB.

4. Formatting Overhead

Additionally, after formatting the drive:

Some space is used for file system structures and metadata (e.g., NTFS, exFAT, ext4), reducing the usable capacity further.

5. Practical Impact

The actual capacity you see (7,452.02 GB) is correct and normal for an 8 TB SSD.
You haven’t lost any capacity—it’s just how the numbers are calculated and represented.

Wrap with HTML tags in VSCode

2024-11-14T00:00:00-06:00

Using the Emmet capabilities in VSCode (see previous TIL) you can highlight a section of HTML code and quickly wrap it with other HTML using Emmet shortcuts.

This is particularly useful when writing plain HTML and then returning to style it.

You can access this functionality in the command palette

Smarter, still, is to create a keyboard shortcut for this. We’ll bind ALT+M to this functionality.

Now, simply highlight the HTML code you want to wrap, hit ALT+M and start typing the Emmet abbreviations for the tags you want to wrap the code with. For example, we’ll wrap this simple HTML form with some BulmaCSS classes to improve layout …

et voila!

main.py file in a project

2024-10-15T00:00:00-05:00

This is, to a Python project / module, what if __name__ == '__main__ is to an individual Python file.

In Python projects, a __main__.py file is used to define what happens when a package or module is run directly as a script. It allows you to make the package executable from the command line, similar to how you would run a single Python script.

Here’s how it works:

Executed when the package is run as a script: When you run a package like this:

python -m package_name

The __main__.py file inside the package is executed, just like a script. This is similar to how a main() function works in other programming languages.

Entry point for a package: It serves as an entry point to the package’s functionality. You can add logic here that should be executed when the package is run.
Example Structure:

my_package/
├── __init__.py
├── __main__.py
├── module1.py
└── module2.py

If you run:

python -m my_package

Python will execute the __main__.py file in my_package.

Typical Use Case: It’s often used to bootstrap or run the application, e.g., starting a server, running an app, or triggering some core functionality when the package is invoked from the command line.

For example, in a __main__.py file, you might have:

def main():
    print("Running the application!")

if __name__ == "__main__":
    main()

This ensures that the package can be run both as a standalone script or imported as a module without accidentally triggering the main execution logic.

A nice workflow for creating Python projects with AI

2024-10-15T00:00:00-05:00

First, create a new project using poetry

poetry new asciinema-editor

This gives you a nice, clean new project structure.

.
├── README.md
├── asciinema_editor
│   ├── __init__.py
├── poetry.lock
├── pyproject.toml
└── tests
    └── __init__.py

Now, let the model know that you have this project structure and ask it to write you code. Claude does a nice job of providing the relevant bits to make your project work nicely.

e.g. __main__.py (see TIL on this)
Exposes the relevant main function in __init__.py making it easier to import and use the package in other scripts if needed.

neofetch for Linux system info summary

2024-10-15T00:00:00-05:00

neofetch provides a nice snapshot of system information. This is what you were wanting to add to the --sysinfo output for gpt-engineer.

❯ neofetch
.-/+oossssoo+/-.               captivus@brisbane
`:+ssssssssssssssssss+:`           -----------------
-+ssssssssssssssssssyyssss+-         OS: Ubuntu 22.04.5 LTS on Windows 10 x86_64
.ossssssssssssssssssdMMMNysssso.       Kernel: 5.15.153.1-microsoft-standard-WSL2
/ssssssssssshdmmNNmmyNMMMMhssssss/      Uptime: 2 days, 20 hours, 58 mins
+ssssssssshmydMMMMMMMNddddyssssssss+     Packages: 961 (dpkg), 10 (snap)
/sssssssshNMMMyhhyyyyhmNMMMNhssssssss/    Shell: zsh 5.8.1
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Theme: Adwaita [GTK3]
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   Icons: Adwaita [GTK3]
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   Terminal: Windows Terminal
ossyNMMMNyMMhsssssssssssssshmmmhssssssso   CPU: 13th Gen Intel i7-13700F (24) @ 2.111GHz
+sssshhhyNMMNyssssssssssssyNMMMysssssss+   GPU: d474:00:00.0 Microsoft Corporation Device 008e
.ssssssssdMMMNhsssssssssshNMMMdssssssss.   Memory: 3098MiB / 15921MiB
/sssssssshNMMMyhhyyyyhdNMMMNhssssssss/
+sssssssssdmydMMMMMMMMddddyssssssss+
/ssssssssssshdmNNNNmyNMMMMhssssss/
.ossssssssssssssssssdMMMNysssso.
-+sssssssssssssssssyyyssss+-
`:+ssssssssssssssssss+:`
.-/+oossssoo+/-.

Emmet Abbreviations in VSCode

2024-10-10T00:00:00-05:00

Apparently there’s this plugin for many IDEs called Emmet which makes writing HTML very fast!

Here’s the Emmet Cheat Sheet

Here’s a great video on it:

https://www.youtube.com/watch?v=V8vizNQKtx0

Start all HTML projects by just typing !:

Then, hit ENTER (not Tab) to complete to this boilerplate!

Start by typing the name of the tag you want … then hit ENTER (not Tab) to complete.

e.g. div (ignore Cursor’s autocomplete suggestions behind it)

Then, ENTER:

That works with any HTML element. It gets even better, though. You can quickly generate CSS and other HTML attributes.

For example, a <span> with the CSS class purple:

ENTER completes it to:

Similarly, adding an id= attribute to an element:

ENTER completes to:

You can complete all of the attributes you want for an element by chaining them as so:

ENTER completes to:

You can write these things the way that CSS Selectors are written.

So, a <div> with a class of purple and a child <span> with class cyan is:

ENTER completes to:

You can nest all of your elements similarly:

Becomes:

If you wanted 3 <li> elements as children of the <ul>, you can use * multiplication:

Becomes:

Want to add text to each of those <li> elements? Use the { } operators:

Becomes:

Add numbers to those list items:

Becomes:

The $ operator can be used in any other attribute also. e.g. .class$ becomes .class-1, .class-2, etc …

Add additional $ to zero-pad the numbers.

Becomes:

Add sibling elements using the + operator.

Becomes:

Group elements together to implement more complex, yet readable, structures. For example, if we wanted a <header> with a <nav> child, then the <main> and <footer> siblings as before, we can group with the ( ) operators.

Becomes:

Let’s combo it up! We want:

Header
- H2, with text
- Nav
  - Ordered List
  - 5 list elements, with links (anchor tags)
Main
Footer

To accomplish this, we write:

Which becomes:

Building Forms using Emmet

Installing Racket (Scheme) in WSL

2024-10-06T00:00:00-05:00

We’ll install Racket from apt as follows, as the snap version ended up with many hassles

Note to self — don’t install Racket from the Snap Store!

**sudo apt update
sudo apt install racket**

Then:

❯ racket --version
Welcome to Racket v8.2 [cs].

We’re going to use VSCode (obviously) so we’ll install the Magic Racket extension (which is the best of the various extensions). Follow the instructions provided by the extension:

NB, this is, specifically:

raco pkg install racket-langserver

Respond “y” when prompted to install dependencies …

❯ raco pkg install racket-langserver
Resolving "racket-langserver" via https://download.racket-lang.org/releases/8.2/catalog/
Resolving "racket-langserver" via https://pkgs.racket-lang.org
Downloading repository https://github.com/jeapostrophe/racket-langserver
The following uninstalled packages are listed as dependencies of racket-langserver:
   html-parsing
   chk-lib
Would you like to install these dependencies? [Y/n/a/c/?] y
00: Resolving "html-parsing" via https://download.racket-lang.org/releases/8.2/catalog/
00: Resolving "html-parsing" via https://pkgs.racket-lang.org
Resolving "chk-lib" via https://download.racket-lang.org/releases/8.2/catalog/
Resolving "chk-lib" via https://pkgs.racket-lang.org
Downloading https://www.neilvandyke.org/racket/html-parsing.zip
Downloading repository git://github.com/jeapostrophe/chk?path=chk-lib
The following uninstalled packages are listed as dependencies of html-parsing:
   mcfly
   overeasy
Would you like to install these dependencies? [Y/n/a/c/?] y
00: Resolving "mcfly" via https://download.racket-lang.org/releases/8.2/catalog/
00: Resolving "mcfly" via https://pkgs.racket-lang.org
Resolving "overeasy" via https://download.racket-lang.org/releases/8.2/catalog/
Resolving "overeasy" via https://pkgs.racket-lang.org
Downloading https://www.neilvandyke.org/racket/mcfly.zip
Downloading https://www.neilvandyke.org/racket/overeasy.zip
The following uninstalled packages were listed as dependencies
and they were installed:
 dependencies of racket-langserver:
   html-parsing
   chk-lib
 dependencies of html-parsing:
   mcfly
   overeasy
raco setup: version: 8.2
raco setup: platform: x86_64-linux [cs]
raco setup: target machine: ta6le
raco setup: installation name: 8.2
raco setup: variants: cs
raco setup: main collects: /usr/share/racket/collects/
raco setup: collects paths:
raco setup:   /home/captivus/.local/share/racket/8.2/collects
raco setup:   /usr/share/racket/collects/
raco setup: main pkgs: /usr/share/racket/pkgs
raco setup: pkgs paths:
raco setup:   /usr/share/racket/pkgs
raco setup:   /home/captivus/.local/share/racket/8.2/pkgs
raco setup: links files:
raco setup:   /usr/share/racket/links.rktd
raco setup:   /home/captivus/.local/share/racket/8.2/links.rktd
raco setup: compiled-file roots:
raco setup:   same
raco setup:   /usr/lib/racket/compiled
raco setup: main docs: /usr/share/doc/racket
raco setup: --- updating info-domain tables ---                    [20:14:51]
raco setup: updating: /home/captivus/.local/share/racket/8.2/share/info-cache.rktd
raco setup: --- pre-installing collections ---                     [20:14:51]
raco setup: --- installing foreign libraries ---                   [20:14:51]
raco setup: --- installing shared files ---                        [20:14:51]
raco setup: --- compiling collections ---                          [20:14:51]
raco setup: --- parallel build using 8 jobs ---                    [20:14:51]
raco setup: 7 making: <pkgs>/chk-lib/chk
raco setup: 6 making: <pkgs>/html-parsing (html-parsing)
raco setup: 5 making: <pkgs>/mcfly (McFly Runtime)
raco setup: 4 making: <pkgs>/overeasy (Overeasy)
raco setup: 3 making: <pkgs>/racket-langserver
raco setup: 3 making: <pkgs>/racket-langserver/scribblings
raco setup: 3 making: <pkgs>/racket-langserver/tests
raco setup: 3 making: <pkgs>/racket-langserver/tests/lifecycle
raco setup: 3 making: <pkgs>/racket-langserver/tests/sync
raco setup: 3 making: <pkgs>/racket-langserver/tests/textDocument
raco setup: 3 making: <pkgs>/racket-langserver/tests/textDocument/code-action
raco setup: 3 making: <pkgs>/racket-langserver/tests/textDocument/completion
raco setup: 3 making: <pkgs>/racket-langserver/tests/textDocument/find-symbol
raco setup: 3 making: <pkgs>/racket-langserver/tests/textDocument/rename
raco setup: --- creating launchers ---                             [20:14:56]
raco setup: --- installing man pages ---                           [20:14:56]
raco setup: --- building documentation ---                         [20:14:56]
raco setup: 7 running: <pkgs>/html-parsing/html-parsing.scrbl
raco setup: 5 running: <pkgs>/racket-index/scribblings/main/user/local-redirect.scrbl
raco setup: 1 running: <pkgs>/mcfly/mcfly.scrbl
raco setup: 6 running: <pkgs>/overeasy/overeasy.scrbl
raco setup: 3 running: <pkgs>/racket-langserver/scribblings/racket-langserver.scrbl
raco setup: 2 running: <pkgs>/racket-index/scribblings/main/user/release.scrbl
raco setup: 0 running: <pkgs>/racket-index/scribblings/main/user/search.scrbl
raco setup: 4 running: <pkgs>/racket-index/scribblings/main/user/start.scrbl
raco setup: 7 rendering: <pkgs>/html-parsing/html-parsing.scrbl
raco setup: 6 rendering: <pkgs>/racket-index/scribblings/main/user/local-redirect.scrbl
raco setup: 5 rendering: <pkgs>/mcfly/mcfly.scrbl
raco setup: 4 rendering: <pkgs>/overeasy/overeasy.scrbl
raco setup: 3 rendering: <pkgs>/racket-langserver/scribblings/racket-langserver.scrbl
raco setup: 2 rendering: <pkgs>/racket-index/scribblings/main/user/release.scrbl
raco setup: 1 rendering: <pkgs>/racket-index/scribblings/main/user/search.scrbl
raco setup: 0 rendering: <pkgs>/racket-index/scribblings/main/user/start.scrbl
raco setup: --- installing collections ---                         [20:15:02]
raco setup: --- post-installing collections ---                    [20:15:02]

Notes on running this in WSL

raco: Racket's Package Management System (and More)

2024-10-06T00:00:00-05:00

In the Racket programming language, raco is a command-line tool used to interact with Racket's package management system, run tests, build documentation, and perform other tasks related to Racket development. It's a versatile utility that automates various tasks in Racket projects.

Some of the common uses of raco include:

Package Management: You can install, remove, or update Racket packages using commands like:

raco pkg install <package-name>
raco pkg remove <package-name>
raco pkg update <package-name>

Compiling Racket Programs: raco can compile Racket files to bytecode or even to an executable:

raco make <file.rkt>

Running Tests: Racket includes support for automated tests, and you can run tests using raco test:

raco test <file.rkt>

Building Documentation: Racket supports generating documentation, and raco can be used to build the documentation for a package:

raco docs <package-name>

Creating Executables: You can use raco exe to create standalone executables from Racket code:

raco exe <file.rkt>

raco simplifies project management by providing a unified interface for all these tasks, making it essential for anyone working on Racket projects.

Working through SICP in Jupyter Notebooks

2024-10-06T00:00:00-05:00

I’m reading Structure and Interpretation of Computer Programs and want to write Racket in VSCode in Jupyter notebooks in a dialect of Scheme closest to the book. This requires this declaration at the top of each Racket file:

#lang r5rs

HOWEVER, Jupyter doesn’t like this …

The Jupyter kernel for Racket doesn't directly support changing the language with #lang declarations in individual cells. Claude to the rescue, though … eventually … what a pain to finally get to this!

We need to find our where iracket is installed.

❯ find ~/.local -name iracket.rkt
/home/captivus/.local/share/racket/8.2/pkgs/iracket/iracket.rkt

Now, we edit this file (home/captivus/.local/share/racket/8.2/pkgs/iracket/iracket.rkt) as follows:

#lang racket/base

;; ============================================================
;; Enabling R5RS mode for compatibility with SICP code
(require racket/sandbox)

(define orig-current-namespace (current-namespace))
(define r5rs-namespace (make-base-namespace))
(parameterize ([current-namespace r5rs-namespace])
  (namespace-require 'r5rs))

(define (switch-to-r5rs)
  (current-namespace r5rs-namespace))

(define (switch-to-orig)
  (current-namespace orig-current-namespace))

(switch-to-r5rs)

;; The rest of the original iracket.rkt content should follow here ...
;; ============================================================

Now, restart your VSCode and create a Jupyter notebook as previously described in your TIL article … and you’re up!

Writing Racket (Scheme) in Jupyter Notebooks

2024-10-06T00:00:00-05:00

First, we’re going to install Jupyter. As we don’t want to contaminate our global Python installation, we’ll do this in a venv

python -m venv venv

Now, let’s activate that bad boy:

source ./venv/bin/activate

Assuming you have Racket installed (see previous TIL — NB do not install via the Snap store … use apt!), install the Racket kernel for Jupyter:

raco pkg install iracket

Should your system complain about not having libzmq5 installed:

sudo apt install libzmq5

Then, install the Racket kernel into Jupyter:

raco iracket install

Then, confirm that Jupyter sees the Racket kernel:

❯ jupyter kernelspec list
Available kernels:
  python3    /home/captivus/projects/sicp/venv/share/jupyter/kernels/python3
  racket     /home/captivus/.local/share/jupyter/kernels/racket

Now … we create a new Jupyter notebook in VSCode using the Command Palette:

In the top right of the notebook, select the kernel (it won’t say Racket when you start):

You’ll be presented with these options:

Select “Jupyter Kernel”, then select “Racket”:

… and you’re up!

Debugging Python CLI Arguments in VSCode

2024-09-15T00:00:00-05:00

Command Palette (CTRL+SHIFT+P): Debug: Add Configuration

Then, select “Python Debugger”

Then, select “Python File with Arguments”

VSCode then creates the .vscode directory in your project, if it doesn’t exist already, and adds launch.json as follows:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://code.visualstudio.com/docs/debugtest/debugging
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Python Debugger: Current File with Arguments",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "args": "${command:pickArgs}"
        }
    ]
}

If you don’t change anything, when you debug a file VSCode will prompt you for arguments to pass the file every time you run the debugger.

Can also click, bottom right, “Add Configuration” which will stub out adding more configs in the JSON so that you can tailor it to your specific application. For example:

{
    // Use IntelliSense to learn about possible attributes.
    // Hover to view descriptions of existing attributes.
    // For more information, visit: https://code.visualstudio.com/docs/debugtest/debugging
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Blue Horseshoe Debugger",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "args": ["run", 
            "--players", "5", 
            "--duration", "2", 
            "--round-end", "fixed", 
            "--config", "strategy_config.yaml", 
            "--simulation", "true", 
            "--strategies", "conservative,conservative,conservative,momentum,arbitrage"]
        },
        {
            "name": "Python Debugger: Current File with Arguments",
            "type": "debugpy",
            "request": "launch",
            "program": "${file}",
            "console": "integratedTerminal",
            "args": "${command:pickArgs}"
        }
    ]
}

Debugging Python Tests in VSCode

2024-09-15T00:00:00-05:00

VSCode offers native test capabilities that are quite useful. Testing Python in Visual Studio Code

It doesn’t, however, recognize your Python tests by default. You must configure it to do so for each project.

Command Palette (CTRL+SHIFT+P) → “Python: Configure Tests” →

Select the framework you’re using:

Now, you get all of the testy-goodness that VSCode has to offer for Python.

In this case, I wanted to debug a specific test. Super easy! Just drop breakpoints in the test where you want them, then run “Debug Test”.

The “Test Explorer” is also quite useful.

Suppressing Rich Tracebacks in Typer Apps

2024-09-11T00:00:00-05:00

# suppress rich traceback
app = typer.Typer(pretty_exceptions_show_locals=False)

This must be applied to the configuration of the Typer object created for the CLI.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Create a futuristic, abstract digital control panel featuring holographic panels and cascading streams of Python code, including a glimpse of the code snippet "app = typer.Typer(pretty_exceptions_show_locals=False)". The scene should evoke the concept of suppressing rich tracebacks, with flowing neon light trails, glitch effects, and a background filled with dark urban circuitry and digital grids, representing the seamless configuration of a high-tech CLI environment.

Uninstalling Poetry Completions for zsh

2024-08-31T00:00:00-05:00

Here’s how to enable: https://python-poetry.org/docs/#enable-tab-completion-for-bash-fish-or-zsh

Uninstalling a specific completion works as follows:

Navigate to `~/.zfunc

❯ cd .zfunc
❯ ll
Permissions Size User     Date Modified Name
.rw-r--r--   243 captivus 31 Aug 10:28  _convert-mp4-to-wav
.rw-r--r--   13k captivus 10 Oct  2023  _poetry
.rw-r--r--   273 captivus 28 Aug 14:20  _stccc-directory-scraper

To uninstall the Zsh completions for _convert-mp4-to-wav, follow these steps:

Remove the Completion File:

Delete the _convert-mp4-to-wav file from your ~/.zfunc directory:

rm ~/.zfunc/_convert-mp4-to-wav

Update the Zsh Completion System:

After removing the file, you should update the Zsh completion system by running the following command:

compinit

Reload Your Zsh Configuration:

Finally, reload your Zsh configuration to ensure that the changes take effect:

source ~/.zshrc

This will remove the completion for _convert-mp4-to-wav from your Zsh setup.

Optionally, you can leave #2 and #3 for later, as the shell should reinitialize compinit when restarted.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a sleek, futuristic terminal screen floating in a dark, digital space. The terminal features luminous commands and code snippets like "cd ~/.zfunc", "rm ~/.zfunc/_convert-mp4-to-wav", and "source ~/.zshrc" in vibrant neon text. Surround the screen with abstract, holographic gridlines and circuit-like patterns, evoking the essence of a high-tech hacking interface. The overall vibe is minimalist yet distinctly technical, capturing the spirit of a cyberpunk command line environment without any human presence.

Ubuntu Snaps in WSL & zsh

2024-08-26T00:00:00-05:00

What on earth are “Snaps”?

These are fully containerized versions of applications that can be easily “installed” from the Snap Store. This store is run by canonical, but it looks like they support other distributions of Linux.

From the link above:

Introduction to snaps

Snaps are a secure and scalable way to embed applications on Linux devices. A snap is an application containerised with all its dependencies. A snap can be installed using a single command on any device running Linux. With snaps, software updates are automatic and resilient. Applications run fully isolated in their own sandbox, thus minimising security risks.

Snaps are hosted in the global Snap Store, an application repository hosted and managed by Canonical, and are free for anyone to download. Snaps can be created by anyone - existing software can be packaged as a snap or new software can be built from scratch using snap packaging. There is also an active, vibrant community of developers and users, with a forum where anyone can ask questions.

Packaging IoT applications as snaps bring the following benefits:


Reliable	Snaps use transactional updates, meaning that if for any reason an update you push to your snap fails, the snap will roll back to its last stable state
Modular	Snaps are reusable, they enable a loosely-coupled software architecture for embedded software and are compatible across architectures
Robust	With snaps, software updates are automatic and over-the-air (OTA), meaning your software is never out-of-date
Optimised	Snaps harness delta updates, minimising the storage and bandwidth needed when updating software. Read more about differential updates here

I wanted to install the great DBeaver database tool, but it isn’t in the mainline apt repos (and I don’t trust third-party apt repos). It is available as a Snap, though.

sudo snap install dbeaver-ce

Now, though … how to run it, as it wasn’t found in my $PATH.

export PATH=$PATH:/snap/bin

That did the trick! I added the above to my .zshrc file also, for persistence.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Picture an ultra-modern, futuristic digital landscape that symbolizes Ubuntu Snaps and WSL integration. Feature a sleek, neon-lit Linux terminal screen with abstract container icons and flowing circuit patterns, interlaced with holographic overlays representing secure and isolated application environments. In the backdrop, depict a sprawling cyberpunk cityscape of glowing data streams and network grids that evoke the modular, robust, and automated world of snap updates—all rendered in vivid neon shades of pink, blue, and purple.

Running Google Chrome GUI in WSL

2024-08-25T00:00:00-05:00

This was working for me a while ago but today, when I tried, it was not.

❯ google-chrome
[541540:541540:0825/114954.251188:ERROR:ozone_platform_x11.cc(244)] Missing X server or $DISPLAY
[541540:541540:0825/114954.251464:ERROR:env.cc(258)] The platform failed to initialize.  Exiting.

There’s a good wiki on this available here.

The solution was simple and immediate:

export DISPLAY=:0

Importantly, this doesn’t carry through to a Jupyter notebook. To make this work in Jupyter notebooks, you need to use the %env magic command.

%env DISPLAY=:0

Let me re-read your post carefully to match your voice.Your style is direct and practical — problem statement, error output, brief context, solution, done. No fluff. You show rather than explain, and you trust the reader to follow along. Here's an update that matches:

Update: December 2025

Different failure mode this time. Chrome launched but no window appeared.

❯ google-chrome
[3365071:3365071:1205/114205.717359:ERROR:dbus/object_proxy.cc:573] Failed to call method: org.freedesktop.DBus.Properties.GetAll: object_path= /org/freedesktop/UPower/devices/DisplayDevice: org.freedesktop.DBus.Error.ServiceUnknown: The name org.freedesktop.UPower was not provided by any .service files
Created TensorFlow Lite XNNPACK delegate for CPU.

No X11 error. $DISPLAY was set correctly. But xeyes also ran without showing a window. The process was running, just invisible.

The wiki says /tmp/.X11-unix should be a symlink to /mnt/wslg/.X11-unix. Mine was a directory:

❯ ls -la /tmp/ | grep X11
drwxrwxrwx    - root     24 Nov 12:12 .X11-unix

Tried to remove it:

❯ sudo rm -r /tmp/.X11-unix
rm: cannot remove '/tmp/.X11-unix/X0': Read-only file system

A read-only tmpfs was mounted on top:

❯ mount | grep X11
none on /tmp/.X11-unix type tmpfs (ro,relatime)

The culprit is a race condition between two systemd mechanisms. The xserver-common package includes /usr/lib/tmpfiles.d/x11.conf:

D! /tmp/.X11-unix 1777 root root 10d

This creates /tmp/.X11-unix as a directory during systemd-tmpfiles-setup.service. WSLg's wslg.service runs after and tries to bind-mount over it, resulting in the broken read-only state. Apps connect to what looks like an X socket but renders nowhere.

The fix is to override the tmpfiles rule so it creates a symlink instead:

echo 'L+ /tmp/.X11-unix - - - - /mnt/wslg/.X11-unix' | sudo tee /etc/tmpfiles.d/wslg-x11.conf

Then fix the current state:

sudo umount /tmp/.X11-unix
sudo rm -r /tmp/.X11-unix
ln -s /mnt/wslg/.X11-unix /tmp/.X11-unix

This survives wsl --shutdown. The L+ directive removes whatever exists and creates a symlink, so tmpfiles and WSLg stop fighting.

Preserving the above-linked wiki for posterity below

Diagnosing "cannot open display" type issues with WSLg

Steve Pronovost edited this page on May 7, 2021 · 3 revisions

On category of issues that we have seen popping up is folks having trouble getting their GUI application to properly connect to WSLg's X server. This page is meant as a quick guide to diagnose this type of connection issue as well as list the currently known problem we're working on fixing.

Verify you are running on Windows build 21364+

From a Windows command prompt, type ver to verify which build you are running.

E:\wsl>ver

Microsoft Windows [Version 10.0.21367.1000]

You must be running on Windows build version 21364+ for WSLg to work. This version of Windows is currently only available through the Windows Insider program. See https://www.microsoft.com/en-us/windowsinsider/ to join the insider program and help us validate pre-released version of Windows.

DISPLAY environment variable

WSLg's X server is running on display 0. The DISPLAY environment variable must have the value :0 for GUI application to connect to the right display. You can verify what the value of your DISPLAY environment variable is per below.

spronovo@OFFICE:~$ echo $DISPLAY
:0

This environment variable is initialize as part of WSL's INIT. If it is unset or has a value other than :0, than you likely have a profile script that is changing it's value that you'll want to hunt down. You can also reset that environment variable like below.

export DISPLAY=:0

X11 display socket

X servers create their socket under /tmp/.X11-Unix. This directory must exist and must be linked to /mnt/wslg/.X11-Unix where WSLg built-in X server create it's socket. You can verify the mapping exist and is the expected link per below.

spronovo@OFFICE:~$ ls -la /tmp/.X11-unix
lrwxrwxrwx 1 spronovo spronovo 19 Apr 21 15:28 /tmp/.X11-unix -> /mnt/wslg/.X11-unix

This link is setup during WSL's INIT. If this directory doesn't exist, something likely caused it be removed in your environment that needs to be tracked down.

You can re-create the link manually to try things out.

sudo rm -r /tmp/.X11-unix
ln -s /mnt/wslg/.X11-unix /tmp/.X11-unix

X11 server running?

If the X server is running, you should see an X0 socket

spronovo@OFFICE:~$ ls /tmp/.X11-unix
X0

If you don't please open an issue and attach /mnt/wslg/weston.log to the bug.

Known issues

You can verify the version of WSLg you are running per below:

spronovo@OFFICE:~$ cat /mnt/wslg/versions.txt
WSLg ( x86_64 ): 1.0.17+3.Branch.master.Sha.a526dfd5ad03d126bb2d8c528f6c3563e86a40da
Mariner: VERSION="1.0.20210224"
FreeRDP: e4a2fc2053bd8c5f99455fcd08ffee7e5591567a
weston: fd961f5cd116c9358d82ce94d139c1578e21bd00
pulseaudio: 2f0f0b8c3872780f15e275fc12899f4564f01bd5
mesa:

Complex monitor arrangement (Fixed in WSLg 1.0.19)

There is a known issue in WSLg 1.0.17 that if you have a combination of vertically and horizontally aligned monitor, Weston may hit an invalid assert and restart. Effectively crashing and restarting the X server on every connection attempt.

You can verify if this is what you are hitting per below

cat /mnt/wslg/weston.log | grep isConnected_V

if you see something like

weston: ../libweston/backend-rdp/rdpdisp.c:481: disp_monitor_validate_and_compute_layout: Assertion `isConnected_V == true' failed.

Then you are hitting this problem. The workaround at the moment is to stack all of your monitor either vertically, or horizontally, but not use a mix of both.

Setting /tmp in /etc/fstab

There is a known issue at the moment (https://github.com/microsoft/wslg/issues/43) where configuring /tmp in /etc/fstab will overwrite the /tmp/.X11-unix link previously described. The workaround at the moment is to either avoid configuring /tmp, or manually recreating the link

ln -s /mnt/wslg/.X11-unix /tmp/.X11-unix

Still having a problem?

Please open an issue and include the following

Run the following command and provide the output:

spronovo@OFFICE:~$ cat /mnt/wslg/versions.txt
WSLg ( x86_64 ): <current>
Mariner: VERSION="1.0.20210224"
FreeRDP: 5f083fa0b97d433d6204985f6047886e29c1c61e
weston: 16de531f00aa3dfd17e0de74c8f49e9fd7cec617
pulseaudio: 2f0f0b8c3872780f15e275fc12899f4564f01bd5
mesa: 2ad0684038f5732f7e4bd1a391ec9d833685fb48

spronovo@OFFICE:~$ echo $DISPLAY
:0

spronovo@OFFICE:~$ ls -la /tmp/.X11-unix
lrwxrwxrwx 1 root root 19 Apr 21 12:12 /tmp/.X11-unix -> /mnt/wslg/.X11-unix

spronovo@OFFICE:~$ ls -la /tmp/.X11-unix/
total 0
drwxrwxrwx 2 root     root   60 Apr 21 12:22 .
drwxrwxrwt 5 root     root  220 Apr 21 12:22 ..
srwxrwxrwx 1 spronovo users   0 Apr 21 12:22 X0

Attach your /mnt/wslg/weston.log file

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a futuristic, shadowy workstation floating in a dark, neon-drenched room. A translucent, holographic Linux terminal displays cryptic code and error messages in glowing neon, casting a blue and purple light across sleek chrome interfaces and digital circuitry that pulses like living art. Abstract, geometric symbols of connectivity and digital mystery intertwine with glitch-effect overlays, evoking the enigmatic energy of a high-tech realm where advanced WSL environments and futuristic software merge into one surreal, incandescent scene.

Comments are Failures in Coding

2024-08-24T00:00:00-05:00

“The proper use of comments is to compensate for our failure to express ourselves in code.” — Robert C. Martin, Clean Code

Every time you express yourself in code, you should pat yourself on the back.

Every time you write a comment, you should grimace and feel the failure of your ability of expression.

Find a way to express your intent in the code, directly. The only truly good comment, is the comment you found a way not to write.

Truth can only be found in one place — the code.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Incorporate a futuristic, digital landscape filled with glowing code snippets, circuit board patterns, and abstract text fragments resembling failed comments in programming. Let the environment evoke the feeling of a cityscape made of luminous data streams and neon-lit architecture, reflecting the idea that true expression lies in the pure essence of code rather than in the annotations.

Launching a Web App from Stream Deck

2024-08-24T00:00:00-05:00

This is actually much more of a hassle than I’d have expected …

The trick, in my case, is that I have a virtual desktop called Media and, on it, an always open Microsoft Edge browser window also called Media in which I keep the Brain.fm web player open. I want to be able to press a button on the Stream Deck and play or pause Brain.fm from anywhere.

I ended up writing a PowerShell script to accomplish this, using Claude to help. Claude did a relatively terrible job prompting him regularly. The way I ultimately got there quickly was to borrow from Manuel Odendahl’s brilliant suggestions (video and PDF handout) of having the model create a Domain Specific Language (DSL) for the tasks at hand, and then implement specific parts of the DSL.

The resulting utility is available on GitHub here. This was, at first, a hassle but then really effing cool to see how much better the model does with the DSL! h/t Manuel!

<#
.SYNOPSIS
Automates playing Brain.fm in Microsoft Edge on a specific virtual desktop.

.DESCRIPTION
This script navigates to a named virtual desktop, focuses on Microsoft Edge,
switches to the Brain.fm tab, and starts playback. It uses the VirtualDesktop
module and Windows API calls for desktop and window management.

.NOTES
Requires the VirtualDesktop module to be installed.
Feature_Image: ./images/feature_images/20240824_-_Launching_a_Web_App_from_Stream_Deck_3.png

#>

# Import the VirtualDesktop module
# Must be installed as admin by running "Install-Module VirtualDesktop -Scope CurrentUser"
Import-Module VirtualDesktop

Add-Type -AssemblyName System.Windows.Forms

# Add Windows API function declarations for window management
Add-Type @"
using System;
using System.Runtime.InteropServices;

public class User32 {
    [DllImport("user32.dll")]
    public static extern bool ShowWindow(IntPtr hWnd, int nCmdShow);

    [DllImport("user32.dll")]
    [return: MarshalAs(UnmanagedType.Bool)]
    public static extern bool SetForegroundWindow(IntPtr hWnd);
}
"@

<#
.SYNOPSIS
Switches to a named virtual desktop.

.PARAMETER name
The name of the virtual desktop to switch to.

.EXAMPLE
Switch-ToNamedDesktop -name "Media"
#>
function Switch-ToNamedDesktop {
    param([string]$name)
    $desktops = Get-DesktopList
    $targetDesktop = $desktops | Where-Object { $_.Name -eq $name }
    if ($targetDesktop) {
        try {
            Switch-Desktop -Desktop $targetDesktop.Number
            Write-Host "Switched to virtual desktop: $name"
        } catch {
            Write-Host "Error switching to desktop: $_"
        }
    } else {
        Write-Host "Virtual desktop '$name' not found."
        Write-Host "Available desktops: $($desktops | ForEach-Object { $_.Name } | Join-String -Separator ', ')"
    }
}

<#
.SYNOPSIS
Sets focus on an application window.

.PARAMETER name
The name of the application to focus on.

.EXAMPLE
Set-ApplicationFocus -name "Edge"
#>
function Set-ApplicationFocus {
    param([string]$name)
    $processes = Get-Process | Where-Object { $_.MainWindowTitle -ne "" }
    Write-Host "Searching for application with name containing '$name'"
    Write-Host "Running processes with non-empty window titles:"
    $processes | ForEach-Object { Write-Host "  - $($_.ProcessName): $($_.MainWindowTitle)" }

    $app = $processes | Where-Object { $_.ProcessName -like "*$name*" -or $_.MainWindowTitle -like "*$name*" } | Select-Object -First 1
    if ($app) {
        [User32]::ShowWindow($app.MainWindowHandle, 9) # 9 = SW_RESTORE
        [User32]::SetForegroundWindow($app.MainWindowHandle)
        Write-Host "Set focus on application: $($app.ProcessName) (Window Title: $($app.MainWindowTitle))"
        return $true
    } else {
        Write-Host "Application with name containing '$name' not found."
        return $false
    }
}

<#
.SYNOPSIS
Switches to a specific tab in Microsoft Edge.

.PARAMETER tabTitle
The title of the tab to switch to.

.EXAMPLE
Switch-EdgeTab -tabTitle "brain.fm"
#>
function Switch-EdgeTab {
    param(
        [string]$tabTitle
    )
    $edge = Get-Process | Where-Object { $_.ProcessName -eq "msedge" -and $_.MainWindowTitle -ne "" } | Select-Object -First 1
    if ($edge) {
        # Activate the Edge window
        [User32]::ShowWindow($edge.MainWindowHandle, 9) # 9 = SW_RESTORE
        [User32]::SetForegroundWindow($edge.MainWindowHandle)
        Write-Host "Activated Edge window: $($edge.MainWindowTitle)"

        # Use correct keyboard shortcut to search tabs
        $wshell = New-Object -ComObject wscript.shell
        $wshell.SendKeys("^+a")  # Ctrl+Shift+A to open tab search
        Start-Sleep -Milliseconds 500  # Wait for search to open
        $wshell.SendKeys($tabTitle)
        Start-Sleep -Milliseconds 500  # Wait for search results
        $wshell.SendKeys("{ENTER}")

        Write-Host "Attempted to switch to tab '$tabTitle' in Edge"
        return $true
    } else {
        Write-Host "No Edge window with a non-empty title found."
        return $false
    }
}

# Main execution
try {
    # Switch to the "Media" virtual desktop
    Switch-ToNamedDesktop -name "Media"

    # Set focus on the Edge browser window
    $focusSuccess = Set-ApplicationFocus -name "Media"

    # Only proceed if focus was successful
    if ($focusSuccess) {
        # Switch to the tab with brain.fm
        $switchSuccess = Switch-EdgeTab -tabTitle "brain.fm"

        if ($switchSuccess) {
            Write-Host "Navigation completed successfully."
            # Wait for 0.5 seconds
            Start-Sleep -Milliseconds 500
            # Send a space key press to start playing Brain.fm
            [System.Windows.Forms.SendKeys]::SendWait(' ')

        } else {
            Write-Host "Navigation partially completed. Failed to switch to the desired tab."
        }
    } else {
        Write-Host "Navigation failed. Could not focus on Microsoft Edge."
    }
} catch {
    Write-Host "An error occurred: $_"
}

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Envision a futuristic digital control nexus where sleek, high-tech interfaces merge with holographic displays that float against a dark, reflective backdrop. The scene is filled with dynamic arrays of abstract code, streaming data ribbons, and luminescent virtual desktops that evoke a sophisticated automation environment reminiscent of a Stream Deck’s command center. Intricate neon circuitry and shimmering panels pulse with energy, creating an atmosphere of digital mastery and modern mystique.

Programs Do Things

2024-08-24T00:00:00-05:00

Programs Do Things

“Every system is built from a domain specific language designed by the programmers to describe that system. Functions are the verbs of that language, and classes are the nouns.” — Robert C. Martin, Clean Code.

Functions are the verbs of your programs. They “do” work.

Classes are the nouns of your programs. They are the “things” that functions do work on.

Feature Image Prompt:

Generate an image. The aesthetic should be cyberpunk with colors of neon pink, blue and purple. Do not add any people. Imagine a dark, futuristic digital landscape where abstract, glowing circuit patterns and symbolic nodes form a sprawling network. In this environment, bursts of neon energy represent dynamic functions—energetic, vivid flashes that trigger action—while structured, luminous blocks symbolize classes, solid and foundational elements that organize the chaos. The composition should evoke a high-tech, cybernetic world where programming languages transform into vibrant, mesmerizing visuals, reminiscent of a digital metropolis pulsing with life.

Learning Machine

Letting Codex Agents Commit: Making .git Writable in the workspace-write Sandbox

The trick: name the .git path itself

Wiring it into a shell function

Caveats

The Dark Arts of Web Automation

How I Fixed Handy's Memory Leak Without Knowing Rust

For the TikTok attention span ("TL;DR")

Have you tried Handy for voice transcription?

I'm not a Rust programmer though ...

Okay, so what is this ✌️"The Framework"✌️?

Right, so what did that adaptation actually look like?

1. The autonomy harness

2. The instrumentation

3. Narrowed systematically until only one candidate remained

4. Fixed it three ways, tested each piece on its own and together

5. Three rounds of adversarial review

6. The meatbag-in-the-loop soak

What's next, then?

You, too, can be an AI agents ninja!

Unlocking Codex GPT-5.4's Full 1M-Token Context Window

The profile that unlocks 1M

Verifying it works

Why this works now (and didn't before)

What about GPT-5.5?

Update -- July 2026 (Codex 0.144.4): this technique no longer works

1. The model_context_window override is ignored

2. The catalog now drives only the displayed window -- not the enforced one

3. There's now a hard 1 MiB cap on a single input message

What the windows actually are now

Getting Handy to work in GNOME Wayland (Ubuntu 25.10+)

Background & what we’re trying to achieve

The solution overview

A note on the Tauri Typing Tool dropdown

Implementing the fixes

Problem 1: global hotkey delivery

Step 1: Bind a GNOME custom keyboard shortcut (with absolute path)

Step 2: Ensure Handy daemon is running

Problem 2: text delivery via ydotool

Step 3: Install ydotool and grant uinput access

Step 4: Write the External Script

Step 5: Configure Handy

Verification

Troubleshooting / failure modes

Appendix: Why simpler fixes don't work

Is HTML "Strictly Better" Than Markdown for Claude Code?

TL;DR

What Thariq actually said

There are actually two questions here

How I ran this

Design exploration: this is where HTML earns its keep

What should we call you?

What kind of work do you do?

You are in.

Welcome, Sam

Get set up (1 of 4)

Who are you, mostly?

PR review: a fair fight, and HTML costs about 40% more for the polish

PR Review &mdash; Respect .gitignore when target has no .git directory

Background: how gitignore resolution actually works here

Two functions, two directions of walk

Before this PR (no .git upstream)

After this PR (no .git upstream)

Annotated diff &mdash; src/toks/scanner.py

Annotated diff &mdash; tests/test_scanner.py

Findings summary

Behavioral coverage matrix

Recommendation

PR Review — toks c6d70f9

TL;DR

Severity legend

Bracing the gitignore / path-resolution logic

Walking the four cases

What the fallback does NOT do

The diff, annotated

Hunk 1 — scanner.py lines 104-112

Hunk 2 — scanner.py lines 128-135

Hunk 3 — tests/test_scanner.py lines 99-110

Findings

[MEDIUM] F-1 — Test is silently dependent on tmp_path having no ancestor .git

The trick: name the `.git` path itself

1. The `model_context_window` override is ignored

PR Review — Respect `.gitignore` when target has no `.git` directory

Before this PR (no `.git` upstream)

After this PR (no `.git` upstream)

Annotated diff — `src/toks/scanner.py`

Annotated diff — `tests/test_scanner.py`

PR Review — `toks` c6d70f9

Hunk 1 — `scanner.py` lines 104-112

Hunk 2 — `scanner.py` lines 128-135

Hunk 3 — `tests/test_scanner.py` lines 99-110

`[MEDIUM]` F-1 — Test is silently dependent on `tmp_path` having no ancestor `.git`

`[MEDIUM]` F-2 — Behavior of fallback should be documented

`[LOW]` F-3 — Unused parameter in `load_gitignore_specs`

`[LOW]` F-4 — Style: `git_root or target` over the ternary

`[LOW]` F-5 — `find_git_root` accepts `.git` as either file or directory; commit message says "directory"

`[POSITIVE]` F-6 — Minimal diff

`[POSITIVE]` F-7 — Test assertions cover both pattern shapes