From the lab

what it does Reading the machine's work

AI writes the code. This reads it back.

Described to an assistant, assembled in an afternoon, shipped on trust — more software than ever is owned by someone who has never read it. One command maps every way into your application, what each one really does, and — on every scan after the first — what changed. Nothing is uploaded. The report is a single file that works offline.

npx what-it-does · free · MIT · runs on your machine

npx what-it-does scan 01scan 02

POST /api/invite sends an email
GET /api/projects reads projects
DELETE /api/projects/[id] checks who is asking · deletes projects no longer checks who is asking

28 ways in · nothing looked wrong 1 new thing worth checking

801Ways in read, one real repo
0Findings on it
1Command to run
0Bytes uploaded

28 ways in 5 findings

Tidepool — the findings demo

A fictional team app, written to be scanned. Five findings of the kind this code really produces — a checkout nobody protected, a delete that lost its guard — each with a fix prompt to paste straight into your AI.

Sample app Fictional code · real analysis

The top of the Tidepool report: 28 ways in, 5 worth checking.

801 ways in 0 findings

dub — the quiet report

The same scanner on dub's real open-source monorepo — 4,000 files of production code. It reports nothing, because there is nothing there to report. A tool you can trust is one that stays quiet on good code — and we measured what happens when it does not.

Real monorepo 0 findings

The top of the dub report: 801 ways in, 0 worth checking.

You will run it once. That is the problem.

A scan tells you what your application does today. Next week somebody merges a change and it does something else, and nobody runs the command again. That is not a discipline failure — it is what happens to every tool that has to be remembered. So the pull request check does not need remembering. It reads the base branch, reads yours, and speaks only when the answer changed.

What it leaves on the pull request

what it does

This changes what your application can do in 1 way, compared with main.

1 new thing worth checking.

DELETE /api/test-slett

Deletes rows from notater — with no visible check on who is asking

app/api/test-slett/route.ts:10

9 other behaviours unchanged. Read locally with npx what-it-does.

The whole of the setup

name: what it does
on: pull_request

jobs:
  behaviour:
    runs-on: ubuntu-latest
    permissions:
      contents: read
      pull-requests: write
    steps:
      - uses: rolfe099-sketch/what-it-does/action@v1
  • It stays quiet. A pull request that changes no behaviour gets no comment at all. When there is something to say it edits the same comment rather than stacking a new one on every push.
  • It fails open. If a scan errors, the check passes and says so in the log. It fails your build over a finding only when you turn on fail-on-new.
  • Your code never leaves your runner. The scan runs inside your own CI. Nothing is sent to us — no repository name, no file, no finding. The only requests are the ones the check needs: installing the scanner from npm and posting the comment to your pull request.

The scan is free. So is the watching.

One command, MIT, no account, nothing uploaded. The pull request check that runs the same comparison on every change is under the same licence, on any repository, with no key to configure. There is nothing on this page to buy.

The scanner

Every way into your application and what each one does, as one HTML file you can read offline. Any project, private or not. No account, and no network at all.

npx what-it-does

The pull request check

Reads the base branch, reads yours, and comments only when the behaviour changed. Silent otherwise, and it fails your build over a finding only if you ask it to. Two minutes to set up.

rolfe099-sketch/what-it-does/action@v1

The field behind this section is tidepool's real dependency graph — 15 resources, 24 connections, laid out by the scanner itself from the report above. The pull request comment is a real one, lifted from a real pull request. Nothing in this section is a mockup.

Aion The same idea, elsewhere

Aion — batch work for coding agents.

Not the work that pays the bills — it is the most technically ambitious thing here, and the source is public. The same instinct runs through it: Aion reads files and calls language models, and deliberately cannot run shell commands.

PythonEngine
432Tests, no network
4Model providers
MITOpen source

Describe the job once. Run it over everything.

An agent asked to review forty files reads them one at a time, filling the conversation with material nobody will read and billing every file to an expensive model. Aion takes the mechanical part: one call per file, run at once, merged into a single answer. Across nine real jobs it kept 64.7% of what was produced out of the conversation.

Then the harder question, and the one most tools never ask: will an agent actually reach for it? A benchmark harness was built to find out, and the first answer was unflattering.

Does the agent actually reach for it?

Available as a skill22%
Named where the agent already reads100%

Eighteen scenarios. Zero false positives — never reached for where it should not have been.

Produced 22,451 chars
Reached the conversation 315 chars

One forty-file review. 98.6% of what it produced never entered the context window — 64.7% aggregate across nine jobs.