step 3/6 failed docs link broke 6/6 verified

Docs that work (literally).

Paste a docs URL. Your coding agent runs the quickstart in a clean container, finds the step that breaks, fixes it, and proves the fix.

Try

A partial URL works. So does a tool name. Nothing leaves your browser.

Untested Docs In. Proven Docs Out.

Every quickstart is a guess until someone runs it. FirstRun runs it first.

Never run by anyone
pip install fails at step 3
404 in the install link
Python version never stated
Rage quit. No bug filed.

FirstRunthe gate

Failure foundthe first broken step, named
Fix patcheda minimal docs change
Proven cleana second fresh container passes
PR openedafter you approve once
Run tracedneatlogs spans
Session recordedan Entire checkpoint

A terminal command, firstrun run with a docs URL, above six numbered quickstart steps in a clean container

Read

FirstRun reads the page and plans one step per command.

Six quickstart steps. The first two pass and step 3, run demo, fails for a new developer

Run

Every step runs in a clean container, like a new developer. The first failure stops the run.

Prove

A fix is not done until a second clean container passes the whole quickstart.

Watch a Real Run.

Every run ends with a share link. Replay one.

Known-broken fixtureA pip pin that cannot install

$ pip install git+https://github.com/HighnessAtharva/firstrun $ firstrun https://docs.getdbt.com/guides/duckdb $ firstrun pr <run> # prints the gh commands for the docs PR

Prefer the terminal? You need Docker and a Claude token from claude setup-token.

Three Tools. One Job Each.

neatlogs

Traces every run.

The planner, executor and verifier show up as spans, so a bad run has a cause you can read.

neatlogs.com

neatlogs trace of the FirstRun planner: 3 steps in 12 seconds, with the runbook JSON it produced — The first FirstRun trace, Sat Oct 10.

Entire

Records every session.

Each commit carries the Claude Code session that wrote it, so any line traces to a prompt.

entire.io

Entire trails view for the firstrun repository with trail number 1 in Planning — Trail #1 for the firstrun repo.

cfo.ai

Turns cost into a plan.

The measured cost per run from neatlogs goes to Ari, the cfo.ai agent, which models the pricing.

cfo.ai

One Engine, Three Answers

A clean container and stored commands answer three questions a docs owner asks.

Does my quickstart work?

Break → Fix → Prove

Runs the page in a fresh container, patches what breaks, and proves the patch.

firstrun <url> Replay a broken fixture

Which release broke it?

0.2.10.3.10.4.60.4.100.4.14

The dbt guide step jafgen --years 6 broke in jafgen 0.4.6, 11 months before the guide shipped.

firstrun bisect <run> --package jafgen Replay the dbt bisect

What does it silently assume?

curlca-certificatesgnupgsudoproven 5/5

The AccuKnox install steps need four packages the docs never name. A fresh container proved the list.

firstrun prereqs <run> Replay the AccuKnox proof

Bisect and Prerequisite Proof replay stored commands, so they cost $0 in model calls.

The Numbers Come From Real Runs.

4upstream docs PRs opened on 3 doc sites. Each fixes a step a new developer hits.dbt #10149 dbt #10150 Entire #2724 AccuKnox #671

5 of 5open-source quickstarts verified in a fresh container

2 → 3of 4 quickstarts finish, v1 to v2

0 → 3failed steps recovered

$0.02model cost per run, measured by neatlogs (list price, not billed)

Every PR, in Public.

Opened upstream on real doc sites. Status and stars read live from GitHub.

Screenshot of pull request 10149 on dbt-labs/docs.getdbt.com: Fixing the jafgen command in the DuckDB quickstart, status Open

dbt-labs/docs.getdbt.comOpen

Fix the jafgen command in the DuckDB quickstart #10149

Brokejafgen --years 6 fails with No such option: --years.

Fixedyears is now positional, so both mentions change.

+2 −2 in 1 file
218 repo stars

View on GitHub

Screenshot of pull request 10150 on dbt-labs/docs.getdbt.com: Stating the Python version the DuckDB quickstart needs, status Open

dbt-labs/docs.getdbt.comOpen

State the Python version the DuckDB quickstart needs #10150

Brokethe cloned repo pins networkx==3.7, which needs Python 3.12.

Fixedthe guide now names the Python version.

+1 −0 in 1 file
218 repo stars

View on GitHub

Screenshot of pull request 2724 on entireio/cli: Saying where install.sh puts entire on Linux, status Open

entireio/cliOpen

Say where install.sh puts entire on Linux #2724

Brokeafter a good install, entire version fails with command not found.

Fixedthe docs say the binaries land in ~/.local/bin, which a clean shell lacks on PATH.

+2 −0 in 1 file
5,169 repo stars

View on GitHub

Screenshot of pull request 671 on accuknox/help: Pointing knoxctl install at the latest release and listing APT prerequisites, status Open

accuknox/helpOpen

knoxctl install: point at the latest release and list APT prerequisites #671

Brokethe page names knoxctl 0.9.0 as latest, but the installer already fetches 0.9.58.

Fixedthe version line and four download links move to 0.9.58, and the APT steps list curl, gnupg and sudo.

+7 −5 in 1 file

View on GitHub

Also run, no PR needed MkDocs verified 9/10 Dagster verified, no docs PR after a source check

How FirstRun Compares.

Feature	FirstRun	Claude Code	GitHub Actions	GitHub Agents	Custom workflow
Runs the quickstart as a new developer	Yes. Every step runs in a fresh Docker container	Runs on your machine unless you set up a sandbox [FILL: sandbox setup]	Yes. A fresh runner per job, but you script each step	[FILL: execution environment]	Depends on the build [FILL: environment]
Plans steps from a docs URL	Yes. The planner writes one step per command	Yes, when you prompt it	No. You write each step	[FILL: input it accepts]	You write the parser
Improvises when a step failsStands out	Yes. It classifies the failure, patches, and retries up to 3 times per step	Yes, inside one session	No. The job fails and stops	[FILL: retry behavior]	Only what you code
Proves the fix	Yes. A second clean container must pass the whole quickstart	Only if you ask	Only if you script it	[FILL: verification step]	Only if you code it
Telemetry per runStands out	Yes. neatlogs spans for planner, executor and verifier. Measured cost of $0.02 per run	[FILL: telemetry setup]	Logs and run history. No agent traces	[FILL: telemetry]	You build it
Opens the docs PR	Yes, after you approve once	Yes, when you ask	Only if you script it	[FILL: PR behavior]	Only if you code it
Ease of useStands out	Paste a docs URL, then paste the prompt into your coding agent	One prompt, no failure classes	A YAML file per quickstart	[FILL: setup steps]	Highest effort
Proof in public repos	4 upstream PRs, all open (see the record above)	[FILL: examples]	[FILL: examples]	[FILL: examples]	[FILL: examples]

FirstRun reads a docs quickstart and runs every step in a clean container, the way a new developer would. It stops at the first failure and classifies it. It tries a fix up to 3 times, then proves the fix with a second clean run. It asks once before it opens a PR.

FirstRun opened 4 upstream docs PRs, and all are open today. The dbt DuckDB quickstart had a broken jafgen command (#10149) and no stated Python version (#10150). The Entire CLI installer put entire on a path that a clean shell lacks (entireio/cli#2724). The AccuKnox knoxctl page named 0.9.0 as the latest release while its installer fetched 0.9.58, and its APT steps assume curl, gnupg and sudo (accuknox/help#671). MkDocs and Dagster passed with no docs change, so a run also shows when docs hold up.

Every run is a neatlogs trace, so a bad run has a readable cause. A docs bug surfaces in a run instead of in a reader who leaves without filing an issue. [FILL: merge status after the maintainers respond]

The Build, in Public.

Every step is a post on @cultist_dev.

Sat Oct 10, 10:45

my neatlogs hackathon project has already caught doc issues for a chroma (24K+ github stars). chromadb's deprecated-config error sends you to its own migration guide. that page returns 404.
Read the post on X
Sat Oct 10, 12:30

#neatHack kickoff, 12:30 IST. FirstRun code before kickoff: 0 lines. neatlogs doctor probe: 7/7 PASS. Solo build. Submission opens Monday, 7 PM IST.
Read the post on X
Sat Oct 10, 13:14

#neatHack, day one. all three required tools are up. Entire: trail #1 created. neatlogs: first agent trace, 3 steps, 12s. cfo.ai: Ari read my public profile.
Read the post on X
Sat Oct 10, 13:26

your quickstart is the only part of your product tested by people who leave instead of filing a bug. so for #neatHack i'm building FirstRun. break. fix. prove.
Watch the video on X
Sat Oct 10, 15:15

a quickstart breaks the way an agent breaks. quietly. so the plan for this weekend: every FirstRun run becomes a trace in @neatlogs.
Read the post on X
Sun Oct 11 to Mon Oct 12More posts land here as the weekend goes.

Follow the rest on X, @cultist_dev

solo build

Atharva Shah

Atharva is a DevRel engineer. He owns developer docs at AccuKnox and works with Altimate AI.

A quickstart that breaks without a sound is a docs bug nobody reports. FirstRun is his answer. He built it alone during the hackathon, in public.

atharvashah.com GitHub @cultist_dev

Docs that work (literally).

Run it fresh with your agent

Untested Docs In. Proven Docs Out.

Read

Run

Prove

Watch a Real Run.

Three Tools. One Job Each.

One Engine, Three Answers

Does my quickstart work?

Which release broke it?

What does it silently assume?

The Numbers Come From Real Runs.

Every PR, in Public.

Fix the jafgen command in the DuckDB quickstart #10149

State the Python version the DuckDB quickstart needs #10150

Say where install.sh puts entire on Linux #2724

knoxctl install: point at the latest release and list APT prerequisites #671

How FirstRun Compares.

The Build, in Public.

Atharva Shah