Andrew Dumit describes supply-chain graphs used to calculate product carbon footprints. Expert judgment can produce several defensible answers, so a plausible final number is not enough: reviewers also need to know whether the method was valid. Specialized tool calls struggled when the task expanded to many richly annotated graphs.
Andrew Dumit reports that coding agents made graph exploration and bulk editing more efficient, but also introduced unexpected languages, direct data edits without lineage and false completion claims. Non-programmer users could not reliably inspect the generated code.
Andrew Dumit constrains consequential edits through a typed TypeScript SDK while preserving freedom to reason and write code. A system-owned final executor lints the script, detects conflicting changes, runs it and validates its output before generating a human-readable review artifact. Invalid results return to the agent instead of being accepted on its word.
Andrew Dumit reports improved internal task outcomes from roughly 43% to 92% after prompt, example, SDK and planning changes. Those figures describe Watershed's own evaluations, not a general performance guarantee. Process validation remains valuable even when the task's ground truth involves disputed expert judgments.
Watch on YouTube
