The toil pattern catalog
Source article for ambientagents.net regeneration. Author: Jeff Sylvan. Every pattern below is a candidate extracted from real Black Skies work, but frequencies and costs are unverified until I fill them in. Rule for the site: real counts or nothing. No pattern ships to the page without its [FILL] slots resolved.
Why a catalog
The compiler is only as good as its seed corpus. These patterns are that corpus: recurring manual interventions from months of running a multi-agent pipeline on a real distributed-systems project. Each one is written as trigger, policy, verification, because that's the shape the compiler consumes. A pattern that can't be expressed in that shape isn't ready to automate, which is itself useful information.
Template for every entry:
- Trigger: the event that starts it
- Manual today: what I actually do by hand
- Policy: what the system should decide without me
- Gate: where a human stays in the loop
- Frequency and cost: [FILL] occurrences per week, [FILL] minutes each
- Automation risk: the specific way automating this goes wrong
The patterns
1. Ratified design delta fan-out [CONFIRM]
Trigger: a design or canon document is ratified with changes. Manual today: I diff the ratified doc against in-flight tasks and queued specs, mark what's stale, and write rework tasks by hand. Policy: maintain a mapping from document sections to DAG subtrees; on ratification, compute the invalidation set and propose rework nodes. Gate: adversarial review of the invalidation set before any preemption fires. Frequency and cost: [FILL]. Automation risk: over-invalidation, which turns one design decision into a week of thrash.
2. Working-doc versus devblog alignment [CONFIRM]
Trigger: an edit lands on either the working architecture document or a published devblog post. Manual today: row-by-row reconciliation of an alignment matrix between the two. Policy: generate a drift report and propose edits to the lagging side. Gate: hard gate; nothing public-facing changes without my ratification. Frequency and cost: [FILL]. Automation risk: the system editing public content without approval. This gate never relaxes.
3. PR review fan-in [CONFIRM]
Trigger: a PR opens or a review is submitted. Manual today: I read agent-authored PRs myself and sometimes manually solicit a second model's opinion. Policy: assign reviewers across model families, capture dissent explicitly rather than averaging it away. Gate: unresolved dissent above threshold blocks merge. Frequency and cost: [FILL]. Automation risk: review loops with vendor bots; lineage filters are mandatory before this turns on.
4. Issue triage and dedup [CONFIRM]
Trigger: an issue opens or receives a comment. Manual today: I check whether it duplicates something already fixed, label it, and slot it into the plan. Policy: similarity search against closed issues and recent PRs; either append a triaged node or propose close-with-reference. Gate: closing an issue requires my approval in v1. Frequency and cost: [FILL]. Automation risk: false dedup that silently discards a real bug.
5. Blocked-on-human parking and resume [CONFIRM]
Trigger: a worker signals it needs a human answer (the exit-10 pattern). Manual today: I notice eventually, answer, and rebuild the context the worker lost while waiting. Policy: park the node durably, route the question to one inbox with priority and expiry, resume with the answer injected into intact context. Gate: the inbox is the gate. Frequency and cost: [FILL]. Automation risk: the question queue growing silently; parked work needs SLOs and escalation, not just persistence.
6. Budget watch and runaway kill [CONFIRM]
Trigger: spend telemetry crosses a threshold. Manual today: I notice a session burning tokens on a doomed path, usually late. Policy: warn, throttle, or kill per policy; refund the node's remaining budget to its mission. Gate: kills above [FILL: dollar threshold] require my acknowledgment. Frequency and cost: [FILL]. Automation risk: killing a productive long run; minimum work quanta exist to prevent exactly this.
7. Stale worktree and branch hygiene [CONFIRM]
Trigger: an invalidation bumps a subtree epoch. Manual today: I hunt down zombie branches and worktrees left by preempted tasks. Policy: enumerate artifacts stamped with stale epochs; archive or delete per policy. Gate: destructive deletes are batched for review. Frequency and cost: [FILL]. Automation risk: deleting salvageable work; quarantine before delete, always.
8. Cross-service interface propagation [CONFIRM]
Trigger: a schema or interface change merges in one service of the fan-out path (bridge, relay, checkpointer). Manual today: I find the dependents and open rework tasks for each. Policy: dependency-map lookup; append rework nodes with correct edges. Gate: normal review only. Frequency and cost: [FILL]. Automation risk: an incomplete dependency map producing false confidence; the system must report map coverage, not just results.
Measurement protocol
Before any of this automates, one baseline week of tally marks: every manual intervention, which pattern, minutes spent. The tally sheet becomes the before column, and it's the only honest source for the ROI numbers the site will eventually claim. After the January dogfood, the same sheet becomes the after column. If the delta is small, that gets published too.