Agent-Human Collaboration
The design space for keeping humans in the loop on complex agent work — framed around two axes: trust (can I let it run?) and control (can I steer it?).
Last updated: 2026-04-23
Overview
As agents take on longer-running, higher-stakes work, the question shifts from “does the model produce good outputs?” to “how do humans and agents divide judgment?” Jacob Lauritzen (Legora CTO) proposes two organizing axes:
- Trust: how reliably does the agent do the right thing — can I let it run unsupervised? Low trust = review every trace; high trust = let it rip.
- Control: how effectively can I impose my judgment at any point in the agent’s work? Low control = can only correct after delivery; high control = can steer at every node.
Neither axis is always maximized. The combination determines the right interface, planning strategy, and oversight pattern.
Work as a DAG
Complex agent work is a tree (technically a DAG) of subtasks: research → review clauses → draft → finalize. The key insight about control is structural: where in the DAG can a human impose judgment?
- Chat: collapses the entire DAG into a single linear thread. You can only impose judgment at the root (the original prompt) or by interrupting afterward. Very low control.
- Planning: lets you define the DAG upfront before work starts. Better — but requires you to anticipate every contingency before seeing the data, and misses edge cases that only appear mid-execution.
- Skills: encode human judgment at specific DAG nodes. When the agent hits a termination clause with an unusual EU law provision, the skill already knows how to handle it. Progressive discovery — contingencies are handled regardless of planning order.
- Elicitation + decision log: agent encounters uncertainty mid-flight, makes a decision, logs it, and keeps going rather than blocking. Human reviews the decision log after completion and reverses specific decisions as needed. Best of planning and skills — handles novel cases skills don’t cover.
The Verifier’s Rule for Agents
Jason Wei’s verifier’s rule states: if a task is solvable and easy to verify, AI will solve it. Lauritzen applies this to agent loops, not just training: if you can make a task verifiable, you can run an agent in a loop with feedback (“you did this wrong, fix it”) and it will eventually get there.
Different tasks sit at different points on the solvability/verifiability spectrum even within a single domain. Legal example:
- Checking definitions in a contract → easy to verify → AI solves it well
- Writing a contract → easy to do, very hard to verify (only courts can say if the language holds up)
- Litigation strategy → impossible to verify (five lawyers, five answers) → AI’s weakest area
Practical implication: when designing agent workflows, first audit where tasks sit on this spectrum. Then use one of three moves:
- Bring the task down the spectrum: add proxy verification. For contracts, compare against golden contracts you know work. Not ground truth, but a useful signal.
- Decompose: split one hard-to-verify task into several easy-to-verify subtasks. Formatting and definition linting can be verified mechanically even when contract quality can’t.
- Add guardrails: limit what the agent can do to increase trust in the subset it executes. Constrain file access, tool set, or search scope.
High-Bandwidth Artifacts > Chat
The core interface thesis: chat is the wrong primary collaboration surface for complex agent work.
Chat is one-dimensional. It collapses the agent’s multi-branch work tree into a linear thread. When a human reviews a 30-minute agent run in a chat interface, they can’t tell at a glance:
- Which clauses were changed (not just which one they asked about)
- What decisions the agent made autonomously
- Where the agent was uncertain
The answer is high-bandwidth, persistent, domain-specific artifacts:
- A document where you can highlight clause three and ask the agent to change only that clause, add inline comments, tag collaborators, tag specialized agents
- A tabular review where the agent populates a structured view of all contracts reviewed, flags specific items needing judgment, and the human can scan and resolve efficiently
These look different by vertical. Legal has documents and review tables. Coding has PR diffs and test results. Finance has financial models and variance reports. The interface must match the domain’s natural review primitive.
Language-as-input is excellent — flexible, familiar, low-barrier. Language-as-the-only-collaboration-surface is a limitation we’re importing from human communication into a context where one participant isn’t a human.
The Planning Trap
Planning (specifying what the agent should do before it starts) sounds like high control. It’s actually lower than it appears, for two reasons:
- Full specification burden: to give good planning instructions, you need to already know what the agent will encounter. The agent hasn’t seen the data yet. You can’t tell it how to handle the special EU clause in contract #7 because you haven’t seen contract #7.
- No mid-flight course correction: planning is a front-loaded specification. If the agent’s execution diverges from your mental model, you only find out at the end.
Lauritzen’s framing: planning is like assigning work to a coworker who takes a brief, aligns with you on the approach, then disappears until they hand in the final deliverable. Functional, but low-bandwidth collaboration.
Skills + elicitation/decision-log are strictly superior for complex work because they handle contingencies at the node level rather than requiring upfront specification of every possible branch.
Context Rot and the Review Problem
Long-running agents run into compaction: when context gets too long, the model compresses or forgets earlier state. This creates a specific review failure mode: the human asks the agent to fix clause three; the agent hits compaction and modifies the entire document. The human now can’t tell what changed.
This compounds with low-bandwidth interfaces. In a chat thread, there’s no structural way to see what changed between versions. In a document with tracked changes, it’s obvious.
Compaction is a reason to keep agent work steps short and artifact-centric: write to a document, not to memory. The document is the persistent state; the agent’s context window is ephemeral.
Connections
- jacob-lauritzen — source of this framework; Legora as vertical AI implementation
- claude-code-skills — skills are the highest-rated control mechanism; encode judgment at DAG nodes
- agent-first-software — high-bandwidth artifacts are the agent-first UI design answer; complements Levie/Andreessen’s “death of the UI” thesis
- agentic-system-failure-modes — context rot and compaction compound the review problem
- agent-evaluation — verifier’s rule connects to Karpathy’s model jaggedness; verifiable tasks ↔ RL improvement
- dark-code — low-control, low-trust agent interfaces produce dark code: unattributable behavior nobody can review
- agentic-engineering — Hoekstra’s orchestration and guardrails layers are the implementation of the control/trust axes
- ai-agents — broader topic area
Sources
- Agents need more than a chat — Jacob Lauritzen, CTO Legora — conference talk, added 2026-04-23