Skip to content
预计 1 分钟右侧目录可快速定位

Shadow Harness — v0.1 → v1.0 持续工程主线 ​

Shadow Harness 是 13 周路线的工程验证载体。每周在同一个仓库、同一条 Run lifecycle 上增量演进;禁止另起独立 demo 替代上一周版本。 版本号表达“当前已经建立哪些 invariant”,不是产品发布语义。

版本演进 ​

text
v0.1 Model → Final / Action Intent
  ↓
v0.2 Model → validated ToolCall → ToolResult
  ↓
v0.3 parallel Tool + timeout/cancel + Observation
  ↓
v0.4 ContextManager + budget + compaction + snapshot
  ↓
v0.5 Retriever source + context injection
  ↓
v0.6 AgentLoop + RunStateMachine + termination
  ↓
v0.7 Planner → validated Graph → Scheduler
  ↓
v0.8 retry + idempotency + policy + approval
  ↓
v0.9 checkpoint + DB transaction + outbox + resume
  ↓
v0.10 Run API + SSE + Queue/Worker + Lease/Fencing
  ↓
v0.11 MCP Adapter + Workspace + Capability/Security boundary
  ↓
v0.12 Trace + deterministic faults + Eval + Benchmark
  ↓
v1.0 deploy + graceful drain + crash/resume + full regression

最终依赖图 ​

text
API / Event Stream
        ↓
Run Runtime / State Machine
├─ ContextManager ← Retriever / Memory
├─ ModelAdapter → Planner → GraphCompiler / Scheduler
├─ ToolExecutor ← ToolRegistry ← MCP Adapter / Workspace
├─ Policy / Approval / Capability
├─ StateStore / CheckpointStore / EventLog / Outbox
├─ Queue / Worker / Lease / Fencing
└─ Trace / Test / Eval / Budget / Benchmark
        ↓
Deployment / Graceful Drain / Resume

每周增量、接口与 invariant ​

版本新增核心接口/状态本版本必须成立的 invariant典型 failure injection
v0.1ModelClient, Run, Turn, ActionIntentModel output 只是候选决策;Runtime 才提交事实unknown action type
v0.2ToolSpec, ToolRegistry, ToolCall, ToolResultToolCall 执行前完成 schema/domain validationmalformed ToolCall
v0.3ToolExecutor, ToolExecutionId, Observation每个 execution 有独立 lifecycle;timeout/cancel 可区分success + timeout + cancel 并发
v0.4ContextManager, TokenBudget, ContextSnapshotModel Context 是 Runtime state 的受控投影overflow + protected state compaction
v0.5Retriever, evidence/provenanceretrieval 是 Context source,不是 execution truthhigh-similarity wrong evidence
v0.6RunStateMachine, TurnSettlementterminal state 由可验证 evidence 驱动model falsely claims done
v0.7StructuredPlan, PlanValidator, GraphCompiler, SchedulerPlan 不直接执行;Graph 先 deterministic validationcycle / missing dependency
v0.8RetryPolicy, ExecutionRecord, IdempotencyKey, ApprovalGateunknown side effect 不能被 LLM 自行重试timeout after possible side effect
v0.9StateStore, CheckpointStore, Outbox, resume statecrash 恢复不丢 execution facts、不重复已提交副作用crash-before-checkpoint / commit-before-dispatch
v0.10Run API, SSE cursor, dispatch envelope, lease/fencing tokendisconnect≠cancel;redelivery/双 claim 不产生重复提交stream disconnect / stale worker commit
v0.11MCP Adapter, workspace capability, permission boundary协议/ToolResult 不能提升自身权限prompt/tool-result injection / capability escape
v0.12trace correlation, fault suite, eval, benchmark每个关键 failure 可复盘;容量边界可测overload / latency / retry storm
v1.0RunManifest, drain protocol, deployment configrolling deploy 中 active run 可安全 drain/resumeSIGTERM during active run

兼容性规则 ​

每次升级都必须复用上一版本 domain identity 与 regression tests:

  • run_id、turn_id、tool_execution_id 一旦成为 execution identity,不因新版本改名而丢失语义;
  • 新增 state 字段必须有默认值或 migration,旧 checkpoint fixture 仍可恢复;
  • 不能为了接入 MCP/Queue/HTTP 而让 Provider/MCP SDK object 穿透 Runtime domain;
  • 新增 capability 必须通过 Adapter / Policy 接入,不绕过既有 validation、execution record 与 trace;
  • 上一周 failure injection 必须继续通过,本周只新增新的失败维度。

Golden Run:同一个任务贯穿 13 周 ​

固定案例:修复 auth token expiration bug,并运行测试。

阶段观察点
W1User → Model → Action → Observation 的最小边界
W2search_code/read_file/apply_patch Tool Schema
W3parallel search、timeout/cancel、ToolResult→Observation
W4哪些文件/Tool/state 被投影到 Context
W5retrieval evidence 怎样进入 Context
W6Observation 如何推动 Run/Turn state
W7修复计划怎样变为 validated Graph
W8patch/test 是否安全 retry;何时 approval
W9patch 成功后 checkpoint 前 crash 怎样恢复
W10API 启动、stream 断开、Queue redelivery、worker fencing
W11workspace/shell/MCP capability 与安全边界
W12同一 run_id 的 model/tool/retry/queue trace + eval + benchmark
W13rolling deploy 中 active run 如何 drain/resume

每周页面必须回答三项 ​

  1. 本周以前系统做不到什么? — 作为学习动机;
  2. 本周只新增哪些抽象? — 防止提前过度设计;
  3. 本周新增的 failure injection 是什么? — 证明能力不是 happy path。

闭环验收 ​

AI Engineering · Agent · Harness · Backend Systems