# agent-value-alignment (Unified Skill)

## Core Instructions (SKILL.md)

# Agent Value Alignment

Audit or establish the closed loop that anchors an AI agent's (or tool's) behavior to its intended value. Operates across four layers — Value, Governance, Runtime Prompt, Infrastructure — bound by a shared vocabulary and traceability matrix.

The core thesis: **goal drift (agent) and value drift (team) are the same failure at different scales** — the absence of structured, repeatable mechanisms to anchor behavior to intent. The governing invariant is that *every governance decision must have a corresponding prompt clause, and every prompt clause a corresponding eval test*.

## Setup

- Determine the **target**: the agent or tool being aligned.
- Determine the **mode**:
  - **AUDIT** — evaluate an existing agent/tool against the four-layer framework; produce a gap report.
  - **ESTABLISH** — stand up the artifacts for a new or un-governed agent/tool, following the one-week adoption sequence.
- If a value proposition, system prompt, eval suite, or behavioral spec already exists for the target, collect them before starting.

For **AUDIT** mode, each "Confirm" step is a check, not a gate: a missing or
failing artifact is recorded as a finding (not a blocker) — continue through
all layers before reporting. For **ESTABLISH** mode, follow the day-map below.

## Procedure

1. **Layer 1 — Value.** Confirm the target has a *falsifiable* value proposition (user segment + outcome + amount + timeframe + metric + baseline), kill criteria with a named owner, and a measurement plan distinguishing leading from lagging indicators. Read `references/layer-1-value.md`.
2. **Layer 2 — Governance.** Confirm a Behavioral Specification Document (BSD) / Agent Charter defines authorized and prohibited behavior, escalation triggers, and success metrics; confirm scope gates and review ceremonies exist. Read `references/layer-2-governance.md`.
3. **Layer 3 — Runtime Prompt.** Confirm the system prompt anchors the objective (Goal Sandwich), fences scope, encodes Minimal Footprint, includes drift-detection/self-check loops, and is verified by a value-focused eval suite (not output-correctness evals). Read `references/layer-3-runtime-prompt.md`.
4. **Layer 4 — Infrastructure.** Confirm alignment-oriented checkpointing (not just fault tolerance), a deliberate memory architecture, HITL interrupts at irreversible boundaries, and drift observability. Read `references/layer-4-infrastructure.md`.
5. **Integration.** Confirm the layers are bound: shared vocabulary document, verbatim propagation of the value proposition across system prompt + BSD + review agenda, traceability matrix linking each governance decision → prompt clause → eval test, and eval results feeding the review cadence. Read `references/integration.md`.
6. **Report.** Use `references/report-template.md`. The verdict and
   layer-status vocabulary (ALIGNED/PARTIAL/NOT_ALIGNED; 🟢/🟡/🔴) are defined
   in the report template — size findings accordingly as you go, not just at
   report time.

For **ESTABLISH** mode, the one-week sequence is distributed across the layer
references — each contains its own ESTABLISH days:

| Day | Artifact(s) | Reference |
|---|---|---|
| 1 | Falsifiable Value Proposition | `references/layer-1-value.md` |
| 2 | Kill Criteria + Agent Charter (+ BSD draft) | `references/layer-1-value.md`, `references/layer-2-governance.md` |
| 3 | Goal Sandwich system prompt | `references/layer-3-runtime-prompt.md` |
| 4 | 10 eval test cases | `references/layer-3-runtime-prompt.md` |
| 5 | Value Realization Review scheduled + decay dashboard | `references/layer-4-infrastructure.md` |

Produce the minimum viable artifact set day-by-day, then run steps 5–6 to
confirm the loop is closed.

If artifacts already exist but are incomplete or unfalsifiable (the common
partial-governance case), run **AUDIT** first to baseline the gaps, then
ESTABLISH the missing pieces — do not treat Day 1's "stop if the team cannot
agree on the VP" as blocking a retrofit of an existing vague VP.

## Rules

- Drift is the default assumption. The burden of proof is on evidence that behavior is anchored to intent — not on evidence that it has drifted.
- Never accept a value proposition that cannot be falsified. "Help users be more productive" is a wish, not an objective.
- Never accept an eval suite that measures output correctness without also measuring outcome achievement ("did this help the user accomplish their goal?").
- Never accept kill criteria without a named decision owner and a specific threshold + date. Unsigned criteria are suggestions.
- Flag any governance decision with no corresponding prompt clause (not enforced), any prompt clause with no eval test (not verified), and any eval result not reviewed in governance (not acted upon).
- Cite the specific artifact and location for every finding. Do not fabricate gaps.
- This skill is advisory — it produces a report and recommended actions; it does not itself edit prompts, evals, or governance documents.
- Where sources disagree on facts (e.g., paper dates), follow the more authoritative citation and note the discrepancy.

## Source

Derived from `content/research-synthesis-agent-value-alignment.md`, which synthesizes three source reports on value delivery verification, agentic alignment/checkpointing, and prompt engineering frameworks. Consult the synthesis for full citations, framework attributions, and worked examples.


---

## Reference: examples.md

# Agent Value Alignment — Worked Examples

Optional reference. Use these to calibrate severity ratings and report format. Two examples: a sample finding showing the severity rubric applied, and a sample completed layer-score table.

## Example 1: A sample finding with severity rationale

A team is auditing "PRBot," an AI agent that reviews pull requests. They have a system prompt and an eval suite but no kill criteria and no BSD.

#### [L1-FINDING-002] [HIGH] — Kill Criteria Document: not present
- **Layer:** 1 — Value
- **What's wrong:** No kill criteria exist. The team has a falsifiable VP ("PRBot will reduce PR time-to-merge by 30% within 90 days, measured by median GitHub PR open→merge vs. the 90-day pre-deployment baseline") but no pre-committed stopping conditions.
- **Why it matters:** Without signed kill criteria, sunk-cost rationalization will keep the tool alive past its evidence threshold — the #1 value-level failure mode (root cause: rationalization under sunk cost). The VP is falsifiable but cannot be honestly *falsified* in practice.
- **Recommendation:** Author ≥3 kill criteria (metric + threshold + date + named owner + consequence) and have them signed by Product and Engineering leads before the next deployment. Owner: Product Manager. Due: 2026-08-14.
- **Invariant violated:** none directly (this is a Layer-1 gap, not an invariant break).
- **Severity rationale:** HIGH, not CRITICAL — a Layer-1 artifact is missing and the loop cannot close without it, but no irreversible-action boundary is unguarded and the invariant is not yet broken.

## Example 2: A CRITICAL finding (invariant break)

#### [INT-FINDING-001] [CRITICAL] — Traceability gap: 3 BSD decisions have no prompt clause
- **Layer:** Integration
- **What's wrong:** The BSD prohibits storing PII (§3.2), requires confirmation before any external communication (§4.1), and limits file access to `/workspace/data/` (§3.1). The system prompt encodes the file-access limit (Scope Fence) but contains **no clause** for the PII prohibition or the external-communication confirmation requirement.
- **Why it matters:** Two governance decisions are unenforced at runtime. The agent could store PII or send external communications with no prompt-level constraint — a direct break of the governing invariant ("every governance decision must have a corresponding prompt clause"). One unguarded boundary (external communication) is irreversible.
- **Recommendation:** Add a Minimal Footprint clause and an explicit prohibition to the system prompt's Scope Fence for both behaviors; add eval cases that verify refusal/escalation on PII-storage and external-comm triggers. Owner: Agent Engineer. Due: immediately (blocks deployment).
- **Invariant violated:** enforcement (decision exists, no prompt clause).

## Example 3: A completed layer-score table

For the same PRBot audit, after all layers reviewed:

| Layer | Status | Critical gaps |
|---|---|---|
| 1 — Value | 🟡 | 0 |
| 2 — Governance | 🔴 | 1 |
| 3 — Runtime Prompt | 🟢 | 0 |
| 4 — Infrastructure | 🟡 | 0 |
| Integration | 🔴 | 1 |

**Overall verdict:** NOT_ALIGNED

Rationale: Layer 2 is 🔴 because the BSD is missing entirely (a HIGH finding, but it escalates the layer to 🔴 since the loop cannot close without it — per the rubric, treat a missing foundational Layer-1/2 artifact as 🔴 for the layer). Integration is 🔴 due to the CRITICAL invariant break in Example 2. Layers 1 and 4 are 🟡 (failures with compensating controls: Layer 1 has no kill criteria but the VP is falsifiable; Layer 4 has observability but no decay dashboard yet). Layer 3 passes all checks.

## Inter-rater calibration notes

- A missing BSD is HIGH as a finding but 🔴 as a layer status — the severity describes the finding; the layer status follows the rubric (≥1 CRITICAL → 🔴; any HIGH/MEDIUM → 🟡). Reconcile by treating a missing foundational Layer-1/2 artifact as 🔴 for the layer even when the finding itself is HIGH, because the loop cannot close without it.
- An unfalsifiable VP is always HIGH (never MEDIUM) — the rubric's "Layer-1 artifact missing/unfalsifiable" clause.
- A missing eval test for a prompt clause is CRITICAL only if the clause guards an irreversible action; otherwise MEDIUM (check fails, compensating control = the prompt clause itself exists and may be manually reviewed).


---

## Reference: integration.md

# Integration: Closing the Loop

**Question:** Are the four layers bound into one auditable system — or are they four separate piles of advice?

The governing invariant:

> **Every governance decision must have a corresponding prompt clause, and every prompt clause must have a corresponding eval test.** If a decision exists in the BSD but has no prompt clause, it is not enforced. If a prompt clause exists but has no eval test, it is not verified. If an eval test exists but is not reviewed in governance, it is not acted upon.

## The vocabulary bridge

The most common integration failure is **vocabulary fragmentation**: the system prompt says "PRIMARY OBJECTIVE," the eval suite says "goal," the value realization review says "value delivery" — all referring to the same thing without anyone knowing it. When language is inconsistent, no behavioral observation can be traced to a governance decision, and no governance decision can be translated into a prompt change.

**Fix:** a shared vocabulary document mapping each concept across system prompt / eval / governance terms, plus a **traceability matrix** linking each governance decision → prompt clause → eval test ID.

| Concept | System prompt term | Eval term | Governance term |
|---|---|---|---|
| What the agent is trying to do | PRIMARY OBJECTIVE | goal | value proposition |
| What the agent is not allowed to do | PROHIBITED ACTIONS | scope violation | out of scope |
| When the agent should stop and ask | ESCALATION TRIGGER | escalation behavior | human oversight event |
| Whether the agent helped the user | OBJECTIVE CHECK | goal achievement score | value delivery |
| Agent doing something unintended | SCOPE BOUNDARY REACHED | behavioral regression | alignment gap |
| Agent gradually shifting behavior | DRIFT CHECK (inner-monologue) | regression in eval suite | behavioral drift |

## Verbatim propagation of the value proposition

The value proposition statement must appear **verbatim** in three places:

1. The system prompt — as the PRIMARY OBJECTIVE in the Goal Sandwich.
2. The BSD — Section 1 (Purpose and Objective).
3. The Value Realization Review agenda — read aloud at the opening of every review.

When the value proposition changes, all three must be updated together, the eval suite re-run, and a Value Realization Review scheduled to re-baseline. This creates a direct, auditable chain from governance commitment to runtime behavior.

## Automated value reporting

Agents can be prompted to emit a structured JSON value report at the end of each task:

```json
{
  "task_id": "...",
  "original_objective": "[restate]",
  "goal_alignment": { "score": 1-5, "reasoning": "...", "gaps": "..." },
  "scope_compliance": { "compliant": true/false, "violations": "...", "escalations": "..." },
  "user_value_delivered": { "score": 1-5, "user_can_now": "..." },
  "resource_usage": { "tools_called": N, "steps_taken": N, "footprint": "minimal|appropriate|excessive" },
  "flags": { "requires_human_review": true/false, "reason": "..." }
}
```

These fields map directly to eval dimensions, feeding the decay detection dashboard and providing evidence for Value Realization Reviews — closing the loop with minimal manual data collection.

## The eval suite as the bridge

The eval suite is the load-bearing structure between Layers 2 and 3. It must be run:

- On every prompt change (before deployment).
- On every model update (before deployment).
- Weekly (as part of the decay dashboard).
- Before every Value Realization Review (as evidence).
- After every behavioral incident (to characterize the failure).

**Sprint-review gate:** do not demo new agent capabilities if eval pass rate has declined from the previous sprint without explanation.

**Value Realization Review question:** *is the relationship between eval pass rate and the primary value metric what we expected? If not, what does that tell us about our eval design?*

## AUDIT checks (integration-specific)

- [ ] Shared vocabulary document exists and is used across all artifacts.
- [ ] Traceability matrix links every governance decision → prompt clause → eval test.
- [ ] Value proposition appears verbatim in system prompt, BSD §1, and review agenda.
- [ ] Eval suite runs on the five required triggers above.
- [ ] Eval results are reviewed in sprint reviews and Value Realization Reviews.
- [ ] (If applicable) automated value reports feed the decay dashboard.


---

## Reference: layer-1-value.md

# Layer 1 — Value: Defining and Verifying the Promise

**Question:** What value was promised, to whom, and how will we know if it was delivered?

## The falsifiable value proposition

A value proposition is only useful if it can be proven false. The canonical structure:

> "[TOOL] will enable [USER SEGMENT] to [OUTCOME VERB] [OUTCOME OBJECT] by [AMOUNT] within [TIMEFRAME], as measured by [METRIC]."

Append **"compared to [BASELINE]"** as a falsifiability requirement — improvement is meaningless without a baseline.

If you cannot fill every slot, you have an aspiration, not a value proposition. Post-hoc rationalization ("we missed the metric but look at this other one") is the most common honesty failure.

## Output vs. outcome vs. impact

- **Outputs** — what was built (within the team's control; always looks successful).
- **Outcomes** — what changed in user behavior.
- **Impact** — what changed at the business level.

Measure at all three. Outputs are leading indicators of outcomes; outcomes of impact. A tool with high output volume but low outcome change is in the **activity trap**.

## Measurement disciplines

1. **Resist Goodhart's Law.** When a measure becomes a target, it ceases to be a good measure. Four failure types: regressive, extremal, causal, adversarial (Cobra Effect). Counter: multiple metrics in healthy tension; metrics requiring genuine behavioral change; monitor trends not point values; audit when metrics move unexpectedly.
2. **Leading before lagging.** A code assistant's "acceptance rate without modification" is *leading*; "PR cycle time reduction" is *lagging*. By the time lagging confirms failure, months are sunk.
3. **Counterfactual measurement.** Gold standard = randomized controlled trial. Where infeasible: difference-in-differences, regression discontinuity, instrumental variables. Self-reported productivity gains are not evidence.

## Frameworks (when to reach for which)

- **JTBD / ODI** (Ulwick/Christensen) — define value solution-free; 86% success vs. 17% industry avg.
- **Impact Mapping** (Adzic) — trace every deliverable → impact → goal; kills unmapped features.
- **Opportunity Solution Trees** (Torres) — outcome → opportunities → solutions → assumption tests.
- **HEART + GSM** (Google) — force multi-dimensional measurement; resists engagement over-indexing.
- **North Star Metric** — single leading indicator; not directly movable by the team.
- **OKRs** — outcome-based key results, quarterly, divorced from compensation.
- **Working Backwards / PR-FAQ** (Amazon) — articulate success + the hard questions before building.
- **Benefits Realization Management** (Jenner) — formal benefits register, independent verification.

## Accountability

- **Kill criteria** — written *before* development; four components: metric, threshold, timeframe, consequence; named decision owner; signed. Unsigned/vague criteria are not criteria.
- **Independent evaluation** — the building team must not be the sole measurer (structural conflict of interest, not honesty).
- **Value realization reviews** — quarterly ceremony (distinct from sprint review/retro): restate VP → review evidence → gap analysis → causal analysis → explicit continue/pivot/kill.
- **Portfolio governance** — forces honest comparison across tools; sunsetting is a pre-committed plan, not a political judgment.

## Activity trap diagnostic

Ask: *if this tool disappeared tomorrow, what would users lose?* If the honest answer is "not much," the tool is busy but not valuable.

## AUDIT checks

- [ ] Falsifiable VP exists with all slots filled (including baseline).
- [ ] Kill criteria exist, are signed, pre-development, with named owner + specific threshold + date.
- [ ] Measurement plan tracks leading *and* lagging indicators.
- [ ] Value realization review is scheduled on a cadence shorter than the drift half-life.
- [ ] Evaluation is independent of the building team.

## ESTABLISH (Day 1–2)

- Day 1 (2h): Write the falsifiable value proposition. *If the team cannot agree on the single sentence, the disagreement is the most important discovery — do not proceed.*
- Day 2 (1h): Write ≥3 kill criteria with specific numbers, dates, owners; sign them.


---

## Reference: layer-2-governance.md

# Layer 2 — Governance: Specifying Authorized Behavior

**Question:** What behavior is authorized, prohibited, and escalated — and who reviews it?

## Why agent governance is different

In traditional software, behavior = code; correct code ⇒ correct behavior. In agent systems, behavior emerges from code × model × prompts × tools × execution context — none fully specifiable in advance. Traditional governance (code review, automated tests, deploy gates) is necessary but insufficient. You need mechanisms that detect behavioral drift *even when the code is correct*.

## Core artifacts

| Artifact | Purpose |
|---|---|
| **Behavioral Specification Document (BSD)** | Authoritative reference for *how* the agent should behave; source of truth for prompts, evals, audits. Covers: purpose, authorized behaviors, prohibited behaviors, escalation criteria, behavioral principles, success metrics, known limitations/edge cases. |
| **Agent Charter** | One-page "constitution" readable by non-technical stakeholders: purpose, scope, principles, prohibitions, escalation, success/failure criteria, governance. |
| **Kill Criteria Document** | Pre-committed stopping conditions, signed before development (see Layer 1). |
| **Behavioral ADRs (BADRs)** | Record of intent for significant behavioral decisions — context, decision, options, rationale, consequences, implementation. Distinguishes bug from feature when behavior is questioned. |
| **Falsifiable Value Proposition** | Propagates verbatim into BSD §1, system prompt, and review agenda (see Integration). |
| **Pre-mortem** | Imagine failure in advance; generate mitigations with owners. |

## The "scope creep by proxy" anti-pattern

The signature governance failure for agents: capabilities expand through accumulation of *individually reasonable* changes — tool proliferation, edge-case capability expansion, behavioral drift from model updates, prompt accumulation. Each change is justified by a use case; the cumulative expansion was never approved. Hard to detect in real time.

**Countermeasure:** explicit scope gates at defined milestones — behavioral scope, tool access, data access, output scope reviews — with deviations either formally approved (via BADR) or rolled back.

## Review ceremonies (augment traditional sprints)

- **Behavioral regression testing** at each sprint boundary — checks alignment, not just functional correctness.
- **Red-team reviews** at regular intervals — attempt to induce goal drift, specification gaming, scope violations.
- **Alignment reviews** by team members *not* in the recent sprint — external perspective catches gradual drift invisible to builders.
- **Incident post-mortems** that classify incidents as goal drift / specification gaming / scope violation, with prevention recommendations. Treat agent behavior incidents with the rigor of security incidents.

## Open-source / distributed governance

Contributors carry different understandings of the objective; PR review is poorly suited to evaluating behavioral alignment. Best practices: versioned behavioral specs, behavioral contribution guidelines, behavioral CI/CD on every PR, designated *alignment maintainers* distinct from technical maintainers.

## Multi-agent systems

If the target is a multi-agent system (orchestrator/worker hierarchies), the
single-agent checklists are necessary but not sufficient — drift can occur at
four levels invisible to per-agent review: **orchestrator drift** (task
decomposition diverges from the objective), **worker drift** (a worker
diverges from its sub-task), **coordination drift** (orchestrator–worker
interaction diverges), and **emergent drift** (collective behavior diverges
in ways not visible at the individual level).

**Additional AUDIT checks for multi-agent targets:**
- [ ] Every worker sub-task has an explicit, verifiable specification (not a
      high-level instruction) and a defined success criterion.
- [ ] The orchestrator verifies worker results against the sub-task spec before
      incorporating them.
- [ ] Cross-agent consistency checks exist (outputs of different workers are
      consistent with each other and the overall objective).
- [ ] HITL review is configured at both orchestrator (task decomposition) and
      worker (sub-task execution) levels.

See the synthesis §5.6 for the orchestrator/worker alignment patterns and the
multi-agent alignment prompts.

## AUDIT checks

- [ ] BSD exists and is version-controlled with the same rigor as code.
- [ ] Agent Charter exists (≤1 page, readable by non-technical stakeholders).
- [ ] Prohibited behaviors are explicit (negation), not implicit.
- [ ] Escalation triggers are specific (not "when uncertain" but the 7 named triggers — see Layer 3).
- [ ] Scope gates exist at milestones; deviations require BADR.
- [ ] Behavioral regression tests run at sprint boundaries.
- [ ] Red-team reviews occur on a cadence.
- [ ] *(Multi-agent only)* Sub-task specs, result verification, and cross-agent
      consistency checks exist (see Multi-agent systems above).

## ESTABLISH (Day 2)

- Day 2 (1h): Write the Agent Charter (≤1 page). Draft the BSD (can be expanded later, but purpose/scope/prohibitions/escalation must exist from day one).


---

## Reference: layer-3-runtime-prompt.md

# Layer 3 — Runtime Prompt: Enforcing Alignment at Execution Time

**Question:** Do the runtime instructions actually anchor behavior to the objective — and are they verified?

## The foundational constraint: lost in the middle

LLM performance follows a **U-shaped curve**: models attend well to the beginning (primacy) and end (recency) of context and systematically underweight the middle (Liu et al., 2023; TACL 2024). Context length *alone* degrades performance 13.9–85% even with perfect retrieval (Du et al., 2025). An objective stated once at the top of a long system prompt, after tool results accumulate, is buried in the middle and statistically underweighted. This is structural, not an intelligence failure.

## Goal-anchoring patterns

| Pattern | Mechanism | Addresses |
|---|---|---|
| **Goal Sandwich** | Primary objective verbatim at top *and* bottom of system prompt | U-shaped attention |
| **Objective Echo** | Agent restates objective before each major action | Active retrieval vs. passive presence |
| **Scope Fence** | Explicit negation of prohibited actions | Implicit-scope unreliability |
| **Minimal Footprint Clause** | Resource/side-effect constraints | Instrumental convergence pressure |
| **Constitutional self-critique** | Critique-then-revise against stated principles (Bai et al., 2022) | Goal alignment at output time |
| **Persistent objective store** | Objective retrieved from outside the context window | Compression/summarization drift |

**Instruction hierarchy:** primary objective in first and last 5 lines; scope fence + minimal footprint early; operational detail in the (lower-attention) middle; constitutional critique principles near the end. Any constraint that must be reliably followed appears in the first or last 20% of the prompt.

**Critical failure mode:** placing safety/scope constraints only in the middle of a long prompt.

## Drift detection and self-check loops

- **ReAct + alignment verification** — extend Thought → Action → Observation with OBJECTIVE/PROGRESS/ALIGNMENT/SCOPE checks before each action. (ReAct outperformed pure CoT by 34% on ALFWorld.) *Failure mode:* context accumulation; mitigate with observation compaction, step limits, periodic goal re-injection.
- **OODA self-correction** — Observe/Orient/Decide/Act as a continuous self-monitoring loop after each major action.
- **Uncertainty/escalation prompts** — structured decision rules forcing stop-and-ask on: ambiguity, scope uncertainty, irreversibility, unexpected state, conflicting instructions, high stakes, confidence below threshold. Implements Anthropic's Minimal Footprint Principle.

## Eval design — the most important shift

From **output correctness** ("is the answer right?") to **outcome achievement** ("did this help the user accomplish their goal?"). A value-focused eval scores 0–4 (DID NOT HELP → FULLY ENABLED) and requires a gap analysis: what would reach Level 4?

**Judges:**
- **LLM-as-judge** (Zheng et al., 2023) — ~80% agreement with humans, matching human-human. *Mandatory mitigations:* evaluate pairwise in both orders (position bias); instruct against verbosity bias; **never use the same model as both agent and judge** (self-enhancement bias up to 25%).
- **G-Eval** (Liu et al., 2023) — chain-of-thought form-filling; generates evaluation steps before scoring.

**Behavioral regression tests** detect when a model/prompt/context change breaks previously-passing behavior. Require a golden dataset with ≥30% "failure class" examples (inputs that should trigger escalation/refusal/scope denial) — happy-path-only suites cannot detect regression in failure handling.

## Meta-prompting for goal fidelity

- **Reflexion** (Shinn et al., 2023) — verbal self-reflection in an episodic buffer, prepended to retries; 91% pass@1 on HumanEval vs. GPT-4's prior 80%.
- **Self-consistency** (Wang et al., 2023) — sample multiple reasoning paths, select the most goal-aligned; +17.9% on GSM8K.
- **Checkpoint output prompts** — force the agent to emit current goal/state/confidence before each major action (auditable record).
- **Meta-alignment prompts** — agent generates its own task-specific checklist before beginning.

## The alignment trade-off

- **RLHF / Constitutional AI** — embeds constraints in weights; robust to drift/injection; expensive, can be gamed if reward model is imperfect.
- **Prompt-based** — flexible, interpretable; subject to drift, overridable.

Production uses both: RLHF/CAI for foundational constraints, prompt-based for task-specific scope. The Claude model specification (principal hierarchy: Anthropic → operators → users → agent; Minimal Footprint, corrigibility, consistency) is the current state of the art.

## AUDIT checks

- [ ] Objective appears verbatim at top *and* bottom (Goal Sandwich); identical wording.
- [ ] Prohibited actions are explicit negations, not implicit.
- [ ] Minimal Footprint clause present.
- [ ] Escalation triggers are the 7 named types, not "when uncertain."
- [ ] Eval suite measures outcome achievement, not just output correctness.
- [ ] Golden dataset includes ≥30% failure-class examples.
- [ ] LLM-as-judge uses a *different* model than the evaluated agent.
- [ ] Behavioral regression tests run on every prompt/model change.

## ESTABLISH (Day 3–4)

- Day 3 (2h): Implement Goal Sandwich + Scope Fence (≥5 prohibited) + Minimal Footprint + Escalation Triggers. This is the minimum viable goal-anchoring prompt.
- Day 4 (3h): Write 10 eval cases — 5 happy-path, 3 scope-boundary, 2 drift. Run manually; fix failures before deploy.


---

## Reference: layer-4-infrastructure.md

# Layer 4 — Infrastructure: State, Memory, and Observability

**Question:** How is state preserved, memory managed, and drift observed in production?

## Checkpointing: fault tolerance vs. alignment

- **Fault-tolerant checkpointing** asks: "can we resume after a crash?"
- **Alignment-oriented checkpointing** asks: "is the agent still pursuing the right goal, and can we roll back to a known-good state if it has drifted?"

The latter requires **goal-state verification at each checkpoint**, not just state persistence.

**LangGraph** (1.0 GA, Oct 2025) is the most mature production option: checkpoints at every super-step, pluggable backends (`PostgresSaver` for production), first-class `interrupt()` API for HITL resumable via `Command(resume=...)`, time-travel debugging, state mutation. Powers agents at ~400 companies (LinkedIn, Uber, Replit) with 8% latency overhead. **MemGPT/Letta** provides OS-inspired tiered memory (main context / recall DB / archival vector store) with LLM-managed tier movement.

Caveat: OpenAI Agents SDK and AutoGen lack native durable execution — teams must add Temporal/DBOS. Google ADK loses in-memory session state on Cloud Run restarts. Configure persistent storage from day one; retrofitting is significantly harder.

## Memory architecture as a first-class concern

"Memory deserves the same level of engineering investment as the LLM itself" (Du, 2026). Three dimensions:

- **Temporal scope** — working (subject to lost-in-the-middle), episodic (concrete experience records), semantic (abstracted knowledge; over-generalization risk), procedural (reusable skills; Voyager without its skill library showed 15.3× slower progression).
- **Representational substrate** — context-resident text (compression drift), vector stores, structured stores, executable repositories.
- **Control policy** — heuristic, prompted self-control (MemGPT; silent orchestration failure risk), learned/RL (e.g., AgeMem, per Du 2026; most capable, requires training infra).

Empirical stakes: Generative Agents without reflection degenerated to repetitive responses within 48 simulated hours; on MemoryArena (2026), active memory agents achieved 80%+ task completion vs. ~45% for long-context-only baselines. Models near-saturating LoCoMo plummet to 40–60% on MemoryArena — a deep gap between passive recall and active, decision-relevant memory.

## Safe stopping, rollback, and HITL

Safe stopping criteria (distinct from error conditions): uncertainty threshold, scope boundary, resource limit, anomaly detection, human review trigger. LangGraph's `interrupt()` provides the mechanism; the *design* challenge is choosing when to pause, what to surface, and how to incorporate feedback: approve / modify / rollback-and-retry / abort.

**Cost calculus favors false positives:** unnecessary human reviews are far cheaper than undetected goal drift leading to irreversible actions. Configure HITL at all irreversible action boundaries.

## Observability for drift detection

**Quantitative signals:** task completion rate, goal retention score, scope violation rate, rollback rate, HITL trigger rate, token efficiency (useful work / total tokens — flags unproductive spinning).

**Qualitative signals:** execution traces, reasoning traces, state snapshots, anomaly detection.

**Decay detection dashboard** tiers these into:
- Behavioral health (weekly) — eval pass rate, goal achievement, scope violations, escalations, errors.
- Adoption health (weekly) — WAU, tasks/user, retention, satisfaction.
- Value health (monthly) — primary value metric, cost/task, cost/user.

**Automatic review triggers:** eval pass rate drops >10% week-over-week; any red metric persists 2+ weeks; model or prompt change deployed (re-run full eval suite).

## AUDIT checks

- [ ] Checkpointing is alignment-oriented (goal-state verification, not just resume).
- [ ] Persistent backend configured from day one (not in-memory in production).
- [ ] Memory architecture is a deliberate choice across all three dimensions, not an afterthought.
- [ ] HITL interrupts configured at all irreversible action boundaries.
- [ ] Tracing/observability deployed before production (LangSmith or equivalent).
- [ ] Decay detection dashboard exists with the three tiers and automatic triggers.
- [ ] Full eval suite re-runs on every model/prompt change.

## ESTABLISH (Day 5)

- Day 5a (30 min): Schedule the first Value Realization Review for 90 days out.
- Day 5b (30 min – ½ day): Stand up the decay detection dashboard. *30 min only if metrics are already instrumented; up to half a day if instrumentation must be built.* Do not skip — without it, drift cannot be detected before lagging outcome metrics confirm it.


---

## Reference: report-template.md

# Agent Value Alignment — Report Template

Use this for both AUDIT and ESTABLISH modes. Replace `[ ]` placeholders.

## Severity rubric

Apply consistently to every finding so two auditors rate the same gap alike:

| Severity | Definition |
|---|---|
| **CRITICAL** | Breaks the governing invariant (an unenforced decision / unverified clause / unacted eval result) **or** leaves an irreversible-action boundary unguarded (no HITL, no rollback) |
| **HIGH** | A Layer-1 or Layer-2 artifact is missing, unfalsifiable, or unsigned (e.g., no kill criteria, no BSD, vague VP) — the loop cannot close without it |
| **MEDIUM** | An AUDIT check fails but a compensating control exists (e.g., eval suite present but golden dataset lacks failure-class examples) |
| **LOW** | Cadence or documentation gap (e.g., Value Realization Review not yet scheduled, dashboard not yet live) |

A layer is 🔴 if it has ≥1 CRITICAL finding; 🟡 if it has ≥1 HIGH/MEDIUM but no CRITICAL; 🟢 if all AUDIT checks pass.

---

## Agent Value Alignment Report

**Target:** [agent/tool name and version]
**Mode:** [AUDIT | ESTABLISH]
**Date:** [YYYY-MM-DD]
**Reviewer:** [name/role]
**Sources consulted:** [system prompt, BSD, eval suite, dashboard, etc.]

---

### Executive Summary

[2–4 sentences. State the single most important finding: is the closed loop intact, and if not, where is it broken?]

**Overall verdict:** [ALIGNED | PARTIAL | NOT_ALIGNED]

### Layer Scores

| Layer | Status | Critical gaps |
|---|---|---|
| 1 — Value | [🟢/🟡/🔴] | [count] |
| 2 — Governance | [🟢/🟡/🔴] | [count] |
| 3 — Runtime Prompt | [🟢/🟡/🔴] | [count] |
| 4 — Infrastructure | [🟢/🟡/🔴] | [count] |
| Integration | [🟢/🟡/🔴] | [count] |

Status: 🟢 all AUDIT checks pass · 🟡 ≥1 check fails, no critical · 🔴 ≥1 critical gap.

---

### Findings

#### [LAYER-N-FINDING-001] [CRITICAL|HIGH|MEDIUM|LOW] — [Location]
- **Layer:** [1 Value | 2 Governance | 3 Runtime | 4 Infra | Integration]
- **What's wrong:** [specific gap, citing the artifact and location]
- **Why it matters:** [which root cause or failure mode this enables]
- **Recommendation:** [specific action — which artifact to create/change, by whom]
- **Invariant violated:** [enforcement | verification | action] (if applicable)

[repeat for each finding]

---

### Invariant Check

The governing invariant: *every governance decision → prompt clause → eval test → governance review*.

- Decisions with no prompt clause (not enforced): [list, or "none"]
- Prompt clauses with no eval test (not verified): [list, or "none"]
- Eval results not reviewed in governance (not acted upon): [list, or "none"]

---

### Value Proposition (as currently stated)

> "[verbatim current VP, or 'NOT DEFINED']"

**Falsifiable?** [Yes/No — which slot is missing]
**Propagated verbatim across prompt + BSD + review agenda?** [Yes/No]

---

### Kill Criteria Status

[Cite each criterion, its threshold, the current value, and status: on-track / at-risk / triggered. Or "NOT DEFINED".]

---

### Recommended Actions (priority order)

1. [Action] — Owner: [role] — Layer: [N] — Due: [date]
2. [Action] — Owner: [role] — Layer: [N] — Due: [date]
3. [Action] — Owner: [role] — Layer: [N] — Due: [date]

### Next Review

**Value Realization Review scheduled:** [date, or "NOT SCHEDULED — schedule within 90 days"]
**Decay dashboard live:** [Yes/No]

---

### For ESTABLISH mode only: Minimum Viable Artifact Set

| Artifact | Status | Location |
|---|---|---|
| Falsifiable Value Proposition | [Done/In progress/Not started] | [path] |
| Kill Criteria (signed) | | |
| Agent Charter | | |
| Goal Sandwich system prompt | | |
| 10 eval test cases | | |
| Value Realization Review scheduled | | |
| Decay detection dashboard | | |

*Do not consider the loop closed until every artifact above is "Done" AND the invariant check shows zero unenforced/unverified/unacted items.*


---