Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

Document status: Draft, not yet approved. The source file livin-ears-spec.md is authoritative.

EARS Specification: Livin

Livin (as in “livin’ on the edge”) is a cross-language CLI that checks whether a test suite actually exercises the boundary values a piece of code’s own shape implies exist — zero, empty, singleton, duplicate, overflow, every arm of a sum type — rather than only the values a property-based generator happens to sample.

This document is the normative product-requirements source. It is self-contained and assumes no prior conversation, no other document, and no familiarity with any other tool, though Section 1 notes where Livin optionally consumes data already produced by a composability checker called Vampiro, if one is present. OpenSpec changes select implementable slices of these requirements; if an OpenSpec artifact conflicts with this document, this document governs until it is explicitly amended.


0. Background & Motivation

Property-based testing (PBT) generates many inputs and checks that a declared property holds for all of them, which is effective for finding unanticipated combinations of ordinary values that happen to violate a law. It is systematically weak at a different, complementary thing: boundary values are a vanishingly small target in a type’s overall space, so a uniform or shrinking-based generator can run thousands of iterations and still never construct the empty collection, the duplicate key, the exact-max-cardinality case, or the identity element — precisely the values most likely to break an invariant, because they are the values most likely to have been handled by a special case someone forgot to write.

Fixed example-based fixtures fix this for known laws (an associativity check can and should include an explicit empty-input example), but that only covers code with a declared law suite. Most functions in a codebase have no declared algebraic law at all, and still have a shape — a domain and codomain with numeric ranges, collection structures, sum types, and so on — that implies a set of boundary values worth exercising regardless of whether any formal property is declared for that function.

Livin targets that gap specifically: given a function’s shape, what are the canonical boundary instantiations for a value of that shape, and does the test suite exercise them anywhere? This is a coverage question about tests, not a composability question about call sites, which is why it is a separate tool rather than an axis inside a composability checker: a composability checker asks whether two pieces fit together; Livin asks whether the test suite hit the values most likely to reveal that they don’t. It is also distinct from mutation testing (which asks “would the suite catch an injected bug”) — Livin asks whether the suite reaches the values most likely to contain a bug in the first place, independent of whether any mutation was ever injected.

Livin explicitly does not cover concurrency-interleaving edge cases (races, retry-under-concurrent-write, partial-failure interleavings). Those are boundaries of execution order, not of a value’s shape, require a different extraction entirely, and are better addressed by a model-checking tool (e.g. TLA+) than by test-suite coverage analysis.


1. Scope & Definitions

  • Shape: a structural signature of a value — described by shape kind (below) plus enough structural detail (numeric range, collection element type, union arms, field set) to determine which boundary values apply to it. Livin does not require full nominal types, only this structural description.
  • Shape kind: a closed-but-extensible taxonomy tag on a shape: numeric, collection, sum/union, recursive/self-referential, string/text, record/product, or opaque (shape could not be classified into any other kind).
  • Boundary value: one canonical, named instantiation of a shape kind associated with disproportionate real-world defect rates — for example, for numeric: zero, the type’s minimum/maximum, the overflow point, NaN/Inf where the type admits them; for collection: empty, singleton, a duplicate-key/duplicate-element instance, the maximum declared cardinality; for sum/union: each arm, treated as a separate obligation; for recursive/self-referential: a self-reference or cycle instance; for string/text: the empty string and an encoding boundary (multi-byte or unmatched-surrogate input) where relevant.
  • Boundary catalogue: a versioned, per-shape-kind table mapping shape kind to its finite set of named boundary values, extensible the same way an idiom table is extensible, and validated by conformance fixtures.
  • Literal/constant-expression match: Livin’s matching discipline for deciding whether a test exercises a boundary value: syntactic detection of a literal value, or a well-known zero/empty/identity constructor, passed as an argument (or asserted as a produced value) at a call site whose shape includes that boundary’s shape kind. This is deliberately not full symbolic execution, to keep the check deterministic.
  • Exercised: true for a given (node, boundary value) pair if some test in the scanned suite contains a literal/constant-expression match for it.
  • Obligation state: exactly one of covered (at least one definite literal/constant-expression match exists), uncovered (all candidate matches were definitely non-matches, including the case where there are no candidates), or indeterminate (no definite match exists and at least one candidate cannot be decided statically, or the language/matcher is unsupported). indeterminate is not a coverage gap and is never treated as covered or uncovered.
  • Coverage gap: a (node, boundary value) pair whose obligation state is uncovered. An indeterminate obligation is not exercised, but is not a coverage gap because available static evidence cannot establish absence.
  • Reachability class: reachable (no upstream constraint rules the boundary value out at this call site), unreachable (an upstream validation or type refinement statically rules it out — e.g. a parameter constrained to positive integers cannot receive zero), or unknown (reachability cannot be determined from available shape information).
  • Shape source: vampiro (shape data consumed from an installed composability checker’s extracted call graph) or standalone (Livin’s own reduced-precision shape extraction, used when no such checker is present). Every finding records which source produced its shape data.
  • Suite: the corpus of test files/functions Livin scans for literal/constant-expression matches.
  • Initial MVP language scope: Rust source accepted by stable rustc, with tests discovered only from built-in #[test] functions in the analyzed Cargo workspace. Macro-generated tests, doctests, external test frameworks, and every non-Rust language/framework are unsupported in the initial MVP. Unsupported files or test constructs are reported explicitly and any affected obligations are indeterminate; they are never silently skipped.

2. Ubiquitous Requirements

  • REQ-1: The tool shall obtain domain/codomain shapes for analyzed nodes, preferentially by consuming an installed composability checker’s extracted shape data when available, and shall record shape source on every run.
  • REQ-2: The tool shall classify every consumed shape into exactly one shape kind from the closed-but-extensible taxonomy in Section 1.
  • REQ-3: The tool shall maintain a versioned boundary catalogue mapping each shape kind to a finite set of named boundary values, validated by conformance fixtures using the same mechanism as an idiom-table check: a fixed set of synthetic shapes with known-correct expected boundary sets.
  • REQ-4: The tool shall scan the configured test suite for literal/constant-expression matches between test call arguments or assertions and each (node, boundary value) obligation implied by that node’s shape.
  • REQ-5: Every finding shall carry a rule ID, the file and line of the under-tested node, the boundary value’s name, its reachability class, and its shape source.
  • REQ-6: The tool shall never generate or modify test files; it shall only report coverage gaps, consistent with this tool family’s enforcement-not-generation design.

3. Event-Driven Requirements

  • REQ-7: When a node’s shape kind has one or more applicable boundary values and a given boundary value’s obligation state is uncovered, the tool shall raise a coverage-gap finding naming the unexercised boundary. It shall not raise a coverage-gap finding for covered or indeterminate obligations.

    Example: apply_discount(amount: Decimal, pct: float) where no test in the suite calls it with pct == 0 or amount == Decimal(0) raises two coverage-gap findings, one per unexercised numeric boundary.

  • REQ-8: When a shape kind is sum/union, the tool shall treat each arm as a separate boundary obligation and raise one coverage-gap finding per uncovered arm, independent of whether other arms are covered.

    Example: fn summarize(items: Vec<Order>) -> Result<Summary, SummarizeError> where the test suite calls it only with non-empty input and only exercises the Ok arm raises two findings: an unexercised empty collection boundary, and an unexercised Err arm.

  • REQ-9: When a coverage gap’s reachability class is unreachable, the tool shall still report the finding, at a downgraded severity tier, rather than omit it. If the shape’s unreachability rests on a refined type (a smart constructor, per a companion composability checker’s boundary-validation model), that downgrade shall only be trusted if the constructor’s own boundary-value coverage is itself fully exercised; otherwise the finding shall be reported at reachability class unknown rather than unreachable, since an unconfirmed refinement is not evidence the boundary actually holds.

    Example: a TypeScript parameter typed as a branded PositiveInt cannot receive zero or a negative value if the brand’s constructor genuinely rejects them. Livin still reports the gap — nothing in the suite confirms the brand’s construction actually enforces that — at reachability class unreachable only once the constructor itself has full boundary coverage; until then it is reported as unknown.

  • REQ-10: When no installed composability checker’s shape data is available, the tool shall fall back to its own standalone shape extraction (REQ-14) and shall record shape-source: standalone on every finding produced in that run.


4. State-Driven Requirements

  • REQ-11: While gate mode is gate, the tool shall exit non-zero if any reachable-class, uncovered coverage gap on a diff-touched node meets or exceeds the configured severity threshold. covered and indeterminate obligations shall not contribute to the gate exit decision.
  • REQ-12: While gate mode is guidance, the tool shall report all findings without forcing a non-zero exit.
  • REQ-13: While a node’s shape kind is opaque, the tool shall report it as unclassified explicitly, rather than silently exclude it from output.

5. Optional Feature Requirements

  • REQ-14: Where no composability checker’s shape data is installed or accessible, the tool shall perform a standalone shape extraction sufficient to classify shape kind, at reduced precision (e.g. no cross-module shape unification), so the tool can operate without a hard dependency on any other tool in the family.
  • REQ-15: Where a custom boundary catalogue entry is supplied for a project-specific shape (e.g. a domain type Money with a declared “must not be negative” boundary), the tool shall use it in place of, or in addition to, the built-in catalogue.
  • REQ-16: Where diff-scoped mode is selected, the tool shall restrict coverage-gap checking to nodes touched by the diff.

6. Unwanted Behavior Requirements

  • REQ-17: If a boundary catalogue entry has no defined literal/constant-expression matcher for the active language, the tool shall report matcher:undefined for that entry rather than silently treat it as covered or uncovered.
  • REQ-18: If the same node is reachable via multiple call paths with different reachability classes for the same boundary value, the tool shall report the most permissive (most-reachable) class observed, rather than average the classes or pick one arbitrarily.
  • REQ-19: If literal/constant-expression matching cannot determine whether a dynamically constructed test value equals a boundary value (the value is computed at runtime rather than written as a literal), and no definite match exists for the same obligation, the tool shall set the obligation state to indeterminate, report an indeterminate diagnostic naming the boundary and reason, and shall not emit a coverage-gap finding or fail the gate for that obligation. A definite match takes precedence and makes the obligation covered.

7. Complex Requirements

  • REQ-20: While shape source is standalone, when a coverage-gap finding’s reachability class would otherwise be unreachable, the tool shall instead report unknown unless standalone extraction can itself establish the upstream constraint, since standalone mode’s reduced precision cannot reliably confirm unreachability on its own.
  • REQ-21: When a node participates in a redundancy/fallback chain with divergent codomain shapes across branches, the tool shall evaluate boundary coverage independently per branch shape, rather than merging the branches into one shape before checking coverage.

8. Non-Functional Requirements

  • REQ-22: A release-build diff-scoped scan of the committed performance fixture (500 diff-touched Rust function nodes and 100 built-in #[test] functions) shall have a median wall-clock time of at most 9.0 seconds over 10 measured runs after one unmeasured warm-up, on an otherwise idle x86-64 machine with at least 2 dedicated CPU cores, 4 GiB RAM, and local SSD. The repository shall provide one documented command that builds the release binary, verifies the fixture checksum, disables network access during the scan, runs this protocol, and emits the raw timings plus median; CI shall retain that output.
  • REQ-23: The boundary catalogue and its conformance fixtures shall be versioned and reproducible byte-for-byte across repeated runs on unchanged input.

9. Traceability Notes

  • Shape acquisition and classification → REQ-1, REQ-2, REQ-10, REQ-14.
  • Boundary catalogue and conformance → REQ-3, REQ-15, REQ-23.
  • Core coverage-gap detection → REQ-4, REQ-7, REQ-8, REQ-19.
  • Reachability grading → REQ-9, REQ-18, REQ-20.
  • Reporting discipline (never silently omit, never generate tests) → REQ-5, REQ-6, REQ-13, REQ-17.
  • Operational modes → REQ-11, REQ-12, REQ-16, REQ-22.
  • Redundancy-chain interaction → REQ-21.

Appendix: Worked Examples

Numeric boundary gap (Python)

def apply_discount(amount: Decimal, pct: float) -> Decimal: ...

# suite only ever calls apply_discount(Decimal("19.99"), 0.15)

Shape kind numeric for both parameters; catalogue includes a zero boundary for each. Neither is exercised → two REQ-7 findings.

Collection + sum-arm gap (Rust)

#![allow(unused)]
fn main() {
fn summarize(items: Vec<Order>) -> Result<Summary, SummarizeError> { ... }

// tests: summarize(vec![order1, order2]) -> assert Ok(...)
}

items is collection; its empty boundary is unexercised. The return type is sum/union with arms Ok/Err; only Ok is exercised → one REQ-7 finding plus one REQ-8 finding.

Reachability downgrade, confirmed refinement (TypeScript)

type PositiveInt = number & { __brand: "PositiveInt" };
function toPositiveInt(n: number): PositiveInt | null { ... }  // smart constructor
function setQuantity(q: PositiveInt) { ... }

The zero boundary for q is structurally ruled out by the branded type, but only if toPositiveInt genuinely rejects zero. If toPositiveInt itself has full boundary coverage (its own zero case is exercised and confirmed to return null), the setQuantity finding is reported at reachability class unreachable, downgraded severity, per REQ-9. If toPositiveInt’s own boundary coverage is incomplete, the same finding is reported at reachability class unknown instead — the refinement is unconfirmed, so it isn’t trusted to rule anything out.

Standalone mode, unconfirmed reachability (Julia, no Vampiro installed)

function apply_rate(x::Float64, rate::Float64)::Float64
    x * rate
end

Without an installed composability checker, Livin’s standalone extraction (REQ-14) can see the Float64 shape but cannot confirm any upstream constraint on rate. A zero-boundary gap on rate is reported with reachability class unknown per REQ-20, not unreachable, since standalone mode cannot establish the negative case.

Per-branch coverage on a redundancy chain (Python)

try:
    data = primary_source.fetch(id)       # -> FullRecord
except SourceUnavailable:
    data = cache.get(id)                  # -> PartialRecord | None

Per REQ-21, Livin evaluates boundary coverage for FullRecord and for PartialRecord | None independently rather than as one merged shape — an empty/None boundary unexercised on the fallback branch is reported even if the primary branch’s boundaries are fully covered.