“Se me acaba el argumento y la metodología Cada vez que se aparece frente a mí tu anatomía Porque este amor ya no entiende de consejos, ni razones Se alimenta de pretextos y le faltan pantalones” — Shakira

testaruda

testaruda is a language-agnostic test selection engine. Given a code change, it computes the set of tests that must run — modeled as the transpose of a provenance-semiring dependency relation, evaluated incrementally, under a recall-first soundness invariant.

Quick Start

cargo install testaruda

# Initialize the store
testaruda init

# Discover tests through configured adapters
testaruda discover

# Select tests affected by uncommitted changes
testaruda select

# Select tests between two revisions
testaruda select --base main --head feature-branch

# Ingest run results
testaruda ingest results.json

Key Concepts

• Transpose selection: affected = Δ · (D*)ᵀ ∪ always_run
• Soundness: over-approximation — missed selections are bugs
• Semiring abstraction: Boolean (selection), Viterbi (confidence), Tropical (distance)
• Incrementality: the active path scopes evaluation to changed components; component-cache primitives exist but are not yet integrated into selection

Continue

• Follow the complete Getting Started tutorial.
• Look up commands in the CLI Reference.
• Configure language adapters and safety rules in Configuration.
• Integrate structured output using Agent Mode.

Ecosystem

testaruda is part of the charly-vibes Tool Ecosystem — the map shows how it relates to its sibling tools.

Documentation provenance

Rendered docs are generated artifacts; the repository sources are authoritative — edit the repo sources, not the rendered output.

Release Status

Generated artifact (docs.yml, from Cargo.toml + git ref). Do not hand-edit — the repo source files remain authoritative.

• Version: v0.5.7
• Built from: main @ d573796

Getting Started

Prerequisites

• A stable Rust toolchain
• Git (for change detection)

Optional: Julia adapter

To use testaruda with Julia projects, you additionally need:

• Julia 1.9+ (coverage recording requires Julia 1.12+ for LCOV tracefile support, or any 1.x for .jl.cov sidecar files)
• Testimonial.jl — the adapter lives inside this package

julia -e 'using Pkg; Pkg.add("Testimonial")'
# Link the adapter wrapper onto PATH
ln -s ~/.julia/packages/Testimonial/*/bin/testaruda-adapter-julia ~/.local/bin/

The Julia adapter discovers @testitem tests (ReTestItems/TestItems.jl) and @testset blocks (Base.Test). In a project that mixes @testitem with plain @test blocks, both the @testitem and @testset tests are available for selection. For files with no @testitem or @testset blocks, a file-level fallback is used — the entire file is run via include().

Optional: .NET adapter (titi)

To use testaruda with .NET projects, you additionally need:

• titi — a .NET monorepo orchestrator that speaks testaruda’s adapter protocol via its testaruda-adapter subcommand

# Follow the titi installation instructions at the link above
# Then configure the extension mapping in testaruda.toml:
# [adapters.extensions]
# ".cs" = "titi testaruda-adapter"

The .NET adapter is an external binary (not a workspace crate) and is opt-in (not auto-detected), following the same pattern as the Julia adapter. See the Configuration Guide for the full list of .NET extension mappings.

Installation

cargo install testaruda

Or from source:

git clone https://github.com/charly-vibes/testaruda
cd testaruda
cargo install --path .
# Installs testaruda, testaruda-adapter-rust, and testaruda-adapter-python

First Run

# Initialize store + write default config
testaruda init

# Discover tests via adapters (scans project for #[test], test_*.py, etc.)
testaruda discover
# Prints the number of test items stored in .testaruda/

# Select tests affected by uncommitted changes
testaruda select

# Select tests between two revisions
testaruda select --base main --head feature

# Machine-readable JSON plan (both orderings accepted; global flag preferred)
testaruda --json select

# Shadow mode: compute selection but signal "run all tests"
testaruda select --shadow

# Explicit file list
testaruda select --files "src/lib.rs,src/main.rs"

A successful selection prints the selected test records and the reason for any safety fallback. If no dependency data exists yet, testaruda intentionally over-selects rather than risking a missed test.

The calibration ramp (first cycles run everything — by design)

Selection quality depends on run history in the store. A cold store (fresh init) has no evidence about which tests failed recently or how tests relate to each other at runtime, so testaruda errs on the side of recall (SAFE-007): every test with no recorded history lands in the always-run set and the engine falls back to the full suite. Expect this sequence:

On the first exec cycle a full-suite run is expected, not a bug. If selection falls back with

exit code 10: confidence below threshold

that is the confidence-threshold fallback (TIA-SAFE-002) telling you history is still too thin — run the suite once more and the ramp completes. Real measured ramps from the Rust benchmark suite (warm cycle after 1–2 exec runs):

Repositories with partial git history (shallow clones, squashed imports) ramp slower: SAFE-007 keeps tests without any recorded run in the always-run set until each has executed at least once. Two to three exec cycles is the normal ramp; testaruda calibrate reports the ranking gate if you use predictive ranking.

CI safety mode

Use safe mode when selection should execute tests in CI. It performs preflight checks and falls back to the full Cargo test suite if configuration, store data, Git revisions, or confidence are insufficient:

testaruda select --safe --base origin/main --head HEAD

Next Steps

• See the CLI Reference for all commands and options (including calibrate, ingest, graph, import, explain, oracle, discover, metrics, completions, and their flags).
• See the Configuration Guide for adapter setup and testaruda.toml reference.
• See the Agent Mode Guide for structured JSON output intended for LLM coding agents.
• See the Architecture Overview for the high-level system design.

Architecture

testaruda is built on three layers:

1. Core engine — Ascent-embedded Datalog selection query
2. Store — SQLite-backed persistence for the dependency graph
3. CLI — User-facing commands

Selection Pipeline

┌────────────┐     ┌──────────────┐     ┌───────────┐
│  git diff  │────▶│  Change Set  │────▶│   Store   │
└────────────┘     └──────────────┘     └─────┬─────┘
                                              │
                                              ▼
                                     ┌──────────────────┐
                                     │  Ascent Engine   │
                                     │  (40 lines DL)   │
                                     │  lattice columns │
                                     └────────┬─────────┘
                                              │
                                              ▼
                                     ┌──────────────────┐
                                     │   Selection      │
                                     │  (affected tests)│
                                     └──────────────────┘

Semiring Instantiations

Engine

The reference implementation uses Ascent (embedded Datalog with lattice support). The selection rule set is 40 lines and produces affected tests, confidence, distance, and minimal-witness chains from a single evaluation.

Soufflé serves as an out-of-process provenance oracle for validation.

Incrementality

The active selection path uses change scoping: it evaluates the dependency subgraph reachable from changed content units. The store contains content-addressed component-cache primitives, but the current selection path does not yet reuse cached selection decisions across invocations.

See docs/src/tia-srs-ears.md for the full specification.

Command-Line Help for testaruda

This document contains the help content for the testaruda command-line program.

Command Overview:

• testaruda↴
• testaruda init↴
• testaruda select↴
• testaruda exec↴
• testaruda calibrate↴
• testaruda ingest↴
• testaruda graph↴
• testaruda import↴
• testaruda explain↴
• testaruda oracle↴
• testaruda discover↴
• testaruda metrics↴
• testaruda fingerprint↴
• testaruda doctor↴
• testaruda feedback↴
• testaruda completions↴
• testaruda status↴

testaruda

Language-agnostic test selection engine — compute the affected test set from a code change via provenance-semiring dependency analysis

Usage: testaruda [OPTIONS] <COMMAND>

Subcommands:

• init — Initialize store and config in the current project
• select — Select affected tests from a code change
• exec — Full loop: select → run selected tests → ingest → calibrate (gh-26)
• calibrate — Evaluate the predictive ranking calibration gate (TIA-VER-005)
• ingest — Ingest test run results to update the model
• graph — Show the current dependency graph
• import — Import a dependency graph from a JSON export
• explain — Explain why a test was or was not selected
• oracle — Run the Soufflé oracle for cross-validation
• discover — Discover tests via configured adapters
• metrics — Show operational metrics
• fingerprint — Refresh all content unit fingerprints from disk
• doctor — Validate project configuration via genesis suite_linter
• feedback — Submit a feedback issue about an error
• completions — Generate shell completions
• status — Display cross-tool health summary

Options:

• -v, --verbose — Increase output verbosity (-v, -vv, -vvv)
• -q, --quiet — Suppress non-error output
• -j, --json — Output machine-readable JSON (default when stdout is not a TTY)
• --human — Output human-readable text (default when stdout is a TTY)

testaruda init

Initialize store and config in the current project

Usage: testaruda init

testaruda select

Select affected tests from a code change

Usage: testaruda select [OPTIONS]

Options:

• --base <BASE> — Base revision (git ref)

• --head <HEAD> — Head revision (git ref)

• --files <FILES> — Explicit changed-file list (comma-separated)

• --shadow — Shadow mode: compute but report all tests should run (TIA-CI-007)

• --agent — Agent output format: structured JSON for LLM agent consumption (TIA-AGENT-001) Conflicts with –pre-edit

• --pre-edit — Pre-edit blast radius: report affected tests for proposed changes (TIA-AGENT-005) Conflicts with –agent

• --ci — CI mode: run selected tests and ingest results automatically (TIA-CI-008)

• --safe — Safe mode: pre-flight checks then fall back to cargo test if anything is missing (config, store, git refs) or confidence is low. Implies –ci. Recommended: pass –base and –head to test the diff between two git refs; otherwise falls back to uncommitted changes

• --ordering <ORDERING> — Selection ordering mode

  Default value: default

  Possible values:

  • default: No specific ordering — results in Ascent’s internal iteration order
  • deterministic: Byte-stable ordering: sort by test ID (TIA-SEL-005)
  • duration: Order by descending recorded mean duration (TIA-SEL-006)
  • predictive: Order by descending historical failure rate (TIA-SEL-007)

testaruda exec

Full loop: select → run selected tests → ingest → calibrate (gh-26)

Usage: testaruda exec [OPTIONS]

Options:

• --base <BASE> — Base revision (git ref)

• --head <HEAD> — Head revision (git ref)

• --files <FILES> — Explicit changed-file list (comma-separated)

• --ordering <ORDERING> — Selection ordering mode

  Default value: default

  Possible values:

  • default: No specific ordering — results in Ascent’s internal iteration order
  • deterministic: Byte-stable ordering: sort by test ID (TIA-SEL-005)
  • duration: Order by descending recorded mean duration (TIA-SEL-006)
  • predictive: Order by descending historical failure rate (TIA-SEL-007)

• --threshold <THRESHOLD> — Recall threshold (0.0–1.0) for the calibration gate (default: 0.8)

  Default value: 0.8

testaruda calibrate

Evaluate the predictive ranking calibration gate (TIA-VER-005)

Usage: testaruda calibrate [OPTIONS]

Options:

• --threshold <THRESHOLD> — Recall threshold (0.0–1.0) for promotion (default: 0.8)

  Default value: 0.8

testaruda ingest

Ingest test run results to update the model

Usage: testaruda ingest [OPTIONS] <PATH>

Arguments:

• <PATH> — Path to run output file

Options:

• --raw — Raw test output — delegate to the project’s configured adapter for parsing and store runtime edges from the execution
• --adapter <ADAPTER> — Adapter binary to use for raw output parsing (default: auto-detect)

testaruda graph

Show the current dependency graph

Usage: testaruda graph

testaruda import

Import a dependency graph from a JSON export

Usage: testaruda import <PATH>

Arguments:

• <PATH> — Path to graph JSON file

testaruda explain

Explain why a test was or was not selected

Usage: testaruda explain [OPTIONS] <TEST_ID>

Arguments:

• <TEST_ID> — Test node ID

Options:

• --change <CHANGE> — Change set reference

testaruda oracle

Run the Soufflé oracle for cross-validation

Usage: testaruda oracle [OPTIONS]

Options:

• --program <PROGRAM> — Path to Soufflé Datalog program

testaruda discover

Discover tests via configured adapters

Usage: testaruda discover

testaruda metrics

Show operational metrics

Usage: testaruda metrics

testaruda fingerprint

Refresh all content unit fingerprints from disk

Usage: testaruda fingerprint

testaruda doctor

Validate project configuration via genesis suite_linter

Usage: testaruda doctor [OPTIONS]

Options:

• --fix — Apply safe fixes

testaruda feedback

Submit a feedback issue about an error

Usage: testaruda feedback [OPTIONS] <KIND>

Arguments:

• <KIND> — Kind of issue (bug|feature|question)

Options:

• --from-last-error — Use the last error from scratch (–from-last-error)
• --dry-run — Dry run — print what would be submitted
• --title <TITLE> — Override the issue title (wins over derived titles)

testaruda completions

Generate shell completions

Usage: testaruda completions <SHELL>

Arguments:

• <SHELL> — Shell to generate completions for

  Possible values: bash, elvish, fish, powershell, zsh

testaruda status

Display cross-tool health summary

Usage: testaruda status

This document was generated automatically by clap-markdown.

Agent Mode

testaruda’s agent mode (--agent flag on select) produces structured JSON designed for consumption by LLM coding agents and automated tooling. This document explains the output format and how to interpret each field.

Usage

testaruda select --agent [--base main] [--head HEAD] [--files <list>]

The --agent flag conflicts with --json and --pre-edit. Agent mode also implies --ordering deterministic to ensure stable output across runs (TIA-AGENT-007).

Output Format

Agent output is a single JSON object on stdout with the following top-level fields:

Summary (summary)

Changed Unit (changed_units[])

Each changed unit describes a file or symbol that the agent is proposing to modify.

Selected Test (selected[])

Each selected test includes a reason chain that explains why the engine chose it.

Reason Edge (reason_chain[])

Each edge explains one step in the dependency chain.

Skipped Test (skipped[])

Tests that were candidates (they import from changed units) but were not deemed affected.

Coverage Gap (coverage_gaps[])

Symbols that changed but no test covers them (TIA-AGENT-006).

Design Decisions

1. Versioned format string — The format field allows future format changes without breaking existing consumers. Agents should check it and reject unknown versions.

2. Reason chains — Each selected test carries a reason_chain that traces back to changed units. This enables agents to make targeted decisions: “test_user_model changed because it imports User from src/models.py which you modified”.

3. Coverage gaps — Gaps alert the agent to untested changed code, so the agent can add tests or note the risk in its response.

4. Skipped tests — When an agent modifies a widely-imported utility, the engine may exclude transitive dependents with zero distance. The exclusion_reason explains why.

Error Handling

If the store has not been initialized, the agent receives:

Error: Store has not been initialized. Run `testaruda init` first.

If the change set cannot be computed (e.g., invalid git ref), the agent receives a descriptive error on stderr and a non-zero exit code.

Pre-Edit Mode

The --pre-edit flag (TIA-AGENT-005) emits a simpler JSON format for proposed (not yet committed) changes:

testaruda select --pre-edit [--files <list>]

Output format: testaruda-pre-edit-v1 with changed_files and selected_tests arrays plus summary stats. Its contract is published in schemas/pre-edit-output-v1.json. The full agent-mode contract is published separately in schemas/agent-output.schema.json (see docs/research/schemas/).

Adapter Protocol

Language adapters are long-lived subprocesses that communicate with testaruda using one JSON object per line over stdin/stdout. Adapters must reserve stdout for protocol responses and write diagnostics to stderr. The core waits up to 30 seconds for each response unless the caller supplies another timeout.

Handshake

testaruda → adapter: {"command": "handshake"}
adapter → testaruda: {"ok":true,"result":{"name":"python-adapter","version":"0.1.0","protocol":1,"languages":["python"],"granularity":"file","capabilities":{"symbol_model_complete":false,"fingerprinting":true,"runtime_edges":false}}}

The core rejects adapters whose protocol is not 1. Capability fields are:

Commands

Dependency edges have this shape:

{"from":"test node ID","to":"content path","weight":1000000,"origin":"static"}

weight is an integer confidence in parts per million. origin is static, runtime, or manual. Per-test ingestion results contain test_id, outcome, and optional duration_ms and error_text fields.

Errors and lifecycle

Adapters report command errors as a single line:

{"ok":false,"error":"human-readable message"}

Malformed JSON, an explicit error response, timeout, and protocol version mismatch are fatal to the current adapter connection. Early EOF is observed as a timeout. A missing result is fatal for handshake, run-args, and ingest; missing arrays from discover, static-deps, or fingerprint are treated as empty. The core starts with handshake, may issue multiple commands, and terminates the subprocess when the connection is dropped.

See TIA-ADAPT-001 through TIA-ADAPT-013 in the specification.

Python Adapter Edge Case Catalog

Ticket: testaruda-64o

Status: Point-in-time engineering audit

Last updated: 2026-07-21

This document catalogs unusual Python project structures that could break the Python adapter’s discover, static-deps, fingerprint, or run-args commands. Each edge case includes a severity rating and a recommendation.

Behavior statements describe the implementation at the date above. They are engineering evidence, not a compatibility contract; verify resolved tickets and current integration tests before treating a finding as still open.

Severity Scale

Discover

Edge 1: src/ layout (vs flat layout)

Scenario: Source code lives in src/, tests in tests/. Import paths use the package name as root (e.g., from my_package.model import Model).

Affected commands: discover, static-deps

Current behavior:

• Discover: Works correctly. WalkDir finds all files; test_ / _test.py filter correctly excludes source files.
• Static-deps: Broken. The adapter’s file_path_to_module converts "src/my_package/model.py" → "src.my_package.model". But test imports use the package name without the src. prefix (e.g., from my_package.model import Model). The mismatch means no edges are created from changed files in src/ to tests that import them.

Expected behavior: The adapter should detect that src/ is a source root (e.g., via presence of src/ directory with __init__.py or via configuration) and strip it from module path resolution. For static-deps, changing src/my_package/model.py should produce edges to tests importing my_package.model.

Minimal reproduction:

my_project/
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── model.py
└── tests/
    └── test_model.py    # from my_package.model import Model
                         # adapter converts src/my_package/model.py → src.my_package.model
                         # test imports my_package.model → no match

Fixture needed? Yes — the tests/fixtures/python-src-layout/ fixture (testaruda-6yw). The integration test static_deps_src_layout_resolves_modules documents this current limitation.

Severity: major (upgraded from cosmetic after integration testing) Recommendation: fix adapter — add source root detection (e.g., strip known source directories like src/ from module path resolution)

Edge 2: Tests in separate directory (tests/) vs co-located

Scenario: Some projects put tests in a tests/ directory, others co-locate them alongside source files (src/package/test_*.py).

Affected commands: discover

Current behavior: The WalkDir finds all test_*.py / *_test.py files regardless of location. Both patterns work.

Expected behavior: Same as current.

Minimal reproduction: Both layouts work:

# Co-located
src/myapp/test_models.py    # ← discovered
src/myapp/models.py         # ← not discovered (not a test file)

# Separate
tests/test_models.py        # ← discovered

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 3: Namespace packages (PEP 420)

Scenario: A package directory without __init__.py. Python 3.3+ treats these as namespace packages, allowing multiple directories to contribute to the same package.

Affected commands: discover, static-deps

Current behavior: The adapter doesn’t check for __init__.py to determine package boundaries. Discover walks the directory tree and finds all test files. file_path_to_module works on file paths, not package structure, so namespace packages don’t affect path-to-module conversion.

However, import resolution for relative imports could be wrong. The parse_python_imports function resolves relative imports by counting dots and trimming from the base package path. For namespace packages, the resolution assumes __init__.py boundaries, which don’t exist.

Expected behavior: Relative imports in namespace packages should resolve correctly. This requires understanding that namespace packages don’t introduce a package level — they just expose subdirectories.

Minimal reproduction:

project/
├── pkg_a/
│   └── model.py          # no __init__.py
└── pkg_b/
    └── test_model.py     # from ..pkg_a.model import Model
                          # ← 2 dots: should go up 1 level (project root),
                          #    then into pkg_a

Fixture needed? Yes — synthetic fixture

Severity: minor Recommendation: fix adapter — relative import resolution should account for namespace packages

Edge 4: __init__.py with test discovery implications

Scenario: An __init__.py file that contains test functions, or test files placed inside a package directory alongside __init__.py.

Affected commands: discover

Current behavior: The adapter discovers test_*.py / *_test.py files regardless of directory. __init__.py is not a test file (doesn’t match the pattern), so it’s correctly excluded.

Expected behavior: Same as current.

Minimal reproduction:

mypkg/
├── __init__.py
└── test_features.py    # ← discovered correctly

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 5: conftest.py at multiple levels

Scenario: pytest conftest.py files at the project root, tests/ directory, and nested test subdirectories (providing shared fixtures, hooks, and plugins).

Affected commands: discover

Current behavior: The adapter doesn’t treat conftest.py as a test file (doesn’t match test_ or _test.py), so it’s correctly excluded from discovery results. This is fine because conftest files don’t contain tests (conventionally).

Expected behavior: Same as current.

Minimal reproduction:

project/
├── conftest.py
├── tests/
│   ├── conftest.py
│   ├── test_a.py
│   └── sub/
│       ├── conftest.py
│       └── test_b.py

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 6: Non-standard test file naming

Scenario: Test files that don’t use the test_ prefix or _test.py suffix:

• check_*.py (nose convention)
• *_spec.py (RSpec convention)
• test*.py without underscore (e.g., testmain.py)
• *_test.py with extra underscores (e.g., feature_test.py — this one works)

Affected commands: discover

Current behavior: The filter fname.starts_with("test_") && fname.ends_with(".py") || fname.ends_with("_test.py") misses:

• check_*.py files
• *_spec.py files
• test*.py files without underscore prefix (e.g., testmain.py)

Expected behavior: The adapter should also detect check_*.py and *_spec.py patterns, or make the test file pattern configurable.

Recommendation: Two sub-recommendations:

1. Add check_ prefix support for nose compatibility (e.g., fname.starts_with("check_"))
2. Add *_spec.py suffix support for RSpec-like conventions (e.g., fname.ends_with("_spec.py")) Alternatively, make test file patterns configurable per project via testaruda.toml.

Minimal reproduction:

tests/
├── test_main.py        # ← discovered
├── check_models.py    # ← NOT discovered
├── api_spec.py        # ← NOT discovered
└── testSmoke.py       # ← NOT discovered

Fixture needed? Yes — synthetic fixture for check_ and _spec patterns

Severity: major Recommendation: fix adapter — add check_ prefix and _spec.py suffix, or make test file patterns configurable per project

Edge 7: Deeply nested packages

Scenario: Source code organized in deeply nested package directories (e.g., src/a/b/c/d/e/f/model.py).

Affected commands: discover, static-deps

Current behavior: WalkDir handles arbitrary nesting depth. file_path_to_module correctly converts src/a/b/c/d/e/f/model.py → src.a.b.c.d.e.f.model. Relative import resolution correctly handles deep nesting via dot counting.

Expected behavior: Same as current.

Minimal reproduction:

src/a/b/c/d/e/f/
├── __init__.py
└── model.py
tests/a/b/c/d/e/f/
└── test_model.py     # from src.a.b.c.d.e.f.model import Model

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 8: Non-UTF-8 filenames in tests/sources

Scenario: Test or source files with non-ASCII characters in their names (e.g., test_ünicöde.py, test_日本語.py, test_emoji_😀.py).

Affected commands: discover, static-deps, fingerprint

Current behavior: On Linux, file paths are byte sequences. Rust’s std::fs and walkdir yield OsStr paths. The adapter converts them via file_name().to_string_lossy(), which replaces non-UTF-8 byte sequences with \u{FFFD} (the Unicode replacement character). This means:

• Valid UTF-8 Unicode filenames (e.g., test_ünicöde.py, test_日本語.py) work correctly on any modern Linux system with a UTF-8 locale.
• Truly non-UTF-8 filenames (containing arbitrary bytes that don’t form valid UTF-8) will have those bytes replaced with \u{FFFD}, which could break pattern matching in starts_with("test_") and ends_with("_test.py"). This is extremely rare in practice.

Expected behavior: Test files with valid Unicode names should be discovered and fingerprinted correctly. Source files with Unicode names should resolve to valid module paths.

Minimal reproduction:

tests/
├── test_ünicöde.py    # ← works fine on UTF-8 Linux
├── test_日本語.py      # ← works fine on UTF-8 Linux
└── test_ascii.py      # ← always discovered

Fixture needed? Yes — synthetic fixture with Unicode filenames

Severity: minor Recommendation: workaround — document that the adapter works on valid UTF-8 filesystems. Non-UTF-8 byte sequences in filenames may cause issues.

Edge 9: Symlinks to external test directories

Scenario: The tests/ directory is a symlink to an external location (e.g., tests/ -> /shared/test-suite/). Or individual test files are symlinks.

Affected commands: discover, static-deps, fingerprint

Current behavior: WalkDir uses .follow_links(false) by default — it does not follow symlinks to directories. This means:

• Directory symlinks (e.g., tests/ -> /shared/test-suite/) are returned as directory entries but their contents are not traversed. Test files inside the linked directory are silently omitted from discovery.
• File symlinks (individual symlinked test files) are included as regular entries, so they are discovered and fingerprinted correctly.
• file_path_to_module resolves the symlink path as-is (not the real path), which may not match the module’s actual import path if the link points outside the project tree.

Expected behavior: The adapter should either:

• Call .follow_links(true) to traverse directory symlinks and resolve paths to their real (canonical) locations, or
• Provide a configuration option to control symlink behavior

Minimal reproduction:

my_project/
├── tests/ -> /shared/test-suite/   # symlink → NOT traversed
│   └── test_shared.py              # ← silently missed
├── src/
│   └── model.py
└── test_real.py                    # ← discovered

Fixture needed? Yes — synthetic fixture with symlinks

Severity: major Recommendation: fix adapter — add follow_links(true) and canonical path resolution, or make symlink behavior configurable

Edge 10: tox environments with multiple Python versions

Scenario: Project uses tox to test against multiple Python versions. The .tox/ directory contains virtual environments with vendored test file copies.

Affected commands: discover

Current behavior: The adapter explicitly excludes .tox from the WalkDir filter. This is correct — vendored tests in .tox/ should not be discovered.

Expected behavior: Same as current.

Minimal reproduction:

project/
├── .tox/
│   ├── py39/
│   │   └── .../test_vendored.py    # ← excluded
│   └── py310/
│       └── .../test_vendored.py    # ← excluded
├── tests/
│   └── test_real.py                # ← discovered

Fixture needed? No — the exclusion is present in code but untested (only .venv exclusion has a dedicated test)

Severity: cosmetic Recommendation: won’t fix

Edge 11: Editable installs (pip install -e .)

Scenario: When a package is installed in editable mode (pip install -e .), the source code is linked directly into site-packages. The adapter walks the project root, not site-packages, so this doesn’t affect discovery directly.

Affected commands: discover, static-deps

Current behavior: The adapter discovers tests within the project root only. Editable installs of other packages into the project’s environment don’t affect discovery. However, if the project has test files that import from editable-installed dependencies, the static-deps command will look for those dependencies’ source files in the project tree and won’t find them.

Expected behavior: The adapter should either:

• Track dependencies from editable installs (via .egg-link or .pth files),
• Or report them as unresolved dependencies.

Minimal reproduction:

my_project/
├── src/my_project/
│   └── main.py
├── tests/
│   └── test_main.py       # import editable_pkg
└── editable-pkg.egg-link  # points to /other/path/editable_pkg

Fixture needed? Yes — synthetic fixture with .egg-link file

Severity: minor Recommendation: workaround — document that editable installs outside the project root are not tracked. The unresolved dependency list will include them.

Static-deps

Edge 12: Relative imports — all varieties

Scenario: Python relative imports in all their forms:

• from .module import X — sibling module import
• from ..module import X — parent package module import
• from . import X — current package import
• from .. import X — parent package import
• from ...module import X — grandparent + module

Affected commands: static-deps

Current behavior: The parse_python_imports function handles all these varieties. It resolves relative imports by counting dots and trimming the base package path accordingly. The implementation has specific handling for from . import X (current package only) and from .. import X (parent package only, no module). The parse_python_imports_relative_* tests cover these cases.

Expected behavior: Same as current.

Minimal reproduction: Already covered by existing tests:

• test_parse_python_imports_relative_current_package
• test_parse_python_imports_relative_parent_package
• test_parse_python_imports_relative_import_current_package_only
• test_parse_python_imports_relative_from_double_dot
• test_parse_python_imports_relative_deep
• test_parse_python_imports_relative_above_root_skipped

Fixture needed? No — already tested

Severity: cosmetic Recommendation: won’t fix

Edge 13: Conditional imports (try/except ImportError)

Scenario: Imports guarded by try/except ImportError blocks for optional dependencies. For example, try: import orjson; except ImportError: import json.

Affected commands: static-deps

Current behavior: The adapter’s parse_python_imports does a simple line-based scan for import and from . It finds these imports regardless of surrounding control flow. This is correct — both branches’ imports are recorded as dependencies.

Expected behavior: Same as current.

Minimal reproduction:

try:
    import orjson
except ImportError:
    import json

Both orjson and json are correctly found.

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 14: Dynamic imports (importlib.import_module, import)

Scenario: Imports performed at runtime via importlib.import_module() or __import__() with dynamic module names constructed from strings or variables.

Affected commands: static-deps

Current behavior: The adapter only scans for import X and from X import Y syntax patterns. Dynamic imports like importlib.import_module("numpy") are not detected. The argument to import_module may be a string literal (which could be parsed) or a variable (which can’t be statically analyzed).

Expected behavior: The adapter should detect importlib.import_module() calls with string literal arguments as additional dependencies. Variable-based dynamic imports are inherently undetectable and should be documented as a limitation.

Minimal reproduction:

import importlib

# String literal — could be detected:
np = importlib.import_module("numpy")

# Variable — undetectable:
module_name = "pandas"
pd = importlib.import_module(module_name)

Fixture needed? Yes — synthetic fixture with importlib.import_module

Severity: blocker Recommendation: fix adapter — add detection of importlib.import_module() and __import__() calls with string literal arguments

Edge 15: Re-exports (from x import y as z, from x import *)

Scenario: Modules that re-export symbols from other modules:

• from x import y as z — rename on import
• from x import * — wildcard import
• from x import y — used as if y is from the importing module

Affected commands: static-deps

Current behavior: The adapter correctly extracts the source module for all three patterns:

• from x import y as z → dependency on x (the as z part is ignored)
• from x import * → dependency on x
• from x import y → dependency on x

The adapter correctly records the dependency on the source module, not the importing module’s namespace.

Expected behavior: Same as current.

Minimal reproduction:

# package/__init__.py
from .submodule import useful_function as fn
from .another import *

# test_file.py
from package import fn   # ← dependency on package, which re-exports

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 16: Circular imports

Scenario: Two modules that import each other (directly or transitively).

Affected commands: static-deps

Current behavior: The adapter’s parse_python_imports doesn’t perform recursive import resolution. It only records the first-level imports of each test file. Circular imports at the first level would create bidirectional edges, but the adapter doesn’t recurse so it won’t follow cycles.

Expected behavior: The adapter should handle circular imports gracefully by not recursing into them (which it already doesn’t). The static-deps result is correct for first-level imports.

Minimal reproduction:

# a.py
from b import helper

# b.py
from a import config

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 17: Imports in __init__.py (package re-exports)

Scenario: A package’s __init__.py that imports and re-exports symbols from submodules. Test files that import from the package (not the submodule) need to be resolved to the actual submodule for dependency tracking.

Affected commands: static-deps

Current behavior: The adapter records the import as a dependency on the package (e.g., from mypkg import useful_func → depends on mypkg). However, it doesn’t follow the re-export chain to find the actual submodule (e.g., mypkg.submodule). This means:

• If mypkg/__init__.py re-exports from mypkg/submodule.py, and the user changes mypkg/submodule.py, the adapter won’t find test files that import from mypkg (not mypkg.submodule).

This is a different concern from Edge 15, which covers the parser correctly identifying the source module of a re-export. Edge 17 is about the dependency chain not being followed through __init__.py.

Expected behavior: The adapter should optionally follow __init__.py re-exports to build the full dependency graph. This is related to the symbol-level modeling capability.

Minimal reproduction:

mypkg/
├── __init__.py              # from .submodule import useful_func
└── submodule.py             # changes here
tests/
└── test_features.py         # from mypkg import useful_func
                             # ← dependency on mypkg, not mypkg.submodule
                             #    so change to submodule.py won't be detected

Fixture needed? Yes — synthetic fixture with __init__.py re-exports

Severity: major Recommendation: fix adapter — add __init__.py import resolution to follow re-export chains, or document as a limitation linked to symbol_model_complete

Edge 18: C extension modules with .so/.pyd files

Scenario: Projects that use compiled C extensions (.so on Linux, .pyd on Windows) alongside or instead of pure Python modules.

Affected commands: static-deps, fingerprint

Current behavior: The adapter treats .so and .pyd files as opaque binary files. For static-deps, it can’t parse imports from them. For fingerprint, it can hash them (blake3 works on any binary input).

Additionally, file_path_to_module produces a corrupted module name for .so/.pyd files. For example: file_path_to_module("mypkg/_speedups.cpython-312-x86_64-linux-gnu.so") → strip_suffix(".py") has no match (file ends with .so) → replaces / with . → "mypkg._speedups.cpython-312-x86_64-linux-gnu.so" → the .so suffix and platform ABI tag are preserved in the module name, which won’t match any import statement.

Expected behavior: The adapter should:

• For static-deps: strip .so/.pyd and the platform ABI suffix (e.g., .cpython-312-x86_64-linux-gnu) from the module name derivation; skip import parsing on binary files; report them in the unresolved list
• For fingerprint: hash them as-is (already works)

Minimal reproduction:

mypkg/
├── __init__.py
├── _speedups.cpython-312-x86_64-linux-gnu.so   # compiled extension
└── pure.py
tests/
└── test_mypkg.py          # import mypkg._speedups  # ← can't resolve

Fixture needed? Yes — synthetic fixture with a .so file (or simulant)

Severity: blocker Recommendation: fix adapter — strip .so/.pyd and ABI tag suffix from module name derivation; gracefully handle binary extension modules (skip import parsing, include in unresolved list)

Edge 19: Type-checking-only imports (TYPE_CHECKING)

Scenario: Imports placed inside if TYPE_CHECKING: blocks to avoid runtime imports. These are used for type hints only.

Affected commands: static-deps

Current behavior: The adapter’s line-based parser doesn’t distinguish between imports inside if TYPE_CHECKING: blocks and regular imports. This produces false positive dependency edges.

Expected behavior: The adapter should either:

• Skip imports inside TYPE_CHECKING blocks (correct behavior), or
• Include them but mark them with a lower weight (acceptable compromise)

Minimal reproduction:

from typing import TYPE_CHECKING

if TYPE_CHECKING:
    from expensive_module import HeavyClass  # ← false positive dependency

def get_heavy() -> "HeavyClass":
    ...

Fixture needed? Yes — synthetic fixture with TYPE_CHECKING guard

Severity: minor Recommendation: fix adapter — add TYPE_CHECKING block detection in parse_python_imports

Edge 20: Lazy imports inside function/class bodies

Scenario: Imports placed inside function or class bodies to defer import cost until first use.

Affected commands: static-deps

Current behavior: The adapter scans all lines of the file regardless of indentation or nesting. Lazy imports inside function bodies are correctly detected.

Expected behavior: Same as current.

Minimal reproduction:

def get_model():
    from myapp.models import HeavyModel  # ← correctly detected
    return HeavyModel()

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 21: sys.path manipulation

Scenario: Projects that modify sys.path at runtime to add custom import paths. This can happen in:

• conftest.py (inserting project root into sys.path)
• __init__.py (adding sibling directories)
• setup.py / setup.cfg configurations

Affected commands: static-deps

Current behavior: The adapter doesn’t read or track sys.path modifications. It resolves module paths purely from file paths. If a test file imports from a module that’s only accessible via a modified sys.path, the adapter won’t find the import resolution.

Expected behavior: The adapter should detect sys.path modifications in conftest.py and __init__.py files and adjust its module resolution accordingly. This is a complex feature — a simpler alternative is to document the limitation.

Minimal reproduction:

# conftest.py
import sys
sys.path.insert(0, "/path/to/other/project")

# tests/test_integration.py
from other_project import Something  # ← sys.path makes this work
                                       #    but adapter can't resolve it

Fixture needed? Yes — synthetic fixture with sys.path manipulation

Severity: major Recommendation: workaround — document that sys.path manipulation is not tracked. Affected imports will appear in the unresolved list.

Edge 22: egg-link / pth file dependencies

Scenario: Python packages installed via pip install -e create .egg-link files, and .pth files can extend sys.path. These are typically in site-packages, not the project root.

Affected commands: static-deps

Current behavior: The adapter walks only the project root directory. It doesn’t read .egg-link or .pth files. Dependencies that point to external packages (installed via -e flag) will be unresolved.

Expected behavior: The adapter should either:

• Read .egg-link files and add the linked paths to the search scope, or
• Report linked imports as unresolved with a clear message

Minimal reproduction:

# .eggs/some-app.egg-link  (typically in site-packages, not project root)
/path/to/some-app

# tests/test_some_app.py
from some_app import helper  # ← unresolved because adapter can't find some_app

Fixture needed? No — this is a site-packages concern, not a project-level fixture

Severity: minor Recommendation: workaround — document that editable installs outside the project root produce unresolved dependencies

Fingerprint

Edge 23: Very large files

Scenario: Test or source files that are very large (e.g., generated test data, long integration tests, fixtures).

Affected commands: fingerprint

Current behavior: The adapter reads the entire file into memory with std::fs::read(file), then hashes it with blake3. For very large files (>100MB), this could cause:

• High memory usage
• Slow fingerprinting
• Potential OOM on memory-constrained systems

Expected behavior: The adapter should stream the file through blake3 in chunks to bound memory usage.

Minimal reproduction: Create a 1GB Python file and fingerprint it:

# tests/test_huge.py
# ... 1GB of generated test data

Fixture needed? No — use an existing large file or generate one

Severity: minor Recommendation: fix adapter — stream file content through blake3 in chunks (e.g., 64KB buffer)

Edge 24: Binary files checked into repo

Scenario: Repositories that contain binary files alongside Python code (e.g., .png, .jpg, .zip, .pickle test fixtures).

Affected commands: fingerprint

Current behavior: The adapter uses blake3::hash on raw bytes, which works for any file type. Binary files are hashed correctly.

Expected behavior: Same as current.

Minimal reproduction:

tests/
├── test_with_fixture.py
└── fixtures/
    └── test_data.pkl   # ← hashed correctly

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 25: Empty files

Scenario: Test files or source files that are empty (zero bytes).

Affected commands: fingerprint

Current behavior: The adapter reads zero bytes and computes the blake3 hash of an empty input. This is deterministic and correct.

Expected behavior: Same as current.

Minimal reproduction:

tests/
└── test_empty.py   # empty file → blake3("") → deterministic hash

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 26: Files with only comments

Scenario: Files that contain only comments and no executable code.

Affected commands: fingerprint

Current behavior: The adapter hashes the file content as-is. Comments are content, so the hash reflects the full file.

Expected behavior: Same as current. Changing a comment changes the hash, which is correct (the file changed, even if it’s only a comment).

Minimal reproduction:

# tests/test_placeholder.py
# This test is a placeholder
# TODO: write actual tests

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 27: Generated files (protobuf, grpc)

Scenario: Files that are generated by code generators (protobuf, gRPC, OpenAPI generators) and checked into the repository.

Affected commands: fingerprint

Current behavior: The adapter hashes all files without distinguishing between hand-written and generated code. Generated files will be fingerprinted and their hashes will change when regenerated, causing unnecessary re-runs.

Expected behavior: The adapter should either:

• Allow users to exclude generated files from fingerprinting, or
• Detect generated file headers (e.g., @generated marker) and handle them differently

Minimal reproduction:

project/
├── proto/
│   └── model_pb2.py    # generated by protoc — changes on every proto change
├── src/
│   └── service.py      # hand-written
└── tests/
    └── test_service.py

Fixture needed? Yes — synthetic fixture with a generated file marker

Severity: minor Recommendation: workaround — document that generated files can be excluded via the project’s .gitignore or by adding them to the adapter’s exclude list

Run-args

Edge 28: Custom pytest configurations (pytest.ini, pyproject.toml)

Scenario: Projects that configure pytest through pytest.ini, pyproject.toml (under [tool.pytest.ini_options]), setup.cfg, or tox.ini.

Affected commands: run-args

Current behavior: The adapter generates run-args as: pytest <selected_files> -v --junitxml=target/test-results.xml

Pytest automatically discovers and applies configuration files in the project root. The adapter doesn’t need to read or pass them explicitly.

Expected behavior: Same as current.

Minimal reproduction:

# pytest.ini
[pytest]
addopts = -ra --strict-markers
testpaths = tests

Running pytest tests/test_foo.py -v --junitxml=target/test-results.xml will automatically pick up addopts = -ra --strict-markers from pytest.ini.

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Edge 29: Parallel test runners (xdist)

Scenario: Projects that use pytest-xdist for parallel test execution (e.g., pytest -n auto).

Affected commands: run-args

Current behavior: The adapter’s run-args doesn’t include any -n flag. Tests run sequentially by default, which is slower but correct.

Expected behavior: The adapter should either:

• Add -n auto to run-args when xdist is available, or
• Make parallel execution configurable

Minimal reproduction:

# In pyproject.toml or requirements
# pytest-xdist installed

# Current adapter run-args:
# pytest tests/test_foo.py tests/test_bar.py -v --junitxml=target/test-results.xml

# Expected (with xdist):
# pytest tests/test_foo.py tests/test_bar.py -v -n auto --junitxml=target/test-results.xml

Fixture needed? No

Severity: major Recommendation: fix adapter — check for xdist availability and add -n auto to run-args when available, or make the parallel flag configurable

Edge 30: Test markers and parametrization

Scenario: Tests that use pytest markers (@pytest.mark.slow) or parametrization (@pytest.mark.parametrize).

Affected commands: run-args

Current behavior: The adapter passes selected test file paths to pytest. Pytest handles marker filtering and parametrization internally. The adapter doesn’t need to understand markers or parametrization.

Expected behavior: Same as current.

Minimal reproduction:

import pytest

@pytest.mark.slow
@pytest.mark.parametrize("n", range(100))
def test_compute(n):
    assert n ** 2 >= 0

Running pytest tests/test_compute.py -v --junitxml=target/test-results.xml will run all 100 parametrized test cases.

Fixture needed? No

Severity: cosmetic Recommendation: won’t fix

Summary

Priority breakdown

Test strategy

Can be tested with real-world codebases:

• Non-standard test file naming (Edge 6) — testSmoke.py pattern
• Deeply nested packages (Edge 7)
• Imports in __init__.py (Edge 17)
• Lazy imports (Edge 20)
• Custom pytest configs (Edge 28)
• Test markers and parametrization (Edge 30)

Need synthetic fixtures:

• src/ layout (Edge 1)
• Namespace packages (Edge 3)
• Non-standard naming variants (Edge 6) — check_* prefix and _spec.py suffix
• Unicode filenames (Edge 8)
• Symlinks (Edge 9)
• Editable installs (Edge 11)
• Dynamic imports (Edge 14)
• C extension modules (Edge 18)
• TYPE_CHECKING (Edge 19)
• sys.path manipulation (Edge 21)
• Generated files (Edge 27)

Configuration

testaruda uses a testaruda.toml file in the project root.

# Select every test in a component when its minimum reachable confidence falls
# below this value. Default: 0.5.
confidence_threshold = 0.5

[adapters]
# Adapter used when no extension mapping matches.
default = "testaruda-adapter-rust"
# Per-response adapter deadline in milliseconds. A deadline miss kills the
# adapter's whole process group (shim chains included) and surfaces as a
# typed timeout error — a hung adapter child can never wedge exec. Default: 30000.
# timeout_ms = 30000

[adapters.extensions]
# Adapter binaries must be on PATH or specified with absolute paths.
".rs" = "testaruda-adapter-rust"
".py" = "testaruda-adapter-python"
".jl" = "testaruda-adapter-julia"
".ts" = "testaruda-adapter-typescript"
".tsx" = "testaruda-adapter-typescript"
".mts" = "testaruda-adapter-typescript"
".cts" = "testaruda-adapter-typescript"
".clj" = "testaruda-adapter-clojure"
".cljs" = "testaruda-adapter-clojure"
".cljc" = "testaruda-adapter-clojure"
".cs" = "titi testaruda-adapter"         # command string: shell-split into (titi, testaruda-adapter)
".fs" = "titi testaruda-adapter"         # see TIA-ADAPT-024
".vb" = "titi testaruda-adapter"
".csproj" = "titi testaruda-adapter"
".sln" = "titi testaruda-adapter"
".slnx" = "titi testaruda-adapter"

[must_run]
# Map a changed-path glob to test node IDs that must run.
"ci/*" = ["ci-contract-tests"]
".github/**" = ["workflow-tests"]

[periodic_full_run]
# Hours between mandatory full-suite runs. Zero disables the trigger.
interval_hours = 168

[environment]
# Scope runtime observations to this named environment.
name = "default"

[discover]
# Directory or file names excluded from adapter discovery walks.
exclude = ["target", ".git", "node_modules", ".venv", "venv",
           "__pycache__", ".mypy_cache", ".pytest_cache",
           "build", "dist", ".tox"]

The example shows every supported setting. testaruda init writes the adapter and discovery sections, selecting a default adapter from project markers such as Cargo.toml, pyproject.toml, or Project.toml; omitted sections retain the defaults below. If the file is absent, testaruda uses in-memory defaults. The store location is fixed at .testaruda/; it is not currently configurable.

Adapters

Rust and Python adapters are installed with testaruda. The Julia adapter is provided by Testimonial.jl; see Getting Started for installation.

The Julia adapter discovers @testitem tests (ReTestItems/TestItems.jl) and @testset blocks (Base.Test). In a project that mixes @testitem with plain @test blocks, both the @testitem and @testset tests are available for selection. For files with no test blocks, a file-level fallback is used.

.NET adapter (titi)

The .NET adapter is provided by titi. Unlike the language-native adapters, titi is invoked through a multi-token command string: titi testaruda-adapter. This is configured by setting the adapter binary to "titi testaruda-adapter" in the extension mapping. testaruda’s shell-split infrastructure (TIA-ADAPT-024) automatically splits the command string into the binary (titi) and its argument (testaruda-adapter).

[adapters.extensions]".cs" = "titi testaruda-adapter"

When titi is not on PATH, testaruda falls back to full-suite selection per TIA-ADAPT-012. The .NET adapter is opt-in — no automatic detection is attempted.

Defaults

Specification

The full Software Requirements Specification (SRS) uses the EARS (Easy Approach to Requirements Syntax) notation and is documented in docs/src/tia-srs-ears.md.

The SRS is a draft normative target, not an implementation-status report. For current behavior, consult the generated CLI Reference, Configuration, and Architecture pages.

Key Requirement Groups

Design Constraints

The current implementation uses Ascent as the embedded logic engine and can generate and run Soufflé programs for validation. See TIA-ENG-001 through TIA-ENG-016 for the complete target constraints.

testaruda — Software Requirements Specification

Notation: EARS (Easy Approach to Requirements Syntax) · Status: Draft v0.3 · Date: 2026-07-07 Changelog: v0.2 adds §8 (Engine & Reference Architecture design constraints, TIA-ENG-*) and Appendix F (engine evaluation). v0.3 corrects safety-core defects: scopes TIA-ARCH-003 homomorphism to the positive reachability core; fixes confidence inversion for unresolved nodes in Appendix G rule set; rewrites TIA-CONF-002 to unify the dual confidence definitions; tightens TIA-CI-003 exit-code condition; softens TIA-ARCH-006 incrementality claim to match the reference implementation; wires TIA-AGENT-002 to TIA-SEL-005; adds completeness precondition to TIA-CHG-004 with protocol wiring in TIA-ADAPT-002/005; adds run-identity-key mechanism to TIA-RUN-005; clarifies transaction boundary in TIA-REL-002; scopes TIA-SAFE-002 fallback trigger to reachability-selected tests only. Note: Requirement IDs (TIA-*) are retained from the original tia project name for code traceability; the binary was renamed to testaruda. Subject: A language-agnostic CLI whose single responsibility is to compute, from a code change, the set of tests that must run — modeled as the transpose of a provenance-semiring dependency relation, evaluated incrementally, under a recall-first soundness invariant.

1. Conventions

1.1 EARS requirement patterns

Every requirement is written in one of the EARS templates. The generic form is <precondition(s)> <trigger> the <entity> shall <response>.

1.2 System entities

the core (language-agnostic engine), the CLI (user-facing command surface), an adapter (per-language/per-framework plugin), the store (persistence layer), the selector (selection engine), the ingestor (runtime-feedback path).

1.3 Identifiers and verification

Requirement IDs: TIA-<GROUP>-<NNN>. Each requirement carries a verification method: I = Inspection, A = Analysis, D = Demonstration, T = Test. “shall” denotes a binding requirement.

1.4 References

[GKT07] Green, Karvounarakis, Tannen, Provenance Semirings, PODS 2007. · [BSalC] Mokhov, Mitchell, Peyton Jones, Build Systems à la Carte, ICFP 2018. · [CA19] Alvarez-Picallo, Eyers-Taylor, Peyton Jones, Ong, Fixing Incremental Computation, ESOP 2019. · [AI77] Cousot & Cousot, Abstract Interpretation, POPL 1977. · [FCA-RTS] Leelahapant, Predictive Regression Test Selection via Formal Concept Analysis, Concordia 2006.

2. Glossary

• Artifact — any tracked node: source file, symbol/block, config, fixture, lockfile, or external resource.
• Content Unit (CU) — an artifact at the chosen granularity, identified by (component, path, symbol?) and carrying a content fingerprint.
• Test item — an individually selectable/executable test, identified by (component, adapter, node_id).
• Edge — a dependency test_item → CU carrying an origin ∈ {static, runtime, manual}, a confidence, and an environment binding.
• Semiring K — (K, ⊕, ⊗, 0, 1). ⊕ fuses alternative dependency evidence; ⊗ composes along dependency paths.
• Provenance semiring — the free/polynomial semiring ℕ[Edges] (the master semiring); concrete semirings are obtained from it by homomorphism [GKT07].
• K-relation — the dependency data as a matrix over K.
• Closure / star D* — reflexive-transitive closure of the dependency relation; the least fixpoint of the reachability rule.
• Transpose (D*)ᵀ — the reversed relation; selection evaluates the change vector against the transpose.
• Change set Δ — the set of CUs whose fingerprint differs between base and head.
• Affected set — Δ · (D*)ᵀ ⊕ always_run, the tests selected to run.
• Always-run set — previously-failed ∪ newly-added ∪ no-history ∪ quarantined tests.
• Environment — a toolchain/OS/lockfile/env-var partition, keyed by a fingerprint.
• Component — a coarse package/service node (the parallelization and cross-repo unit).
• Manifest — a component’s exported interface fingerprints + version, for cross-repo edges.
• Soundness / over-approximation — the modeled relation ⊒ the true semantic relation; guarantees recall.
• Confidence — a [0,1] signal that gates fallback; not a probability that the suite is correct.
• Missed-selection incident — a full run fails a test that selection would have skipped.
• Shadow mode — compute and record a selection but still run everything (non-gating).

3. Goals and Non-Goals (informative)

Goals: compute the affected test set from a diff; fuse static, runtime, and historical dependency evidence; serve CI, local-dev, and LLM-agent consumers; compose across monorepos and multiple repos; guarantee recall by construction and bound residual risk. Non-Goals: the system is not a test runner, build system, coverage tool, or flaky-test fixer; it does not enforce hermetic builds; it does not mandate machine learning.

4. Architectural Constraints (ARCH)

These bind the design to the unifying abstraction; downstream requirements instantiate them. They are engine-independent; the reference engine that realises them is pinned separately in §8 (TIA-ENG-*).

• TIA-ARCH-001 [U] — The core shall represent all dependency data as a single K-relation valued in a configured commutative semiring. (V: I, A)
• TIA-ARCH-002 [U] — The core shall treat the provenance (polynomial) semiring as the master representation from which all other semiring values are derived. (V: I, A)
• TIA-ARCH-003 [U] — The core shall derive the positive reachability results (selection (Boolean), distance (tropical), cost/scheduling, and explanation (provenance polynomial — the master)) from the provenance representation by semiring homomorphism, such that no positively-derived result can disagree with the master computation. The always-run set (TIA-SAFE-007), confidence-threshold fallback (TIA-SAFE-002), and duration ranking (TIA-SEL-006) involve negation-as-failure or aggregation over external signals; they are not derived by semiring closure but contributed as input facts to the selection relation, and are not required to be homomorphic images of the provenance column. Path confidence (Viterbi) annotates the same paths but is likewise excluded from the homomorphism guarantee. (V: A, T)
• TIA-ARCH-004 [U] — The selector shall compute the affected set by evaluating the change set against the transpose of the transitive-closure relation. (V: I, T)
• TIA-ARCH-005 [U] — The core shall compute transitive dependency as the least fixpoint (semiring star) of the one-step dependency relation. (V: I, T)
• TIA-ARCH-006 [U] — The core shall compute selection as a scope-bounded recomputation restricted to the subgraph from which the change set is backward-reachable (i.e. the transitive reverse-closure of the changed content units), rather than the full graph. (Informative note: for the reference engine, scope-bounding plus the content-addressed component cache (TIA-ENG-007/008) satisfies this requirement; full change-propagation (derivative) incrementality via differential dataflow is the documented scale-up path (TIA-ENG-013).) (V: A, T)
• TIA-ARCH-007 [U] — The core shall fuse static, runtime, and manual edges by semiring addition over a common edge set, without origin-specific merge logic. (V: I, T)
• TIA-ARCH-008 [U] — The core shall contain no language- or framework-specific logic; all such logic shall reside behind the adapter interface. (V: I)
• TIA-ARCH-009 [U] — The core shall not execute tests, compile, or build; it shall only select tests and emit native-runner arguments for an external executor. (V: I, D)
• TIA-ARCH-010 [U] — The core shall compute selection across components and repositories by associative composition of per-unit K-relations, such that the result is independent of composition order. (V: A, T)

5. Functional Requirements

5.1 Core data model (CORE)

• TIA-CORE-001 [U] — The store shall identify each content unit by the tuple (component, path, symbol?). (V: I)
• TIA-CORE-002 [U] — The store shall record, for each content unit, a content fingerprint computed from the normalized unit content. (V: T)
• TIA-CORE-003 [U] — The store shall classify each content unit by kind ∈ {source, config, fixture, lockfile, external}. (V: I)
• TIA-CORE-004 [U] — The store shall identify each test item by the tuple (component, adapter, node_id). (V: I)
• TIA-CORE-005 [U] — The store shall record each dependency edge as (test_item, content_unit, environment, origin, K-value). (V: I)
• TIA-CORE-006 [U] — The store shall permit static, runtime, and manual edges to coexist for the same (test_item, content_unit, environment) triple. (V: I, T)
• TIA-CORE-007 [U] — The store shall maintain a reverse index from content unit to dependent test items. (V: T)
• TIA-CORE-008 [U] — The store shall partition all edges and statistics by environment fingerprint. (V: T)
• TIA-CORE-009 [U] — The store shall record per-test run history including outcome, attempt count, duration, and error signature. (V: I)

5.2 Change detection (CHG)

• TIA-CHG-001 [E] — When invoked with a base and head revision, the core shall derive the changed file set from the version-control diff between them. (V: T)
• TIA-CHG-002 [O] — Where the caller supplies an explicit changed-file list, the core shall use that list as the change source in place of a diff. (V: T)
• TIA-CHG-003 [U] — The core shall compute the change set Δ as the content units whose fingerprint differs between base and head. (V: T)
• TIA-CHG-004 [O] — Where the responsible adapter declares symbol granularity and sets the symbol_model_complete capability flag to true in its handshake (TIA-ADAPT-002) (asserting that no dependency paths exist beyond those it enumerates for the file), the core shall fingerprint only the changed symbols/blocks so that whitespace-only or unrelated edits within a file yield no affected tests for unchanged symbols. If the adapter does not set symbol_model_complete (e.g. due to reflection, macros, or dynamic dispatch), the core shall fall back to file-level granularity for that file to preserve TIA-SAFE-001. (V: T)
• TIA-CHG-005 [U] — The core shall compute each test item’s dependency fingerprint as a function of its dependency content-unit fingerprints and its environment fingerprint, and shall select a test item if and only if its dependency fingerprint changed or it is in the always-run set. (V: T)
• TIA-CHG-006 [U] — The core shall treat lockfiles, configuration, and adapter-declared external resources as content units subject to change detection. (V: T)
• TIA-CHG-007 [UW] — If a changed file has a kind the core cannot model and no known edges, then the core shall raise a fallback signal for the affected component. (V: T)
• TIA-CHG-008 [S] — While operating in CI mode, the core shall not read the local working tree as a change source. (V: I, T)

5.3 Selection engine (SEL)

• TIA-SEL-001 [E] — When a change set is supplied, the selector shall compute the affected set as the union of the transpose-closure of Δ and the always-run set. (V: T)
• TIA-SEL-002 [U] — The selector shall scope reverse traversal to components reachable from the change set before enumerating candidate tests. (V: A, T)
• TIA-SEL-003 [O] — Where a semiring is specified, the selector shall evaluate the same selection query in that semiring (Boolean → affected set, Viterbi → confidence, tropical → change-to-test distance, cost → expected duration). (V: T)
• TIA-SEL-004 [U] — The selector shall never remove a static edge from consideration on the grounds that runtime evidence did not confirm it. (V: I, T)
• TIA-SEL-005 [O] — Where deterministic ordering is requested, the selector shall emit the affected set in a stable, reproducible order. (V: T)
• TIA-SEL-006 [O] — Where a duration ordering is requested, the selector shall order selected tests by descending recorded mean duration. (V: T)
• TIA-SEL-007 [O] — Where predictive ranking is enabled, the selector shall apply ranking only as a re-ordering or cap over the already-computed recall-safe affected set, and shall not remove any always-run member. (V: I, T)

5.4 Provenance and explainability (PROV)

• TIA-PROV-001 [U] — The core shall compute, for each selection, the provenance expression that derived it. (V: T)
• TIA-PROV-002 [E] — When a test item is selected, the core shall be able to produce its reason chain as the set of edges and changed content units that caused its selection, each annotated with origin. (V: T)
• TIA-PROV-003 [E] — When a test item is excluded, the core shall be able to produce an explicit exclusion reason. (V: T)
• TIA-PROV-004 [E] — When queried about a specific test, the core shall report whether and why that test was or was not selected for the current change set. (V: D, T)
• TIA-PROV-005 [U] — The store shall persist provenance such that a past selection can be re-explained without re-running selection. (V: T)

5.5 Runtime feedback (RUN)

• TIA-RUN-001 [E] — When test results and coverage are ingested, the ingestor shall create or update runtime edges with origin runtime. (V: T)
• TIA-RUN-002 [E] — When coverage indicates a test exercised a content unit not linked by any static edge, the ingestor shall record that runtime edge so the dependency becomes selectable on future changes. (V: T)
• TIA-RUN-003 [E] — When an adapter reports external inputs read at runtime (config files, env vars, fixture files, reflectively loaded modules), the ingestor shall record them as runtime edges to the corresponding content units. (V: T)
• TIA-RUN-004 [E] — When results are ingested, the ingestor shall update per-test run history, timing, and failure-rate statistics. (V: T)
• TIA-RUN-005 [U] — The ingestor shall be idempotent with respect to re-ingestion of the same run. Each run payload shall carry a unique run-identity key (a caller-supplied or adapter-generated opaque string, e.g. a UUID). Before performing any write, the ingestor shall check whether that key has already been recorded and skip ingestion if so. If no run-identity key is present in the payload, the ingestor shall reject the request with a diagnostic rather than proceed without dedup protection. (V: T)
• TIA-RUN-006 [S] — While ingesting a run, the ingestor shall record the environment fingerprint under which the run executed. (V: T)

5.6 Soundness and safety (SAFE)

TIA-SAFE-001 is the cardinal requirement; all other safety requirements exist to maintain it.

• TIA-SAFE-001 [U] — The core shall maintain the invariant that the modeled dependency relation over-approximates the true semantic dependency relation, such that any test that could be affected by a change is selected. (V: A, T)
• TIA-SAFE-002 [UW] — If the minimum Viterbi path confidence (TIA-CONF-001) across the reachability-selected tests in a component falls below the configured threshold, then the core shall fall back to selecting all tests in that component. If no reachability-selected tests exist for a component (i.e. the component’s selection consists solely of always-run members from TIA-SAFE-007), TIA-SAFE-002 does not apply — TIA-SAFE-007 already provides sufficient recall for that component. (V: T)
• TIA-SAFE-003 [U] — The core shall scope confidence-driven fallback to the affected component(s) and shall not escalate to a global full run unless every affected component is below threshold. (V: T)
• TIA-SAFE-004 [UW] — If an adapter reports unresolved files it cannot statically analyze, then the core shall treat the dependents of those files conservatively by force-including them or raising a component fallback. (V: T)
• TIA-SAFE-005 [E] — When the environment fingerprint or a lockfile changes, the core shall schedule a full run for the affected environment. (V: T)
• TIA-SAFE-006 [U] — The core shall support a configurable periodic full-run schedule independent of change-based selection. (V: D)
• TIA-SAFE-007 [U] — The core shall include in the always-run set every test that failed in its last recorded run, every newly added test, every test with no recorded history, and every quarantined test. (V: T)
• TIA-SAFE-008 [E] — When a full run fails a test that the most recent selection would have skipped, the core shall record a missed-selection incident and create a manual edge that forces the test on the implicated change in future. (V: T)
• TIA-SAFE-009 [O] — Where the user defines must-run rules (e.g. path globs mapped to tests), the core shall force-select the mapped tests when matching files change. (V: T)
• TIA-SAFE-010 [U] — The core shall treat a quarantined test as selected-and-run while excluding its outcome from pass/fail trust calculations; quarantine shall not mean skip. (V: I, T)
• TIA-SAFE-011 [E] — When a test produces inconsistent outcomes across retried attempts in one run, the core shall record the outcome as flaky and update its flakiness score. (V: T)
• TIA-SAFE-012 [O] — Where predictive ranking training is enabled, the core shall exclude flaky-labeled outcomes from the training labels. (V: I, A)

5.7 Confidence scoring (CONF)

• TIA-CONF-001 [U] — The core shall compute a selection confidence in the range [0,1]. (V: T)
• TIA-CONF-002 [U] — The core shall apply a per-invocation confidence floor derived from at least coverage freshness, adapter resolution ratio, history depth, and environment match, such that the effective path confidence reported by TIA-CONF-001 reflects the quality of the dependency evidence for the current invocation. These are invocation-level quality signals and shall not mutate stored edge weights, ensuring TIA-REL-001 (deterministic selections for identical store state) is preserved. (V: I, A)
• TIA-CONF-003 [U] — The core shall report the confidence value to every consumer interface. (V: T)
• TIA-CONF-004 [U] — The core shall document, wherever confidence is reported, that confidence gates fallback and is not a probability that the suite is correct. (V: I)

5.8 Language adapter interface (ADAPT)

• TIA-ADAPT-001 [U] — The core shall communicate with adapters using JSON request/response over standard input and output, with diagnostics on standard error and status via exit code. (V: I, T)
• TIA-ADAPT-002 [E] — When the core starts an adapter, the adapter shall return a handshake declaring its name, version, supported protocol version, languages, granularity, and capability flags. Capability flags shall include at minimum: symbol_model_complete (boolean — used by TIA-CHG-004 to permit sub-file granularity narrowing). (V: T)
• TIA-ADAPT-003 [U] — An adapter shall implement the commands discover, static-deps, fingerprint, run-args, and ingest. (V: I, T)
• TIA-ADAPT-004 [E] — When invoked with discover, an adapter shall enumerate test items in scope with their node id, suite kind, and file. (V: T)
• TIA-ADAPT-005 [E] — When invoked with static-deps and a changed-file set, an adapter shall return candidate test items, K-valued edges, and a list of files it could not resolve. When symbol_model_complete is true, the adapter shall also return per-symbol edges; otherwise the core treats all edges as file-level (TIA-CHG-004). (V: T)
• TIA-ADAPT-006 [E] — When invoked with fingerprint, an adapter shall return content fingerprints at its declared granularity. (V: T)
• TIA-ADAPT-007 [E] — When invoked with run-args and a selected set, an adapter shall return the native runner arguments and a collection path, and shall not execute the tests. (V: I, T)
• TIA-ADAPT-008 [E] — When invoked with ingest and a run’s output, an adapter shall return runtime edges, per-test results, and observed external inputs. (V: T)
• TIA-ADAPT-009 [U] — An adapter shall emit dependency edges as semiring values, defaulting to the multiplicative identity where it has no finer weight. (V: I)
• TIA-ADAPT-010 [UW] — If an adapter does not declare a capability, then the core shall degrade gracefully for that capability rather than fail, applying conservative defaults. (V: T)
• TIA-ADAPT-011 [UW] — If an adapter’s protocol version is incompatible with the core, then the core shall refuse to use it and report the mismatch. (V: T)
• TIA-ADAPT-012 [UW] — If an adapter fails, times out, or returns malformed output, then the core shall fall back to selecting all tests in the affected component and record the failure. (V: T)
• TIA-ADAPT-013 [U] — The core shall invoke adapters with least privilege and a configurable timeout. (V: I, T)

5.9 Composability (COMP)

• TIA-COMP-001 [U] — The store shall maintain a component graph distinct from the fine-grained test-to-content-unit graph. (V: I)
• TIA-COMP-002 [U] — The store shall record inter-component dependency edges with an origin. (V: I)
• TIA-COMP-003 [E] — When computing selection in a monorepo, the core shall first resolve affected components bottom-up from the change set, then select within them. (V: T)
• TIA-COMP-004 [O] — Where multi-repo operation is configured, each repository shall be able to export a manifest of its components, their public-interface fingerprints, and a version. (V: T)
• TIA-COMP-005 [E] — When a consumer repository records a dependency on another repository, the core shall record an edge to that repository’s published interface fingerprint and version. (V: T)
• TIA-COMP-006 [E] — When a published interface fingerprint changes, the core shall mark dependent tests in consumer repositories as affected. (V: T)
• TIA-COMP-007 [U] — The core shall aggregate manifests across repositories without requiring a global lock or single shared database. (V: A, D)
• TIA-COMP-008 [U] — The core shall compute per-component selection in parallel. (V: D, T)
• TIA-COMP-009 [U] — The core shall produce identical affected sets regardless of the order in which components or repositories are composed. (V: A, T)
• TIA-COMP-010 [U] — The core shall key a component’s cached selection decision on its dependency fingerprint, and shall reuse the cached decision when the fingerprint is unchanged. (V: T)
• TIA-COMP-011 [O] — Where a remote cache is configured, the core shall share and retrieve cached selection decisions across machines through a local-then-remote lookup. (V: T)
• TIA-COMP-012 [O] — Where sharding is requested, the core shall emit a balanced shard plan computed over recorded test durations. (V: T)

5.10 Persistence and store (STORE)

• TIA-STORE-001 [U] — The store shall persist the queryable index in an embedded transactional database. (V: I)
• TIA-STORE-002 [U] — The store shall persist large per-run payloads in a content-addressed blob store, deduplicated by content hash. (V: T)
• TIA-STORE-003 [U] — The store shall support export and import of the dependency graph and provenance in a documented interchange format. (V: T)
• TIA-STORE-004 [E] — When the store schema version differs from the running core, the core shall migrate or refuse with a clear diagnostic rather than corrupt data. (V: T)
• TIA-STORE-005 [U] — The store shall support concurrent reads during an in-progress write. (V: T)

5.11 CI consumer (CI)

• TIA-CI-001 [E] — When selection completes successfully, the CLI shall exit with code 0. (V: T)
• TIA-CI-002 [UW] — If confidence requires a full run, then the CLI shall exit with code 10 to signal “run everything.” (V: T)
• TIA-CI-003 [E] — When the full selection set (reachability ∪ always-run ∪ fallback, where fallback is scoped per affected component per TIA-SAFE-003) is empty, the CLI shall exit with code 20 to signal “safe to skip.” (V: T)
• TIA-CI-004 [UW] — If a non-recoverable error occurs, then the CLI shall exit with a non-zero code distinct from 10 and 20. (V: T)
• TIA-CI-005 [U] — The CLI shall treat exit code 10 and any unknown condition as “run all tests,” never as “run no tests.” (V: I, T)
• TIA-CI-006 [O] — Where a machine-readable format is requested, the CLI shall emit the selection plan as JSON or a runner-native plan. (V: T)
• TIA-CI-007 [O] — Where shadow mode is enabled, the CLI shall compute and record the selection but report that all tests should run. (V: D, T)
• TIA-CI-008 [E] — When a CI run finishes, the CLI shall accept ingestion of its results to update the model. (V: T)

5.12 Local-developer consumer (LOCAL)

• TIA-LOCAL-001 [E] — When invoked locally against the working tree, the CLI shall compute selection from uncommitted changes. (V: T)
• TIA-LOCAL-002 [O] — Where a daemon is running, the CLI shall reuse a cached in-memory graph to return selection with low latency. (V: D, T)
• TIA-LOCAL-003 [O] — Where watch mode is enabled, the CLI shall recompute the affected set on each saved change. (V: D)
• TIA-LOCAL-004 [U] — The CLI shall always re-run tests that failed in the developer’s most recent local run. (V: T)
• TIA-LOCAL-005 [U] — The CLI shall operate without network access in local mode. (V: T)

5.13 LLM-agent consumer (AGENT)

• TIA-AGENT-001 [O] — Where the agent output format is requested, the CLI shall emit a structured JSON object containing the selection, per-test reasons, confidence, changed units, and summary statistics. (V: T)
• TIA-AGENT-002 [U] — Given the same change set and store state, the CLI in agent mode shall produce byte-stable output. Agent mode shall implicitly enforce deterministic output ordering (TIA-SEL-005) regardless of parallel evaluation (TIA-COMP-008), without requiring a separate flag from the caller. (V: T)
• TIA-AGENT-003 [E] — When the agent requests an explanation, the CLI shall include for each selected test its reason chain and for each skipped test its exclusion reason. (V: T)
• TIA-AGENT-004 [E] — When the agent queries a specific test, the CLI shall answer why that test was or was not selected. (V: T)
• TIA-AGENT-005 [O] — Where pre-edit mode is requested, the CLI shall report the blast radius of a proposed change without requiring the edit to be applied. (V: D, T)
• TIA-AGENT-006 [E] — When a changed symbol has no covering test, the CLI shall surface that coverage gap in agent output. (V: T)
• TIA-AGENT-007 [S] — While serving an agent or a merge gate, the CLI shall default to deterministic selection and shall not apply non-deterministic predictive ranking unless explicitly enabled. (V: I, T)

5.14 Observability (OBS)

• TIA-OBS-001 [E] — When requested, the core shall export the current dependency graph in a documented format. (V: T)
• TIA-OBS-002 [U] — The core shall be able to explain any selection it produced. (V: D)
• TIA-OBS-003 [U] — The core shall emit metrics including selection rate, estimated time saved, fallback rate, flakiness rate, and missed-selection count. (V: T)
• TIA-OBS-004 [U] — The core shall emit structured logs for each selection and ingestion. (V: I)

6. Non-Functional Requirements

6.1 Performance (PERF)

• TIA-PERF-001 [U] — The core shall scale incremental selection time with the size of the change set rather than the size of the full test suite. (V: A, T)
• TIA-PERF-002 [S] — While a warm local daemon is available, the CLI shall return selection for a single-file change within an interactive latency budget. (V: T)
• TIA-PERF-003 [U] — The core shall bound its own selection overhead to a small fraction of the time saved by running fewer tests. (V: A)

6.2 Reliability (REL)

• TIA-REL-001 [S] — While in deterministic mode, the core shall produce identical selections for identical inputs and store state. (V: T)
• TIA-REL-002 [U] — The core shall make ingestion crash-safe, leaving the store consistent after an interrupted operation. Each ingestion shall execute within a single database transaction (or equivalent write-ahead-log mechanism) so that a crash during ingestion leaves the store in its pre-ingestion state. Idempotency of re-ingestion is provided by the run-identity key (TIA-RUN-005). (V: T)

6.3 Security (SEC)

• TIA-SEC-001 [U] — The core shall not execute arbitrary repository code as part of selection. (V: I, A)
• TIA-SEC-002 [U] — The core shall run adapters under least privilege with bounded resource limits. (V: I, T)
• TIA-SEC-003 [U] — The core shall include only an allowlisted, hashed representation of environment variables in cache and environment keys, and shall not store raw secret values. (V: I, T)

6.4 Portability and maintainability (PORT)

• TIA-PORT-001 [U] — The core shall be usable across languages and test frameworks solely through adapters, with no core changes required to add a language. (V: I, D)
• TIA-PORT-002 [U] — The adapter protocol shall permit adapters to be implemented in any language and versioned independently of the core. (V: I)
• TIA-PORT-003 [U] — The core shall declare a protocol-compatibility policy and reject adapters outside the supported version range. (V: I, T)

6.5 Scalability (SCALE)

• TIA-SCALE-001 [U] — The core shall support monorepos containing many components without recomputing the full graph per change. (V: A, T)
• TIA-SCALE-002 [U] — The core shall support federation across multiple repositories without a single shared write bottleneck. (V: A, D)

7. Verification and Rollout

• TIA-VER-001 [U] — The system shall be verifiable in shadow mode, computing selections without gating, before it is permitted to gate. (V: D)
• TIA-VER-002 [U] — The system shall record zero missed-selection incidents over a defined evaluation window as the precondition for enabling enforcing mode. (V: A, T)
• TIA-VER-003 [U] — The system shall use periodic full-run reconciliation as a continuous verification mechanism for the over-approximation invariant. (V: A, D)
• TIA-VER-004 [U] — The soundness invariant (TIA-SAFE-001) shall be verified by a seeded-fault recall test in which every seeded regression’s fault-revealing test is selected. (V: T)
• TIA-VER-005 [O] — Where predictive ranking is enabled, it shall pass a calibration gate meeting defined test-failure-recall and change-recall targets on a held-out recent window before promotion. (V: T)

8. Engine and Reference Architecture (Design Constraints)

§4 (ARCH) is binding and engine-independent: it constrains the abstraction (a provenance-semiring K-relation, selection as its transpose, soundness as over-approximation). This section pins the reference implementation that realises §4 with the least code. TIA-ENG-* requirements bind the reference build; they do not constrain the abstraction, which remains portable per TIA-ENG-012.

8.1 Decision

The reference implementation is a single Rust binary embedding Ascent (a Datalog-style logic language with lattice support) as the in-process selection engine. Change scoping is active today; the store’s content-addressed component-cache primitives (TIA-COMP-010) are not yet integrated into the selection path. Soufflé serves as an out-of-process provenance oracle for validation and full why-provenance. DBSP/Feldera is the documented scale-up path for long-running, fully-incremental, retraction-aware selection. DDlog is rejected (archived/unmaintained).

8.2 Concern → mechanism

8.3 Design-constraint requirements (ENG)

• TIA-ENG-001 [U] — The reference implementation shall be distributed as a single statically linked binary that embeds the selection engine in-process. (V: I, D)
• TIA-ENG-002 [U] — The reference implementation shall not require a separate engine process to compute a selection. (V: I)
• TIA-ENG-003 [U] — The reference implementation shall use Ascent as the embedded logic engine for the selection query. (V: I)
• TIA-ENG-004 [U] — The reference implementation shall represent each semiring as a lattice column so that selection, confidence, and distance are produced by the same rule set under different lattice types. (V: I, T)
• TIA-ENG-005 [U] — The reference implementation shall compute transitive dependency closure using a union-find–backed relation. (V: T)
• TIA-ENG-006 [U] — The reference implementation shall evaluate per-component selection using the engine’s data-parallel mode. (V: D, T)
• TIA-ENG-007 [U] — The reference implementation shall provide cross-invocation incrementality through change scoping and the content-addressed component cache (TIA-COMP-010) rather than a persistent streaming engine. (V: A, T)
• TIA-ENG-008 [U] — The reference implementation shall evaluate the selection query over only the change-scoped residual subgraph. (V: A, T)
• TIA-ENG-009 [U] — For explanation, the reference implementation shall compute a minimal-witness derivation (a single shortest reason chain) as a lattice value sufficient to satisfy TIA-PROV-002 and TIA-AGENT-003, without requiring full why-provenance for routine selection. (V: T)
• TIA-ENG-010 [O] — Where full why-provenance or independent validation is required, the reference implementation shall be able to evaluate the same rule set through an out-of-process Soufflé oracle. (V: D)
• TIA-ENG-011 [S] — While operating in shadow mode (TIA-VER-001), the reference implementation shall be able to cross-check its selections against the Soufflé oracle and flag divergences. (V: D, T)
• TIA-ENG-012 [U] — The selection rule set shall be engine-independent, such that it can be re-targeted to an alternative engine without changing the dependency model or any TIA-ARCH-* behaviour. (V: A)
• TIA-ENG-013 [O] — Where long-running, fully-incremental selection at monorepo scale with retractions is required, the system shall be re-targetable to a streaming incremental engine (e.g. DBSP) as a documented scale-up path. (V: A)
• TIA-ENG-014 [UW] — If a selected engine becomes unmaintained, then the system shall be re-targetable to an alternative engine through the engine-independent rule set without loss of TIA-ARCH-* behaviour. (V: A)
• TIA-ENG-015 [U] — The reference implementation shall persist the dependency graph and statistics in the store of §5.10 independently of the engine’s in-memory working state. (V: I)
• TIA-ENG-016 [U] — The reference implementation core shall be memory-safe and shall isolate untrusted adapter execution to subprocesses (cf. TIA-SEC-002), keeping the engine free of repository code execution. (V: I, A)

8.4 Rejected alternatives (informative)

DDlog — cleanest conceptual match (incremental Datalog on differential dataflow) but archived; rejected on maintenance risk (see TIA-ENG-014). Pure DBSP/differential-dataflow for v1 — true IVM, but Z-sets are an abelian group (signed weights) whereas the master provenance semiring has no additive inverses, making why-provenance awkward; also heavier than the cache-based incrementality requires. Retained as scale-up only (TIA-ENG-013). Soufflé as the primary engine — gold-standard built-in provenance, but compiles to C++ and is not a Rust-embeddable library; retained as oracle (TIA-ENG-010). CozoDB — bundles engine + persistence, but semiring/provenance are not first-class; viable fast-prototype substitute, not the reference.

Appendix A — Semiring instantiations

The positively-derived columns (selection, distance, scheduling, and explanation) are homomorphic images of the provenance column (TIA-ARCH-003). Path confidence annotates the same paths but is not a homomorphic image of the provenance column. Always-run and fallback members of the affected set are contributed as input facts to the selection relation, not derived by semiring closure (see TIA-ARCH-003).

Appendix B — CLI exit codes

Appendix C — Adapter command summary

Appendix D — Verification legend

I Inspection · A Analysis · D Demonstration · T Test.

Appendix E — EARS pattern legend

[U] Ubiquitous · [S] State-driven (While) · [E] Event-driven (When) · [O] Optional-feature (Where) · [UW] Unwanted-behavior (If/Then) · [C] Complex.

Appendix F — Engine evaluation (informative)

Evaluated against the abstraction in §4 and the “easy to implement” priority. The decisive reframing: “incremental” is satisfiable by change-scoping + the component cache (COMP-010), not only by a streaming IVM engine — which favours an embeddable batch engine with lattice support over a heavier streaming one.

Why Ascent for the reference build: its lattice feature is the semiring layer (its shortest-path Dual<u32> example is the tropical distance semiring), it embeds with no FFI or separate process, it parallelises via rayon, and BYODS union-find gives efficient transitive closure — so transitive closure, the semiring instantiations (SEL-003), and parallel selection (COMP-008) fall out of the tool. The one gap, provenance, is closed with a minimal-witness lattice column (TIA-ENG-009); full why-provenance is delegated to the Soufflé oracle (TIA-ENG-010). Because the rule set is engine-independent (TIA-ENG-012), this choice is reversible.

Appendix G — Reference rule set (verified)

The entire core selection path is 40 lines of Ascent declarations + rules (104 lines including a worked example and assertions). It compiles and runs on rustc/cargo 1.75 with ascent 0.8; the full runnable file is tia_core_selection.rs. The rule block:

#![allow(unused)]
fn main() {
ascent! {
    // EDB (scoped residual): facts loaded from the store
    relation changed(u32);                    relation unresolved(u32);
    relation cu_dep(u32, u32, Origin, u32);   relation test_dep(u32, u32, Origin, u32);
    relation always_run(u32);  relation comp_fallback(u32);  relation test_comp(u32, u32);

    // Boolean selection = reverse reachability (ARCH-004/005, SEL-001)
    relation impacted(u32);  relation affected(u32);
    impacted(c) <-- changed(c);
    impacted(c) <-- unresolved(c);                          // over-approximate (SAFE-004)
    impacted(a) <-- cu_dep(a, b, _, _), impacted(b);
    affected(t) <-- test_dep(t, c, _, _), impacted(c);
    affected(t) <-- always_run(t);                          // SAFE-007 union
    affected(t) <-- comp_fallback(k), test_comp(t, k);      // fallback

    // Confidence (Viterbi: lub = max, product along path) — ppm integers
    lattice impact_conf(u32, u32);  lattice test_conf(u32, u32);
    impact_conf(c, ONE) <-- changed(c);
    impact_conf(c, 0)   <-- unresolved(c);     // unresolved → zero confidence; triggers SAFE-002 fallback
    // Note: changed(c) and unresolved(c) are mutually exclusive by definition — a file is either
    // fingerprint-diffed (changed) or unanalysable (unresolved), never both.
    impact_conf(a, ((*w as u64 * *d as u64)/ONE as u64) as u32) <-- cu_dep(a,b,_,w), impact_conf(b,d);
    test_conf(t,  ((*w as u64 * *d as u64)/ONE as u64) as u32) <-- test_dep(t,c,_,w), impact_conf(c,d);
    // No test_conf seed for always_run(t): these tests are selected by SAFE-007 (input fact), not
    // by path evidence. Absent test_conf entry means SAFE-002 does not apply to them (TIA-SAFE-002).

    // Distance (tropical min-plus: Dual flips lub to min)
    lattice impact_dist(u32, Dual<u32>);  lattice test_dist(u32, Dual<u32>);
    impact_dist(c, Dual(0)) <-- changed(c);
    impact_dist(c, Dual(0)) <-- unresolved(c);
    impact_dist(a, Dual(d+1)) <-- cu_dep(a,b,_,_), impact_dist(b, ?Dual(d));
    test_dist(t, Dual(d+1)) <-- test_dep(t,c,_,_), impact_dist(c, ?Dual(d));

    // Minimal-witness predecessors (TIA-ENG-009): edges on a shortest reason chain
    relation cu_pred(u32, u32, Origin);  relation test_pred(u32, u32, Origin);
    cu_pred(a,b,o) <-- cu_dep(a,b,o,_), impact_dist(a, ?Dual(da)), impact_dist(b, ?Dual(db)), if *da == *db+1;
    test_pred(t,c,o) <-- test_dep(t,c,o,_), test_dist(t, ?Dual(dt)), impact_dist(c, ?Dual(dc)), if *dt == *dc+1;
}
}

Worked example output (developer edits session.py; test_totp reaches it only through a runtime-observed edge that static import analysis cannot see — TIA-RUN-002):

affected = [100, 102, 900]
  test 100: conf=0.640 dist=Some(2) witness=[(11,Static),(10,Static)]   # static path
  test 102: conf=1.000 dist=Some(2) witness=[(20,Runtime),(10,Runtime)] # RUNTIME-only path
  test 900: conf=none  dist=None    witness=None                        # always-run (input fact; no test_conf entry; SAFE-002 does not apply)
// Assertions: reachability-selected tests 100 and 102 verified; test 900 has no conf entry by design.

test_invoice (101) is correctly excluded (precision); test_totp (102) is caught solely by the runtime edge (the fusion thesis); confidence 0.640 = 0.8×0.8 is the Viterbi product; distance is the tropical hop count; the witness is the minimal reason chain. One EDB load + p.run() per invocation; swap ascent!→ascent_par! for the parallel evaluator (COMP-008). Cross-run incrementality is the cache (TIA-ENG-007), not the engine.

testaruda — Evaluation Reports

Independent evaluation reports covering implementation state, specification correctness, and release readiness for testaruda.

These are historical, commit-scoped engineering records. Findings in an early round may be resolved by a later retest or by commits after the report’s subject commit; they do not describe the current release unless explicitly restated.

Key findings (short version)

Implementation

• I11 (P0) — Adapter handshake envelope mismatch: AdapterIO::spawn expects top-level fields but adapters wrap responses in {"ok":..., "result":...}. This single bug blocks all adapter communication.
• I12 (P1) — Backlog divergence: tracker reports adapter-protocol tickets as done but the core loop is still broken.
• I1–I2 (P0) — Data ingress gap and FK violation; both are downstream symptoms of I11.

Specification (all fixed in SRS v0.3)

• S1: Homomorphism claim too strong (scoped to positive reachability)
• S2: Confidence inversion for unresolved nodes (fixed to zero confidence)
• S3: Two incompatible confidence definitions (resolved by subordination)
• S4: Exit-20 contradicts always-run (fixed to union condition)
• S5: Derivative-incrementality overclaim (softened to scope-bounded)
• S6: AGENT/SEL/COMP unwired (fixed with implicit ordering)
• S7: CHG-004 missing precondition (gated on capability flag)
• S8: RUN-005/REL-002 no enabling mechanism (run-identity key and WAL boundary)

testaruda — implementation-state and specification evaluation

Historical report: Every finding is scoped to the subject commit named in its evaluation round. Consult later retest sections and current user documentation before applying a finding to the present implementation.

What this document is. An independent, self-contained evaluation of testaruda, a software project that was scaffolded using the charly-vibes developer-tool suite as its mandated workflow. It is a companion to the separate dev-tooling-evaluation.md report (which evaluates the tools); this document evaluates the project those tools produced — both the state of its code and, in depth, the quality of its written specification. It is written to the “amnesia test”: every tool, term, and scenario is defined here, so it is understandable with no prior context.

Date of evaluation: 2026-07-07. Evaluator: an autonomous LLM coding agent (“pi”). Subject commit: 75414cb (“initial scaffold: testaruda test selection engine”). Verdict in one line: the specification is A-grade in form and the core algorithm is correct, but the implementation is a non-functional scaffold, and the specification hides several genuine correctness defects in its safety-critical core that its polished EARS formatting can mask.

1. What testaruda is

testaruda is a Rust command-line program that aims to be a language-agnostic test selection engine: given a code change (a diff), it computes the subset of a test suite that must run, so CI and local developers can skip tests that a change cannot possibly affect. Its stated model is mathematically specific — it treats the dependency data as a provenance-semiring K-relation (see the glossary below) and computes the affected tests as the transpose of the transitive closure of that relation, under a recall-first invariant (it must never skip a test that a change could have broken).

The repository contains two very different things, and the gap between them is the story of this evaluation:

1. A specification — docs/src/tia-srs-ears.md, a ~130-requirement Software Requirements Specification written in EARS notation (Easy Approach to Requirements Syntax, a template discipline where every requirement is typed as Ubiquitous / Event-driven / State-driven / Optional / Unwanted-behaviour). It is titled “tia — SRS” (an earlier name for the project) and is thorough, literature-grounded, and well-structured.

2. An implementation — ~816 lines of Rust across src/ that builds cleanly and passes 6 trivial unit tests, but, as shown below, cannot actually select a test end-to-end.

Glossary (terms used throughout)

• Semiring — an algebraic structure (K, ⊕, ⊗, 0, 1): a set with an “add” operation ⊕, a “multiply” operation ⊗, and identity elements. Unlike ordinary arithmetic, a semiring has no subtraction (no additive inverse). This missing-subtraction property is central to finding S1 below.
• Provenance semiring — the “free” semiring ℕ[Edges] (formally, polynomials whose variables are dependency edges). A classic result (Green-Karvounarakis-Tannen, Provenance Semirings, PODS 2007) says this is universal: for positive database queries (no negation, no aggregation), every other semiring result can be obtained from it by a unique structure-preserving map (a homomorphism). testaruda calls this its “master” semiring.
• K-relation — the dependency graph represented as a matrix of semiring values.
• Transpose / reverse reachability — dependencies point test → code; to find which tests a code change affects you follow the edges backwards. This reversed traversal is the transpose, and it is the correct core idea.
• Recall-first / over-approximation — the system must model at least every true dependency (it may model spurious extra ones). This guarantees no affected test is ever missed (“recall”), at the cost of sometimes running more than strictly necessary.
• Ascent — a Rust library that embeds a Datalog-style logic language (rules + a fixpoint solver) directly in a Rust binary, with support for lattice columns (values that combine by a least-upper-bound). testaruda uses Ascent as its selection engine.
• EDB / IDB — in Datalog, the Extensional DataBase is the input facts; the Intensional DataBase is the relations derived by the rules.

The charly-vibes tools wired into the project

testaruda is set up to be driven by the same four-tool suite evaluated in the companion report. For self-containment: wai (wai) tracks the why behind decisions; dont (dont) forces claims to be grounded in file+line evidence; pretender (pretender) checks per-function code complexity; espectacular (ah) verifies that written Given/When/Then specs map to real tests. Two supporting tools also appear: beads (bd, a git-native issue tracker) and openspec (a spec-driven-development scaffolder whose openspec/ directory espectacular reads).

2. Evaluation scenario, method, environment

• What was evaluated: the checked-in state of the testaruda repository at commit 75414cb — its Rust sources, its build/test behaviour, its own tooling wiring, and its specification document.
• Method: (a) build the binary (cargo build --release) and run its own test suite; (b) exercise every CLI subcommand end-to-end against a throwaway git repo; (c) run the project’s own quality tools (pretender, ah); (d) read the full SRS and check its mathematical and logical claims against the referenced literature and against the project’s own reference rule set.
• Environment: macOS on Apple Silicon; Rust toolchain installed from source (the charly-vibes tools ship no prebuilt binaries — see companion report F9). clippy was not installable in the toolchain, so lint could not be run.

Severity legend: 🔴 blocker · 🟡 friction/defect · 🟢 papercut/documentation · 💡 suggestion. Findings are namespaced: I# = implementation, S# = specification.

3. Implementation-state findings

Build and unit tests pass:

$ cargo build --release        # clean, ~53s
$ cargo test --release         # 6 passed (all trivial: semiring algebra + arg parsing)

The 6 tests only exercise pure helper functions (the three semiring add/mul laws, an argument-split, and “the Ascent program type exists”). None exercises an actual selection.

🔴 I1 — There is no data-ingress path for the dependency graph; selection can never fire.

The entire purpose is to select tests from a dependency graph. But every INSERT statement in the codebase writes to only two of the six tables — content_units and run_history. Nothing ever writes test_items, dependency_edges, or reverse_index. Consequently the test_dep relation the engine reads is always empty, so the rule that selects a test (affected(t) <-- test_dep(t, c, _, _), impacted(c)) can never produce output. The language-adapter protocol — the only specified mechanism to populate edges (see §5.8 of the SRS) — is entirely unimplemented (no subprocess is ever spawned). Demonstrated:

$ testaruda select --files "src/foo.rs,src/bar.rs"
{ "changed_count": 0, "selected_count": 0, "tests": [] }

Selection is structurally incapable of returning a non-empty set. This is the headline defect: the tool cannot do the one thing it exists to do.

🔴 I2 — ingest is broken by a foreign-key violation.

ingest writes to run_history, which has a foreign key to test_items, but never ensures the referenced test item exists (and per I1 nothing else creates test items):

$ echo '{"run_id":"r1","tests":[{"id":1,"outcome":"failed"}]}' > res.json
$ testaruda ingest res.json
Error:   × Failed to insert run result: FOREIGN KEY constraint failed

The runtime-feedback path (SRS §5.5) and the always-run set (which reads previously-failed tests from run_history) are therefore both unreachable.

🔴 I3 — CI exit codes are not implemented (violates SRS §5.11).

The SRS and docs/src/cli.md promise a CI contract: exit 20 = “no tests affected, safe to skip”, exit 10 = “low confidence, run everything”, other-nonzero = hard error. Actual:

$ testaruda select --files "x.rs"; echo $?   # no tests affected → spec requires 20
0

Only 0 (success) and 1 (error) are ever returned. The safety-critical exit semantics — including the rule “treat unknown conditions as run-all, never run-none” — are absent.

🔴 I4 — select misclassifies genuinely-changed files as “unresolved” on first sight.

On first encounter a changed path is not yet in content_units, so it is bucketed as unresolved (and inserted). A second run of the same file then reports it as changed:

$ testaruda select --files "src/foo.rs"   # 1st run → changed_count: 0
$ testaruda select --files "src/foo.rs"   # 2nd run → changed_count: 1

Same input, store-dependent output that flips classification. Worse, unresolved files are supposed to trigger conservative force-include or component fallback (the recall guard); here they are silently created and produce nothing, and the count is never surfaced. This is a recall-relevant bug against the cardinal invariant.

🟡 I5 — espectacular (ah) is wired but the project scaffold is incomplete.

The project ships .espectacular/config.toml pointing specs = "openspec/specs" and changes = "openspec/changes", but those directories were never created (the openspec/ folder contains only AGENTS.md and project.md). So the project’s declared spec-gate cannot run:

$ ah check
No such file or directory (os error 2)   # exit 2 (correct failure code)

Two sub-points: (a) this is primarily a project defect — the scaffold is half-finished; (b) it exposes a tool papercut recorded separately as F26 in the companion report: ah check’s error is a bare errno message that names neither the missing path nor the misconfigured setting, whereas ah doctor diagnoses it precisely (missing-path: specs directory not found: …/openspec/specs). ah check exits 2 correctly — there is no silent-success bug here (an earlier note to the contrary was a shell-pipe artifact and is retracted).

🟡 I6 — The justfile does not parse.

just is the task runner referenced by the README and the project’s own instructions. Its justfile has Markdown prose and a fenced ```bash code block prepended above the real recipes, so the parser dies:

$ just --list
error: unknown start of token '<'  ——▶ justfile:36:1   (an HTML comment on line 36)

Every recipe (just test, just build, just check, just ah) is unusable. This looks like a documentation snippet accidentally committed into the recipe file.

🟢 I7 — A fully-documented config file is never read.

docs/src/configuration.md specifies a detailed testaruda.toml ([store], [confidence] threshold, [semiring] default, [ci] shadow, [adapters], [always_run] patterns), and the crate docs advertise “Config: testaruda.toml”. No code parses it — the toml and serde_yaml crates are unused, init never writes it, and the confidence-threshold fallback it configures is unimplemented. init’s message “Create store and config” is false; it only creates the store.

🟢 I8 — Several declared dependencies are entirely unused.

blake3 (content fingerprinting — a specified feature), similar (diffing), uuid, rayon + ascent-byods-rels (the parallel / union-find engine path the SRS reference architecture mandates), toml, and serde_yaml are all in Cargo.toml but referenced nowhere in src/. Content fingerprinting is stubbed to the literal string "unknown", so change-detection-by-fingerprint (a core requirement) does not exist.

🟢 I9 — pretender advisories (minor, non-blocking).

pretender check src/ flagged 5 functions over the 40-line / ABC-30 advisory thresholds (main 67 lines, load_selection_context 68, from_diff 64, initialize 53, engine::select 49) and exited 0. All advisory; reasonable to defer. This is the one tool that worked exactly as intended here (a corroboration of the companion report’s positive note on pretender).

🟢 I10 — The oracle subcommand is a no-op stub.

testaruda oracle prints a banner and exits 0; with a program argument it shells out to souffle and dumps stdout with no parsing or cross-validation. The Soufflé validation oracle and shadow-mode divergence check (SRS §8.3) are not implemented.

4. Specification evaluation

The SRS was reviewed on its own merits (independent of the code) for correctness (are the claims true?), coherence (do requirements contradict each other?), and completeness (what is missing?). The verdict is NEEDS RESOLUTION: the form is excellent, but there are real correctness defects concentrated in the safety-critical core, where they are easy to miss precisely because the surrounding formatting is so clean.

🔴 S1 — The homomorphism claim (ARCH-003) is provably too strong; the safety machinery lives outside it.

Requirement TIA-ARCH-003 (a binding Ubiquitous requirement) states that concrete results are derived “by semiring homomorphism, such that no derived result can disagree with the master computation.” The universal property that would justify this (Green-Karvounarakis-Tannen) holds only for positive queries — no negation, no aggregation. But the spec’s own safety features are exactly negation/aggregation:

• the always-run set (SAFE-007) is defined by “tests with no recorded history”, “newly added tests”, “test that failed in its last run” — negation-as-failure and temporal argmax;
• the confidence fallback (SAFE-002) is a threshold predicate over an aggregate;
• duration ranking (SEL-006) is an aggregation.

The spec’s own reference rule set proves the point: it selects always-run tests with the rule affected(t) <-- always_run(t), where always_run is an independent input relation, not something derived from the provenance polynomial. So the Boolean “affected” set already contains members with no provenance in the master — a derived result exceeds the master by construction. ARCH-003 is contradicted by SAFE-007 and by the spec’s own Appendix G. (§8.4 already concedes the “no additive inverses” problem when rejecting a streaming engine; the same missing-subtraction gap invalidates the universality claim wherever negation enters.) Fix: scope ARCH-003 to “the positive reachability core”, and state explicitly that always-run and fallback are unioned outside the semiring derivation.

🔴 S2 — The confidence column inverts the very signal the fallback depends on.

The spec’s reference rule set seeds confidence at the maximum value (ONE = 1.0) for unresolved content units (files the analyzer could not resolve) and for always_run tests (forced in because there is no evidence). But confidence is what gates the safety fallback (SAFE-002: “if confidence < threshold, run everything”). So the rule set assigns highest confidence exactly where the model is weakest, which would suppress the fallback that is meant to protect those cases. This is a concrete, latent soundness bug in the specification’s own “verified” reference rule set. Fix: unresolved / always-run nodes must lower confidence (or be excluded from the confidence column), never set it to 1.0.

🔴 S3 — “Confidence” is defined two incompatible ways.

• Appendix A + SEL-003 + the reference rule set define confidence as a Viterbi path-product of edge weights (a per-test graph quantity; the worked example computes 0.8 × 0.8 = 0.64).
• CONF-002 defines confidence as derived from “coverage freshness, adapter resolution ratio, history depth, and environment match” — a per-selection aggregate of four external signals, none of which appears anywhere in the rule set.

These are different quantities with different types. SAFE-002 triggers fallback on “confidence” without saying which — and it is the safety-critical trigger. Fix: pick one definition or name them distinctly (e.g. path_confidence vs selection_confidence) and state which one gates fallback.

🔴 S4 — The “safe to skip” exit code (CI-003) contradicts the always-run rule (SAFE-007).

CI-003 says: “When no tests are affected, the CLI shall exit 20” (safe to skip the test stage). But SAFE-007 forces the always-run set to include every previously-failed, newly-added, no-history, and quarantined test. On a fresh store every test has no history, so always-run is the whole suite; in general it is rarely empty. “No tests affected” (an empty reachability set) can be true while always-run is non-empty — yet CI-003 would emit “safe to skip”, dropping tests SAFE-007 mandates. For a recall-first tool, telling CI to run nothing is the single highest-risk output, and it is guarded only on the reachability set. Fix: condition exit-20 on the full selection (reachability ∪ always-run ∪ fallback) being empty.

🟡 S5 — ARCH-006 mandates derivative incrementality; the reference engine only does scoped batch.

ARCH-006 (binding) requires computing selection “as the change-propagation (derivative) of the reachability query, recomputing only relations affected by the change set.” The reference-engine requirements (ENG-007/008) and Appendix F instead redefine “incremental” as change-scoping + a result cache over a residual subgraph, explicitly “rather than a streaming engine”. A batch Ascent run over a scoped subgraph still recomputes relations the change did not touch within that scope — that is re-scoping, not the derivative ARCH-006 literally requires. The spec resolves this by redefining the word in an appendix, not by satisfying the requirement as written; as stated ARCH-006 is unverifiable against the chosen engine. Fix: soften ARCH-006 to “scope-bounded recomputation”, or state that the reference build only partially satisfies it.

🟡 S6 — Unconditional byte-stability (AGENT-002) versus optional ordering (SEL-005) under mandated parallelism (COMP-008).

AGENT-002 demands byte-stable output for identical inputs — always. SEL-005 makes stable ordering happen only “where requested.” COMP-008/ENG-006 mandate parallel per-component evaluation, which reorders results. Byte-stability requires a total order the spec only promises on request, so AGENT mode must implicitly force SEL-005 or AGENT-002 is unsatisfiable under parallelism. The dependency is not wired.

🟡 S7 — Symbol-granularity narrowing (CHG-004) has an unstated soundness precondition.

CHG-004 lets the system fingerprint individual symbols so that “unrelated edits within a file yield no affected tests for unchanged symbols.” This narrows selection below file granularity and is sound only if the symbol-level dependency model is complete for that file — but reflection, macros, dynamic dispatch, and string references routinely defeat symbol resolution. A test depending on symbol X through an unmodeled path, when sibling symbol Y changes, would be wrongly excluded, violating the cardinal recall invariant. The existing guard (SAFE-004) covers unresolved files, not partially-resolved ones. Fix: state the precondition — symbol narrowing is permitted only under a declared-complete symbol model, else fall back to file granularity.

🟡 S8 — Idempotency (RUN-005) and crash-safety (REL-002) are asserted with no enabling mechanism.

Ingesting a run updates running statistics (mean duration, failure rate, flakiness) — inherently non-idempotent — yet RUN-005 requires idempotent re-ingestion with no required run-identity key or dedup/event-log mechanism to make it achievable. REL-002 (crash-safe, consistent after interruption) has the same shape: a property with no stated transactional boundary.

🟢 Lower-severity specification gaps

• No retraction/deletion semantics. When a content unit or test is deleted, nothing specifies pruning the graph or reverse index; retraction appears only as a future scale-up path. Stale edges over-select (recall-safe) but corrupt explanations and metrics. State “no retraction in v1; stale = conservative” explicitly.
• Scope coherence. The spec calls this a “single responsibility” CLI, then specifies a daemon, watch mode, remote cache, sharding, ML ranking, multi-repo federation, and a blob store. The non-goals exclude “test runner / build system”, but the surface rivals one — flag for minimum-viable-product scoping.
• Naming drift. The SRS is still headed “tia — SRS” with all TIA-* requirement IDs, though the project was renamed to testaruda; the document itself was never re-headed. Since the project conventions ask for TIA-* IDs in code doc-comments, this is a traceability smell.

5. What is genuinely strong

• EARS discipline is real. Every requirement is typed and carries an explicit verification method (Inspection / Analysis / Demonstration / Test). This is better than most production SRSs.
• The abstraction/reference split (binding, engine-independent architecture in §4 versus a separately-justified, explicitly-reversible reference engine in §8) is excellent design hygiene.
• The core selection semantics are correct. Reverse reachability = transpose closure for selection is right; the Boolean / tropical-distance / Viterbi-confidence lattice instantiation is a legitimate and elegant realization; the engine-evaluation appendix is honest about trade-offs.
• Recall-first framing is coherent as a philosophy — which is exactly why the confidence-inversion (S2) and exit-code (S4) defects matter: they quietly betray the stated invariant.
• pretender worked exactly as intended against this code (I9).

6. Prioritized recommendations

1. Resolve the safety-core specification defects first (S1–S4). They are small textual fixes, but each one currently undermines the recall-first invariant that is the tool’s entire reason to exist. Do these before any further implementation, because they change what “correct” means.
2. Then the coherence items (S5–S8): reconcile ARCH-006 with the batch engine, wire AGENT-002 to SEL-005, state CHG-004’s precondition, and give RUN-005/REL-002 an enabling mechanism.
3. For the implementation, close the ingress gap (I1) and the ingest FK bug (I2) before anything else — until edges and test items can be created, the engine is a demo. Then wire the CI exit-code contract (I3, small and safety-critical) and fix the changed-vs-unresolved classification (I4, a soundness bug).
4. Finish or delete the half-built scaffolding: complete the openspec/ layout so ah runs (I5), repair the justfile (I6), and either implement testaruda.toml (I7) and the unused dependencies (I8) or remove the promises so the docs match reality.

This document is the self-contained, standalone companion to dev-tooling-evaluation.md. The tool-level papercut surfaced during this evaluation is recorded there as finding F26.

7. Retest round — 2026-07-10: the project has become spec-and-ticket-only

Subject commit: 94fcc5c (up from 75414cb). Read-only constraint: per this evaluation round’s scope, testaruda’s repository (like the other four charly-vibes tools) was treated as read-only — this section is a pure retest against the pulled state, with zero commits or edits made to the project.

Headline finding: in the five commits since the last evaluation, zero lines of src/, Cargo.toml, or justfile changed. git diff --stat 75414cb..94fcc5c touches 177 files / +3223 −2 lines, and every one of them is under openspec/specs/, .espectacular/ (contract files), .beads/ (the issue tracker’s own store), or docs (docs/src/tia-srs-ears.md). The implementation is byte-for-byte the same 816-line, 6-test skeleton evaluated in §2–3 above. In plain terms: all work in this window was specification and ticket-writing; the code did not move. This matches the project’s own beads backlog exactly — 19 of 20 tracked issues are still open/in_progress, and 4 of those are priority-1 feature tickets covering exactly the missing pieces this section re-confirms (testaruda-6js adapter protocol, testaruda-gbs CI exit codes, testaruda-9uw adapter subprocess management, testaruda-imo content fingerprinting).

The specification surface grew by 17 formal spec documents — and confirms 0/155 scenarios have real test coverage

The openspec/specs/ tree now has 18 requirement areas (adapter-protocol, agent-mode, architectural-constraints, change-detection, ci-integration, composability, confidence-scoring, core-data-model, engine, local-mode, non-functional, observability, persistence, provenance, runtime-feedback, safety, selection-engine, verification; 1,539 total lines), each with an espectacular (ah) contract file per scenario — 154 scenario contracts, one more than the SRS’s original ~130 EARS requirements. This is a genuine specification-completeness improvement: I5 (“scaffold half-finished, ah check cannot even run”) from §3 is fixed — the specs/ and changes/ directories .espectacular/config.toml requires now exist and ah check runs (does not error out).

However, running it end-to-end (ah check) surfaces the real state precisely:

$ ah check
... (155 findings, all kind "no-tests-ran")
summary: { "structural": 0, "execution": 155, "passed": 0,
           "counts_by_kind": { "no-tests-ran": 155 } }

0 of 155 declared scenarios have a passing, scenario-specific test. Every contract’s test.command is the same blanket cargo test invocation (verified by inspecting the contract TOML files: each declares command = "cargo test" rather than a filtered test name), and since the codebase’s only 6 real tests are generic semiring-algebra/arg-parsing unit tests with no per-scenario naming convention, ah’s discovery logic cannot match any of them to a declared scenario — so every one of the 155 reports the no-tests-ran execution failure, regardless of whether the underlying feature exists. This is exactly what “only specs and tickets” means in practice: a fully-formed, EARS-grounded requirements tree and a matching contract-per-requirement scaffold, wired to zero verified behavior. The polish of the specification (still deserved — see §5) makes this easy to miss if one reads only the spec tree and not ah check’s output.

I1–I4 (the safety-critical implementation blockers) all still reproduce, unchanged

Retested against a fresh scratch repo with the rebuilt 94fcc5c binary:

• I1 (selection can never fire) — testaruda select --files "foo.rs" still returns {"changed_count": 0, "selected_count": 0, "tests": []} on first sight of a file.
• I4 (changed-vs-unresolved flip) — running the identical select a second time still flips changed_count from 0 to 1 for the same input, confirming the store-dependent misclassification is unchanged.
• I2 (ingest FK violation) — testaruda ingest res.json still fails identically: Error: × Failed to insert run result: FOREIGN KEY constraint failed.
• I6 (justfile does not parse) — just --list still fails with the identical error: unknown start of token '<' at the same line (a leftover Markdown/HTML comment block above the real recipes).
• I3, I7–I8 were not independently re-run this round but are corroborated by the zero src//Cargo.toml diff: the CI exit-code contract, testaruda.toml parsing, and the unused-dependency list (blake3, similar, uuid, rayon, ascent-byods-rels, toml, serde_yaml) are line-for-line the same code, so they are unchanged by construction.

Net assessment

The user-facing framing for this round (“testaruda should now have only specs and tickets”) is factually confirmed, with one caveat worth flagging: the repository is not purely specs and tickets — the original 816-line implementation skeleton from §2–3 is still checked in and still builds — but no implementation work has occurred since the last evaluation; every commit in the intervening window added specification, contracts, or ticket-tracking metadata. The specification-side investment (§4’s S1–S8 findings, still unresolved and unretested this round since no spec-content commits touched those specific clauses per a quick diff check) has not yet been spent on paying down I1/I2/I4/I6, which remain the actual blockers to the tool doing anything. The most valuable next increment, unchanged from §6’s recommendation, is still: wire a data-ingress path (I1) so the 154 freshly-written scenario contracts have something to test against — at that point, ah check’s 155/155 “no-tests-ran” figure becomes a real, trustworthy completeness metric instead of a placeholder.

8. Deep evaluation — is the specification actually going to produce a correct tool?

Scope of this section. §7 established that testaruda is currently “specs and tickets only” (zero implementation movement across the last 5 commits). This section asks the next question: if the current backlog is executed as written, will it actually produce a correct, safe test-selection engine — or does the plan itself have defects that would carry through into the code? This required (a) verifying that the SRS fixes claimed in the v0.3 changelog are real and coherent, not just changelog text; (b) checking the openspec/specs/ tree (which drives the 154 test contracts) is not stale relative to the SRS; (c) reading the beads ticket backlog’s dependency graph and acceptance criteria for whether they actually close the implementation gaps (I1–I10) in the right order; and (d) hunting for new defects the previous two evaluation rounds didn’t have reason to look for. Read-only throughout — no changes made to the project.

8.1 The headline positive finding: SRS v0.3 genuinely fixed all eight §4 specification defects (S1–S8)

This is not just a changelog claim. Every fix was independently verified against the actual requirement text and, for S1/S2, against the reference Ascent rule set in Appendix G:

• S1 (ARCH-003 homomorphism claim too strong) — now explicitly scoped: “The always-run set (TIA-SAFE-007), confidence-threshold fallback (TIA-SAFE-002), and duration ranking (TIA-SEL-006) … are not derived by semiring closure but contributed as input facts … and are not required to be homomorphic images.” Exactly the suggested fix.
• S2 (confidence inversion for unresolved nodes) — verified in the actual rule set, not just prose: impact_conf(c, 0) <-- unresolved(c); (was seeded at ONE before). Unresolved nodes now correctly get zero confidence, which triggers the SAFE-002 fallback instead of suppressing it.
• S3 (two incompatible confidence definitions) — CONF-002 is now explicitly a modifier of CONF-001’s Viterbi value (“such that the effective path confidence reported by TIA-CONF-001 reflects the quality of the dependency evidence”), not a competing definition. The dual-definition contradiction is resolved by subordination, not by picking one arbitrarily.
• S4 (exit-20 contradicts always-run) — CI-003 now reads “When the full selection set (reachability ∪ always-run ∪ fallback) is empty, … exit 20” — the exact union the original review said was missing.
• S5 (ARCH-006 derivative-incrementality overclaim) — softened to “scope-bounded recomputation restricted to the subgraph … rather than the full graph,” with an informative note naming the reference engine’s actual mechanism (component cache) and differential dataflow as an explicit future scale-up path, not a present claim.
• S6 (AGENT-002/SEL-005/COMP-008 unwired) — AGENT-002 now reads “Agent mode SHALL implicitly enforce deterministic output ordering (TIA-SEL-005) regardless of parallel evaluation (TIA-COMP-008), without requiring a separate flag.”
• S7 (CHG-004 missing precondition) — now gated on a new capability flag, symbol_model_complete, wired through matching new sub-requirements TIA-ADAPT-002/005, with an explicit fallback to file-level granularity when the flag is absent or false.
• S8 (RUN-005/REL-002 no enabling mechanism) — RUN-005 now specifies a mandatory run-identity key with reject-if-absent semantics; REL-002 now specifies a transaction/WAL boundary and explicitly delegates idempotency to RUN-005’s key rather than asserting it independently.

This is an unusually disciplined correction pass — every fix addresses the root cause named in the original finding (not a surface patch), and none introduces an observable new contradiction (checked pairwise against the related requirements each one touches, e.g. SAFE-002 ↔ SAFE-007, CONF-001 ↔ CONF-002, RUN-005 ↔ REL-002).

8.2 The openspec/specs/ tree is in perfect sync with the SRS — no drift

Given the SRS and the openspec specs are two separate documents (the SRS predates openspec adoption; openspec’s 154 contracts are generated from openspec/specs/*.md, not from the SRS directly), there was a real risk that the SRS v0.3 fixes landed in one document but not the other. Checked exhaustively:

$ # requirement IDs defined in openspec/specs/*/spec.md
$ rg -o 'Requirement: (TIA-[A-Z]+-[0-9]+)' -r '$1' openspec/specs/*/spec.md | sort -u | wc -l
149
$ # requirement IDs with a [tag] in the SRS
$ rg -o '\*\*(TIA-[A-Z]+-[0-9]+) \[' -r '$1'  docs/src/tia-srs-ears.md | sort -u | wc -l
149
$ # set difference both directions
$ comm -23 srs_ids.txt openspec_ids.txt   # in SRS, missing from openspec
(empty)
$ comm -13 srs_ids.txt openspec_ids.txt   # in openspec, missing from SRS
(empty)

149/149 exact correspondence, zero missing either direction. Spot-checked the requirement text (not just the ID) for all eight S1–S8-affected requirements (ARCH-003, CI-003, CHG-004, AGENT-002, RUN-005, REL-002, CONF-001/002) — all are byte-identical or effectively identical between the two documents. Additionally checked every TIA-* cross-reference used in prose within openspec/specs/ (e.g. CI-003’s reference to TIA-SAFE-003, CHG-004’s reference to TIA-ADAPT-002) — zero dangling references: every ID referenced anywhere resolves to a real, defined requirement somewhere in the tree. This is a genuinely well-maintained specification corpus; the risk that the project is building 154 test contracts against a stale or diverged copy of the requirements is not realized.

8.3 The ticket backlog’s dependency graph correctly targets I1 (the core blocker) in the right order

The four priority-1 tickets are, encouragingly, exactly the right ones:

• testaruda-9uw (Implement adapter protocol subprocess management) — acceptance criteria explicitly include “discover enumerates test items stored in DB,” which directly closes the “nothing ever writes test_items” half of I1.
• testaruda-6js (static-deps + fingerprint commands), correctly blocks-dependent on 9uw — its acceptance would populate dependency_edges/reverse_index, the other half of I1.
• testaruda-p84 (run-args + ingest commands), correctly dependent on 6js — completes the adapter protocol’s write path with runtime edges.
• testaruda-gbs (CI exit codes + shadow mode), correctly dependent on x1i (deterministic ordering) — targets I3 directly, and its acceptance criteria (0/10/20/error-distinct) match the now-fixed CI-003 union condition exactly, not the old broken version.

This sequencing is sound: the original I1 finding was specifically that the engine’s read-side Datalog rules were already correct (affected(t) <-- test_dep(t, c, _, _), impacted(c)) — the bug was purely that nothing ever populated the EDB. The ticket plan does not propose touching the (already-correct) rule set; it proposes wiring up exactly the missing write path, via the adapter protocol, in the minimal necessary order (discover → static-deps → ingest). This is the right fix shape, not a rewrite in search of one.

The reference rule set itself (Appendix G) also shows a specific, deliberate design choice that reduces future risk: confidence is computed in fixed-point ppm integers (parts-per-million), not floating point. Combined with Ascent’s lattice-join semantics being genuinely commutative/associative for max/min/multiply-along-path, this means the mandated parallel evaluation (TIA-COMP-008) cannot introduce floating-point non-associativity nondeterminism into confidence scores — a failure mode that would have been easy to miss and hard to diagnose (silently different confidence values between two parallel runs of the identical selection). This was not explicitly called out as a risk in the original review; it is verified here as a risk that was avoided.

8.4 New findings: three gaps in the plan that weren’t visible in §3/§4

S9 (new, 🟡) — no specification requirement covers a content unit’s classification on its very first observation, which is exactly the condition under which I4 (the changed-vs-unresolved flip) manifests. TIA-CHG-003 defines the change set as “content units whose fingerprint differs between base and head” — this presupposes a prior fingerprint of record to compare against. TIA-REL-001 promises deterministic selections “for identical store state” — but the store state genuinely differs between I4’s two runs (the first run inserts a new, previously-absent content_units row), so REL-001 does not formally cover this case either. No contract or scenario in any of the 18 spec areas addresses “what classification does a never-before-seen content unit receive, and is that classification stable on immediate re-invocation with no intervening change?” Practically: even after testaruda-imo (content fingerprinting) lands, there is no acceptance criterion or espectacular scenario that would catch a regression of I4 — the bug could be “fixed” by accident as a side effect of proper fingerprinting, or could persist silently, and either way nothing in the current plan would notice. Suggested fix: add a TIA-CHG-009 (or similar) requiring idempotent classification of a content unit across repeated invocations absent any intervening change, with an explicit rule for the cold-start case (no prior fingerprint of record ⇒ treat as unresolved, consistently, not changed on a coin-flip of insertion order).

Backlog gap A (🟡) — testaruda-mfx (idempotent ingestion) has no formal dependency on testaruda-9uw, but its correctness depends on test_items already existing. I2 (the original FK-violation defect: ingest writes to run_history, which has a foreign key to test_items, but nothing ensures the referenced row exists) is not explicitly re-targeted by any single ticket — the closest candidate, testaruda-mfx, is scoped purely to dedup/idempotency-key semantics and crash safety (TIA-RUN-005/REL-002); its acceptance criteria test “duplicate key skipped,” “missing key rejected,” and “crash leaves pre-ingestion state” — none of them test “ingest succeeds once the referenced test item exists”, i.e. none of them re-verify I2 is actually closed. In practice mfx would very likely be implemented after 9uw simply because it is priority-2 and 9uw is priority-1, so the FK-violation symptom will probably disappear as a side effect — but the beads dependency graph does not enforce this ordering (mfx shows deps: []), and no acceptance criterion anywhere explicitly re-tests the original I2 reproduction command. Suggested fix: add mfx --deps blocks:9uw (or equivalent, using beads’ explicit dependency syntax — see the companion dev-tooling-evaluation.md finding F14 for the direction-of---deps footgun to watch for when adding this) and add an acceptance criterion to mfx (or a new contract) that directly re-runs I2’s original reproduction: testaruda ingest against a run payload referencing a test item that does exist, after discover has populated it.

Backlog gap B (🟡) — I6 (the justfile that cannot be parsed, blocking every just recipe including just test/just check) has no ticket anywhere in the backlog and is not mentioned in any tracked document. Searched the full beads issue set (all 20 issues, titles and descriptions) and every tracked file in the repository for “justfile” outside of the justfile itself — zero hits. This is a previously-documented (§3, I6), independently-reproduced (§7) defect that blocks the project’s own stated task-runner workflow, and it has fallen out of tracking entirely between evaluation rounds — neither carried forward as a ticket nor superseded by a spec requirement (task-runner hygiene is not modeled by any TIA-* requirement, correctly, since it’s tooling rather than product behavior — but it still needs a ticket). Suggested fix: file a beads issue for I6; it is a small, self-contained fix (remove the accidentally-committed Markdown/HTML block above the real recipes).

8.5 The empirical validation mechanism for the core invariant exists and is correctly scoped, but not yet elevated

TIA-VER-004 (“Seeded-fault recall test… SHALL be verified by a seeded-fault recall test in which every seeded regression’s fault-revealing test is selected”) is exactly the right mechanism to empirically validate the tool’s entire reason to exist — the recall-first / over-approximation invariant (TIA-SAFE-001) — via mutation-style testing rather than by inspection alone. Its ticket, testaruda-q39, correctly frames it as “the verification gate for the soundness invariant.” It is scoped at priority 2, which is defensible (it needs the adapter protocol’s priority-1 tickets to exist first, since a seeded-fault test needs real dependency edges to select against) — but it is worth calling out explicitly, since it is the one ticket in the entire backlog whose job is to prove that everything else actually achieves the invariant the whole tool exists to guarantee, and it should not slip behind priority-3 feature work (predictive ranking, Soufflé oracle, environment fingerprinting) once priority-1/2 adapter work lands.

8.6 Updated verdict

The plan, as currently written, is sound and is targeting the right things in the right order. The specification is complete (149/149 SRS↔openspec correspondence), internally consistent (zero dangling cross-references), and — most importantly — the eight safety-critical defects found in the previous evaluation round were genuinely fixed at the root cause, not patched over, and the fixes propagate correctly through the derived contract tree. The ticket backlog’s priority-1 sequence (discover → static-deps/fingerprint → run-args/ingest, then CI exit codes) is exactly the minimal path to closing I1, the single defect that made the tool structurally incapable of its core job. The three new findings in §8.4 (S9, and backlog gaps A/B) are all real but all low-severity and cheap to fix — they are the kind of gaps a careful pre-implementation review is supposed to catch, which is itself a point in favor of the specification discipline on display here. Net recommendation: proceed with the current priority-1 ticket sequence as planned; file three small follow-up tickets (S9’s cold-start classification rule, mfx’s dependency edge onto 9uw plus a direct I2 regression check, and I6’s justfile fix) before or alongside that work, and keep testaruda-q39 (seeded-fault recall) from sliding behind priority-3 feature tickets once the adapter protocol lands.

9. Retest round — 2026-07-14: a real implementation landed, but one wire-format bug still blocks the core loop end-to-end

Subject commit: da0072b (tagged v0.1.0), up from 94fcc5c — 16 commits, +7,520/−320 lines across src/adapter.rs, src/agent.rs, two new adapter binaries (src/bin/adapter-{rust,python}.rs), src/config.rs, src/store.rs, and two new test files (tests/ordering.rs, tests/seeded_fault.rs). Unlike the §7 retest (spec/tickets only, zero code movement), this window is the opposite: real, substantial implementation work. Read-write for the update step only — git pull --ff-only then cargo build --release; no source edits were made to testaruda for this evaluation, everything below is black-box CLI exercise plus read-only source inspection to find root causes. Unit test count rose from 6 to 86 (77 lib + 7 bin + 1 ordering + 1 seeded-fault), all passing. The beads backlog is 18/21 issues closed (17 more than §7’s snapshot, which recorded 1 closed of 20 total); only 3 open, all priority-3 (Soufflé oracle bridge, environment fingerprinting, graph export/import) — i.e., by the tracker’s own account, every priority-1/2 safety-critical item is done.

9.1 Confirmed fixed: I3, I4, I6, I7, and part of I8

Each retested end-to-end against a fresh scratch git repo with the rebuilt da0072b binary:

• I3 (CI exit codes absent) — ✅ fixed. select with an empty selection now exits 20; select --shadow exits 10. Matches the SRS’s now-fixed CI-003 union condition (S4).
• I4 (changed-vs-unresolved classification flip) — ✅ fixed. Running select twice on the same unchanged file now reports changed_count: 0 both times (was 0 then 1); editing the file’s content correctly flips it to changed_count: 1 on the next run. This matches the S9 fix suggested in §8.4 (idempotent cold-start classification) — verified via the store’s content_units table, which now records one stable row with a real fingerprint (see I8) rather than being created inconsistently.
• I6 (justfile does not parse) — ✅ fixed. just --list now lists all 16 recipes cleanly (ah, build, check, clean, clippy, default, doc, explain, fmt, fmt-check, ingest, init, release, select, test, test-v); the stray Markdown/HTML block above the recipes is gone.
• I7 (testaruda.toml never read) — ✅ fixed. src/config.rs now parses testaruda.toml via toml::from_str and drives adapter-extension resolution; confirmed by reading the file and by init’s testaruda.toml actually being consulted at adapter-spawn time.
• I8 (unused dependencies) — 🟡 partially fixed. blake3 (content fingerprinting, called via blake3::hash(...) in 3 files — both adapter binaries and store.rs) and toml (config parsing, called via toml::from_str(...) in 1 file, config.rs) are now genuinely used — the two dependencies that mattered most for I1/I7 are wired in. similar, uuid, rayon, ascent-byods-rels, and serde_yaml remain declared in Cargo.toml and referenced nowhere in src/ (checked via each crate’s actual :: call syntax, not a bare-word search — a bare-word grep for “toml”/“blake3” overcounts by also matching unrelated mentions of the filename testaruda.toml in comments and strings).

9.2 🔴 I11 (new) — I1 and I2 still reproduce end-to-end, but the root cause has moved: a single response-envelope mismatch breaks every adapter handshake

This is the headline finding. The discover/static-deps/fingerprint/run-args/ingest logic that I1 originally found completely absent has genuinely been implemented (src/adapter.rs, 932 lines; two real adapter binaries that each correctly discover tests and emit fingerprints when invoked directly) — but the pipeline still cannot select a single test against a real adapter (a separate binary, one per language, that testaruda spawns as a subprocess and talks to over JSON lines on stdin/stdout — see §1’s “Adapters” table), because of one concrete bug in the very first exchange of that conversation, the handshake: before doing anything else, testaruda sends {"command":"handshake"} and expects the adapter to answer with its name, protocol version, and capabilities, so testaruda knows it is safe to proceed. Here is that exchange failing:

$ testaruda select --files "src/foo.rs"
  ⚠️  Failed to spawn testaruda-adapter-rust: JSON error: missing field `name`
ℹ️  no tests selected
{ "changed_count": 1, "selected_count": 0, "tests": [] }

Root cause, isolated by reading src/adapter.rs and invoking the adapter binary directly:

$ echo '{"command":"handshake"}' | testaruda-adapter-rust
{"ok":true,"result":{"capabilities":{...},"granularity":"symbol",
  "languages":["rust"],"name":"rust-adapter","protocol":1,"version":"0.1.0"}}

Both adapter binaries correctly wrap every response in an {"ok": bool, "result": {...}} envelope (verified in src/bin/adapter-rust.rs: json!({"ok": true, "result": result}), and this envelope shape is exactly what DiscoverResponse/StaticDepsResponse/etc. in adapter.rs expect — each has a result: Option<...> field). But the handshake path is the one exception: AdapterIO::spawn does

#![allow(unused)]
fn main() {
let hs: Handshake = io.send(&AdapterCommand::handshake())?;   // adapter.rs:159
}

and Handshake is defined with name/version/protocol as top-level fields (no result wrapper) — so deserializing the real envelope into Handshake fails with exactly the observed missing field \name`error,spawn()returnsErrbeforediscover()is ever called, and the entire adapter session is aborted at the first round-trip. Confirmed with a minimal Python repro that deserializes the adapter's raw stdout directly:‘name’ in respisFalse, ‘name’ in resp[‘result’]isTrue. The bug is in AdapterIO::spawnitself, which is shared code invoked identically regardless of which adapter binary is spawned, and bothtestaruda-adapter-rustandtestaruda-adapter-pythonwere independently confirmed (by piping the handshake command to each directly) to answer with the same enveloped shape — so the defect affects every configured adapter, not just Rust's. One caveat worth naming precisely: in a scratch repo with both a.rsand a.pyfile staged,discover/selectonly ever reported thetestaruda-adapter-rustfailure and silently never attemptedtestaruda-adapter-pythonat all (confirmed by temporarily removing the rust binary fromPATH`, which changed the error from a JSON-parse failure to a “binary not found” failure — still only for rust). Whether that is a second, adapter-fan-out defect or simply this test repo’s layout not tripping the python path was not tracked down further here; it does not change the diagnosis above, which was verified directly against both binaries in isolation, not only inferred by analogy.

This one bug is now the sole cause of both I1 and I2: dropped a test item directly into store.db with sqlite3 (bypassing the broken handshake) and re-ran ingest against the same payload that previously hit FOREIGN KEY constraint failed — it succeeded immediately (✅ Run ingested, exit 0). So I2 is not an independent defect anymore; it is a downstream symptom of I11, exactly as I1 always was. Fixing the envelope mismatch in the handshake deserialization path (either strip the envelope before decoding, or give Handshake the same result: Option<...> shape as every other response type) should resolve both I1 and I2 in one change.

Why this survived 86 passing tests and a closed, fully-checked-off ticket (testaruda-9uw, closed 2026-07-13, acceptance criterion “discover enumerates test items stored in DB” checked [x]): the unit test that exercises this exact deserialization, test_handshake_deserialization (adapter.rs:637), hand-constructs the JSON without the envelope and asserts it decodes — it never round-trips through a spawned subprocess, so it cannot see the mismatch. The espectacular contract meant to guard this scenario, .espectacular/adapter-protocol/handshake-response.toml, runs exactly that unit test (cargo test adapter::tests::test_handshake) and nothing that spawns a real adapter binary. This generalizes a concern from §7/§8 in a sharper form: even where a contract was narrowed from the old blanket cargo test (closing part of the earlier “no-tests-ran everywhere” criticism — ah check now reports 8/155 scenarios passing, up from 0/155), the specific named test it was narrowed to still never crosses the process boundary the feature is actually about. A contract can be “passing” and specific while still not testing the thing that matters. Suggested fix: (a) fix the envelope bug (small, one struct or one deserialization step); (b) add — and point the handshake-response/test-discovery contracts at — an integration test that spawns the real testaruda-adapter-rust binary via AdapterIO::spawn and asserts a live discover call returns test items, not a hand-built JSON string fed straight to serde_json::from_str.

9.3 🟡 I12 (new) — the beads backlog reports the safety-critical work as fully done; it is not

testaruda-9uw, testaruda-6js, testaruda-p84, and testaruda-gbs — the four priority-1 tickets §8.3 identified as “exactly the right ones” to close I1 — are all closed, each with every acceptance-criterion checkbox marked [x], including 9uw’s “discover enumerates test items stored in DB.” Per §9.2, that specific criterion is currently false in end-to-end use: testaruda discover against a real project discovers zero test items (✅ Discovered 0 test items) because the handshake fails before discover() is reached. The ticket tracker and reality have diverged, and nothing in the tracker currently flags it — the only open issues are unrelated priority-3 features. This is not a beads defect (the tool faithfully recorded what the closing commit’s author asserted); it is a project-process gap: the acceptance criteria were verified against unit tests rather than the live CLI path they describe. Suggested fix: reopen testaruda-9uw (or file a new P1 regression ticket) for the handshake envelope bug, and add a live-CLI smoke check (testaruda init && testaruda discover against a real fixture repo, asserting a non-zero discovered count) to the acceptance criteria template for any future adapter-protocol ticket.

9.4 Other observations

• ah check improved from 0/155 to 8/155 passing, and gained 2 new orphan-toml findings for the two S9-motivated cold-start contracts — real, if incremental, progress on the specification-coverage gap flagged in §7.
• dont was never initialized for this project (dont check --ungrounded → error: no .dont/ project found) — a sharper version of the companion report’s F20 (dont decaying to zero use): here it never even reached day-1 init, despite the README’s own “Tools” table listing it as part of the required suite. Zero claims exist to ground any of the correctness assumptions this evaluation had to independently re-derive (e.g., “the handshake response is enveloped like every other response” would have been exactly the kind of claim dont ground --file src/bin/adapter-rust.rs was built to pin down before it diverged from adapter.rs’s expectations).
• pretender check src/ still returns advisory-only findings, exit 0 (mostly long test functions, e.g. test_quarantined_flag_in_selection_result at 88 lines / ABC 56.5) — consistent with I9’s original corroboration that pretender behaves correctly here; nothing new to report.

9.5 Updated verdict

testaruda crossed the line from “non-functional scaffold” (§3) and “specs-and-tickets-only” (§7) to “a real, substantially-implemented tool with one precise, high-leverage bug blocking its core loop.” That is genuine, measurable progress: 86 passing tests (up from 6), 18/21 backlog issues closed, four of the eight §4 specification defects’ code-level implications (I3/I4/I6/I7) now demonstrably fixed, and ah check showing real (if small) movement off zero. But the tool still cannot select a single test against a real adapter today — the single handshake-envelope mismatch found in §9.2 sits directly on the critical path every other fix in this round depends on, and it was invisible to both the test suite and the ticket tracker that both currently report the adapter protocol as complete and verified. Net recommendation: fix the I11 envelope bug next — it is small, isolated to one deserialization call, and its fix should mechanically also close I2 — then add the live-subprocess integration test suggested in §9.2 before closing any further adapter-protocol tickets, so “tests pass” and “the feature works” stop being able to diverge this far apart again.

Contributing

Workflow

This project uses spec-driven development via OpenSpec.

1. Check existing specs: openspec list --specs
2. Create a change proposal: see openspec/AGENTS.md
3. Implement following TDD
4. Validate: openspec validate <change-id> --strict
5. Archive after deployment

Development

# Build
cargo build

# Test
cargo test
cargo clippy --all-targets -- -D warnings
cargo fmt --check

# Run code quality checks
pretender check src/

# Run the epistemic gate (dont claims)
just check-claims

Epistemic Gate (dont)

This repo keeps a real claim corpus in .dont/ and has adopted the repo-local epistemic gate per the dont-bpuo ADR (see dont’s enforcement model):

• just check-claims runs dont prime, the terminal check — it exits 1 while any claim in .dont/ is Doubted, regardless of project mode.
• A lefthook pre-commit hook runs just check-claims; commits are rejected while a claim is doubted (run dont undoubt <claim-id> after resolving the doubt, or dont flag to verify with evidence).
• CI runs check-claims as part of just ci, so a doubted claim fails the pipeline.
• Register claims as you make them during development (see dont’s grounding workflow); unverified claims do not block in permissive mode, doubted ones always do.

Project Conventions

• Rust 2021 edition
• No unwrap() in library code
• Reference SRS requirement IDs in doc comments
• Commit messages: conventional commits

Status

beta — core works and is dogfooded in anger; surface may shift before stable.

Implemented

Adapters

In progress

• Adapter coverage round; dogfood-matrix installs (DDL-1a0).

Mapped to specs

• openspec/ proposals in-repo; validated strict in CI.

Dogfooding

• Runs pretender (pinned), dont, espectacular, plus planned wai/openspec/ddl installs per versions.ddl.toml
• Its own CI runs just ci; docs book built from root book.toml

Adapter Protocol

Purpose

Language/framework adapter interface — JSON request/response protocol over stdin/stdout with handshake, discovery, dependency analysis, and ingestion commands.

Requirements

Requirement: TIA-ADAPT-001 — JSON protocol

The core SHALL communicate with adapters using JSON request/response over standard input and output, with diagnostics on standard error and status via exit code.

Scenario: Adapter communication

• GIVEN the core and an adapter process
• WHEN the core sends a command
• THEN the adapter SHALL receive JSON on stdin
• AND SHALL respond with JSON on stdout
• AND SHALL emit diagnostics on stderr
• AND SHALL indicate status via exit code

Requirement: TIA-ADAPT-002 — Adapter handshake

When the core starts an adapter, the adapter SHALL return a handshake declaring its name, version, supported protocol version, languages, granularity, and capability flags. Capability flags SHALL include at minimum: symbol_model_complete (boolean — used by TIA-CHG-004 to permit sub-file granularity narrowing).

Scenario: Handshake response

• GIVEN an adapter is started
• WHEN the core requests its capabilities
• THEN the adapter SHALL return name, version, protocol version, languages, granularity, and capability flags
• AND symbol_model_complete SHALL be included among the capability flags

Requirement: TIA-ADAPT-003 — Required commands

An adapter SHALL implement the commands discover, static-deps, fingerprint, run-args, and ingest.

Scenario: Required command set

• GIVEN an adapter implementation
• WHEN the core sends any of the required commands
• THEN the adapter SHALL respond appropriately for discover, static-deps, fingerprint, run-args, and ingest

Requirement: TIA-ADAPT-004 — Discover command

When invoked with discover, an adapter SHALL enumerate test items in scope with their node id, suite kind, and file.

Scenario: Test discovery

• GIVEN an adapter with access to test files
• WHEN the discover command is invoked
• THEN it SHALL return test items with node id, suite kind, and file path

Requirement: TIA-ADAPT-005 — Static-deps command

When invoked with static-deps and a changed-file set, an adapter SHALL return candidate test items, K-valued edges, and a list of files it could not resolve. When symbol_model_complete is true, the adapter SHALL also return per-symbol edges; otherwise the core treats all edges as file-level (TIA-CHG-004).

Scenario: Static dependency analysis

• GIVEN a changed-file set
• WHEN the static-deps command is invoked
• THEN the adapter SHALL return candidate tests, K-valued edges, and unresolved files
• AND if symbol_model_complete is true, SHALL also return per-symbol edges

Requirement: TIA-ADAPT-006 — Fingerprint command

When invoked with fingerprint, an adapter SHALL return content fingerprints at its declared granularity.

Scenario: Content fingerprinting

• GIVEN files or symbols to fingerprint
• WHEN the fingerprint command is invoked
• THEN the adapter SHALL return content fingerprints at its declared granularity

Requirement: TIA-ADAPT-007 — Run-args command

When invoked with run-args and a selected set, an adapter SHALL return the native runner arguments and a collection path, and SHALL NOT execute the tests.

Scenario: Run arguments generation

• GIVEN a selected set of tests
• WHEN the run-args command is invoked
• THEN the adapter SHALL return native runner arguments and a collection path
• AND SHALL NOT execute the tests

Requirement: TIA-ADAPT-008 — Ingest command

When invoked with ingest and a run’s output, an adapter SHALL return runtime edges, per-test results, and observed external inputs.

Scenario: Run output ingestion

• GIVEN a test run’s output
• WHEN the ingest command is invoked
• THEN the adapter SHALL return runtime edges, per-test results, and observed external inputs

Requirement: TIA-ADAPT-009 — Semiring edge values

An adapter SHALL emit dependency edges as semiring values, defaulting to the multiplicative identity where it has no finer weight.

Scenario: Default semiring weight

• GIVEN a dependency edge with no specific weight
• WHEN the adapter emits it
• THEN it SHALL use the multiplicative identity as the default weight

Requirement: TIA-ADAPT-010 — Graceful degradation

If an adapter does not declare a capability, then the core SHALL degrade gracefully for that capability rather than fail, applying conservative defaults.

Scenario: Missing capability

• GIVEN an adapter that does not declare a capability
• WHEN the core needs that capability
• THEN the core SHALL NOT fail
• AND SHALL apply conservative defaults instead

Requirement: TIA-ADAPT-011 — Protocol incompatibility

If an adapter’s protocol version is incompatible with the core, then the core SHALL refuse to use it and report the mismatch.

Scenario: Version mismatch

• GIVEN an adapter with an incompatible protocol version
• WHEN the core attempts to use it
• THEN the core SHALL refuse to use it
• AND SHALL report the version mismatch

Requirement: TIA-ADAPT-012 — Adapter failure fallback

If an adapter fails, times out, or returns malformed output, then the core SHALL fall back to selecting all tests in the affected component and record the failure. The core SHALL handle pre-spawn failures (adapter binary not found, command not found) and startup failures (adapter crashes before responding) identically.

Scenario: Adapter timeout

• GIVEN an adapter that times out
• WHEN the core is waiting for a response
• THEN the core SHALL fall back to selecting all tests in the affected component
• AND SHALL record the adapter failure

Scenario: Adapter binary not found

• GIVEN the adapter binary is not installed or not found at the configured path
• WHEN the core attempts to spawn the adapter
• THEN the core SHALL report the missing binary
• AND SHALL fall back to selecting all tests
• AND SHALL record the adapter failure

Requirement: TIA-ADAPT-013 — Least privilege and timeout

The core SHALL invoke adapters with least privilege and a configurable timeout.

Scenario: Adapter sandboxing

• GIVEN an adapter process
• WHEN it is invoked
• THEN it SHALL be run with least privilege
• AND a configurable timeout SHALL be enforced

Requirement: TIA-ADAPT-017 — Clojure adapter handshake (static deps)

A Clojure adapter SHALL declare "clojure" as one of its supported languages in the handshake. The adapter SHALL declare symbol_model_complete: false and runtime_edges: false in its handshake capabilities. The adapter SHALL declare granularity: "file".

The adapter SHALL be a Rust binary in the testaruda workspace (not a separate repository), as a separate crate named adapter-clojure producing the binary testaruda-adapter-clojure. The testaruda.toml adapter config SHALL map .clj and .cljs extensions to this binary.

Scenario: Clojure handshake with static deps

• GIVEN a Clojure adapter binary is spawned
• WHEN the handshake command is invoked
• THEN the adapter SHALL include "clojure" in its languages array
• AND SHALL declare symbol_model_complete: false
• AND SHALL declare runtime_edges: false
• AND SHALL declare granularity: "file"

Scenario: Missing Clojure CLI or Leiningen

• GIVEN a system with the adapter binary installed but without Clojure CLI (clojure) or Leiningen (lein), and the adapter cannot detect a supported runner
• WHEN the run-args command is invoked
• THEN the adapter SHALL return an error

Requirement: TIA-ADAPT-018 — Clojure discover scope (deftest-based)

A Clojure adapter SHALL discover test items by walking the project’s test directories (as declared in deps.edn :test-paths, project.clj :test-paths, or defaulting to test/) and scanning .clj files for deftest and deftest- top-level forms. Each deftest SHALL be one test item.

Node IDs SHALL follow the format <namespace>, derived from the file’s ns declaration (or, where absent, the file path relative to the source root). The namespace-level ID is used because the Cognitect test runner cannot run individual deftest forms — it runs all tests in a namespace.

The adapter SHALL discover deftest forms in all .clj files under the test paths, including nested subdirectories. The adapter SHALL NOT discover tests in directories outside the declared test paths.

Scenario: Discover Clojure tests from deftest forms

• GIVEN a Clojure project with test files containing deftest forms
• WHEN the discover command is invoked
• THEN the adapter SHALL return test items
• AND each item SHALL be derived from one deftest or deftest- form
• AND node IDs SHALL follow the format <namespace>
• AND suite kind SHALL be "clojure.test"

Scenario: No test directory

• GIVEN a Clojure project with no test directory
• WHEN the discover command is invoked
• THEN the adapter SHALL return an empty items list

Scenario: deftest in non-test directory

• GIVEN a Clojure project with deftest forms in both src/ and test/
• WHEN the discover command is invoked
• THEN the adapter SHALL only discover tests under the declared test paths
• AND SHALL NOT discover deftest forms under src/

Scenario: Nested test directories

• GIVEN a Clojure project with tests in test/unit/ and test/integration/
• WHEN the discover command is invoked
• THEN the adapter SHALL discover tests in both subdirectories
• AND each test item SHALL include the correct file path

Requirement: TIA-ADAPT-019 — Clojure dependency analysis (require-based static deps)

A Clojure adapter SHALL build dependency edges by parsing (:require ...) and (:use ...) forms from the ns declaration of each changed source file. Each uniquely required namespace SHALL produce a dependency edge from the importing test to the source file.

The adapter SHALL NOT resolve aliases, :refer lists, or :rename mappings for dependency edge construction — all required namespaces are treated as file-level dependencies. The adapter SHALL NOT create dependency edges for (:import ...) entries — changes to Java imports are captured via the existing edge from the test to the source file.

The adapter SHALL emit edges at file-level granularity with origin: "static".

Scenario: Static dependency from require

• GIVEN a changed source file src/my_project/core.clj with namespace my-project.core
• AND a test file test/my_project/core_test.clj with (:require [my-project.core :as sut])
• WHEN the static-deps command is invoked with changed_files: ["src/my_project/core.clj"]
• THEN the adapter SHALL return an edge from the test item to src/my_project/core.clj
• AND the edge SHALL have origin: "static"

Scenario: Multiple requires produce multiple edges

• GIVEN a test file that requires two different namespaces
• WHEN the static-deps command is invoked with changes to both
• THEN the adapter SHALL return edges from the test to each changed file

Scenario: Deduplicate edges for same namespace

• GIVEN a test file with two separate require entries for the same namespace (e.g., (:require [my-project.core :as core] [my-project.core :as c]))
• WHEN the static-deps command is invoked
• THEN the adapter SHALL emit a single edge for that namespace

Scenario: Changed source file with no test coverage (Clojure)

• GIVEN a changed source file whose namespace is not required by any test
• WHEN the static-deps command is invoked
• THEN the adapter SHALL include the source file in the unresolved list
• AND the core SHALL apply the fallback mechanism (TIA-SAFE-004)

Scenario: Non-Clojure file silently ignored

• GIVEN a changed file that is not a .clj, .cljs, or .cljc file (e.g. .edn, .md)
• WHEN the static-deps command is invoked
• THEN the adapter SHALL NOT include the file in its response
• AND the core SHALL handle the unmodeled file via existing fallback mechanisms

Scenario: Dependency on Java interop

• GIVEN a source file with (:import java.util.Date) in its ns form
• AND a test depending on that source file via :require
• WHEN the static-deps command is invoked
• THEN the adapter SHALL NOT create a separate edge for java.util.Date
• AND the existing edge from the test to the source file SHALL remain

Requirement: TIA-ADAPT-020 — TypeScript adapter handshake (tree-sitter deps)

A TypeScript adapter SHALL declare "typescript" as one of its supported languages in the handshake. The adapter SHALL declare symbol_model_complete: false and runtime_edges: false in its handshake capabilities. The adapter SHALL declare granularity: "file".

The adapter SHALL be a Rust binary in the testaruda workspace (not a separate repository), as a separate crate named adapter-typescript producing the binary testaruda-adapter-typescript. The testaruda.toml adapter config SHALL map .ts, .tsx, .mts, and .cts extensions to this binary.

The adapter SHALL support both TypeScript and TSX syntax. For .ts files, the adapter SHALL use the tree-sitter-typescript typescript grammar. For .tsx files, the adapter SHALL use the tree-sitter-typescript tsx grammar. For .mts and .cts files, the adapter SHALL use the typescript grammar (same as .ts).

Scenario: TypeScript handshake with static deps

• GIVEN a TypeScript adapter binary is spawned
• WHEN the handshake command is invoked
• THEN the adapter SHALL include "typescript" in its languages array
• AND SHALL declare symbol_model_complete: false
• AND SHALL declare runtime_edges: false
• AND SHALL declare granularity: "file"

Scenario: Missing test runner

• GIVEN a system with the adapter binary installed but without Vitest or Jest available (no npx, no local node_modules/.bin/vitest or jest)
• WHEN the run-args command is invoked
• THEN the adapter SHALL return an error indicating no supported runner was found

Scenario: TSX file uses tsx grammar

• GIVEN a .tsx file with JSX syntax
• WHEN the adapter parses it via tree-sitter
• THEN the adapter SHALL use the tsx grammar from tree-sitter-typescript
• AND SHALL correctly parse JSX elements, fragments, and expressions

Scenario: MTS/CTS files use typescript grammar

• GIVEN a .mts or .cts file
• WHEN the adapter parses it via tree-sitter
• THEN the adapter SHALL use the typescript grammar (same as .ts)
• AND SHALL correctly parse the file

Requirement: TIA-ADAPT-021 — TypeScript discover scope (describe/it/test + file conventions)

A TypeScript adapter SHALL discover test items by scanning the project for .ts, .tsx, .mts, and .cts files and applying two strategies:

1. Declaration-based discovery: Parse each file with tree-sitter and match describe, it, test, describe.each, it.each, and test.each call expressions using tree-sitter queries. Each test-function call with a string literal first argument SHALL produce one test item. For .each calls, the string argument is at position 2 (after the data table).

2. File-convention discovery: Any file matching *.test.ts, *.test.tsx, *.spec.ts, *.spec.tsx, *.test.mts, *.spec.mts, *.test.cts, *.spec.cts, or located in a __tests__/ or __test__/ directory SHALL produce a file-level test item, even if no test declarations are found in it.

Node IDs SHALL follow the format <file-path>::<test-name>, where <file-path> is relative to the project root. For nested describe blocks, <test-name> SHALL be the ::-separated chain of description strings from outermost to innermost (e.g., file::Calculator::adds two numbers for describe("Calculator", () => { it("adds two numbers", ...) })). For file-convention test items, the node ID SHALL be the file path alone.

Suite kind: the adapter SHALL always return suite_kind: "unit" for all discovered test items.

The adapter SHALL discover test files under the src/ and test/ directories (or their equivalents), including nested subdirectories. The adapter SHALL NOT discover files in node_modules/, dist/, build/, .next/, or .git/.

Scenario: Discover Vitest tests from describe/it blocks

• GIVEN a TypeScript project with a file tests/calculator.test.ts containing:

  describe("Calculator", () => {
    it("adds two numbers", () => { expect(1+1).toBe(2); });
  });
  

• WHEN the discover command is invoked
• THEN the adapter SHALL return two test items:
  • tests/calculator.test.ts::Calculator (from describe)
  • tests/calculator.test.ts::Calculator::adds two numbers (from it nested in describe)
• AND each item SHALL have suite_kind: "unit"
• AND each item SHALL have the correct file path

Scenario: Discover test from top-level test() call

• GIVEN a file src/utils.test.ts with test("isEven returns true for even numbers", () => { ... })
• WHEN the discover command is invoked
• THEN the adapter SHALL return a test item with node ID src/utils.test.ts::isEven returns true for even numbers

Scenario: Discover parameterized tests with describe.each

• GIVEN a file tests/calculator.test.ts with:

  describe.each([1, 2, 3])("value %i", (n) => {
    it(`works with ${n}`, () => {});
  });
  

• WHEN the discover command is invoked
• THEN the adapter SHALL return a test item with node ID containing value %i (from describe.each)
• AND SHALL return test items for each it block nested inside

Scenario: Discover parameterized tests with test.each

• GIVEN a file tests/utils.test.ts with test.each([{a:1,b:2,expected:3}])("$a + $b = $expected", ({a,b,expected}) => {})
• WHEN the discover command is invoked
• THEN the adapter SHALL return a test item with node ID containing $a + $b = $expected

Scenario: File-convention discovery with no test declarations

• GIVEN a file __tests__/integration.ts that imports and calls test functions dynamically but has no describe/it/test declarations
• WHEN the discover command is invoked
• THEN the adapter SHALL return a file-level test item with the file path as the node ID

Scenario: Exclude node_modules

• GIVEN a TypeScript project with a node_modules directory containing .test.ts files
• WHEN the discover command is invoked
• THEN the adapter SHALL NOT return any test items from node_modules
• AND SHALL NOT discover tests from dist/, build/, or .git/

Requirement: TIA-ADAPT-022 — TypeScript dependency analysis (import-based static deps)

A TypeScript adapter SHALL build dependency edges by parsing export declarations from changed source files and import declarations from test files using tree-sitter queries. For each changed source file, the adapter SHALL:

1. Parse its export declarations to determine which symbols it exports
2. Scan all discovered test files for import declarations (ES import ... from, CommonJS require(), dynamic import()) that reference the changed file’s path
3. For each matching test-to-source pair, emit a dependency edge from the test item to the source file

Each uniquely resolved source file that is imported by a test SHALL produce a dependency edge from the test item to the source file.

Import path resolution: The adapter SHALL resolve relative import paths (starting with ./, ../, or /) to file paths by:

1. Removing the leading ./ or ../ prefix and computing the canonical path relative to the importing file
2. Appending the file extension if absent — trying in order: .ts, .tsx, /index.ts, /index.tsx
3. Replacing hyphens with underscores is NOT needed for TypeScript (unlike Clojure) — TypeScript file names match import paths directly

Non-relative imports (e.g., lodash, @angular/core) SHALL be treated as external package imports, emitted as-is without file-resolution attempts.

tsconfig.json path resolution: The adapter SHALL read tsconfig.json (if present) and apply compilerOptions.paths and compilerOptions.baseUrl to resolve path-alias imports (e.g., @/components/Button → src/components/Button.ts). The adapter SHALL strip JSON comments before parsing. If tsconfig.json is absent, path-aliased imports SHALL be treated as external and SHALL NOT produce dependency edges; the adapter SHALL emit a warning to stderr.

Non-code imports: import "./styles.css" or import logo from "./logo.png" — the adapter SHALL resolve these to their target file paths (if the files exist on disk) and include them as dependency edges.

Circular import safety: When resolving re-export chains, the adapter MUST maintain a visited-set of file paths and MUST NOT re-visit a file already in the chain, preventing infinite loops from circular imports.

Re-exports (export { X } from "./module") SHALL be followed for one hop only: the adapter resolves the immediate re-export source of the direct import target. Deeper chains are not followed. Wildcard re-exports (export * from "./module") are treated the same way — one hop of resolution.

The adapter SHALL emit edges at file-level granularity with origin: "static".

Scenario: Static dependency from ES import

• GIVEN a changed source file src/calculator.ts exporting a Calculator class
• AND a test file tests/calculator.test.ts with import { Calculator } from "../src/calculator"
• WHEN the static-deps command is invoked with changed_files: ["src/calculator.ts"]
• THEN the adapter SHALL parse src/calculator.ts for export declarations
• AND SHALL scan test files for import declarations referencing ../src/calculator
• AND SHALL return an edge from the test item to src/calculator.ts
• AND the edge SHALL have origin: "static"
• AND the edge SHALL have weight: 1000000

Scenario: Static dependency from require() call

• GIVEN a changed source file src/utils.ts
• AND a test file tests/utils.test.ts with const { helper } = require("../src/utils")
• WHEN the static-deps command is invoked with changed_files: ["src/utils.ts"]
• THEN the adapter SHALL parse src/utils.ts for exports
• AND scan test imports for require("../src/utils")
• AND SHALL return an edge from the test item to src/utils.ts

Scenario: Multiple imports produce multiple edges

• GIVEN a test file that imports from two different source modules
• WHEN the static-deps command is invoked with changes to both
• THEN the adapter SHALL return edges from the test to each changed file

Scenario: Extension-less import resolution

• GIVEN a changed file src/models/user.ts
• AND a test that imports import { User } from "../src/models/user" (no .ts extension)
• WHEN the static-deps command is invoked
• THEN the adapter SHALL resolve ../src/models/user to src/models/user.ts
• AND SHALL return an edge from the test to src/models/user.ts

Scenario: Barrel file import resolution (index.ts)

• GIVEN a barrel file src/models/index.ts that re-exports from src/models/user.ts
• AND a test that imports import { User } from "../src/models" (resolves to src/models/index.ts)
• WHEN the static-deps command is invoked with changed_files: ["src/models/user.ts"]
• THEN the adapter SHALL follow the re-export and return an edge from the test to src/models/user.ts

Scenario: Re-export chain (one level)

• GIVEN a file src/core.ts exporting a function
• AND a barrel file src/index.ts with export { coreFn } from "./core"
• AND a test that imports import { coreFn } from "../src" (resolves to src/index.ts)
• WHEN the static-deps command is invoked with changed_files: ["src/core.ts"]
• THEN the adapter SHALL follow the re-export
• AND SHALL return an edge from the test to src/core.ts

Scenario: Non-relative (package) import emitted as-is

• GIVEN a test file with import { Component } from "@angular/core"
• WHEN the static-deps command is invoked
• THEN the adapter SHALL NOT resolve @angular/core to a file path
• AND the import SHALL NOT produce a local dependency edge

Scenario: Changed source file with no test coverage (TypeScript)

• GIVEN a changed source file that no test imports
• WHEN the static-deps command is invoked
• THEN the adapter SHALL include the source file in the unresolved list
• AND the core SHALL apply the fallback mechanism (TIA-SAFE-004)

Scenario: Type-only imports produce edges

• GIVEN a source file src/types.ts with type definitions
• AND a test file with import type { MyType } from "../src/types"
• WHEN the static-deps command is invoked
• THEN the adapter SHALL return an edge from the test to src/types.ts
• AND the edge SHALL have origin: "static"
• AND weight: 1000000

Scenario: Dynamic import() produces edge

• GIVEN a test file with const mod = await import("../src/dynamic_module")
• WHEN the static-deps command is invoked
• THEN the adapter SHALL include a static edge from the test to src/dynamic_module.ts

Scenario: tsconfig.json path alias resolution

• GIVEN a project with tsconfig.json containing "paths": {"@/*": ["src/*"]} and "baseUrl": "."
• AND a changed source file src/components/Button.ts
• AND a test file with import { Button } from "@/components/Button"
• WHEN the static-deps command is invoked with changed_files: ["src/components/Button.ts"]
• THEN the adapter SHALL resolve @/components/Button to src/components/Button.ts using tsconfig.json paths
• AND SHALL return an edge from the test to src/components/Button.ts

Scenario: tsconfig.json absent — path-alias imports produce warning

• GIVEN a project with NO tsconfig.json
• AND a test file with import { X } from "@/components/X"
• WHEN the static-deps command is invoked
• THEN the adapter SHALL NOT resolve @/components/X to a file path
• AND the adapter SHALL emit a warning to stderr indicating that tsconfig.json was not found and path-alias resolution is disabled
• AND the import SHALL NOT produce a local dependency edge

Scenario: Non-code import (CSS) produces edge

• GIVEN a source file src/styles.css
• AND a test file with import "./styles.css"
• WHEN the static-deps command is invoked with changed_files: ["src/styles.css"]
• THEN the adapter SHALL resolve ./styles.css to src/styles.css
• AND SHALL return an edge from the test to src/styles.css
• AND the edge SHALL have origin: "static"

Scenario: Non-TypeScript file silently ignored

• GIVEN a changed file that is not a .ts, .tsx, .mts, .cts file (e.g. .json, .md)
• WHEN the static-deps command is invoked
• THEN the adapter SHALL NOT include the file in its response
• AND the core SHALL handle the unmodeled file via existing fallback mechanisms

Scenario: TypeScript adapter does not deduplicate identical import paths

• GIVEN a changed source file
• AND two different test files that both import from it
• WHEN the static-deps command is invoked
• THEN the adapter SHALL return two edges (one per importing test)
• AND each edge SHALL reference the correct test node ID

Scenario: Circular re-export chain handled safely

• GIVEN file a.ts re-exports from b.ts, and b.ts re-exports from a.ts (circular)
• WHEN the adapter resolves re-exports during static-deps
• THEN the adapter SHALL NOT loop infinitely
• AND SHALL produce at most one edge per unique test-to-source pair

Scenario: Wildcard re-export from barrel produces edge

• GIVEN a barrel file src/index.ts with export * from "./core"
• AND a test importing import { coreFn } from "../src"
• AND src/core.ts is changed
• WHEN the static-deps command is invoked
• THEN the adapter SHALL follow the wildcard re-export
• AND SHALL return an edge from the test to src/core.ts

LLM Agent Mode

Purpose

Selection output format for consumption by LLM coding agents — structured JSON, byte-stable output, reason chains, and coverage gap surfacing.

Requirements

Requirement: TIA-AGENT-001 — Structured agent output

Where the agent output format is requested, the CLI SHALL emit a structured JSON object containing the selection, per-test reasons, confidence, changed units, and summary statistics. Each per-test entry SHALL include the test’s numeric id, string node_id (file path), boolean always_run, optional fallback_reason string explaining why the test is always-run when dependency edges are missing, an array reason_chain, and numeric confidence.

Scenario: Agent JSON output with node_id and fallback_reason

• GIVEN a request for agent output format with missing dependency edges
• WHEN selection is computed
• THEN the CLI SHALL emit a JSON object with selection, per-test reasons, confidence, changed units, and summary statistics
• AND each per-test entry SHALL include id, node_id, always_run, optional fallback_reason, reason_chain, and confidence
• AND when always_run is true and edges are missing, fallback_reason SHALL explain why

Scenario: Agent JSON output

• GIVEN a request for agent output format
• WHEN selection is computed
• THEN the CLI SHALL emit a structured JSON object on stdout
• AND the output SHALL contain selection, changed_units, and summary

Requirement: TIA-AGENT-002 — Byte-stable agent output

Given the same change set and store state, the CLI in agent mode SHALL produce byte-stable output. Agent mode SHALL implicitly enforce deterministic output ordering (TIA-SEL-005) regardless of parallel evaluation (TIA-COMP-008), without requiring a separate flag from the caller.

Scenario: Deterministic agent output

• GIVEN identical inputs and store state
• WHEN agent mode is used twice
• THEN both outputs SHALL be byte-identical

Requirement: TIA-AGENT-003 — Reason chains for selected and skipped

When the agent requests an explanation, the CLI SHALL include for each selected test its reason chain and for each skipped test its exclusion reason.

Scenario: Full explanation

• GIVEN an agent explanation request
• WHEN selection output is produced
• THEN each selected test SHALL include its reason chain
• AND each skipped test SHALL include its exclusion reason

Requirement: TIA-AGENT-004 — Specific test query

When the agent queries a specific test, the CLI SHALL answer why that test was or was not selected.

Scenario: Single test query

• GIVEN an agent query for a specific test ID
• WHEN the CLI processes the query
• THEN it SHALL return whether the test was selected
• AND SHALL explain why

Requirement: TIA-AGENT-005 — Pre-edit blast radius (structured JSON)

Where pre-edit mode is requested, the CLI SHALL report the blast radius of a proposed change without requiring the edit to be applied. The output SHALL be a structured JSON object, following the same pattern as --agent output ("format": "testaruda-agent-v1"), containing at minimum: the set of changed files, the set of affected tests, and a summary of changed/affected counts.

Scenario: Pre-edit analysis

• GIVEN proposed file changes and a pre-edit mode request
• WHEN the CLI evaluates the blast radius
• THEN it SHALL report which tests would be affected without applying the edit
• AND the output SHALL be a structured JSON object with a versioned format field
• AND the output SHALL include changed files, affected tests, and summary counts

Requirement: TIA-AGENT-006 — Coverage gap surfacing

When a changed symbol has no covering test, the CLI SHALL surface that coverage gap in agent output.

Scenario: Coverage gap detection

• GIVEN a changed symbol with no covering tests
• WHEN agent output is produced
• THEN the CLI SHALL surface the coverage gap

Requirement: TIA-AGENT-007 — Deterministic default for agent/gate

While serving an agent or a merge gate, the CLI SHALL default to deterministic selection and SHALL NOT apply non-deterministic predictive ranking unless explicitly enabled.

Scenario: Deterministic agent mode

• GIVEN agent or merge gate mode
• WHEN selection is computed
• THEN the CLI SHALL default to deterministic selection
• AND SHALL NOT apply predictive ranking unless explicitly enabled

Requirement: TIA-AGENT-008 — Schema and agent-facing documentation

The agent output format ("format": "testaruda-agent-v1") SHALL have an accompanying JSON Schema file in the repository describing the full payload shape. The repository SHALL include a docs/agent-mode.md document explaining how AI coding agents should consume testaruda’s agent output — including interpreting has_coverage_gaps, fallback_reason, coverage_gaps fields and integrating them into a decision loop (e.g. “write a test before merging if has_coverage_gaps: true”).

Scenario: Schema shipped

• GIVEN the agent output format "format": "testaruda-agent-v1"
• WHEN the release artifact is built
• THEN a JSON Schema file SHALL exist in the repository at a canonical path (e.g. schemas/agent-output-v1.json)
• AND the schema SHALL describe all fields of the agent output payload

Scenario: Agent documentation

• GIVEN the repository
• WHEN looking for agent integration guidance
• THEN docs/agent-mode.md SHALL exist
• AND it SHALL explain each field’s semantics
• AND it SHALL describe how an AI agent should use the tool

Architectural Constraints

Purpose

Foundational, engine-independent architectural constraints that bind the design of the entire system. These are the unifying abstraction that all other requirements instantiate.

Requirements

Requirement: TIA-ARCH-001 — K-relation representation

The core SHALL represent all dependency data as a single K-relation valued in a configured commutative semiring.

Scenario: Dependency data as K-relation

• GIVEN dependency data loaded from the store
• WHEN the core processes it for selection
• THEN it SHALL represent it as a single K-relation valued in a configured commutative semiring
• AND the semiring SHALL be configurable per invocation

Requirement: TIA-ARCH-002 — Provenance semiring as master

The core SHALL treat the provenance (polynomial) semiring as the master representation from which all other semiring values are derived.

Scenario: Derived semiring values

• GIVEN dependency edges stored in the store
• WHEN any semiring value is needed
• THEN the provenance (polynomial) semiring SHALL be the master representation
• AND all other semiring values SHALL be derived from it

Requirement: TIA-ARCH-003 — Positive reachability by homomorphism

The core SHALL derive the positive reachability results (selection (Boolean), distance (tropical), cost/scheduling, and explanation (provenance polynomial — the master)) from the provenance representation by semiring homomorphism, such that no positively-derived result can disagree with the master computation. The always-run set (TIA-SAFE-007), confidence-threshold fallback (TIA-SAFE-002), and duration ranking (TIA-SEL-006) involve negation-as-failure or aggregation over external signals; they are not derived by semiring closure but contributed as input facts to the selection relation, and are not required to be homomorphic images of the provenance column. Path confidence (Viterbi) annotates the same paths but is likewise excluded from the homomorphism guarantee.

Scenario: Homomorphism correctness

• GIVEN a provenance polynomial for a test item
• WHEN the Boolean selection is derived via semiring homomorphism
• THEN the Boolean result SHALL NOT be false if the provenance polynomial has any positive term
• AND the tropical distance SHALL be consistent with the shortest path in the provenance polynomial

Scenario: Always-run exclusion from homomorphism

• GIVEN a test in the always-run set
• WHEN selection is computed
• THEN its selection SHALL NOT be required to be a homomorphic image of the provenance column
• AND it SHALL be contributed as an input fact to the selection relation

Requirement: TIA-ARCH-004 — Selection as transpose of closure

The selector SHALL compute the affected set by evaluating the change set against the transpose of the transitive-closure relation.

Scenario: Transpose evaluation

• GIVEN a change set Δ and a transitive-closure relation D*
• WHEN computing the affected set
• THEN the selector SHALL evaluate Δ · (D*)ᵀ

Requirement: TIA-ARCH-005 — Least fixpoint for transitive dependency

The core SHALL compute transitive dependency as the least fixpoint (semiring star) of the one-step dependency relation.

Scenario: Fixpoint computation

• GIVEN a one-step dependency relation
• WHEN computing transitive closure
• THEN the core SHALL compute it as the least fixpoint of the one-step relation

Requirement: TIA-ARCH-006 — Scope-bounded recomputation

The core SHALL compute selection as a scope-bounded recomputation restricted to the subgraph from which the change set is backward-reachable (i.e. the transitive reverse-closure of the changed content units), rather than the full graph.

Scenario: Scope bounding

• GIVEN a change set Δ in a large dependency graph
• WHEN computing selection
• THEN the core SHALL restrict recomputation to the transitive reverse-closure of the changed content units
• AND SHALL NOT traverse the full graph

Requirement: TIA-ARCH-007 — Fusion by semiring addition

The core SHALL fuse static, runtime, and manual edges by semiring addition over a common edge set, without origin-specific merge logic.

Scenario: Edge fusion

• GIVEN static, runtime, and manual edges for the same (test_item, content_unit, environment) triple
• WHEN computing selection
• THEN the core SHALL fuse them by semiring addition
• AND SHALL NOT use origin-specific merge logic

Requirement: TIA-ARCH-008 — Language-agnostic core

The core SHALL contain no language- or framework-specific logic; all such logic SHALL reside behind the adapter interface.

Scenario: Adapter isolation

• GIVEN requirements for a new language
• WHEN adding support for it
• THEN no changes SHALL be required to the core
• AND all language-specific logic SHALL be implemented in an adapter

Requirement: TIA-ARCH-009 — No test execution

The core SHALL NOT execute tests, compile, or build; it SHALL only select tests and emit native-runner arguments for an external executor.

Scenario: Selection-only operation

• GIVEN an invocation of the selection command
• WHEN execution completes
• THEN the core SHALL NOT have executed any tests
• AND SHALL only have emitted native-runner arguments

Requirement: TIA-ARCH-010 — Associative composition

The core SHALL compute selection across components and repositories by associative composition of per-unit K-relations, such that the result is independent of composition order.

Scenario: Composition order independence

• GIVEN components A, B, and C with dependency edges between them
• WHEN computing selection by composing (A·B)·C and A·(B·C)
• THEN both compositions SHALL produce identical affected sets

Change Detection

Purpose

How changes are detected and the change set Δ is computed from VCS diffs, explicit file lists, and content fingerprints.

Requirements

Requirement: TIA-CHG-001 — VCS diff

When invoked with a base and head revision, the core SHALL derive the changed file set from the version-control diff between them.

Scenario: Git diff between revisions

• GIVEN a base revision main and a head revision feature-branch
• WHEN the core is invoked with both revisions
• THEN it SHALL run git diff --name-only main feature-branch (or equivalent)
• AND use the output as the changed file set

Requirement: TIA-CHG-002 — Explicit file list

Where the caller supplies an explicit changed-file list, the core SHALL use that list as the change source in place of a diff.

Scenario: Explicit file override

• GIVEN an explicit list of changed files
• WHEN selection is invoked
• THEN the core SHALL use that list directly
• AND SHALL NOT run a VCS diff

Requirement: TIA-CHG-003 — Fingerprint-based change set

The core SHALL compute the change set Δ as the content units whose fingerprint differs between base and head.

Scenario: Fingerprint comparison

• GIVEN base and head fingerprints for the same content unit
• WHEN the fingerprints differ
• THEN the content unit SHALL be included in Δ
• AND when fingerprints match, it SHALL be excluded from Δ

Requirement: TIA-CHG-004 — Symbol-level granularity

Where the responsible adapter declares symbol granularity and sets the symbol_model_complete capability flag to true in its handshake (TIA-ADAPT-002) (asserting that no dependency paths exist beyond those it enumerates for the file), the core SHALL fingerprint only the changed symbols/blocks so that whitespace-only or unrelated edits within a file yield no affected tests for unchanged symbols. If the adapter does not set symbol_model_complete (e.g. due to reflection, macros, or dynamic dispatch), the core SHALL fall back to file-level granularity for that file to preserve TIA-SAFE-001.

Scenario: Symbol-level with complete model

• GIVEN an adapter with symbol_model_complete: true
• WHEN a symbol within a file changes but other symbols do not
• THEN only tests depending on the changed symbol SHALL be selected
• AND tests depending only on unchanged symbols SHALL NOT be selected

Scenario: Fallback to file-level

• GIVEN an adapter without symbol_model_complete or with symbol_model_complete: false
• WHEN any change occurs in a file
• THEN the core SHALL treat the entire file as changed (file-level granularity)

Requirement: TIA-CHG-005 — Dependency fingerprint check

The core SHALL compute each test item’s dependency fingerprint as a function of its dependency content-unit fingerprints and its environment fingerprint, and SHALL select a test item if and only if its dependency fingerprint changed or it is in the always-run set.

Scenario: Dependency fingerprint change

• GIVEN a test whose dependency content fingerprints have changed
• WHEN selection is computed
• THEN the test SHALL be selected
• AND a test whose dependency fingerprints are unchanged and is not in always-run SHALL NOT be selected

Requirement: TIA-CHG-006 — External resource change detection

The core SHALL treat lockfiles, configuration, and adapter-declared external resources as content units subject to change detection.

Scenario: Lockfile change

• GIVEN a changed lockfile
• WHEN selection is computed
• THEN the lockfile SHALL be treated as a changed content unit
• AND tests depending on it SHALL be selected

Requirement: TIA-CHG-007 — Unknown file kind fallback

If a changed file has a kind the core cannot model and no known edges, then the core SHALL raise a fallback signal for the affected component.

Scenario: Unmodeled file kind

• GIVEN a changed file of an unknown kind with no known dependency edges
• WHEN selection is computed
• THEN the core SHALL raise a fallback signal for the affected component

Requirement: TIA-CHG-008 — CI mode working tree exclusion

While operating in CI mode, the core SHALL NOT read the local working tree as a change source.

Scenario: CI mode change source

• GIVEN CI mode is active
• WHEN selection is invoked
• THEN the core SHALL only use the provided base/head revisions or explicit file list
• AND SHALL NOT read the local working tree

Requirement: TIA-CHG-009 — Init-time language detection

The adapter SHALL probe for the following project files in well-known locations: (1) Python: pyproject.toml, setup.py, setup.cfg; (2) Rust: Cargo.toml; (3) JavaScript/TypeScript: package.json, vitest.config.ts, jest.config.ts, tsconfig.json; (4) Go: go.mod; (5) Clojure: deps.edn, project.clj.

When a marker is found (and no other project file takes priority), the core SHALL set the default adapter to the corresponding binary.

The core SHALL use the following priority order (first match wins, most specific first):

1. Cargo.toml SHALL map to Rust testaruda-adapter-rust
2. vitest.config.ts or jest.config.ts SHALL map to TypeScript testaruda-adapter-typescript
3. package.json (with vitest or jest in devDependencies) SHALL map to TypeScript testaruda-adapter-typescript
4. pyproject.toml, setup.py, setup.cfg SHALL map to Python testaruda-adapter-python
5. deps.edn, project.clj SHALL map to Clojure testaruda-adapter-clojure
6. go.mod SHALL map to Go adapter (when implemented)
7. package.json (without vitest/jest) SHALL result in fallback (no auto-detection)

Scenario: TypeScript project detection via vitest.config.ts

• GIVEN a directory with vitest.config.ts but no Cargo.toml or pyproject.toml
• WHEN init is invoked
• THEN the generated testaruda.toml SHALL have default = "testaruda-adapter-typescript"

Scenario: TypeScript project detection via jest.config.ts

• GIVEN a directory with jest.config.ts but no vitest.config.ts, Cargo.toml, or pyproject.toml
• WHEN init is invoked
• THEN the generated testaruda.toml SHALL have default = "testaruda-adapter-typescript"

Scenario: TypeScript project detection via package.json devDependencies

• GIVEN a directory with package.json containing "vitest" or "jest" in devDependencies but no vitest.config.ts, jest.config.ts, Cargo.toml, or pyproject.toml
• WHEN init is invoked
• THEN the generated testaruda.toml SHALL have default = "testaruda-adapter-typescript"

Scenario: TypeScript config wins over Python

• GIVEN a directory with both vitest.config.ts and pyproject.toml
• WHEN init is invoked
• THEN the generated testaruda.toml SHALL have default = "testaruda-adapter-typescript"
• AND the Python adapter SHALL still be registered as a secondary extension mapping for .py files

Requirement: TIA-CHG-010 — Cold-start content-unit classification

A content unit with no prior fingerprint of record (i.e., first observed by the system) SHALL be classified as unresolved, with confidence set to zero. Repeated invocations with no intervening fingerprint change SHALL produce the same classification as the first observation.

Scenario: Cold-start content unit classified as unresolved

• GIVEN a content unit with no prior fingerprint of record in the store
• WHEN the change set Δ is computed
• THEN the content unit SHALL be classified as unresolved
• AND its confidence SHALL be set to zero, triggering the confidence-based fallback (TIA-SAFE-002)

Scenario: Idempotent classification across repeated invocations

• GIVEN a content unit that was previously classified as unresolved on first observation
• AND no intervening change has occurred
• WHEN the change set Δ is computed again
• THEN the content unit SHALL receive the same classification (unresolved) as on the first invocation
• AND its confidence SHALL remain zero

CI Integration

Purpose

Selection behavior in CI pipeline mode — exit codes, shadow mode, machine-readable plan output, and result ingestion.

Requirements

Requirement: TIA-CI-001 — Success exit code

When selection completes successfully, the CLI SHALL exit with code 0.

Scenario: Successful selection

• GIVEN a selection that completes without errors
• WHEN the CLI exits
• THEN the exit code SHALL be 0

Requirement: TIA-CI-002 — Full-run signal

If confidence requires a full run, then the CLI SHALL exit with code 10 to signal “run everything.”

Scenario: Low confidence full run

• GIVEN confidence below threshold requires a full run
• WHEN the CLI exits
• THEN the exit code SHALL be 10

Requirement: TIA-CI-003 — Empty selection signal

When the full selection set (reachability ∪ always-run ∪ fallback, where fallback is scoped per affected component per TIA-SAFE-003) is empty, the CLI SHALL exit with code 20 to signal “safe to skip.”

Scenario: Empty selection

• GIVEN no tests are selected (reachability, always-run, and fallback all empty)
• WHEN the CLI exits
• THEN the exit code SHALL be 20

Requirement: TIA-CI-004 — Error exit code

If a non-recoverable error occurs, then the CLI SHALL exit with a non-zero code distinct from 10 and 20.

Scenario: Hard error

• GIVEN a non-recoverable error (e.g. database corruption)
• WHEN the CLI exits
• THEN the exit code SHALL be non-zero
• AND SHALL NOT be 10 or 20

Requirement: TIA-CI-005 — Fallback default action

The CLI SHALL treat exit code 10 and any unknown condition as “run all tests,” never as “run no tests.”

Scenario: Unknown condition safety

• GIVEN an unknown or unexpected exit condition
• WHEN the CI pipeline interprets the result
• THEN it SHALL run all tests
• AND SHALL NOT skip any tests

Requirement: TIA-CI-006 — Machine-readable plan

Where a machine-readable format is requested, the CLI SHALL emit the selection plan as JSON or a runner-native plan.

Scenario: JSON plan output

• GIVEN a request for machine-readable output
• WHEN selection is computed
• THEN the CLI SHALL emit the plan as JSON or a runner-native format

Requirement: TIA-CI-007 — Shadow mode

Where shadow mode is enabled, the CLI SHALL compute and record the selection but report that all tests should run.

Scenario: Shadow mode selection

• GIVEN shadow mode is enabled
• WHEN selection is computed
• THEN the CLI SHALL compute and record the selection
• BUT SHALL report that all tests should run

Requirement: TIA-CI-008 — Result ingestion

When a CI run finishes, the CLI SHALL accept ingestion of its results to update the model.

Scenario: CI results ingestion

• GIVEN a completed CI run with results
• WHEN the ingest command is invoked
• THEN the CLI SHALL accept and process the results to update the model

cli-core Specification

Purpose

TBD - created by archiving change upgrade-genesis. Update Purpose after archive.

Requirements

Requirement: CLI scaffold uses Guide

The tool’s main.rs CLI setup SHALL use genesis::guide::Guide::builder() when the tool adopts genesis::guide. Adoption is determined by the tool’s maintainer — this spec defines the contract for how to do it.

Scenario: Guide builder replaces ad-hoc setup

• GIVEN the tool adopts genesis::guide
• WHEN main.rs is updated to use Guide::builder(...)
• THEN command handlers SHOULD return Output<T> or use ErrorSink
• AND cargo test SHALL pass

Scenario: ErrorSink for self-healing errors

• GIVEN the tool adopts genesis::guide
• WHEN a command returns an error
• THEN ErrorSink SHOULD print the error with a suggestion footer
• AND it SHOULD write to the error scratch (for --from-last-error)

Composability

Purpose

Multi-component and multi-repository composition — bottom-up resolution, manifest export, parallel evaluation, and caching.

Requirements

Requirement: TIA-COMP-001 — Component graph

The store SHALL maintain a component graph distinct from the fine-grained test-to-content-unit graph.

Scenario: Separate graph structures

• GIVEN the store
• WHEN querying the component graph
• THEN it SHALL be distinct from the test-to-content-unit graph

Requirement: TIA-COMP-002 — Inter-component edges

The store SHALL record inter-component dependency edges with an origin.

Scenario: Cross-component recording

• GIVEN a dependency between components
• WHEN it is stored
• THEN the edge SHALL include its origin (static, runtime, or manual)

Requirement: TIA-COMP-003 — Bottom-up component resolution

When computing selection in a monorepo, the core SHALL first resolve affected components bottom-up from the change set, then select within them.

Scenario: Bottom-up resolution

• GIVEN a change set in a monorepo with multiple components
• WHEN selection is computed
• THEN affected components SHALL be resolved bottom-up from the change set
• AND selection SHALL be computed within resolved components

Requirement: TIA-COMP-004 — Multi-repo manifest

Where multi-repo operation is configured, each repository SHALL be able to export a manifest of its components, their public-interface fingerprints, and a version.

Scenario: Manifest export

• GIVEN multi-repo operation configured
• WHEN a repository is queried
• THEN it SHALL export a manifest containing its components, their public-interface fingerprints, and version

Requirement: TIA-COMP-005 — Cross-repo edge recording

When a consumer repository records a dependency on another repository, the core SHALL record an edge to that repository’s published interface fingerprint and version.

Scenario: Cross-repo edge

• GIVEN a consumer repository depending on another repository
• WHEN the dependency is recorded
• THEN the edge SHALL reference the published interface fingerprint and version

Requirement: TIA-COMP-006 — Interface change propagation

When a published interface fingerprint changes, the core SHALL mark dependent tests in consumer repositories as affected.

Scenario: Cross-repo change propagation

• GIVEN a repository whose published interface fingerprint changes
• WHEN selection is computed in a consumer repository
• THEN the dependent tests SHALL be marked as affected

Requirement: TIA-COMP-007 — Aggregation without global lock

The core SHALL aggregate manifests across repositories without requiring a global lock or single shared database.

Scenario: Decentralized aggregation

• GIVEN manifests from multiple repositories
• WHEN they are aggregated
• THEN the core SHALL NOT require a global lock or shared database

Requirement: TIA-COMP-008 — Parallel per-component selection

The core SHALL compute per-component selection in parallel.

Scenario: Parallel evaluation

• GIVEN multiple components to evaluate
• WHEN selection runs
• THEN per-component selection SHALL be computed in parallel

Requirement: TIA-COMP-009 — Order-independent results

The core SHALL produce identical affected sets regardless of the order in which components or repositories are composed.

Scenario: Order invariance

• GIVEN the same dependency data processed in different component orders
• WHEN selection is computed
• THEN the affected set SHALL be identical regardless of composition order

Requirement: TIA-COMP-010 — Cached selection decisions

The core SHALL key a component’s cached selection decision on its dependency fingerprint, and SHALL reuse the cached decision when the fingerprint is unchanged.

Scenario: Cache reuse

• GIVEN a component whose dependency fingerprint matches a cached value
• WHEN selection is computed
• THEN the cached selection decision SHALL be reused

Requirement: TIA-COMP-011 — Remote cache

Where a remote cache is configured, the core SHALL share and retrieve cached selection decisions across machines through a local-then-remote lookup.

Scenario: Remote cache lookup

• GIVEN a configured remote cache
• WHEN a selection decision is needed
• THEN the core SHALL look up the local cache first, then the remote cache

Requirement: TIA-COMP-012 — Shard plan

Where sharding is requested, the core SHALL emit a balanced shard plan computed over recorded test durations.

Scenario: Duration-balanced sharding

• GIVEN a request for sharding
• WHEN the shard plan is computed
• THEN it SHALL balance shards based on recorded test durations

Confidence Scoring

Purpose

Confidence computation from dependency evidence quality — Viterbi path confidence, invocation-level quality signals, and fallback gating.

Requirements

Requirement: TIA-CONF-001 — Confidence range

The core SHALL compute a selection confidence in the range [0, 1].

Scenario: Confidence output

• GIVEN a selected test
• WHEN its selection confidence is computed
• THEN the confidence SHALL be in the range [0, 1]

Requirement: TIA-CONF-002 — Per-invocation confidence floor

The core SHALL apply a per-invocation confidence floor derived from at least coverage freshness, adapter resolution ratio, history depth, and environment match, such that the effective path confidence reported by TIA-CONF-001 reflects the quality of the dependency evidence for the current invocation. These are invocation-level quality signals and SHALL NOT mutate stored edge weights, ensuring TIA-REL-001 (deterministic selections for identical store state) is preserved.

Scenario: Invocation-level quality adjustment

• GIVEN stale coverage data, low adapter resolution, and poor environment match
• WHEN selection confidence is computed
• THEN the effective confidence SHALL be adjusted downward from stored edge weights
• AND stored edge weights SHALL remain unchanged

Requirement: TIA-CONF-003 — Confidence reporting

The core SHALL report the confidence value to every consumer interface.

Scenario: Confidence in output

• GIVEN any consumer interface (CLI, JSON, agent output)
• WHEN selection results are emitted
• THEN the confidence value SHALL be included in the output

Requirement: TIA-CONF-004 — Confidence meaning documentation

The core SHALL document, wherever confidence is reported, that confidence gates fallback and is not a probability that the suite is correct.

Scenario: Confidence disclaimer

• GIVEN confidence values reported in output
• WHEN documentation accompanies the output
• THEN the documentation SHALL state that confidence gates fallback and is not a probability of correctness

config Specification

Purpose

TBD - created by archiving change upgrade-genesis. Update Purpose after archive.

Requirements

Requirement: Shared config management

The tool SHALL adopt genesis::config for shared config management.

Scenario: config struct implements ConfigFile

• GIVEN the tool needs config management
• WHEN the tool adopts genesis::config
• THEN the tool’s config SHALL implement genesis::config::ConfigFile
• AND all config file I/O (read, write, parse) SHALL delegate to genesis
• AND cargo test SHALL pass

Scenario: config registered at startup

• GIVEN the tool starts up
• WHEN it initializes
• THEN it SHALL register its config struct with ConfigRegistry
• AND it SHOULD use ConfigStore for config discovery and validation

Scenario: dead config code removed

• GIVEN the tool has adopted genesis::config
• WHEN the old config parsing code is removed
• THEN cargo clippy SHALL introduce no new warnings
• AND cargo test SHALL pass

Core Data Model

Purpose

Entities and relationships that form the dependency graph data model — content units, test items, dependency edges, and run history.

Requirements

Requirement: TIA-CORE-001 — Content unit identification

The store SHALL identify each content unit by the tuple (component, path, symbol?).

Scenario: Content unit lookup

• GIVEN a content unit in a component
• WHEN it is stored or retrieved
• THEN its identity SHALL be the tuple (component, path, symbol?)
• AND the symbol field SHALL be optional (nullable)

Requirement: TIA-CORE-002 — Content fingerprint

The store SHALL record, for each content unit, a content fingerprint computed from the normalized unit content.

Scenario: Fingerprint storage

• GIVEN a content unit with known content
• WHEN it is inserted into the store
• THEN a content fingerprint SHALL be computed from the normalized content
• AND the fingerprint SHALL be recorded alongside the content unit

Requirement: TIA-CORE-003 — Content unit kind classification

The store SHALL classify each content unit by kind ∈ {source, config, fixture, lockfile, external}.

Scenario: Kind classification

• GIVEN a content unit being inserted
• WHEN the store records it
• THEN it SHALL be classified as one of: source, config, fixture, lockfile, or external

Requirement: TIA-CORE-004 — Test item identification

The store SHALL identify each test item by the tuple (component, adapter, node_id).

Scenario: Test item lookup

• GIVEN a test item in a component
• WHEN it is stored or retrieved
• THEN its identity SHALL be the tuple (component, adapter, node_id)

Requirement: TIA-CORE-005 — Dependency edge record

The store SHALL record each dependency edge as (test_item, content_unit, environment, origin, K-value).

Scenario: Edge recording

• GIVEN a dependency between a test and a content unit
• WHEN the dependency is recorded
• THEN it SHALL include test_item, content_unit, environment, origin, and K-value

Requirement: TIA-CORE-006 — Coexisting edges

The store SHALL permit static, runtime, and manual edges to coexist for the same (test_item, content_unit, environment) triple.

Scenario: Multiple origins per triple

• GIVEN a test_item, content_unit, and environment triple
• WHEN edges of different origins exist
• THEN the store SHALL permit all three origins (static, runtime, manual) to coexist

Requirement: TIA-CORE-007 — Reverse index

The store SHALL maintain a reverse index from content unit to dependent test items.

Scenario: Reverse lookup

• GIVEN a content unit
• WHEN querying for dependent tests
• THEN the store SHALL return all test items that depend on it via the reverse index

Requirement: TIA-CORE-008 — Environment partition

The store SHALL partition all edges and statistics by environment fingerprint.

Scenario: Environment-scoped queries

• GIVEN multiple environments with different fingerprints
• WHEN querying edges or statistics
• THEN the store SHALL return results scoped to the specified environment fingerprint

Requirement: TIA-CORE-009 — Run history

The store SHALL record per-test run history including outcome, attempt count, duration, and error signature.

Scenario: Run history recording

• GIVEN a completed test run
• WHEN results are ingested
• THEN the store SHALL record the outcome, attempt count, duration, and error signature for each test

doctor Specification

Purpose

TBD - created by archiving change adopt-genesis. Update Purpose after archive.

Requirements

Requirement: Doctor command

testaruda SHALL provide a doctor command (the true-suite-minimum verb it currently lacks — tool-craft §2.1) backed by genesis::suite_linter, running health checks and printing a per-check pass/fail verdict, with --fix applying safe fixes.

Scenario: doctor runs suite linter checks

• WHEN testaruda doctor is run
• THEN it SHALL run genesis::suite_linter checks: testaruda.toml schema conformance, pretender.toml presence when pretender is wired, ah check in pre-commit when .espectacular/ exists, dont check in pre-push when .dont/ exists, and managed-block/badge match
• AND SHALL exit 0 only when all non-skipped checks pass, 1 otherwise.

Scenario: doctor –fix applies safe fixes

• WHEN testaruda doctor --fix is run and a check carries a fix function
• THEN the fix SHALL be applied via genesis::suite_linter fix fns
• AND checks without a fix SHALL remain as ✗ with a ContextHint.

Requirement: feedback subcommand

testaruda SHALL provide a feedback subcommand that files a structured issue against testaruda’s upstream repo via gh, wrapping genesis::feedback for the redactor, context-bundle, error-scratch, and gh-invocation machinery.

Scenario: agent files a bug with last error

• WHEN testaruda feedback bug --from-last-error --yes is run after a non-zero exit
• THEN testaruda SHALL read its own error scratch
• AND SHALL assemble and redact the body via genesis::feedback
• AND SHALL invoke gh issue create against testaruda’s Cargo.toml repository with labels agent-reported, bug, has-repro.

Scenario: error with no self-healing fix

• WHEN testaruda exits non-zero and no genesis::suggestions::Fix is available
• THEN the error footer SHALL print Feedback: testaruda feedback bug --from-last-error.

Engine

Purpose

Reference implementation design constraints — the Ascent-embedded selection engine, its rule set, incrementality mechanism, and scale-up path.

Requirements

Requirement: TIA-ENG-001 — Single static binary

The reference implementation SHALL be distributed as a single statically linked binary that embeds the selection engine in-process.

Scenario: Binary distribution

• GIVEN the reference implementation
• WHEN it is built
• THEN it SHALL be a single statically linked binary
• AND the selection engine SHALL be embedded in-process

Requirement: TIA-ENG-002 — No separate engine process

The reference implementation SHALL NOT require a separate engine process to compute a selection.

Scenario: Self-contained selection

• GIVEN the reference implementation
• WHEN a selection is computed
• THEN no separate engine process SHALL be required

Requirement: TIA-ENG-003 — Ascent engine

The reference implementation SHALL use Ascent as the embedded logic engine for the selection query.

Scenario: Ascent dependency

• GIVEN the reference implementation’s Cargo.toml
• WHEN inspecting dependencies
• THEN ascent SHALL be listed as a dependency
• AND the selection query SHALL be embedded via the ascent! macro

Requirement: TIA-ENG-004 — Lattice columns for semirings

The reference implementation SHALL represent each semiring as a lattice column so that selection, confidence, and distance are produced by the same rule set under different lattice types.

Scenario: Lattice-based semirings

• GIVEN the Ascent rule set
• WHEN examining relation types
• THEN each semiring SHALL be represented as a lattice column
• AND the same rules SHALL produce selection, confidence, and distance under different lattice types

Requirement: TIA-ENG-005 — Union-find transitive closure

The reference implementation SHALL compute transitive dependency closure using a union-find–backed relation.

Scenario: Union-find for closure

• GIVEN a dependency graph
• WHEN computing transitive closure
• THEN the implementation SHALL use a union-find–backed relation

Requirement: TIA-ENG-006 — Data-parallel per-component selection

The reference implementation SHALL evaluate per-component selection using the engine’s data-parallel mode.

Scenario: Data-parallel engine evaluation

• GIVEN multiple components to evaluate
• WHEN selection is computed
• THEN the implementation SHALL use the engine’s data-parallel mode (e.g. ascent_par!)

Requirement: TIA-ENG-007 — Change scoping and component cache

The reference implementation SHALL provide cross-invocation incrementality through change scoping and the content-addressed component cache (TIA-COMP-010) rather than a persistent streaming engine.

Scenario: Cache-based incrementality

• GIVEN repeated invocations with overlapping inputs
• WHEN a new selection is computed
• THEN the implementation SHALL reuse cached component decisions when fingerprints match
• AND SHALL NOT use a persistent streaming engine

Requirement: TIA-ENG-008 — Scoped residual evaluation

The reference implementation SHALL evaluate the selection query over only the change-scoped residual subgraph.

Scenario: Scoped query

• GIVEN a change set Δ
• WHEN the Ascent program runs
• THEN it SHALL only load facts for the scope-bounded subgraph
• AND SHALL NOT load the full graph

Requirement: TIA-ENG-009 — Minimal-witness explanation

For explanation, the reference implementation SHALL compute a minimal-witness derivation (a single shortest reason chain) as a lattice value sufficient to satisfy TIA-PROV-002 and TIA-AGENT-003, without requiring full why-provenance for routine selection.

Scenario: Shortest-path witness

• GIVEN a selected test
• WHEN its reason is requested
• THEN the implementation SHALL return a minimal-witness derivation (shortest reason chain)
• AND SHALL NOT require full why-provenance

Requirement: TIA-ENG-010 — Soufflé oracle for full provenance

Where full why-provenance or independent validation is required, the reference implementation SHALL be able to evaluate the same rule set through an out-of-process Soufflé oracle.

Scenario: Soufflé validation

• GIVEN a need for full why-provenance
• WHEN the oracle command is invoked
• THEN the same rule set SHALL be evaluated through Soufflé
• AND results SHALL be comparable to the Ascent evaluation

Requirement: TIA-ENG-011 — Shadow mode cross-check

While operating in shadow mode (TIA-VER-001), the reference implementation SHALL be able to cross-check its selections against the Soufflé oracle and flag divergences.

Scenario: Shadow mode divergence detection

• GIVEN shadow mode is active
• WHEN the Ascent selection completes
• THEN it MAY be cross-checked against the Soufflé oracle
• AND any divergences SHALL be flagged

Requirement: TIA-ENG-012 — Engine-independent rule set

The selection rule set SHALL be engine-independent, such that it can be re-targeted to an alternative engine without changing the dependency model or any TIA-ARCH-* behaviour.

Scenario: Rule set portability

• GIVEN the selection rule set
• WHEN targeting a different engine (e.g. Soufflé, DBSP)
• THEN the dependency model and ARCH behaviours SHALL remain unchanged
• AND only the rule syntax SHALL need adaptation

Requirement: TIA-ENG-013 — DBSP scale-up path

Where long-running, fully-incremental selection at monorepo scale with retractions is required, the system SHALL be re-targetable to a streaming incremental engine (e.g. DBSP) as a documented scale-up path.

Scenario: Scale-up documentation

• GIVEN a need for streaming IVM at scale
• WHEN evaluating scale-up options
• THEN DBSP/Feldera SHALL be the documented scale-up path

Requirement: TIA-ENG-014 — Engine migration path

If a selected engine becomes unmaintained, then the system SHALL be re-targetable to an alternative engine through the engine-independent rule set without loss of TIA-ARCH-* behaviour.

Scenario: Engine migration

• GIVEN an unmaintained engine
• WHEN migrating to a replacement
• THEN the ARCH behaviours SHALL be preserved
• AND only the rule syntax SHALL need updating

Requirement: TIA-ENG-015 — Store independence

The reference implementation SHALL persist the dependency graph and statistics in the store of §5.10 independently of the engine’s in-memory working state.

Scenario: Independent persistence

• GIVEN the engine’s in-memory state
• WHEN the process exits
• THEN the dependency graph and statistics SHALL have been persisted in the store
• AND the in-memory state SHALL be ephemeral

Requirement: TIA-ENG-016 — Memory safety and subprocess isolation

The reference implementation core SHALL be memory-safe and SHALL isolate untrusted adapter execution to subprocesses (cf. TIA-SEC-002), keeping the engine free of repository code execution.

Scenario: Subprocess isolation

• GIVEN untrusted adapter code
• WHEN it is executed
• THEN it SHALL run in a separate subprocess
• AND the engine process SHALL remain free of repository code execution

Local Developer Mode

Purpose

Selection behavior when used locally against the working tree — uncommitted changes, daemon mode, watch mode, and offline operation.

Requirements

Requirement: TIA-LOCAL-001 — Working tree selection

When invoked locally against the working tree, the CLI SHALL compute selection from uncommitted changes.

Scenario: Local uncommitted changes

• GIVEN uncommitted changes in the working tree
• WHEN the CLI is invoked locally
• THEN it SHALL compute selection from those uncommitted changes

Requirement: TIA-LOCAL-002 — Daemon mode

Where a daemon is running, the CLI SHALL reuse a cached in-memory graph to return selection with low latency.

Scenario: Warm daemon cache

• GIVEN a running daemon with a cached in-memory graph
• WHEN selection is requested
• THEN the CLI SHALL reuse the cached graph for low-latency selection

Requirement: TIA-LOCAL-003 — Watch mode

Where watch mode is enabled, the CLI SHALL recompute the affected set on each saved change.

Scenario: File save triggers recomputation

• GIVEN watch mode is enabled
• WHEN a file is saved
• THEN the CLI SHALL recompute the affected set

Requirement: TIA-LOCAL-004 — Local failure rerun

The CLI SHALL always re-run tests that failed in the developer’s most recent local run.

Scenario: Local failure rerun

• GIVEN tests that failed in the most recent local run
• WHEN a new local selection is computed
• THEN those tests SHALL be included in the always-run set

Requirement: TIA-LOCAL-005 — Offline operation

The CLI SHALL operate without network access in local mode.

Scenario: Offline selection

• GIVEN no network access
• WHEN the CLI is used in local mode
• THEN it SHALL still compute selection successfully

Requirement: TIA-LOCAL-006 — Store-readiness precondition

Before performing any store-dependent operation (select, discover, ingest, explain, graph), the CLI SHALL verify that the testaruda store is initialized (.testaruda/store.db exists with the expected schema). If the store is not initialized, the CLI SHALL emit a human-actionable error message suggesting the user run testaruda init first, and SHALL NOT attempt any SQL operation.

Scenario: Select before init

• GIVEN a directory with no testaruda init run
• WHEN select is invoked
• THEN the CLI SHALL detect the missing store
• AND SHALL emit an error message indicating the store is missing and suggesting testaruda init (e.g. Error: no testaruda store found — run 'testaruda init' first)
• AND SHALL NOT attempt any SQL operation

Scenario: Other commands before init

• GIVEN a directory with no testaruda init run
• WHEN discover, ingest, explain, or graph is invoked
• THEN the CLI SHALL detect the missing store
• AND SHALL emit an actionable error message

Scenario: Corrupted store

• GIVEN a directory where .testaruda/store.db exists but is corrupted or has an incompatible schema version
• WHEN any store-dependent command is invoked
• THEN the CLI SHALL detect the corruption or version mismatch
• AND SHALL emit an error message explaining the issue
• AND SHALL suggest re-running testaruda init or using a migration tool

Non-Functional Requirements

Purpose

Cross-cutting quality attributes that constrain the implementation but do not specify functional behaviour: performance, reliability, security, portability, and scalability.

Requirements

Requirement: TIA-PERF-001 — Change-set scaling

The core SHALL scale incremental selection time with the size of the change set rather than the size of the full test suite.

Scenario: Incremental time scaling

• GIVEN a full test suite of size N
• WHEN a change of size c is computed
• THEN selection time SHALL scale with c, not with N

Requirement: TIA-PERF-002 — Interactive latency

While a warm local daemon is available, the CLI SHALL return selection for a single-file change within an interactive latency budget.

Scenario: Interactive selection

• GIVEN a warm local daemon
• WHEN a single-file change is evaluated
• THEN selection SHALL complete within an interactive latency budget (e.g. < 100ms)

Requirement: TIA-PERF-003 — Overhead bound

The core SHALL bound its own selection overhead to a small fraction of the time saved by running fewer tests.

Scenario: Overhead vs savings

• GIVEN a selection that reduces the test run by T seconds
• WHEN selection overhead is measured
• THEN the overhead SHALL be a small fraction of T

Requirement: TIA-REL-001 — Deterministic mode

While in deterministic mode, the core SHALL produce identical selections for identical inputs and store state.

Scenario: Deterministic behavior

• GIVEN identical inputs and store state in deterministic mode
• WHEN selection is computed twice
• THEN both selections SHALL be identical

Requirement: TIA-REL-002 — Crash-safe ingestion

The core SHALL make ingestion crash-safe, leaving the store consistent after an interrupted operation. Each ingestion SHALL execute within a single database transaction (or equivalent write-ahead-log mechanism) so that a crash during ingestion leaves the store in its pre-ingestion state. Idempotency of re-ingestion is provided by the run-identity key (TIA-RUN-005).

Scenario: Crash during ingestion

• GIVEN an ingestion operation in progress
• WHEN a crash occurs midway
• THEN the store SHALL remain in its pre-ingestion state
• AND the run-identity key SHALL enable safe re-ingestion

Requirement: TIA-SEC-001 — No code execution

The core SHALL NOT execute arbitrary repository code as part of selection.

Scenario: Safe selection

• GIVEN a repository with arbitrary code
• WHEN selection is computed
• THEN the core SHALL NOT execute any repository code

Requirement: TIA-SEC-002 — Least privilege adapters

The core SHALL run adapters under least privilege with bounded resource limits.

Scenario: Sandboxed adapter

• GIVEN an adapter process
• WHEN it is spawned by the core
• THEN it SHALL run with least privilege
• AND SHALL have bounded resource limits

Requirement: TIA-SEC-003 — Hashed environment representation

The core SHALL include only an allowlisted, hashed representation of environment variables in cache and environment keys, and SHALL NOT store raw secret values.

Scenario: Secret-free environment keys

• GIVEN environment variables containing secrets
• WHEN they are incorporated into cache or environment keys
• THEN only an allowlisted, hashed representation SHALL be used
• AND raw secret values SHALL NOT be stored

Requirement: TIA-PORT-001 — Language-agnostic via adapters

The core SHALL be usable across languages and test frameworks solely through adapters, with no core changes required to add a language.

Scenario: New language via adapter

• GIVEN a new language to support
• WHEN an adapter is written for it
• THEN no core changes SHALL be needed

Requirement: TIA-PORT-002 — Independent adapter versioning

The adapter protocol SHALL permit adapters to be implemented in any language and versioned independently of the core.

Scenario: Adapter in any language

• GIVEN a new adapter
• WHEN it is implemented
• THEN it MAY be written in any language
• AND versioned independently of the core

Requirement: TIA-PORT-003 — Protocol compatibility policy

The core SHALL declare a protocol-compatibility policy and reject adapters outside the supported version range.

Scenario: Version range enforcement

• GIVEN an adapter outside the supported protocol version range
• WHEN the core attempts to use it
• THEN the core SHALL reject it

Requirement: TIA-PORT-004 — CLI reference documentation freshness

The repository SHALL keep docs/cli.md (or equivalent human-facing CLI reference) in sync with the real --help output of the installed binary. The documentation SHALL NOT be considered up-to-date if it:

• Omits a real subcommand or flag, or
• Documents a subcommand or flag that does not exist in the real --help output.

Note: Common approaches to satisfy this requirement include generating docs/cli.md from clap’s help text (e.g. clap-markdown) or adding a CI check (e.g. just doc-cli-sync) that compares documented subcommands and flags against --help output and fails on drift.

Scenario: Missing subcommand detected

• GIVEN a real subcommand added to the CLI
• WHEN docs/cli.md is not updated
• THEN the CI doc-freshness check SHALL fail
• AND the failure SHALL identify the undocumented subcommand

Scenario: Generated reference

• GIVEN the CLI binary
• WHEN docs/cli.md is generated from --help output
• THEN all real subcommands and flags SHALL be present
• AND no phantom subcommands or flags SHALL appear

Requirement: TIA-SCALE-001 — Monorepo scaling

The core SHALL support monorepos containing many components without recomputing the full graph per change.

Scenario: Monorepo efficiency

• GIVEN a monorepo with many components
• WHEN a change is made in a subset of components
• THEN the core SHALL NOT recompute the full graph

Requirement: TIA-SCALE-002 — Federated repositories

The core SHALL support federation across multiple repositories without a single shared write bottleneck.

Scenario: No write bottleneck

• GIVEN multiple federated repositories
• WHEN they are composed
• THEN no single shared write bottleneck SHALL exist

Observability

Purpose

Metrics, structured logs, and graph introspection — exporting the dependency graph, explaining selections, and emitting operational metrics.

Requirements

Requirement: TIA-OBS-001 — Dependency graph export

testaruda’s --json output for dependency-graph export and selection explanation SHALL wrap its payload in genesis::envelope::Envelope, so its JSON shape matches wai/dont/pretender/espectacular across the suite.

Scenario: select emits shared envelope

• WHEN testaruda select --json is run after adopting genesis
• THEN the emitted JSON SHALL have top-level keys ok, envelope_version, cli_version, envelope_kind, data, warnings, hints, meta
• AND the selected-test set SHALL be nested under data.

Requirement: TIA-OBS-002 — Selection explanation

The core SHALL be able to explain any selection it produced.

Scenario: Post-hoc explanation

• GIVEN a prior selection
• WHEN explanation is requested
• THEN the core SHALL be able to explain it

Requirement: TIA-OBS-003 — Metrics emission

The core SHALL emit metrics including selection rate, estimated time saved, fallback rate, flakiness rate, and missed-selection count.

Scenario: Metrics output

• GIVEN a selection is computed
• WHEN metrics are requested or logged
• THEN the core SHALL emit selection rate, estimated time saved, fallback rate, flakiness rate, and missed-selection count

Requirement: TIA-OBS-004 — Structured logs

The core SHALL emit structured logs for each selection and ingestion.

Scenario: Logging on operations

• GIVEN a selection or ingestion operation
• WHEN it completes
• THEN structured logs SHALL be emitted for the operation

Requirement: TIA-OBS-005 — Log routing to stderr in machine-readable modes

When a machine-readable output mode (--json, --agent, or --pre-edit) is active, the core SHALL route all structured log output to stderr, leaving stdout clean for parseable JSON. When no machine-readable mode is active, the core MAY emit logs to either stderr or stdout; stderr is recommended.

Scenario: Machine-readable mode log routing

• GIVEN a machine-readable output mode (--json, --agent, or --pre-edit) is active
• WHEN the core emits structured logs
• THEN all log output SHALL appear on stderr
• AND stdout SHALL contain only the requested machine-readable output

Scenario: Human mode log placement

• GIVEN no machine-readable output mode is active
• WHEN the core emits structured logs
• THEN it MAY emit logs to stderr or stdout
• AND stderr is the recommended target

Persistence

Purpose

Database storage, blob store, schema management, and concurrent access — the store layer.

Requirements

Requirement: TIA-STORE-001 — Embedded transactional database

The store SHALL persist the queryable index in an embedded transactional database.

Scenario: Embedded database

• GIVEN the store
• WHEN persisting the queryable index
• THEN it SHALL use an embedded transactional database (e.g. SQLite)

Requirement: TIA-STORE-002 — Content-addressed blob store

The store SHALL persist large per-run payloads in a content-addressed blob store, deduplicated by content hash.

Scenario: Deduplicated blob storage

• GIVEN a per-run payload
• WHEN it is persisted
• THEN it SHALL be stored in a content-addressed blob store
• AND duplicates SHALL be deduplicated by content hash

Requirement: TIA-STORE-003 — Export and import

The store SHALL support export and import of the dependency graph and provenance in a documented interchange format.

Scenario: Graph export

• GIVEN a populated store
• WHEN export is requested
• THEN the dependency graph and provenance SHALL be exported in a documented interchange format

Requirement: TIA-STORE-004 — Schema migration

When the store schema version differs from the running core, the core SHALL migrate or refuse with a clear diagnostic rather than corrupt data.

Scenario: Schema version mismatch

• GIVEN a store with an older schema version than the running core
• WHEN the core attempts to use it
• THEN the core SHALL either migrate the schema or refuse with a clear diagnostic
• AND SHALL NOT corrupt data

Requirement: TIA-STORE-005 — Concurrent reads during write

The store SHALL support concurrent reads during an in-progress write.

Scenario: Read during write

• GIVEN an in-progress write to the store
• WHEN a read request arrives
• THEN the store SHALL allow the read to proceed concurrently

Provenance and Explainability

Purpose

Computing reason chains for selections, enabling query-time explanation of why each test was or was not selected.

Requirements

Requirement: TIA-PROV-001 — Provenance derivation

The core SHALL compute, for each selection, the provenance expression that derived it.

Scenario: Provenance per test

• GIVEN a change set and dependency graph
• WHEN a test is selected
• THEN the core SHALL have derived a provenance expression for it

Requirement: TIA-PROV-002 — Reason chain

When a test item is selected, the core SHALL be able to produce its reason chain as the set of edges and changed content units that caused its selection, each annotated with origin.

Scenario: Reason chain output

• GIVEN a selected test item
• WHEN its reason is requested
• THEN the core SHALL return the chain of edges and changed content units that caused selection
• AND each edge SHALL be annotated with its origin (static, runtime, or manual)

Requirement: TIA-PROV-003 — Exclusion reason

When a test item is excluded, the core SHALL be able to produce an explicit exclusion reason.

Scenario: Exclusion explanation

• GIVEN a test item that was not selected
• WHEN its exclusion reason is requested
• THEN the core SHALL produce an explicit reason (e.g. no change reaches it, no edges exist)

Requirement: TIA-PROV-004 — Explain a specific test

When queried about a specific test, the core SHALL report whether and why that test was or was not selected for the current change set.

Scenario: Explain endpoint

• GIVEN a test ID and a change set
• WHEN the explain command is used
• THEN the core SHALL report whether the test was or was not selected
• AND SHALL explain why

Requirement: TIA-PROV-005 — Persisted provenance

The store SHALL persist provenance such that a past selection can be re-explained without re-running selection.

Scenario: Historical explanation

• GIVEN a past selection with persisted provenance
• WHEN its explanation is requested later
• THEN the store SHALL provide the explanation without re-running the selection query

Runtime Feedback

Purpose

Ingesting test results and observed dependency evidence to update the model — runtime edges, run history, and idempotent ingestion.

Requirements

Requirement: TIA-RUN-001 — Runtime edge creation

When test results and coverage are ingested, the ingestor SHALL create or update runtime edges with origin runtime.

Scenario: Runtime edge ingestion

• GIVEN test results with coverage data
• WHEN they are ingested
• THEN the ingestor SHALL create or update edges with origin runtime

Requirement: TIA-RUN-002 — Coverage-discovered edges

When coverage indicates a test exercised a content unit not linked by any static edge, the ingestor SHALL record that runtime edge so the dependency becomes selectable on future changes.

Scenario: New runtime dependency

• GIVEN coverage data showing a test exercised a content unit not in its static edges
• WHEN results are ingested
• THEN the ingestor SHALL record a new runtime edge between them

Requirement: TIA-RUN-003 — External input recording

When an adapter reports external inputs read at runtime (config files, env vars, fixture files, reflectively loaded modules), the ingestor SHALL record them as runtime edges to the corresponding content units.

Scenario: External input edge

• GIVEN an adapter reporting external inputs consumed at runtime
• WHEN results are ingested
• THEN the ingestor SHALL record runtime edges to those external inputs

Requirement: TIA-RUN-004 — Run history update

When results are ingested, the ingestor SHALL update per-test run history, timing, and failure-rate statistics.

Scenario: History update

• GIVEN ingested test results
• WHEN ingestion completes
• THEN per-test run history, timing, and failure-rate statistics SHALL be updated

Requirement: TIA-RUN-005 — Idempotent ingestion

The ingestor SHALL be idempotent with respect to re-ingestion of the same run. Each run payload SHALL carry a unique run-identity key (a caller-supplied or adapter-generated opaque string, e.g. a UUID). Before performing any write, the ingestor SHALL check whether that key has already been recorded and skip ingestion if so. If no run-identity key is present in the payload, the ingestor SHALL reject the request with a diagnostic rather than proceed without dedup protection.

Scenario: Duplicate ingestion skipped

• GIVEN a run payload with a run-identity key already recorded
• WHEN the payload is ingested again
• THEN the ingestor SHALL skip all writes
• AND report the duplicate

Scenario: Missing run-identity key rejection

• GIVEN a run payload without a run-identity key
• WHEN ingestion is attempted
• THEN the ingestor SHALL reject the request with a diagnostic

Requirement: TIA-RUN-006 — Environment recording

While ingesting a run, the ingestor SHALL record the environment fingerprint under which the run executed.

Scenario: Environment fingerprint on ingest

• GIVEN a run payload with environment metadata
• WHEN ingestion is performed
• THEN the environment fingerprint SHALL be recorded alongside the results

Safety

Purpose

Soundness invariant, fallback mechanisms, and forced-selection rules that ensure recall — the over-approximation guarantee.

Requirements

Requirement: TIA-SAFE-001 — Over-approximation invariant

The core SHALL maintain the invariant that the modeled dependency relation over-approximates the true semantic dependency relation, such that any test that could be affected by a change is selected.

Scenario: Over-approximation guarantee

• GIVEN a change that semantically affects a test
• WHEN selection is computed
• THEN the test SHALL be selected (over-approximation ensures recall)

Requirement: TIA-SAFE-002 — Confidence fallback

If the minimum Viterbi path confidence (TIA-CONF-001) across the reachability-selected tests in a component falls below the configured threshold, then the core SHALL fall back to selecting all tests in that component. If no reachability-selected tests exist for a component (i.e. the component’s selection consists solely of always-run members from TIA-SAFE-007), TIA-SAFE-002 does not apply — TIA-SAFE-007 already provides sufficient recall for that component.

Scenario: Confidence below threshold

• GIVEN a component where min reachability-selected test confidence < configured threshold
• WHEN selection is computed
• THEN the core SHALL fall back to selecting all tests in that component

Scenario: Always-run-only component

• GIVEN a component whose selection consists solely of always-run members
• WHEN confidence is checked
• THEN TIA-SAFE-002 SHALL NOT apply
• AND no fallback is triggered

Requirement: TIA-SAFE-003 — Scoped fallback

The core SHALL scope confidence-driven fallback to the affected component(s) and SHALL NOT escalate to a global full run unless every affected component is below threshold.

Scenario: Component-scoped fallback