Audit & score history
Where the CI gates fire on a single change (per-PR), the audit spine is the over-time, standing-repo sweep — a graded, profile-based assessment of a whole concern, turned into a trended deliverable. These commands are version-locked in the package; the orchestration is your agent's work, the deterministic spine is these commands.
.valinor/audit/ (the per-run findings, which may carry source snippets) is gitignored; only the grade-only score-history.jsonl is committed. Every command fails closed on a missing or malformed input.
audit-list
List the audit-skill ids a consumer can invoke — the 3rd governance-triad leg (the skills/valinor-audit-<id>/ audit skills, bundled version-locked inside the installed package, not scaffold-copied). One id per line on stdout (a clean, pipeable list); the count summary goes to stderr.
List audit skills
npx @cmbrcreative/valinor audit-list
audit-show
Print that concern's bundled SKILL.md to stdout, for your agent to read and run as a standing-repo audit. Read-only — no LLM, no mutation. Fails closed: an unknown/absent concern exits 1 with an error listing the available ids; a missing arg exits 2 (usage). Run audit-list to discover the ids.
- Name
concern- Type
- string (required)
- Description
The audit-skill id to print (e.g.
doctrine).
Show an audit skill
npx @cmbrcreative/valinor audit-show doctrine
audit-verify
The deterministic QA gate over a persisted audit run. Fails closed on a baseless finding (evidence absent at its cited file:line), a coverage gap, a grade that doesn't recompute, or a blocked/incomplete review. The report is rendered only after this passes — so the deliverable is verified by construction.
- Name
path- Type
- string
- Description
The persisted audit dir. Default:
.valinor/audit.
Verify an audit run
npx @cmbrcreative/valinor audit-verify
audit-record
Append a privacy-safe, grade-only ScoreRecord (composite / letter / sub-scores + provenance — never a finding, file, or snippet) from the latest run to the committed .valinor/score-history.jsonl. The deliberate "record the score" step that makes the quality score visible over time in git.
Record the score
npx @cmbrcreative/valinor audit-record
audit-trend
Read the committed score history (oldest→newest) and print a deterministic Markdown trend table — latest-vs-previous and latest-vs-baseline composite / sub-score / letter-band changes with ▲ / ▼ / → arrows. No LLM, no prose.
Render the trend
npx @cmbrcreative/valinor audit-trend
The whole-repo runtime, end to end: valinor-audit (the orchestrator skill) scopes +
inventories the repo and fans out per-concern subagents → the run is persisted under
.valinor/audit/ → audit-verify gates it → audit-record appends the grade-only
score → audit-trend renders the score-over-time table.
gate-health-record
Append a privacy-safe GateHealthRecord (per-gate pass/fail conclusions + counts + provenance — gate ids only, never a finding) for a run to the committed .valinor/gate-health.jsonl. The outcomes.json (the run's {gate, conclusion} list) is the thin per-run source the CI job writes. The L5 meta layer — "record gate health".
- Name
outcomes.json- Type
- string (required)
- Description
The run's
{gate, conclusion}outcomes file.
Record gate health
npx @cmbrcreative/valinor gate-health-record outcomes.json
gate-health-trend
Read the committed gate-health history (oldest→newest) and print a deterministic Markdown table — per-gate pass rate + a flake (↺) marker for a gate that flipped between its two most recent runs. No LLM, no prose.
Render gate-health trend
npx @cmbrcreative/valinor gate-health-trend
eval-check
The calibration-eval regression gate — fails closed if any floored audit concern's recorded accuracy regressed below its committed floor (or a floored concern has no recorded result). Keeps the audit finders honest over time.
Run the eval gate
npx @cmbrcreative/valinor eval-check
eval-record
Score a corpus-findings file against the committed labels (eval/audit-corpus/labels.json) and write the per-concern accuracy to eval/results/<ISO-date>.json.
- Name
findings.json- Type
- string (required)
- Description
The corpus-findings file to score.
Record an eval result
npx @cmbrcreative/valinor eval-record findings.json