Tarn — AI Workflows

Use JSON intentionally

$ tarn run --format json --json-mode compact
$ tarn run --format human --format json=reports/run.json --format curl=reports/failures.sh

Verbose JSON: richer assertion detail for forensics.
Compact JSON: slimmer payload for LLM and automation loops.
curl export: replays failed or full-suite requests without re-reading YAML.

JSON report structure

The JSON output from tarn run --format json has this structure:

{
  "schema_version": 1,
  "summary": {
    "status": "PASSED",
    "total": 10,
    "passed": 9,
    "failed": 1
  },
  "files": [
    {
      "file": "tests/users.tarn.yaml",
      "tests": [
        {
          "name": "User API",
          "status": "FAILED",
          "steps": [
            {
              "name": "Create user",
              "status": "FAILED",
              "failure_category": "assertion_failed",
              "assertions": {
                "failures": [
                  {
                    "assertion": "status",
                    "expected": "201",
                    "actual": "400",
                    "message": "Expected status 201, got 400"
                  }
                ]
              }
            }
          ]
        }
      ]
    }
  ]
}

Failure categories

Every failed step includes a failure_category field:

Category	Meaning	How to fix
`assertion_failed`	Request succeeded, assertion mismatch	Adjust assertion to match real response
`connection_error`	DNS/TLS/connect failure before response	Verify server is running and reachable
`timeout`	Exceeded allowed time	Increase timeout, use polling
`parse_error`	Invalid YAML or schema	Fix YAML, run tarn validate
`capture_error`	Assertions passed, capture extraction failed	Update JSONPath, use optional/default

How an agent should process failures

Check summary.status — if PASSED, done
For FAILED, iterate files[].tests[].steps[]
Check failure_category FIRST (not the message string)
If assertion_failed: read response.body, compare with expected, adjust
If connection_error: check request.url for correct endpoint
If capture_error: verify JSONPath against response body
If parse_error: run tarn_validate for details

Using the fix plan

# Get structured remediation
tarn_fix_plan(report: failedReport)

Returns prioritized next actions per failed step.

JSON output modes

Verbose mode (default): full assertion details, all passes and failures

Compact mode (--json-mode compact): drops passed assertion details, truncates bodies to ~200 chars

Agent mode (--agent): compact root-cause-first AgentReport JSON for MCP/automation clients

Fields that matter in the retry loop

failure_category: assertion, response-shape drift, parse, connection, timeout, capture, cascade skip.
error_code: stable machine-readable subtype for routing and heuristics.
remediation_hints: lightweight next-step guidance in the JSON itself.
request and response: included on executed failures so the model sees what actually happened.

Recommended shell loop

Generate or edit the Tarn YAML.
Run tarn validate to catch parser and config issues.
Run tarn run --format json --json-mode compact.
Branch on failure_category and error_code before touching assertions blindly.
Use curl export when you need to replay exact failures outside Tarn.

Streaming and scoped execution

For live UIs and incremental retry loops, tarn run --ndjson streams one JSON event per line to stdout: file_started, step_finished, test_finished, file_finished, done. Pair it with --select FILE[::TEST[::STEP]] (repeatable, ANDs with --tag) to run just the failing step, or --only-failed to replay the previous run's failures. Use --no-progress in CI to keep the stream clean.

Editor intelligence via tarn-lsp

An agent running inside an LSP 3.17 editor — Claude Code, VS Code, Neovim, Helix, Zed, Emacs — gets the same structured signal that tarn run --format json produces, but interleaved with the edit loop rather than after it. Hover over {{ env.api_key }} and the resolved value plus source layer is already there. Type {{ capture. and every in-scope capture autocompletes. Save a broken buffer and diagnostics land on the exact offending token via NAZ-260 range metadata. Invoke the quick-fix code action on a tarn_validation diagnostic and the edit comes straight out of the same tarn::fix_plan engine that powers the tarn_fix_plan MCP tool — one remediation graph, two surfaces.

The JSONPath hover and the workspace/executeCommand-based tarn.evaluateJsonpath are the agent-native affordance: an agent with a recorded response sidecar on disk can ask the server to evaluate a JSONPath without re-running the suite, without re-parsing the YAML, and without reimplementing serde_json_path. That is the shortest edit loop Tarn offers: record, edit, ask, observe, revise.

Install, full feature list, sidecar layout, and the per-client recipes (Claude Code plugin, VS Code extension with tarn.experimentalLspClient, Neovim, Helix, Zed, Emacs) all live on the dedicated tarn-lsp page. This page deliberately stops documenting install here so there is one source of truth for editor setup.

Redaction

Report-time redaction is configurable at DSL level. Tarn can redact sensitive headers, named env vars, and selected captures across JSON, human, JUnit, TAP, HTML, and curl outputs.

Why this matters

An agent should not need to infer whether a failure is DNS, TLS verification, unresolved interpolation, or a plain status mismatch from plain-text output. Tarn ships that distinction directly.

Give the model objects, not stdout.