From first install to
Full Control.

00 Overview

What you'll build

By the end of this tutorial you'll have a working SYZYGY project that captures business requirements, generates a plugin, authors test scenarios, sends requests to a service, validates responses, and produces an auditable run report — all from plain YAML committed to git. No framework to stand up, no server to operate.

git clone https://github.com/syzs-code/syz-cli-demo.git
cd syz-cli-demo
# then follow the steps in DEMO.md

The path is linear. Each step produces an artefact the next step uses:

1. Install & init — get the syz CLI and lay down the .syz/ project structure.
2. Capture domain knowledge — record the requirements that define correct behaviour in DOMAIN.yaml.
3. Generate a plugin — create procedures and templates from an OpenAPI spec with syz generate.
4. Write scenarios — compose procedures into test flows in YAML.
5. Run & audit — execute tests and read the ledger, debug log, and interactive HTML report.
6. Share plugins — install shared plugins from git and use them like local ones.

01 Installation & Setup

Setting up your first SYZYGY project

SYZYGY is a single Node.js CLI — syz, published as syzs-cli. There's nothing else to operate. You install one binary, run syz init once, and everything else lives as plain text in your repo.

Prerequisites

Before installing SYZYGY CLI, ensure you have:

Operating Systems: macOS, Linux, Windows (via WSL or native)

1. 1

   Install the CLI

   Installs syz globally so it's available in any terminal. Requires Node.js 20+.

   npm install -g syzs-cli

   Verify Installation:

   node --version
   # Output should show: v20.x.x or higher
   
   npm --version
   # Output should show: 10.x or higher
   
   syz --version
   # Output: e.g., "0.12.0"

2. 2

   Initialise the project

   Run this from your repo root (any git repo will do). It creates the .syz/ directory and the framework files inside it.

   syz init

   syz init generates:

   • .syz/dossier/ — the knowledge layer: directive router, CLI manual, guidelines, testing how-to, AI-risk notice, migration playbook, plain-language playbook (owned, overwritten on init), plus scaffolded DOMAIN.yaml / TEST-STRATEGY.md stubs that are yours.
   • Pointer files for every AI assistant — CLAUDE.md, .cursor/rules/syzygy.md, .github/copilot-instructions.md, GEMINI.md, and more.
   • .syz/dependency.json — git-installed plugin dependencies, plus the syz_version stamp recording which syz set up the workspace and the syz_format stamp recording which file format your artefacts are written against (see the Appendix).
   • Empty environments/ and scenarios/ folders, plus plugins/ for your work.

   Explicit initialisation required. Run syz init once in your project root to create the .syz/ workspace. After that, all commands (syz run, syz generate, syz install) require the workspace — they locate it by walking up from wherever you run them to the nearest .syz/ (the way git finds a repo), so you can run them from any subdirectory of your project, including from inside .syz/ itself. If no .syz/ exists in the current directory or any parent, they fail fast. Everything prefixed SYZ- is owned by the CLI and overwritten on every syz init; the scaffolded files without the prefix — .syz/README.md (optional), dossier/DOMAIN.yaml, dossier/TEST-STRATEGY.md — are yours and are never overwritten. The single exception is additive: syz init appends a missing syz_format line to your own plugin.yaml files, preserving everything else — see the Appendix.

   What to commit. syz init manages a .gitignore at your project root and keeps syz's regenerable output out of git — the SYZ-* docs, the pointer files, plugins.installed/, and results/ are all rebuilt by the CLI. You commit what you authored: DOMAIN.yaml, TEST-STRATEGY.md, every PLUGIN-GUIDE.yaml, dependency.json, and your plugins, scenarios, environments, and suites.

3. 3

   Explore the structure

   Everything the framework manages lives under .syz/. Discovery is convention-based — there's no config file; the directory shape is the contract.

   <project-root>/
   ├── .gitignore                    # syz adds plugins.installed/ automatically
   └── .syz/
       ├── CLAUDE.md                 # AI assistant pointers (+ others)
       ├── dependency.json           # git plugin deps + syz_version + syz_format
       ├── dossier/
       │   ├── SYZ-INDEX.md           # directive router for AI assistants (owned, overwritten on init)
       │   ├── SYZ-MANUAL.md          # full CLI manual (owned, overwritten)
       │   ├── … other SYZ-*.md docs  # guidelines, testing how-to, AI-risk notice, migration playbook, plain-language playbook (owned, overwritten)
       │   ├── DOMAIN.yaml            # domain requirements (you write this)
       │   └── TEST-STRATEGY.md       # test approach (you write this)
       ├── plugins/                  # hand-authored plugins — committed
       ├── plugins.installed/        # git-installed plugins — gitignored
       ├── environments/             # base URLs + credentials per env
       ├── scenarios/                # component/ integration/ e2e/ uat/
       ├── suites/                   # test playlists
       └── results/                  # one folder per run — gitignored

   • plugins/ — plugins you author by hand. Committed to git like any other code.
   • plugins.installed/ — plugins fetched by syz install. Auto-added to .gitignore.
   • environments/ — one YAML per environment, naming base URLs and credentials.
   • scenarios/ — your tests, organised free-form into test level folders.
   • results/ — execution output, one folder per syz run.
   • dossier/ — the SYZYGY-owned knowledge layer (directive router, CLI manual, guidelines, testing how-to, AI-risk notice, migration playbook, plain-language playbook — overwritten on init) plus your domain requirements and test strategy (you write those).
4. 4

   Write your first environment file

   An environment names the systems under test. Plugin config sits at the top level of the file, keyed by plugin name — base URL, credentials via environment variables, optional timeout.

   .syz/environments/local.yaml

   name: local
   production: false
   payments:
     base_url: "http://localhost:3000"
     token: "{{ os-env-var:PAYMENTS_TOKEN }}"  # resolved at load time; value auto-masked everywhere
     timeout_ms: 5000                           # optional — default 30000

   Then export the secret in your shell (or your CI's secret store) — never in the YAML:

   export PAYMENTS_TOKEN="dev-token"

   {{ os-env-var:NAME }} is inline-capable. It can appear anywhere inside a value — "https://{{ os-env-var:API_HOST }}/v1" works. Every resolved value is automatically masked in logs, debug files, HTML reports, and terminal output. If the variable is unset, syz lint catches it before execution, failing loudly.

Installation Troubleshooting

Problem: syz: command not found

Solution: Node.js or npm isn't installed, or SYZYGY wasn't installed globally.

# Reinstall with explicit global flag
npm install -g syzs-cli

# Verify the installation path
npm list -g syzs-cli

Problem: npm ERR! code EACCES (Permission Denied)

Solution: npm doesn't have permission to write to global directories.

Option 1 — Fix npm permissions (Recommended):

mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
export PATH=~/.npm-global/bin:$PATH
npm install -g syzs-cli

Option 2 — Use sudo (Not Recommended):

sudo npm install -g syzs-cli

Problem: Node.js version too old

Solution: Upgrade to Node.js 20 LTS or later.

# On macOS (with Homebrew)
brew upgrade node

# On Windows/Linux
# Download from https://nodejs.org/

Problem: syz init fails or creates incomplete structure

Solution: Ensure you're running the latest version and have write permissions.

# Update to latest version
npm install -g syzs-cli@latest

# Verify you're in an empty directory or have write permissions
pwd
ls -la

# Try again
syz init

Alternative Installation Methods

Install Locally (Project-Specific)

If you want SYZYGY only for one project:

cd my-integration-tests
npm install --save-dev syzs-cli

# Run via npx
npx syz --version
npx syz init
npx syz run --suite .syz/scenarios --env local

02 Introduction & Core Concepts

What SYZYGY is and how it works

What is SYZYGY?

SYZYGY is a tool to capture, version, and share your integration knowledge — and then prove it with deterministic tests. You don't start by writing test code; you start by recording what correct means for your integrations, in plain language, as durable requirements. Plugins, scenarios, and execution all derive from that captured knowledge and trace back to it.

The Four-Layer Architecture

SYZYGY starts from knowledge, not from tests. Everything below the knowledge layer exists to realise, exercise, and run what the knowledge layer declares:

Read it top-down: you author L4 first (before a plugin or a single test exists), a plugin at L2 realises those requirements against a real API, scenarios at L3 exercise them, and L1 runs everything deterministically. Coverage flows back up — tests link to L4 requirements with covers:, and syz rtm proves the loop is closed.

Running Example: The Payments Plugin

Throughout this tutorial, we'll work with a fictional Payments API that manages financial transactions. Our plugin will support:

• Creating a payment (POST /payments)
• Retrieving payment status (GET /payments/{id})
• Refunding a payment (POST /payments/{id}/refund)

We'll start with capturing business knowledge, an OpenAPI spec, generate a plugin, author test scenarios, run them, and review the results.

03 Part 1: Capture — Define Your Domain

Document business requirements before building tests

What is "Capture"?

Before you generate a plugin or write test scenarios, you need to document your business domain. This means answering questions like:

• What is this API for? (e.g., processing customer transactions)
• What are the business rules? (e.g., refunds must be issued within 30 days)
• What integrations depend on this API? (e.g., order processing, invoicing)
• What are the key test scenarios? (e.g., happy path, error handling, edge cases)

This knowledge lives in two files:

• DOMAIN.yaml — Cross-integration business knowledge (under .syz/dossier/)
• PLUGIN-GUIDE.yaml — Plugin-specific integration rules (one per plugin, under .syz/plugins/payments/)

Understanding DOMAIN.yaml

DOMAIN.yaml is a schema-validated YAML file that captures business and technical requirements shared across your entire integration suite.

.syz/dossier/DOMAIN.yaml

# What the application does end to end, in your own domain language.
summary: >
  Payments processing for customer orders — checkout, status tracking,
  refunds. Integrates with order processing and invoicing.

# The registry of domain / journey labels. A domain requirement id is
# REQ-<AREA>-DOM-<NNN>, where <AREA> must be a key declared here.
areas:
  - key: PAYMENT
    title: Payment processing
    description: charge → track → reconcile
  - key: REFUND
    title: Refunds
    description: reverse a completed payment

# The identifiers that link services together (optional)
entities:
  payment: { id: payment.id, links: [order.payment_id, invoice.payment_id] }

# What "correct" means end to end
requirements:
  - id: REQ-PAYMENT-DOM-001
    behaviour:
      given: a customer order with a valid payment method
      when: the customer pays
      then: the payment settles and the order is confirmed
    flow:                      # the journey in domain terms
      - the customer pays for the order
      - order processing charges the customer
      - invoicing reconciles the settled payment
    rules:
      - payments must be positive amounts
      - only supported currencies are accepted (AUD, CNY, EUR, GBP, INR, JPY, USD)
    notes: >
      The payment journey every other service depends on. Order processing
      charges the customer; invoicing reconciles against the settled payment.

  - id: REQ-REFUND-DOM-001
    behaviour:
      given: a completed payment
      when: a refund is requested within 30 days
      then: the payment is reversed and the order is updated
    rules:
      - refunds are only allowed on completed payments
      - a refund cannot exceed the original amount
      - duplicate refund requests for the same payment are rejected
    notes: >
      Refunds span payments and order processing — the order must reflect the
      reversal.

  - id: _REQ-PAYMENT-DOM-002   # "_" = in progress — excluded from coverage tracking
    behaviour:
      given: a payment in a non-final state
      when: the provider reports a status change
      then: the payment status is updated and downstream services are informed
    notes: >
      Status webhooks are still being designed. Drop the leading "_" once this
      is ready to be tested and tracked — it's the same requirement, so any
      covers: already pointing at it keeps working.

Key Points:

• Business-focused, not technical. Domain language only — no procedure, scenario, or plugin vocabulary. A Product Owner should be able to read and update this file, before a single plugin exists.
• Schema-validated. syz lint checks the shape (id, behaviour, notes are mandatory on every requirement), the id grammar (REQ-<AREA>-DOM-<NNN>, where <AREA> is a declared areas: key), and workspace-wide id uniqueness. Optional fields (rules, inputs, errors, flow, examples, …) come from a fixed catalog — an unrecognised key is flagged.
• Checkable on its own, before anything else exists. syz lint --knowledge validates just this layer — DOMAIN.yaml plus every PLUGIN-GUIDE.yaml — reading no environment, no scenarios and no procedures, so a workspace with clean requirements and nothing else exits 0. Every rule above still applies; the only difference is that requirements without a test yet aren't reported as gaps (with no scenarios written, that would fire on every one), and you get a count of the layer instead. That makes it the check to run while writing requirements, long before there's an environment to point --env at. --plugin <name> narrows it to one guide when several people are authoring in parallel.
• Drives traceability. Scenarios link back to these ids with covers:; syz rtm reports which requirements are covered and which are gaps.
• Lives in .syz/dossier/ — scaffolded by syz init if absent, never overwritten by the CLI.
• Shared across all plugins. If you have multiple plugins (Orders, Payments, Invoicing), they all reference the same DOMAIN.yaml for consistency.

Understanding PLUGIN-GUIDE.yaml

PLUGIN-GUIDE.yaml is a schema-validated YAML file that documents the integration behaviour specific to one plugin — and it ships with the plugin via syz install, so every consumer reads exactly this guide.

.syz/plugins/payments/PLUGIN-GUIDE.yaml

summary: >
  Payments API — create a payment, track its status, refund it.

# The integration's behaviour, ID'd for traceability. A plugin-guide
# requirement id is REQ-PAYMENTS-PG-<NNN> — the scope matches the plugin folder.
requirements:
  - id: REQ-PAYMENTS-PG-001
    behaviour:
      given: a valid amount, currency, and customer
      when: a payment is created
      then: the payment is accepted and its identifier and status are returned
    inputs:
      - { field: amount, type: integer, required: true }    # minor units (cents)
      - { field: currency, type: string, required: true }   # ISO 4217 — AUD, CNY, EUR, GBP, INR, JPY, USD
      - { field: customer_id, type: string, required: true }
      - { field: idempotency_key, type: string, required: false }
    rules:
      - the amount must be positive
      - repeating a request with the same idempotency key returns the original result
    errors:
      - { code: 400, meaning: missing or invalid fields }
      - { code: 409, meaning: duplicate idempotency key }
      - { code: 422, meaning: business rule violated — e.g. zero amount }
    notes: >
      Creating a payment is the entry point of the integration. Payments cannot
      be modified after completion, and the service rate-limits at 100
      requests per minute.

# Binding authoring rules the AI must OBEY when building this integration's
# procedures and scenarios (optional). Unlike requirements, a guideline may name
# headers, executors, and settings — steering the mechanics is its job.
guidelines:
  - id: GL-PAYMENTS-001
    affects: "all requests to the payments service"
    rule: "requests must include the header X-Api-Version: 2"
    cause: "platform upgrade"
    impact: "absent → 400 UNSUPPORTED_VERSION"

Key Points:

• Plugin-specific. Each plugin gets its own PLUGIN-GUIDE.yaml in its folder, and it travels with the plugin on syz install.
• No name or version fields. The folder is the plugin's name and plugin.yaml holds the version — never duplicated here.
• Same validation as DOMAIN.yaml. id, behaviour, notes mandatory; id grammar REQ-<PLUGIN>-PG-<NNN>; syz lint enforces it. The _REQ-… in-progress marker works here too.
• Requirements stay in domain language; guidelines carry the mechanics. A requirement never names headers or executors — a binding authoring rule that must (an API version header, a TLS setting) goes under guidelines:.
• Never overwritten. Scaffolded by syz generate plugin if absent, always preserved — even with --force.
• The guide can exist before the plugin does. syz generate plugin --name <name> --guide-only scaffolds a knowledge-only shell — the guide, an identity plugin.yaml, and a CLAUDE.md pointer, nothing else — so behaviour is recorded before any endpoint or spec exists. syz lint and syz rtm pick the requirements up immediately, and a later --from <spec> run builds the runnable plugin on the same folder, preserving the guide. The --guide-only flag refuses to touch an existing plugin folder and can't be combined with --from, --force, or --overwrite.

# Record behaviour first — no API spec needed
syz generate plugin --name payments --guide-only

# Later, when the spec exists, complete the plugin in place
syz generate plugin --name payments --from openapi:./payments.json

Complete field reference

Both files share the same building blocks — requirements: and guidelines: — and only a small core of each is mandatory. Everything else is optional and drawn from a fixed catalog: pick a field from the menu below rather than inventing one. An unrecognised key isn't a hard error — syz lint flags it as an UNKNOWN_KNOWLEDGE_FIELD warning — but the catalog is what keeps these files consistent across a team. When nothing in the catalog fits, put it under more_details: (a requirement's single free-form escape hatch), not a new top-level key.

A note on enforcement. The columns below mark whether syz lint checks the shape of a field (✓) or only recognises the name and leaves the structure to you (guidance). Only id, behaviour (with its given / when / then), and notes have their shape enforced on a requirement; the rest accept any structure, so the shapes shown are the recommended convention, not a contract.

Top-level keys

requirements[] — every field (identical in both files)

guidelines[] — every field (identical in both files)

areas[] — DOMAIN.yaml only

04 Part 2: Generate — From API Spec to Runnable Plugin

Deterministic plugin generation from OpenAPI or Postman

What is "Generate"?

After documenting your domain, the next step is to generate a plugin from your API specification. SYZYGY reads an OpenAPI or Postman collection and deterministically creates, per operation, a send procedure (a structured HTTP request definition), a status-check validate procedure, and a workflow that composes them — plus data.yaml, a runnable sample scenario per operation, and a seeded environments/local.yaml. Nunjucks craft templates are generated only for multipart/binary upload operations — plain JSON operations carry their request in structured fields, no template involved.

The syz generate Command

Basic Syntax:

syz generate plugin --name <plugin-name> --from openapi:<path> [options]

Full Signature:

syz generate plugin \
  --name payments \
  --from openapi:./payments-api.openapi.yaml \
  --force \
  --no-install

Command Flags Explained

Step-by-Step: Generating the Payments Plugin

Step 1: Prepare Your OpenAPI Spec

Create payments-api.openapi.yaml:

openapi: 3.0.0
info:
  title: Payments API
  version: 1.0.0
servers:
  - url: https://api.payments.example.com/v1

paths:
  /payments:
    post:
      operationId: createPayment
      summary: Create a new payment
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [amount, currency, customer_id]
              properties:
                amount:
                  type: number
                  description: Amount in minor units (cents)
                  example: 1000
                currency:
                  type: string
                  example: GBP
                customer_id:
                  type: string
                  example: cust_123
      responses:
        '201':
          description: Payment created
          content:
            application/json:
              schema:
                type: object
                properties:
                  id:
                    type: string
                  status:
                    type: string
                  amount:
                    type: number

  /payments/{id}:
    get:
      operationId: getPaymentStatus
      summary: Retrieve payment details
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Payment details

Step 2: Run the Generator

syz generate plugin --name payments --from openapi:./payments-api.openapi.yaml

Step 3: Review Generated Files

For every operation the generator emits a send, a validate, and a workflow procedure; it also seeds the sample scenarios and the environment file:

.syz/
├── plugins/payments/
│   ├── plugin.yaml                        # plugin identity
│   ├── PLUGIN-GUIDE.yaml                  # scaffolded, not overwritten
│   ├── CLAUDE.md                          # pointer to PLUGIN-GUIDE.yaml
│   ├── procedures/
│   │   ├── send_create_payment.yaml          # POST /payments (send)
│   │   ├── validate_create_payment.yaml      # status check for POST /payments
│   │   ├── create_payment_send_success.yaml  # workflow: send + no-error validate
│   │   ├── send_get_payment_status.yaml      # GET /payments/{id} (send)
│   │   ├── validate_get_payment_status.yaml
│   │   ├── get_payment_status_send_success.yaml
│   │   └── validate_send_no_error.yaml       # shared no-error validate
│   ├── models/
│   │   └── validate_send_no_error_input.yaml
│   └── data.yaml                          # example test data per operation
├── scenarios/generated/payments/
│   ├── sample_create_payment.yaml
│   └── sample_get_payment_status.yaml
└── environments/
    └── local.yaml                         # seeded with base_url + auth_token per step

Plain JSON operations carry their request in structured http.request fields — there's no templates/ directory unless an operation needs a Nunjucks craft template (multipart/binary upload). If local.yaml already exists, the generator only adds missing keys — it never rewrites values you've set.

Generator Workflow Scenarios

Scenario 1: Fresh Generation

syz generate plugin --name payments --from openapi:./api.yaml

The generator creates all files. Your custom modifications are safe — only the autogenerated procedure/template files will be regenerated later if needed.

Scenario 2: API Updated — Regenerate Everything

syz generate plugin --name payments --from openapi:./api.yaml --force

Scenario 3: API Updated — Preserve Custom Work

syz generate plugin --name payments --from openapi:./api.yaml --overwrite

"Which operations do you want to regenerate?" Pick only the new ones. --overwrite touches nothing outside your selection — so customised procedures on the operations you leave unselected stay exactly as you wrote them. (Select an operation you'd edited, and it gets regenerated too.)

Scenario 4: Preview Before Writing

syz generate plugin --name payments --from openapi:./api.yaml --dry-run

Output shows file paths and operations found, but creates nothing.

05 Part 3: Author — Write Your Test Scenarios

Compose procedures into test flows

What is "Author"?

Once your plugin is generated, you write test scenarios — YAML files that describe a sequence of steps: send a request, validate the response, send another request, and so on. A scenario mirrors a real workflow in your system.

Understanding Step Types

Type 1: Send Steps

A send step makes a request to your API or system:

- type: send
  procedure: payments.send_create_payment
  inputs:
    amount: 5000
    currency: "GBP"
    customer_id: "cust_alice"

Key Points:

• The procedure name (payments.send_create_payment) maps to a YAML file in .syz/plugins/payments/procedures/
• inputs are the concrete values referenced inside the procedure's structured request block as {{ inputs.* }}
• The response is automatically captured in the ledger
• A send step never stops the run on its own — it just captures the response (success or error)

Referencing Previous Responses:

A send step's request and response live under its executor key — reference them with {{ ledger.<plugin>.<procedure>.http.response… }} — and any declared output is available via the {{ ledger.<plugin>.<procedure>.outputs.<name> }} shortcut:

# Create a payment first
- type: send
  procedure: payments.send_create_payment
  inputs:
    amount: 5000

# Then get its status using the ID from the create response
- type: send
  procedure: payments.send_get_payment_status
  inputs:
    id: "{{ ledger.payments.send_create_payment.outputs.id }}"

Type 2: Validate Steps

A validate step runs assertions on a previous response:

- type: validate
  procedure: payments.validate_payment_created
  inputs:
    response: "{{ ledger.payments.send_create_payment.http.response }}"
    expected_status: 201

Key Points:

• Assertions check the response against expectations
• If an assertion fails, the scenario stops (unless --continue-scenario-on-failure is set)
• The validate procedure defines the checks in its YAML

Example Validate Procedure:

name: payments.validate_payment_created
type: validate
description: "Validates a successful payment creation"

input_model: models/validate_payment_created_input.yaml
checks:
  - type: http
    description: "Response status is 201"
    path: "$.fields.status_code"
    operator: equals
    value_from_input: expected_status

  - type: json
    description: "Payment ID is present"
    path: "$.fields.body.id"
    operator: exists

  - type: json
    description: "Status is 'pending' or 'processing'"
    path: "$.fields.body.status"
    operator: in
    value: ["pending", "processing"]

  - type: json
    description: "Amount matches request"
    path: "$.fields.body.amount"
    operator: equals
    value_from_input: expected_amount

Assertion Operators:

This table covers the most common operators for beginners. For the complete list including not_contains, less_than, matches_schema, all, any, and others, see the Quick Reference section at the end of this tutorial.

Anatomy of a Basic Scenario

Scenarios live under .syz/scenarios/. The folder you choose is the level (component/, integration/, e2e/, uat/) — free-form, but those four are the convention.

.syz/scenarios/e2e/checkout.yaml

name: "Checkout — successful order"
level: e2e
plugins:
  - name: payments
    version: "1.0.0"

flow:
  - type: send
    procedure: payments.send_create_payment
    inputs:
      userId:   "u-001"
      amount:   100
      currency: "GBP"

  - type: validate
    procedure: payments.validate_order_response
    inputs:
      response:        "{{ ledger.payments.send_create_payment.http.response }}"
      expected_status: 201

• name — a human-readable title for this scenario; appears in run reports.
• level — labels the test phase (e.g. e2e, smoke, integration); used to group and filter runs.
• plugins — the plugins this scenario depends on, each pinned to a version.
• flow — the ordered list of steps to execute. The validate step reads the send step's response straight from the ledger.

Parameterising with an Outline

Add an examples table and the scenario becomes an outline — the same flow runs once per row, each row an isolated execution with its own ledger.

.syz/scenarios/e2e/checkout-parameterized.yaml

name: "Checkout — parameterised amounts"
level: e2e
plugins:
  - name: payments
    version: "1.0.0"

flow:
  - type: send
    procedure: payments.send_create_payment
    inputs:
      userId:   "{{ outline.userId }}"
      amount:   "{{ outline.amount }}"
      currency: "{{ outline.currency }}"

  - type: validate
    procedure: payments.validate_order_response
    inputs:
      response:        "{{ ledger.payments.send_create_payment.http.response }}"
      expected_status: "{{ outline.expected_status }}"

examples:
  - name: standard-order
    userId: "u-001"
    amount: 100
    currency: "GBP"
    expected_status: 201

  - name: zero-amount-rejected
    userId: "u-001"
    amount: 0
    currency: "GBP"
    expected_status: 400

  - name: large-usd-order
    userId: "u-001"
    amount: 500000
    currency: "USD"
    expected_status: 201

  - name: invalid-currency
    userId: "u-001"
    amount: 100
    currency: "XXX"
    expected_status: 422

Each row runs the whole flow independently. A failing row doesn't halt the others — you see exactly which inputs pass and which fail in a single run.

Advanced: Context Variables

Use ctx.var to store and reference values across steps:

flow:
  # Capture the payment ID
  - type: send
    procedure: payments.send_create_payment
    inputs:
      amount: 5000
    set:
      payment_id: "{{ ledger.payments.send_create_payment.outputs.id }}"

  # Use the stored ID later
  - type: send
    procedure: payments.send_get_payment_status
    inputs:
      id: "{{ ctx.var.payment_id }}"

Advanced: Poll Until Converged (poll:)

Some state becomes true only after a delay — an async settlement lands, a downstream consumer catches up, a row materialises. A fixed syz.wait guesses the duration and is a flakiness smell; a poll: block instead re-runs a workflow until it passes, or a bound trips. This is the primitive for eventual consistency.

A workflow is a reusable, ordered sequence of steps — typically a get-status send paired with a validate — invoked as a single step. poll: attaches to a workflow step only — never send/validate/craft (that's a POLL_ON_NON_WORKFLOW_STEP lint error). The workflow re-runs until its aggregate outcome is success:

flow:
  # Fire the state change
  - type: send
    procedure: payments.send_settle_payment
    inputs: {}

  # Re-run this workflow (a get-status send + a validate-settled pair) until it passes.
  - type: workflow
    procedure: payments.check_payment_settled
    poll:
      max_attempts: 10      # required — finite bound (integer ≥ 1)
      interval_ms: 1000     # optional base delay between attempts (default 0)
      backoff_ms: 500       # optional LINEAR increment added each attempt (default 0 → fixed interval)
      timeout_ms: 60000     # optional secondary wall-clock cap

How the loop decides:

• The polled workflow's own validate sub-step is the convergence condition. success → stop early; failure → retry; error / exception / aborted → halt immediately (a fault is never retried — retrying broken setup only buries the cause).
• Exhaustion (all max_attempts used, or timeout_ms elapsed with no pass) surfaces as the last attempt's failure — it re-enters the normal stop path, so --continue-scenario-on-failure still applies. No special exit code.
• The delay before attempt n+1 is interval_ms + (n−1) × backoff_ms.
• Each attempt re-runs the workflow's sends, so the ledger gains indexed keys (send_get_payment_status, send_get_payment_status[1], …). A bare {{ ledger.<procedure>… }} reference resolves to the latest attempt, so the workflow's validate always reads that attempt's fresh response.
• A polled workflow with no validate sub-step has nothing to converge on — lint warns (POLL_WITHOUT_VALIDATE).

Prefer poll: over a fixed syz.wait whenever you're waiting for the system to reach a state; reserve syz.wait for genuinely fixed windows (a rate-limit window, a TTL expiry).

06 Part 4: Execute — Run Your Tests

Execute scenarios and handle the results

Lint First (Pre-Flight Check)

Before running, pre-flight the scenario. syz lint validates schemas, resolves the environment, confirms every referenced procedure exists, and runs a residual scan for any unresolved {{ os-env-var:... }} expressions — all without making any HTTP calls.

syz lint --env local

Additional flags: --plugin <name> (scope to a specific plugin — required when multiple local plugins exist), --suite <path> (lint only a scenario file, directory, or suite manifest — defaults to all of .syz/scenarios/), --tag <label> (filter by tags), --example <name> (filter outline rows), --json (structured output).

Run It

syz run executes your scenarios. Required flags: --suite (scenario file or directory) and --env (environment to run against).

syz run --suite .syz/scenarios/e2e/checkout.yaml --env local

Point --suite at a directory to run everything under it, recursively:

# run every scenario in the e2e folder
syz run --suite .syz/scenarios/e2e/ --env local

# run the entire suite, every level
syz run --suite .syz/scenarios/ --env local

Useful Flags

Reading the Terminal Output

Each step streams its phases live, then a one-line verdict. Sensitive fields are masked everywhere:

─── STEP 1 of 2: send — payments.send_create_payment ───────────────────

  [Input Resolution]
    base_url    "{{ env.payments.base_url }}"           →  "http://localhost:3000"
    auth_token  "Bearer {{ env.payments.auth_token }}"  →  "Bearer [MASKED]"
    body.amount 100                                        (concrete)

  [Payload Construction]
    Merged, resolved request:
    POST http://localhost:3000/payments
    body (raw): {"userId":"u-001","amount":100,"currency":"GBP"}

  [Executor]
    →  POST http://localhost:3000/payments
    ←  201 Created  145ms
       {"id":"pay_abc123","status":"pending"}

  [Ledger Write]
    Outputs:   { id: "pay_abc123", httpStatus: 201 }
    Duration:  145ms

✓ PASS  145ms

The Results Folder

Every run writes one folder under .syz/results/, named with the execution ID (or your --run-name):

.syz/results/run-2026-07-04T10-53-00-000Z/
├── execution.json    # run-level index — all scenarios + status
├── index.html        # self-contained debug viewer — opens via file://
├── junit.xml         # only with --reporter junit
└── e2e/
    └── checkout/      # mirrors .syz/scenarios/e2e/checkout.yaml
        ├── debug.log  # phase-by-phase trace, plain text
        └── ledger.json  # the audit artefact — every request/response

Handling Failures

By Default (scenario execution stops on the first failure):

syz run --suite .syz/scenarios --env local
# Scenario 1: ✓ pass
# Scenario 2: ✗ fail — run stops here
# Scenario 3: ✓ pass
# Exit code: 1

Continue on Failure (scenario execution does not stop on the first failure):

syz run --suite .syz/scenarios --env local --continue-scenario-on-failure
# Scenario 1: ✓ pass
# Scenario 2: ✗ fail — continue anyway
# Scenario 3: ✓ pass
# Exit code: 1 (still fails overall)

Use this when you want the scenario to continue to the end, instead of stopping at the first problem.

07 Part 5: Audit & Review — Read Your Results

Understand what happened at every step

What is "Audit & Review"?

After running tests, you review the results to understand:

• What passed and what failed? (execution summary)
• What was sent and what was received? (request/response capture)
• Which assertions passed and which failed? (validation details)
• How does this trace back to business requirements? (traceability)

Reading execution.json

.syz/results/run-2026-07-04T10-53-00-000Z/execution.json

The run-level summary of all scenarios and their outcomes:

{
  "execution_id": "run-2026-07-04T10-53-00-000Z",
  "timestamp": "2026-07-04T10:53:00.000Z",
  "environment": "local",
  "suite_path": ".syz/scenarios",
  "total_scenarios": 2,
  "passed": 1,
  "failed": 1,
  "total_duration_ms": 3334,
  "continue_scenario_on_failure": false,
  "scenarios": [
    {
      "scenario_id": "e2e/payment_flow",
      "name": "Payment Flow — Happy Path",
      "level": "e2e",
      "status": "success",
      "duration_ms": 1234,
      "results_path": "e2e/payment_flow",
      "steps": [
        { "step_number": 1, "procedure": "payments.send_create_payment",   "type": "send",     "outcome": "success" },
        { "step_number": 2, "procedure": "payments.validate_payment_created", "type": "validate", "outcome": "success" }
      ]
    },
    {
      "scenario_id": "e2e/payment_refund",
      "name": "Payment Refund — Happy Path",
      "level": "e2e",
      "status": "failure",
      "duration_ms": 2100,
      "results_path": "e2e/payment_refund",
      "steps": [
        { "step_number": 1, "procedure": "payments.send_create_payment",    "type": "send",     "outcome": "success" },
        { "step_number": 2, "procedure": "payments.send_refund_payment",     "type": "send",     "outcome": "success" },
        { "step_number": 3, "procedure": "payments.validate_refund_applied", "type": "validate", "outcome": "failure" }
      ]
    }
  ]
}

Outline scenarios add outline_name / example_name per row; a filtered run adds a selection block recording the suite, tags, and examples that were matched or skipped.

Status & Outcome Types — status is the per-scenario verdict, outcome is the per-step verdict; both draw from the same set:

• success — All steps passed
• failure — ≥1 validation failed (test logic mismatch)
• error — ≥1 step couldn't run (bad reference, wrong syntax)
• exception — External system down (API unreachable, timeout)
• aborted — SYZYGY internal issue

Reading index.html

.syz/results/run-2026-07-04T10-53-00-000Z/index.html

The interactive debug viewer — self-contained, opens in a browser without any server. Double-click the file or run:

open .syz/results/run-2026-07-04T10-53-00-000Z/index.html

Sections you'll find:

1. Header + Execution Summary — Run ID, environment badge, timestamp, and total duration in the header (plus a continue-on-failure badge and doc-skew / upgrade-available banners when applicable). Below it, four stat cards: Total, Passed, Failed, Duration.
2. Controls — Expand All / Collapse All, a status filter (All / Success / Failure / Error / Exception / Aborted, each with a count), and a scenario search box.
3. Scenario List — One expandable card per scenario, showing its status, name, and duration, plus a copy-trace button. Each card links directly to its debug.log and ledger.json for the full step-by-step trace and the structured record.
4. Step Details (inside each scenario) — One expandable block per step, showing status, type, procedure, and duration, with two inline sections:
   • Executor — the request and response as an exchange: a method + URL line, a headers table, and a formatted body. The request offers Copy JSON and, for an HTTP request, Copy as cURL (a ready-to-run curl command); the response shows its status, reason, size, format, and duration.
   • Assertions — each check, pass/fail, with rule, expected, actual, and detail.

   The lower-level resolution detail — input resolution, payload construction, context variables, and the ledger write — isn't shown in the report; it lives in the linked debug.log and ledger.json, so the report stays focused on what was sent, what came back, and what passed. A step's error message renders inline within its block. A workflow renders as a single tree node named after the workflow, with its steps nested beneath it (a polled workflow instead nests its per-attempt sub-steps). Steps that neither send a request nor check a response — a wait or a context-variable set — are omitted, unless one failed, in which case it's kept with its error.

Reading debug.log

.syz/results/run-2026-07-04T10-53-00-000Z/e2e/checkout/debug.log

A plain-text transcript of the entire scenario execution. Useful for CI logs or quick text search:

─── STEP 1: send — payments.send_create_payment ───────────────
  [Input Resolution]
    {{ env.payments.base_url }}  →  http://localhost:3000
  [Payload Construction]
    Merged, resolved request: POST http://localhost:3000/payments
  [Executor]
    POST http://localhost:3000/payments  →  201 Created  145ms
  [Ledger Write]
    Output: id = "pay_abc123"
  ✓ SUCCESS

─── STEP 2: validate — payments.validate_payment_created ──────
  [Assertions]
    http  $.fields.status_code  equals 201     →  PASS
    json  $.fields.body.id      exists          →  PASS
    json  $.fields.body.amount  greater_than 0  →  PASS
  ✓ SUCCESS

Reading ledger.json

.syz/results/run-2026-07-04T10-53-00-000Z/e2e/checkout/ledger.json

The authoritative machine-readable log of the scenario execution. The file is an object keyed by procedure (a repeated procedure gets indexed keys: procedure, procedure[1], …). A send entry nests its request/response under the executor key (http:); the HTTP response envelope carries fields (status code, body) and headers:

{
  "payments.send_create_payment": {
    "procedure": "payments.send_create_payment",
    "type": "send",
    "http": {
      "request": {
        "fields": {
          "base_url": "https://api.payments.example.com/v1",
          "method": "POST",
          "path": "/payments",
          "body": {
            "type": "raw",
            "content": { "amount": 5000, "currency": "GBP" }
          }
        },
        "headers": { "Authorization": "[MASKED]" }
      },
      "response": {
        "fields": {
          "status_code": 201,
          "body": { "id": "pay_abc123xyz", "status": "pending", "amount": 5000 }
        },
        "headers": { "content-type": "application/json" },
        "format": "json",
        "size_bytes": 145
      }
    },
    "outputs": { "id": "pay_abc123xyz", "status": "pending" },
    "outcome": "success",
    "timestamp": "2026-07-04T10:53:00.000Z",
    "duration_ms": 245
  }
}

These entries are exactly what {{ ledger.… }} references resolve against: {{ ledger.payments.send_create_payment.http.response.fields.body.id }} walks this structure, and {{ ledger.payments.send_create_payment.outputs.id }} is the declared-output shortcut.

Debugging Failed Tests

Scenario: Assertion Failed

Step 1: Check the execution summary

cat .syz/results/run-2026-07-04T10-53-00-000Z/execution.json | grep -B2 -A1 "failure"

Step 2: Open index.html in browser, navigate to the failed step, look at the "Assertions" section to see which check failed and why

Step 3: Check the debug.log for context

cat .syz/results/run-2026-07-04T10-53-00-000Z/e2e/payment_flow/debug.log

Step 4: Fix and Re-run

Update the scenario or test data based on what the assertion showed, then:

syz run --suite .syz/scenarios/e2e/payment_flow.yaml --env local

Requirements Traceability Matrix (RTM)

The syz rtm command maps every requirement in your knowledge files — DOMAIN.yaml and each plugin's PLUGIN-GUIDE.yaml — to the tests that cover it. Coverage is declared on the test side with covers:, on a scenario, an outline example row, or an individual step:

name: "Payment Flow — Happy Path"
covers: [REQ-PAYMENT-DOM-001]

Basic Syntax:

syz rtm            # writes .syz/rtm.md — the human-readable matrix

With Options:

syz rtm --json     # writes .syz/rtm.json instead — machine-readable
syz rtm --strict   # exit 1 if any requirement is uncovered or a covers: dangles (the CI gate)
syz rtm --lastrun  # annotate each entry with its outcome from the latest run

Terminal Output:

  ✓ wrote .syz/rtm.md
  ⚠ 1 uncovered requirement
  ✓ 0 dangling covers: references
  • 1 in-progress requirement excluded (ids beginning with '_')

The generated .syz/rtm.md:

# Requirements Traceability Matrix (RTM)

> Derived report — generated by `syz rtm`. Do not hand-edit; set `covers:` on your tests and regenerate.
>
> 1 in-progress requirement excluded (ids beginning with `_`). Drop the leading `_` to track coverage.

| Requirement | Covered by |
|---|---|
| REQ-PAYMENT-DOM-001 | e2e/payment_happy_path; e2e/payment_refund |
| REQ-PAYMENTS-PG-001 | component/validate_currencies ▸ row:invalid_currency |
| REQ-REFUND-DOM-001 | integration/refund_workflow ▸ step:2 (payments.refund) |

## Uncovered (1)
- REQ-PAYMENTS-PG-003

## Dangling covers: (0)

The RTM is a derived report, like the run report — never maintained by hand. With --json, .syz/rtm.json carries the same model plus a summary (total, covered, uncovered, orphans, in_progress_excluded) and a gaps list for scripting.

In-progress requirements: while the work behind a requirement is still in flight, prefix its id with an underscore (_REQ-PAYMENT-DOM-002, as in the DOMAIN.yaml example earlier). It stays fully validated, but it's left out of the coverage floor — syz rtm won't list it as a gap, --strict won't fail on it, and the summary notes how many were excluded. Drop the leading _ when it's ready to track: it's the same requirement, so any covers: already pointing at it keeps working (syz lint warns while a test covers a requirement that's still in progress).

Use this to identify test gaps — which requirements need more scenario coverage?

08 Running Example: Complete Walkthrough

A full end-to-end example from scratch

Scenario: The Payments API Integration

Goal: Set up a plugin for a Payments API, write test scenarios, run them, and review results.

1. 1

   Initialise the Workspace

   mkdir payments-integration
   cd payments-integration
   syz init

2. 2

   Define Your Domain (DOMAIN.yaml)

   Edit .syz/dossier/DOMAIN.yaml — declare the area and the end-to-end requirements that define correct behaviour. This captures what you're testing and why.

   summary: >
     Payments processing for e-commerce orders.
   
   areas:
     - key: PAYMENT
       title: Payment processing
   
   requirements:
     - id: REQ-PAYMENT-DOM-001
       behaviour:
         given: an order with a valid payment method
         when: the customer pays
         then: the payment settles and its status can be tracked
       rules:
         - payments must be positive amounts (in minor units — cents)
         - only AUD, CNY, EUR, GBP, INR, JPY, and USD are accepted
         - a payment's status is one of pending, completed, failed, refunded
       notes: >
         The core journey to test: create a payment and verify its status, run
         the refund workflow, and reject unsupported currencies.

3. 3

   Create OpenAPI Spec

   Create payments-api.openapi.yaml:

   payments-api.openapi.yaml

   openapi: 3.0.0
   info:
     title: Payments API
     version: 1.0.0
   servers:
     - url: https://api.payments.example.com/v1
   
   paths:
     /payments:
       post:
         operationId: createPayment
         summary: Create a payment
         requestBody:
           required: true
           content:
             application/json:
               schema:
                 type: object
                 required: [amount, currency, customer_id]
                 properties:
                   amount:
                     type: integer
                     description: Amount in cents
                     example: 5000
                   currency:
                     type: string
                     example: GBP
                   customer_id:
                     type: string
                     example: cust_123
         responses:
           '201':
             description: Payment created
             content:
               application/json:
                 schema:
                   type: object
                   properties:
                     id:
                       type: string
                     status:
                       type: string
                     amount:
                       type: integer

4. 4

   Generate Plugin

   syz generate plugin --name payments --from openapi:./payments-api.openapi.yaml

5. 5

   Set Up Environment

   syz generate already seeded .syz/environments/local.yaml with a base_url and auth_token per procedure — review it and point it at your staging host:

   name: local
   production: false
   
   payments:
     send_create_payment:
       base_url: "https://api-staging.payments.example.com/v1"
       auth_token: "{{ os-env-var:PAYMENTS_AUTH_TOKEN }}"
     send_get_payment_status:
       base_url: "https://api-staging.payments.example.com/v1"
       auth_token: "{{ os-env-var:PAYMENTS_AUTH_TOKEN }}"

   Both entries name the same OS variable, and the repetition is deliberate: one plugin is one upstream system and therefore one credential set, so the seeded name is <PLUGIN>_<FIELD> — plugin-scoped, not per-operation. One export runs every scenario for the plugin, and rotating the credential is one change rather than one per operation. If a particular operation genuinely needs its own credential, point that one entry at a different variable and export it.

   Set the environment variable the seeded file references — one per plugin, covering every operation in it:

   export PAYMENTS_AUTH_TOKEN="sk_live_abc123xyz"

6. 6

   Adapt the Generated Plugin (add validate checks)

   The generator gives you runnable procedures — and the send procedure already drives its request body from each scenario's inputs:, so there's nothing to wire by hand. The one thing left to author is the richer validate procedures your scenarios call. This step is plugin-side authoring — see the Artefact Reference for the full schema of every field used here.

   1. The send procedure is already wired to your scenario inputs. syz generate emits send_create_payment with a defaults: block — one entry per body field, each seeded from data.yaml — and a request body whose content reads those values back through {{ inputs.* }}. So a scenario runs with inputs: {} and posts the data.yaml sample, or sets inputs: { amount: 500 } to override just that field while the rest stay sampled. No editing required; this is the shape the generator produces:

   .syz/plugins/payments/procedures/send_create_payment.yaml

   procedures/send_create_payment.yaml

   name: payments.send_create_payment
   type: send
   execution_library: http
   description: "Create a new payment"
   outputs:
     id:
       path: "$.fields.body.id"
       description: "ID of the created payment"
   defaults:                              # each body field seeded from data.yaml…
     amount: "{{ plugin.payments.data.create_payment.body.amount }}"
     currency: "{{ plugin.payments.data.create_payment.body.currency }}"
     customer_id: "{{ plugin.payments.data.create_payment.body.customer_id }}"
   http:
     request:
       fields:
         base_url: "{{ env.payments.send_create_payment.base_url }}"
         method: POST
         path: /payments
         body:
           type: raw
           content:                       # …and read back here through the merged inputs
             amount: "{{ inputs.amount }}"
             currency: "{{ inputs.currency }}"
             customer_id: "{{ inputs.customer_id }}"
       headers:
         Authorization: "Bearer {{ env.payments.send_create_payment.auth_token }}"
   sensitive_fields:
     - http.request.headers.Authorization

   A scenario overrides only what it names: inputs: {} sends all three fields from data.yaml; inputs: { amount: 9999 } sends amount: 9999 with currency/customer_id still sampled. (This per-field wiring is emitted for JSON request bodies; file-upload and multipart sends stream a file instead and expose no body inputs.)

   2. Author the two validate procedures the scenarios call. The generator emits a status-only validate_create_payment; the happy-path scenario wants richer checks under its own names. Create these two files under .syz/plugins/payments/procedures/:

   procedures/validate_payment_created.yaml

   procedures/validate_payment_created.yaml

   name: payments.validate_payment_created
   type: validate
   description: "A payment was created — 201 with a well-formed body"
   checks:
     - type: http
       description: "HTTP status matches expected"
       path: "$.fields.status_code"
       operator: equals
       value_from_input: expected_status
     - type: json
       description: "Payment ID is present"
       path: "$.fields.body.id"
       operator: exists
     - type: json
       description: "Status is a known lifecycle state"
       path: "$.fields.body.status"
       operator: in
       value: ["pending", "processing", "completed"]

   procedures/validate_payment_status.yaml

   procedures/validate_payment_status.yaml

   name: payments.validate_payment_status
   type: validate
   description: "The payment reached the expected lifecycle status"
   checks:
     - type: json
       description: "Body status matches expected"
       path: "$.fields.body.status"
       operator: equals
       value_from_input: expected_status

   expected_status means two different things by design: on validate_payment_created it's an HTTP status code (201, checked against $.fields.status_code), and on validate_payment_status it's a body value ("completed", checked against $.fields.body.status). Each validate procedure decides what its inputs assert against.

7. 7

   Write Test Scenarios

   Create .syz/scenarios/e2e/payment_happy_path.yaml:

   e2e/payment_happy_path.yaml

   name: "Payment Flow — Happy Path"
   level: e2e
   plugins:
     - name: payments
       version: "1.0.0"
   
   flow:
     # Create a payment
     - type: send
       procedure: payments.send_create_payment
       inputs:
         amount: 5000
         currency: "GBP"
         customer_id: "cust_alice"
   
     # Verify it was created
     - type: validate
       procedure: payments.validate_payment_created
       inputs:
         response: "{{ ledger.payments.send_create_payment.http.response }}"
         expected_status: 201
   
     # Get the payment status
     - type: send
       procedure: payments.send_get_payment_status
       inputs:
         id: "{{ ledger.payments.send_create_payment.outputs.id }}"
   
     # Verify status is completed
     - type: validate
       procedure: payments.validate_payment_status
       inputs:
         response: "{{ ledger.payments.send_get_payment_status.http.response }}"
         expected_status: "completed"

   Create .syz/scenarios/e2e/payment_parameterized.yaml (test multiple currencies):

   e2e/payment_parameterized.yaml

   name: "Payments — Multiple Currencies"
   level: e2e
   plugins:
     - name: payments
       version: "1.0.0"
   
   flow:
     - type: send
       procedure: payments.send_create_payment
       inputs:
         amount: 5000
         currency: "{{ outline.currency }}"
         customer_id: "cust_alice"
   
     - type: validate
       procedure: payments.validate_payment_created
       inputs:
         response: "{{ ledger.payments.send_create_payment.http.response }}"
         expected_status: "{{ outline.expected_status }}"
   
   examples:
     - name: gbp-success
       currency: "GBP"
       expected_status: 201
   
     - name: usd-success
       currency: "USD"
       expected_status: 201
   
     - name: eur-success
       currency: "EUR"
       expected_status: 201
   
     - name: invalid-xxx
       currency: "XXX"
       expected_status: 422

8. 8

   Run Tests

   syz run --suite .syz/scenarios/e2e/ --env local

9. 9

   Review Results

   # Interactive viewer
   open .syz/results/run-2026-07-04T10-53-00-000Z/index.html
   
   # Or check plain text log
   cat .syz/results/run-2026-07-04T10-53-00-000Z/e2e/payment_happy_path/debug.log
   
   # Or parse the ledger
   cat .syz/results/run-2026-07-04T10-53-00-000Z/e2e/payment_happy_path/ledger.json | jq

10. 10

    Check Requirements Coverage

    syz rtm
    
    # Writes .syz/rtm.md — which requirements from DOMAIN.yaml / PLUGIN-GUIDE.yaml
    # are covered by your tests, and which are still uncovered

    In CI, use syz rtm --strict as the coverage gate — it exits 1 if any requirement is uncovered or a covers: dangles.

09 Common Workflows & Troubleshooting

How to handle common situations

Workflow 1: "I Found a Bug in My API"

Scenario: A test fails because the API returned an unexpected error.

Diagnosis:

1. Open index.html from the failed run
2. Find the failed step
3. Look at the Executor block — what was the request sent and the actual response?
4. Look at the Assertions block — which check failed, and what was expected vs actual?
5. For the inputs and constructed payload behind the request, open the step's debug.log (linked from the scenario card) and read its Input Resolution and Payload Construction sections.

Solution: Usually one of:

• API bug: Report to dev team with ledger.json attached (proves what was sent)
• Test bug: Update scenario inputs or assertions
• Integration bug: Environment/credentials issue

Workflow 2: "I Want to Verify My Tests Are Actually Testing"

Scenario: You want proof that your test would catch a real bug.

Verification (Mutation Testing):

1. Temporarily break the assertion:

checks:
  - type: json
    path: "$.fields.body.id"
    operator: equals
    value: "wrong-id-on-purpose"

2. Run: syz run --suite your-scenario.yaml --env local
3. Verify it fails (if it passes, your test isn't actually checking that field)
4. Revert the change

Workflow 3: "I Need to Test Against Multiple Environments"

Scenario: Same tests, different servers (local, staging, production).

Solution:

# Local
syz run --suite .syz/scenarios/ --env local

# Staging
syz run --suite .syz/scenarios/ --env staging

# Production (read-only, no destructive tests)
syz run --suite .syz/scenarios/e2e/ --env production

Each environment has its own .syz/environments/<name>.yaml with different base URLs and credentials.

Workflow 4: "My Test Passes Locally but Fails in CI"

Scenario: Flaky or environment-dependent test.

Diagnosis:

1. Fetch the CI run folder (.syz/results/<run-id>/) as a build artifact
2. Compare CI ledger.json vs local ledger.json
3. Look for differences in:
   • Response timestamps
   • Base URLs (is CI using the right environment?)
   • Credentials (are env vars set?)
   • Rate limiting (did CI hit a rate limit?)

Common Causes:

• Credentials not exported in CI
• Wrong environment flag missing
• Data cleanup between runs (CI doesn't clean up, next run conflicts)
• Timing issues (requests are too slow or fast)

Workflow 5: "I Need to Mask Sensitive Data"

Scenario: Logs contain API keys that shouldn't be visible.

Solution: Mark sensitive fields in the procedure:

name: payments.send_create_payment
type: send
sensitive_fields:
  - http.request.headers.Authorization
  - http.response.fields.body.access_token

SYZYGY automatically masks these in:

• debug.log (shows [MASKED])
• index.html (shows [MASKED])
• ledger.json (shows [MASKED])
• Terminal output

The actual value is preserved internally and forwarded to later steps, but hidden from logs.

Workflow 6: "I Need to Generate a Report for Management"

Scenario: Convert test results to a format for stakeholders.

Solution:

# Generate JUnit XML for CI tools (Jenkins, GitLab, GitHub)
syz run --suite .syz/scenarios/ --env local --reporter junit --output ./results.xml

# Open the HTML report for visual walkthrough
open .syz/results/run-2026-07-04T10-53-00-000Z/index.html

# Generate RTM for coverage reporting (writes .syz/rtm.json)
syz rtm --json

10 Appendix: Artefact Reference

Author anything by hand

Parts above lean on syz generate and your AI assistant. This appendix is the counterpart: the complete authored shape of every .syz artefact, so you can create or edit any of them by hand with nothing but this page. Each field below is the real schema syz lint enforces.

Everything a plugin ships lives under .syz/plugins/<name>/:

.syz/plugins/payments/
├── plugin.yaml            # identity (name + version + syz_format)
├── PLUGIN-GUIDE.yaml      # knowledge layer (see Capture)
├── procedures/            # send / validate / workflow / craft — one .yaml each
├── models/                # optional input models (referenced by input_model:)
├── templates/             # optional Nunjucks .njk files (craft procedures only)
└── data.yaml              # optional shared constants + example bodies

plugin.yaml — plugin identity

The whole file is three fields. The folder name is the plugin name and must match plugin:; the version is a plain string. There is no name or version anywhere else — PLUGIN-GUIDE.yaml, procedures, and scenarios never repeat them.

plugin: payments
version: 1.0.0
syz_format: 1

syz_format is not something you maintain. It records which version of the syz file format this plugin is written against — not your plugin's own version:, and not the syz version. syz generate plugin fills it in when it creates the folder, and leaves your value alone on every later run, including --force. It changes only if a future syz changes how plugin files are read, and syz will tell you exactly what to do if that day comes. The one time it matters: if you hand-write a plugin.yaml, include syz_format: 1.

Send procedure — procedures/<name>.yaml

A send procedure defines one HTTP request as a structured block (no template). Full field set:

The http block in full:

name: payments.send_create_payment
type: send
execution_library: http
description: "Create a new payment"
outputs:
  id:     { path: "$.fields.body.id",     description: "ID of the created payment" }
  status: { path: "$.fields.body.status", description: "Current status" }
http:
  request:
    fields:
      method: POST                        # GET|POST|PUT|DELETE|PATCH|HEAD|OPTIONS
      base_url: "{{ env.payments.send_create_payment.base_url }}"   # location — OR a full url:
      path: /payments                     # joined to base_url with one slash
      query: { limit: 50, ids: [1, 2] }   # arrays → repeat-key (?ids=1&ids=2) by default
      body:
        type: raw                         # none | raw | form-data | x-www-form-urlencoded | binary
        content:
          amount: "{{ inputs.amount }}"
          currency: "{{ inputs.currency }}"
          customer_id: "{{ inputs.customer_id }}"
    headers:                              # OPEN namespace — any valid header name
      Authorization: "Bearer {{ env.payments.send_create_payment.auth_token }}"
      Accept: "application/json"
  settings:                               # transport tuning (closed set — all optional)
    timeout_ms: 30000                     # default 30000
    follow_redirects: true                # default true
    max_redirects: 5                      # default 5
    query_array_format: repeat            # repeat | bracket | comma
    max_capture_bytes: 5242880            # response-body capture cap (~5 MB default)
    tls:   { verify: true, ca_cert: "./ca.pem", client_cert: "./c.pem", client_key: "./k.pem", client_key_passphrase: "{{ os-env-var:KEY_PASS }}" }
    proxy: { url: "http://corp-proxy:8080", username: "{{ os-env-var:PROXY_USER }}", password: "{{ os-env-var:PROXY_PASS }}" }
sensitive_fields:
  - http.request.headers.Authorization

Body — body.type decides content's shape:

A file is always { file: "<path>" }, never a bare path. A dynamic body (loops/conditionals) is built by a craft step and referenced as content: "{{ ledger.<plugin>.<craft>.response.body }}".

Validate procedure — procedures/<name>.yaml

A validate procedure is a list of checks: run against a response passed in as an input. Field set: name / type (validate) are required; description, an input_model path, and the checks list follow. Each check has a type (http | json | text), a description, an operator, and a target — a path (JSONPath into the response envelope) or a content input key, plus a literal value: or value_from_input: <key>.

name: payments.validate_payment_created
type: validate
description: "Payment created — 201, well-formed body, known currency"
input_model: models/validate_payment_created_input.yaml
checks:
  - type: http                                   # equality / numeric on a JSONPath
    description: "HTTP status matches expected"
    path: "$.fields.status_code"
    operator: equals
    value_from_input: expected_status
  - type: json                                   # string ops + JSONPath into the body/headers
    description: "Payment ID is present"
    path: "$.fields.body.id"
    operator: exists
  - type: json
    description: "Currency is supported"
    path: "$.fields.body.currency"
    operator: in
    value: ["AUD", "CNY", "EUR", "GBP", "INR", "JPY", "USD"]
  - type: json                                   # nested all/any carry sub-checks
    description: "Every line item has a positive amount"
    path: "$.fields.body.items[*].amount"
    operator: all
    checks:
      - { type: json, operator: greater_than, value: 0 }
  - type: json                                   # sandboxed JS over the whole response
    description: "Total equals subtotal plus tax"
    operator: expression
    expression: "response.fields.body.total === response.fields.body.subtotal + response.fields.body.tax"
  - type: json                                   # validate structure against JSON Schema
    description: "Body matches the payment schema"
    path: "$.fields.body"
    operator: matches_schema
    value:
      type: object
      required: [id, amount, currency, status]
      properties:
        id:       { type: string }
        amount:   { type: number }
        currency: { type: string }
        status:   { type: string }

Picking a type: use http for status-code equality and numeric comparisons; use json for anything on the body or headers, including all string operators (contains, matches) — it JSONPaths the whole envelope, so $.headers.content-type is reachable. Every operator evaluates in-process and deterministically — no assertion calls out to anything. The full operator menu is in the Quick Reference.

Workflow procedure — procedures/<name>.yaml

A reusable ordered list of steps. Each step names a procedure: and supplies its inputs: inline. A scenario invokes the whole workflow as one step and customises children with overrides:.

name: payments.create_payment_send_success
type: workflow
description: "createPayment happy path — send, then confirm no transport error"
steps:
  - procedure: payments.send_create_payment
    inputs:
      amount: 5000
      currency: "GBP"
      customer_id: "cust_alice"
  - procedure: payments.validate_send_no_error   # shared no-error validate, one per plugin
    inputs:
      response: "{{ ledger.payments.send_create_payment.http.response }}"

A workflow leaf step may also carry set: (write ctx.var.*), connection: (name a handle), and sensitive_fields: — exactly like a top-level scenario step.

Craft procedure — procedures/<name>.yaml

A craft procedure produces a local artefact (XML, JSON, CSV, …) with no network call. The content lands in the ledger and, optionally, on disk. The crafter: field names the strategy — nunjucks, the default and the only one. There is no execution_library — craft has no executor. output_to_file (supports {{ }}) writes under results/<run>/<scenario>/crafted/<proc>/; absent = in-memory only.

name: accounting.craft_invoice_xml
type: craft
crafter: nunjucks              # optional — the default when absent
description: "Render an invoice XML from inputs"
outputs:
  filePath: { path: "$.metadata.output_file_path", description: "Path to the written file" }
output_to_file: "invoice_{{ inputs.payment_id }}.xml"
template: templates/invoice.xml.njk

nunjucks is the only crafter — a craft step renders a template you wrote and committed, so the same inputs always produce the same artefact.

The rendered content is always at {{ ledger.<plugin>.<craft>.response.body }}; the written path (when output_to_file is set) is at {{ ledger.<plugin>.<craft>.outputs.filePath }} via the declared output.

Nunjucks template — templates/<name>.njk

Used only by craft procedures (HTTP requests use structured fields, not templates). A template receives just the step's resolved concrete inputs — no {{ ledger. }}, {{ env. }}, or {{ data. }} inside a .njk file.

<invoice>
  <paymentId>{{ payment_id }}</paymentId>
  <amount>{{ amount }}</amount>
</invoice>

Input model — models/<name>.yaml

An optional, reusable declaration of a procedure's inputs, referenced by input_model:. Each field carries a type and description.

name: validate_payment_created_input
fields:
  response:
    type: object
    description: "The response envelope from the send step"
  expected_status:
    type: number
    description: "The HTTP status code to assert"

data.yaml — plugin shared data

Optional. Holds example request bodies (one key per operation, written by syz generate) and any hand-authored constants. Referenced as {{ <plugin>.data.<key>... }} from procedures, scenarios, or workflow inputs.

# Generated example bodies — one block per operation
send_create_payment:
  headers: { Content-Type: application/json }
  params: {}
  body:
    amount: 1
    currency: example-currency
    customer_id: example-customer

# Hand-authored constants coexist alongside the generated keys
currencies:
  supported: ["AUD", "CNY", "EUR", "GBP", "INR", "JPY", "USD"]
  default: "GBP"

Environment file — environments/<name>.yaml

Selected at run time by syz run --env <name> (the <name> matches the file's name: field). A scenario never names an environment, so the same scenarios run unchanged against any env.

name: local
production: false                    # stored in report metadata; not enforced at runtime
allow_destructive_operations: true   # same — advisory metadata for your own CI to gate on

# One top-level block per plugin. The keys under it are whatever your procedures
# reference via {{ env.<plugin>.<...> }} — the shape is yours to define.
payments:
  send_create_payment:
    base_url: "https://api-staging.payments.example.com/v1"
    auth_token: "{{ os-env-var:PAYMENTS_AUTH_TOKEN }}"   # secret — see below
  send_get_payment_status:
    base_url: "https://api-staging.payments.example.com/v1"
    auth_token: "{{ os-env-var:PAYMENTS_AUTH_TOKEN }}"   # same var: one plugin = one credential
  timeout_ms: 5000                   # any plain constant you want to reference

# Optional: named connection handles shared across steps
connections:
  payments_api:
    execution_library: http
    lifecycle: ephemeral             # ephemeral (default) | keep_alive
    http:
      request:
        fields:  { base_url: "https://api-staging.payments.example.com/v1" }
        headers: { Authorization: "Bearer {{ os-env-var:PAYMENTS_AUTH_TOKEN }}" }

Env-driven header names (advanced). A header name — not just its value — can come from the environment, so one procedure sends Authorization against one gateway and a different name (or none) against another with no plugin edit. Template the key and quote it (an unquoted {{ corrupts the name):

# procedure
http: { request: { headers: { "{{ env.payments.send_create_payment.auth_header }}": "Bearer {{ env.payments.send_create_payment.auth_token }}" } } }
# environments/local.yaml → auth_header: Authorization
# environments/prod.yaml  → auth_header: null      # explicit null drops the header entirely

An env key that is simply absent (or an unset os-env-var) does not drop the header — it's an unresolved reference and fails the run (UNRESOLVED_TEMPLATE).

Configuring executors

Today the only executor is HTTP, selected with execution_library: http on a send procedure. "Configuring" it means writing its http.request.fields / headers / settings — including tls for mTLS or self-signed certs, proxy, and timeout_ms. There are no drivers to install for HTTP — it's built in.

• Connection identity can live in several layers, deep-merged field-by-field under one nearest-wins rule — lowest to highest: env connections.<name> < procedure connection < procedure http < scenario connections.<name> < step override (i.e. step > scenario > procedure > env). Ship a named handle when identity is shared across steps or must be reused/renamed; use plain {{ env.* }} references for single-procedure identity (what syz generate emits).
• --no-install / driver provisioning. Stateful executors on the roadmap (Kafka, SQL, SFTP, S3, …) need client libraries that syz provisions on demand by scanning the execution_library values your plugins declare — pure-JS drivers install silently, native/browser ones ask consent (--yes to accept non-interactively). --no-install (or SYZ_NO_AUTO_INSTALL=1) turns this off and fails fast with the exact npm install command. An HTTP-only project pulls nothing extra, so these flags are no-ops for it today.

dependency.json — installed plugins

Lives at .syz/dependency.json (committed). Two writers, cleanly split: syz install manages the plugins array — plugins pulled from git (they land in .syz/plugins.installed/<name>/, auto-gitignored), where name is the local alias and must match the source folder and version is a git tag — and syz init stamps syz_version with the CLI version that set up the workspace. Because the SYZ-* docs are regenerable and not committed, syz_version is the committed record of which syz authored your tests; it's what the "your reference docs may be out of date — run syz init" advisory compares against, so a teammate who clones and upgrades still gets the nudge.

{
  "syz_version": "0.12.0",
  "syz_format": 1,
  "plugins": [
    { "name": "payments", "git": "https://github.com/acme/payments-plugin.git", "version": "v1.2.0" }
  ]
}

syz_format is the workspace twin of the field in plugin.yaml above — same name, and the file it sits in says what it covers. This one covers the artefacts you own: your scenarios, environments, and suites. Note how differently it behaves from syz_version right above it: syz_version is a record, re-stamped on every syz init; syz_format is a contract, written once when it's absent and never bumped for you — init has no way to know whether your scenarios were actually updated, so raising the number is your call, not its.

11 Quick Reference & Glossary

Cheat sheet and terminology

Command Reference

File Locations Cheat Sheet

.syz/
├── dossier/
│   ├── DOMAIN.yaml                 ← Domain requirements (you write this)
│   └── TEST-STRATEGY.md            ← Test approach (you write this)
├── plugins/
│   └── payments/
│       ├── plugin.yaml
│       ├── PLUGIN-GUIDE.yaml       ← Plugin rules (you write this)
│       ├── procedures/                 ← send, validate, craft, workflow procedures
│       ├── models/                 ← input field contracts
│       ├── templates/              ← Nunjucks craft templates (multipart/file ops only)
│       └── data.yaml               ← static test data
├── environments/
│   ├── local.yaml                  ← Local env config (you write this)
│   ├── staging.yaml                ← Staging env config
│   └── production.yaml             ← Production env config
├── scenarios/
│   ├── e2e/                        ← Test scenarios (you write these)
│   ├── integration/
│   ├── component/
│   └── uat/
└── results/
    └── run-2026-07-04T10-53-00-000Z/
        ├── execution.json          ← Execution summary
        ├── index.html              ← Interactive viewer
        └── e2e/checkout/
            ├── debug.log           ← Plain-text trace
            └── ledger.json         ← Full request/response log

Glossary

Reference: Expression Syntax

One expression syntax — {{ ... }} — across scenarios, procedure inputs, and templates:

Reference: Assertion Operators

Complete list of operators available in validate procedure checks:

Next Steps

Learn more:

• Read .syz/dossier/SYZ-TESTING.md for scenario authoring best practices
• Read .syz/dossier/SYZ-GUIDELINES.md for plugin design patterns
• Read ARCHITECTURE.md for technical internals and advanced features
• Set up CI/CD to run tests on every commit