How SYZYGY couples requirements to tests, so the traceability matrix is generated rather than maintained. Below: the plugin format, the execution model, and the artefacts each run produces. New to the tool? The hands-on tutorial walks it all through step by step.
01 How it works
A small, sharp model. Three things.
The Execute and Report stages of the lifecycle, up close. Plugins hold the knowledge — procedures, templates, models, fixed test data, and a guide that explains it all. Scenarios describe the flow. Environments name the infrastructure under test. The CLI runs them, deterministically.
Plugin
A versioned bundle of integration knowledge for one system — its procedures (which craft payloads, send requests, and validate responses), templates (attachments and outbound artefacts rendered by craft steps), models (input contracts that document each procedure's inputs), fixed test data, and a schema-validated PLUGIN-GUIDE.yaml that captures all the business, technical, logical or any other knowledge of the system required for testing. Owned by the team that owns the integration, kept in git, tagged like a library, and installed with syz install.
Scenario
A YAML file that names a flow — a sequence of procedures a plugin exposes. The same scenario can be reused across test levels (a mock during component testing, staging during integration, a production-like environment during UAT), or new scenarios can be authored for a new scope, pulling in whichever plugins are needed with syz install.
Environment
A YAML file that names the infrastructure involved — base URLs, credentials (via env vars or secret-manager-rendered files), and any per-system config or environment specific test data. Switching environments is switching one CLI flag.
resolver · runtime data builder · ledger / env / data / outline / ctx.var / os-env-var references · every value from a committed file or a prior steppayload builder · structured request merge (env → procedure → scenario → step) · Nunjucks-rendered craft artefactsexecutor registry · HTTP today · built to plug in Kafka, MQ, SFTP, SSH, SQL, NoSQL, key-value, gRPC, cloud, and browser executors — without touching the runtimeassertion factory · json · text · http · 18 operators, all evaluated in-process · extensible
AUDITimmutable
ledger.json · each step's request/response entries · sensitive data maskeddebug.log · per-scenario execution traceindex.html · run summary dashboard · per-scenario expandable details · single-stop for any execution detailexecution.json · run summary metadata · per-scenario status indexjunit.xml · CI-friendly test results · optional
What happens when you run
command
$syz run--suite <path>--env <name>[options]
--suiterequiredpath to a scenario file or directory of scenarios to run
--envrequiredname matching a file in .syz/environments/ — sets base URLs, credentials, config
01
Load
Zod-validated — schema mismatch aborts before any execution begins
Plugins from .syz/plugins/ (local) and .syz/plugins.installed/ (git-installed), the scenario(s), and the environment — Zod-validated, every file.
readsplugins/plugins.installed/scenarioenv.yaml
02
Pre-flight
All-or-nothing — no partial execution begins with an unresolved name
The pre-flight checker walks the scenario's flow and confirms every step or workflow procedure referenced (including those nested inside workflow procedures) exists in a loaded plugin. No execution begins until every name resolves.
Three procedure types. One shared lifecycle. Distinct middle phase.
The flow runner iterates step by step. Every procedure — regardless of type — runs the same outer lifecycle: input resolution, then the procedure's specific execution, then a ledger write. What changes is the middle phase. A craft procedure resolves its inputs, renders the payload from a committed Nunjucks template, and optionally writes the artefact to disk for a downstream send. A send procedure resolves its inputs, builds the payload from its structured request block, and dispatches it to the target system via the executor — the response envelope is what downstream steps reference. A validate procedure resolves its inputs (including the response passed from the ledger), runs every declared check through the assertion engine, and writes each verdict to the ledger.
craft
resolve inputs→render committed Nunjucks template→write artefact to disk→ledger
send
resolve inputs→build payload from structured request block→execute call to server→ledger
validate
resolve inputs→run assertions on response→ledger
04
Record
Immutable — sensitive fields are masked at write time, not after
Every step writes an immutable, indexed entry to the scenario's ledger. Sensitive fields declared in the procedure YAML are masked on write. Output shortcuts resolve via JSONPath against the response and become {{ ledger.<procedure>.outputs.<field> }} references for downstream steps.
writesledger entrymasked fields{{ ledger.x }}
05
Report
Opens via file:// — no server, no network, no dependencies
The reporter writes index.html (clickable, embedded, opens with file://), per-scenario debug.log (plain-text phase trace), per-scenario ledger.json (the audit artefact), execution.json (run-level index), and optionally a JUnit XML.
For a team that has to produce evidence, determinism is not one feature among many — it is what makes the evidence worth anything.
Every value in a run resolves from a committed file or a previous step's recorded output, so the same scenario sends the same request and reaches the same verdict on the hundredth run as on the first. This is unconditional — there is no flag, no mode, and no configuration under which it stops being true.
What enforces it
Every input is committed. A value comes from a plugin's data.yaml, an environment file, an outline row, or a previous step's recorded output — never from something generated at run time. No sampling, and no wall-clock or random input unless you commit one.
The resolver, executor, assertion engine, and ledger are exact. Each resolves or evaluates its inputs by rule, and the ledger records what was sent and received rather than recomputing it.
The 18 assertion operators are deterministic. All are evaluated in-process against the recorded response — equals, contains, matches, matches_schema, and expression for logic spanning several fields.
Nothing is resolved outside the workspace. The runtime reads your .syz/ directory and the systems you point it at, and nothing else — so a run on a colleague's machine or a CI runner replays from the same committed inputs.
What you can hand an auditor
The rendered traceability matrix — .syz/rtm.md from syz rtm, each requirement against the test that covers it and that test's outcome. --strict makes it a CI gate that fails the build on any uncovered requirement or dangling covers:.
The run report — the self-contained index.html written per run; opens with a double-click, no server.
JUnit XML — --reporter junit, for whatever your organisation already ingests.
The immutable ledger and phase-by-phase debug log — what was sent, what came back, and which rule decided, with sensitive fields masked throughout.
A per-verdict exit code — 0 all passed, 1 a check failed, 2 bad input or setup, 3 an upstream system faulted, 4 no verdict delivered, 130 interrupted.
Because all of it is committed, the evidence pack for a release is a commit range — not a folder of screenshots assembled the week before the audit.
03 Core capabilities
What you get on day one.
Every capability below is shipped today.
01
Capture · Trace
Requirements as a concrete spec — authored by AI, traced automatically
Document what each system must do. Link tests to it. Get a matrix that's always current.
The people who know what "correct" means — product owners, business analysts, integration owners — capture it as schema-validated YAML: PLUGIN-GUIDE.yaml per integration, DOMAIN.yaml for the cross-service journeys. It's written in plain domain language with any AI assistant out of the box, before a single plugin exists — a concrete, structured spec with required fields and a known vocabulary, not free prose, so it stays consistent across people and sessions and is validated by syz lint. And "before a plugin exists" is literal: syz generate plugin --name <name> --guide-only scaffolds the knowledge-only shell when the endpoints themselves don't exist yet — the recorded requirements lint and trace immediately. syz lint --knowledge is the matching check: it validates the requirements on their own, reading no environment, no scenarios and no procedures, so a business analyst can check their work the moment they write it — with no endpoint, no spec and no environment file in existence yet. It applies every id, scope, layer, duplicate and schema rule the full pre-flight applies, so nothing is checked less strictly, and a later --from <spec> run builds the runnable plugin around them on the same folder, never touching the guide. Every requirement carries a stable ID. Link any scenario, outline, row, or step to the requirements it exercises with covers:, and syz rtm generates a Requirements Traceability Matrix on demand — a derived report, like the run report, never maintained by hand. --strict fails CI on any uncovered requirement or dangling link; --lastrun annotates each requirement with its latest execution outcome.
02
Generate
Generate plugins from what you already have
Point syz generate at an OpenAPI spec or a Postman collection — or answer a short interactive prompt. Get a complete, runnable plugin.
The generator is deterministic — no AI in the path, no network call required, always reproducible. It parses the source, builds a normalised SourceModel, and emits the full plugin: for every operation a send procedure (structured HTTP request block), a status-check validate procedure, and a send-success workflow; plus the plugin's data.yaml, a scaffolded PLUGIN-GUIDE.yaml, a happy-path sample scenario per operation, and a seeded environment file. Multipart endpoints with text content types are scaffolded as craft → send → validate (with a Nunjucks craft template); binary multipart endpoints get a filePathsend procedure and validate procedure. Working knowledge-first, before any spec exists? --guide-only scaffolds just the knowledge shell (PLUGIN-GUIDE.yaml plus an identity manifest) — the same command with --from later completes it in place, preserving the guide. OpenAPI and Postman ship today; the same --from interface is built to take more formats (see Roadmap).
03
Execute
Same scenarios, every environment
Author once, run against a mock during component testing, staging during integration, and a production-like environment during UAT.
--env <name> selects the environment file. Plugins read system config from env[pluginName] — base URLs, tokens, timeouts — so switching environments is switching a flag. Cross-references between systems ({{ env.shared.api_key }}) are resolved at load time with cycle detection, so a misconfigured env fails before any execution begins. {{ os-env-var:NAME }} is the inline resolver for OS environment variables — commit env files safely and inject secrets from a vault or CI secret store. Every resolved value is automatically registered and masked across all artefacts (stdout, debug log, ledger, HTML), composing with path-based sensitive_fields masking.
The environment file's counterpart is the plugin's data.yaml — the test data that stays fixed across environments (canonical payloads, reference IDs, sample inputs), addressed in templates as {{ <plugin>.data.x }}. Environment files hold what changes per environment; data.yaml holds what doesn't. Together they form the full test-data substrate, split along the only line that matters: does the value vary with where you run, or with what you test.
04
Author
Plugins are shared like libraries
Versioned in git. Installed by tag. Pinned in a manifest. The same workflow your team already uses for code.
syz install performs a shallow, sparse checkout — git clone --no-checkout --depth 1 --branch <tag> then git sparse-checkout set — and copies only the plugin's folder into .syz/plugins.installed/. No central registry, no publish workflow, no proprietary lock-in. .syz/dependency.json records every dependency by name, URL, and version — committed to source control so every team member and CI runner reproduces the same set. Monorepos with many plugins install efficiently because only the named plugin folder is transferred.
05
Audit + Review
Evidence, not just results
An auditor doesn't ask whether you have tests. They ask which test proves requirement REQ-014, and what it actually sent. Both answers are artefacts here.
Every step's phase-by-phase trace — input resolution, payload construction, executor, assertions, context variables, ledger write — is written to debug.log as plain text (identical to what streamed to the terminal). The HTML report is the review surface: it shows each step's request/response exchange and its assertions inline, with one-click Copy as cURL to replay a request, and links to that step's debug.log and ledger.json for the full trace. It opens via file:// double-click — no server, no network. Two masking mechanisms compose: path-based sensitive_fields declared on the procedure, and a run-scoped secret registry that automatically masks every value resolved from {{ os-env-var:... }} — both apply everywhere (stdout, debug log, ledger, HTML). Each scenario gets its own isolated ledger, so {{ ledger.x }} references never cross scenario boundaries and a failing scenario never corrupts a passing one.
The ledger is immutable and written at execution time — not reconstructed afterwards from logs — so it stands as a record of what was actually sent and what actually came back. Paired with syz rtm, that closes the loop a regulated change process needs: the requirement, the test that covers it, the run that executed it, and the request/response that proves it — four artefacts, all in git, all reviewable in a pull request, none of them maintained by hand.
06
Foundation
Author with the assistant your team already uses.
Your knowledge layer, in your editor, against the current contract.
syz init writes a framework-aware pointer file for every major AI tool, each backed by the versioned .syz/dossier/ knowledge layer. Your assistant reads those files in your editor and can capture requirements and author scenarios that match the framework contract as it stands today — no prompt engineering, no stale copy of the rules pasted into a chat.
Claude Code
Cursor
Copilot
Windsurf
Continue
Amazon Q
JetBrains AI
Gemini Code Assist
Aider
It drafts a concrete requirement, template, data entry, or assertion; you review it in a pull request and commit it; from then on the runtime replays exactly what you committed. That puts a human diff between the draft and your evidence, and it is why a scenario keeps giving the same answer twice — see Determinism.
The dossier is the reason this works without prompt discipline. It is versioned alongside your plugins, so the guidance an assistant reads is the same guidance the CLI validates against — when the contract changes, syz init refreshes the dossier and every assistant picks it up.
04 Quickstart
Requirement to result. Against a live API.
Real HTTP against a live, public API — no mocks, no placeholder host. Capture what the integration must do first, generate its plugin from an OpenAPI spec, run it green, and close on the traceability matrix that ties the result back to the requirement it proved.
You need Node.js 20+ and a git repo (any). The example targets the public Swagger Petstore (petstore3.swagger.io) — it's live, needs no real credentials, and returns a created record your test asserts on. Nothing here is mocked or stubbed.
1
Install
npm install -g syzs-cli
2
Initialise the workspace
syz init
Creates the .syz/ directory, the dossier/ knowledge layer, and a context file for every supported AI tool. It's explicit and idempotent — safe to re-run, and re-running after a CLI upgrade refreshes the framework rules in place. It never overwrites files that are yours: .syz/README.md, dossier/DOMAIN.yaml, dossier/TEST-STRATEGY.md, and each plugin's PLUGIN-GUIDE.yaml.
3
Capture the knowledge first — before any endpoint is wired
Scaffold a guide-only shell: the knowledge layer exists before the plugin does.
syz generate plugin --name petstore --guide-only
This writes a user-owned PLUGIN-GUIDE.yaml (plus an identity manifest) and no procedures — there's nothing to run yet, and that's the point: you record what correct means before you wire how to call it. Add one requirement, in plain language, to .syz/plugins/petstore/PLUGIN-GUIDE.yaml:
requirements:
- id: REQ-PETSTORE-001
behaviour:
given: a valid pet name and status
when: the pet is added
then: the API returns the created pet with a server-assigned id
notes: The id is assigned by the server; the client never supplies it.
Then validate the knowledge layer on its own — no environment, no spec, no network:
syz lint --knowledge
--knowledge applies every id, scope, layer, duplicate and schema rule the full pre-flight applies, reading no environment and no procedures — so the requirement is checked the moment it's written, with no endpoint in existence yet.
4
Generate the plugin from a real OpenAPI spec — onto the same folder
The spec has 19 operations, so syz generate asks the scope. Press n, then 1 to scaffold just POST /pet — the add-a-pet request:
✔ Found 19 operation(s). Generate all? (Y/n) · n
Enter operation numbers to generate (comma-separated, e.g. 1,3): 1
✓ Generated 1 operation(s) on the same folder.
⚠ PLUGIN-GUIDE.yaml — preserved (user-owned)
One operation keeps this to a single green run — POST /pet needs one auth token (step 6) and has no path parameter to wire. Add more as you need them.
syz generate writes the procedures, templates, data.yaml, the sample scenario (.syz/scenarios/generated/petstore/sample_add_pet.yaml), and the environment file it targets (.syz/environments/local.yaml) — and, crucially, leaves your PLUGIN-GUIDE.yaml untouched: the --from run builds the runnable plugin around the requirement you captured in step 3. --dry-run previews without writing.
One edit before you run: the Petstore spec ships a relative server URL, so set the full host as base_url in .syz/environments/local.yaml:
base_url: "https://petstore3.swagger.io/api/v3"
OpenAPI is the path shown here; syz generate --from postman: and syz install <git-url> are the other two ways to acquire a plugin — see the Tutorial.
5
Link the test to the requirement it proves
This is the load-bearing step — the one no collection-based tool can offer. Add a covers: line to the generated sample scenario (.syz/scenarios/generated/petstore/sample_add_pet.yaml) so its result can be traced back to the requirement you captured in step 3:
covers: [REQ-PETSTORE-001]
Any scenario, outline, row, or step can carry covers:; here one scenario-level link is enough. Without it the test still runs — but the requirement shows as uncovered in step 8, which is exactly what an auditor is looking for.
6
Export the secret the generated env references
syz generate names one auth token per plugin (<PLUGIN>_<FIELD>), not per operation: one plugin is one upstream system and one credential set, so a single export runs every generated scenario for it. Here that's PETSTORE_AUTH_TOKEN. If it's unset, syz lint prints the exact name. The Petstore sandbox accepts any value; it's a per-session variable, so pick your shell:
export PETSTORE_AUTH_TOKEN="demo"
$env:PETSTORE_AUTH_TOKEN = "demo"
set PETSTORE_AUTH_TOKEN=demo
7
Lint, run against the live server, open the report
Pre-flight the scenario before running:
syz lint --env local
Run it against the live Petstore server:
syz run --suite .syz/scenarios/generated/petstore/ --env local
Then open the report in your browser. syz run prints the results folder it just wrote — .syz/results/run-<timestamp>/. Open that folder's index.html: double-click it in your file explorer, or drag it into a browser tab. It's fully self-contained and opens over file:// — no server, no commands to run.
syz run makes a real HTTP call to the live Petstore server; the report shows a genuine 200 with the created pet's ID. Every run writes a fresh, uniquely-timestamped folder under .syz/results/ — so re-running never collides and never overwrites the last report.
run-2026-07-04T10-53-00-000Z
2 success0 failed1.4s
✓generated/petstore — Add a pet — created successfully612ms
✓sendpetstore.send_add_pet498ms
✓validatepetstore.validate_add_pet_response114ms
Self-contained index.html — opens via file://, no server.
In CI, pass a meaningful, unique label instead of relying on the timestamp:
syz run --suite .syz/scenarios/generated/petstore/ --env staging --run-name "deploy-$GIT_SHA"
8
Trace the result back to the requirement — the final frame
A passing run is table stakes; this is the artefact that isn't. Derive the matrix from the run you just did:
syz rtm --lastrun
syz rtm reads the covers: links and the latest run, and writes .syz/rtm.md — a derived report, like the run report, never maintained by hand:
That is the loop the whole product exists to close: a requirement captured before the plugin existed, realised as a runnable test, executed for real against a live API, and traced back — four artefacts, all in git. In CI, syz rtm --strict fails the build on any uncovered requirement or dangling link.
That's it. The same scenario runs against staging, UAT, or production by changing the --env flag and the matching environment file — and the matrix travels with it.
05 Anatomy of a plugin
What's in a plugin.
A folder — procedures, workflows, templates, models, test data, and a schema-validated integration guide. All YAML and Nunjucks. No code generation, no DSL, no surprises.
A plugin is a directory under .syz/plugins/<name>/ (if you authored it locally) or .syz/plugins.installed/<name>/ (if syz install fetched it for you). Both shapes are identical.
craft procedure renders a local artefact (XML, CSV, SWIFT MT103) from a committed Nunjucks template, optionally writing the result to disk for a downstream send procedure.
send procedure declares an executor (http), a structured request block (http.request fields + headers), optional named outputs, and a list of sensitive fields.
validate procedure declares an input contract and a list of checks — JSONPath, text, HTTP-metadata, with 18 deterministic operators, all evaluated in-process.
workflow procedure composes other procedures with input wiring, accepting per-step field-level overrides from the scenario that calls it.
models contains YAML input contracts — a named list of typed fields each procedure accepts. Referenced by input_model: in craft, send, and workflow procedures; checked by syz lint and read by AI assistants writing scenarios against the plugin.
templates contains Nunjucks .njk files that render craft artefacts — payloads for multipart and file operations — with access to resolved inputs, environment values, and plugin data. Referenced by template: on craft procedures.
data.yaml holds the test fixtures that don't change with the environment — canonical payloads, reference IDs, sample inputs — addressed as {{ <plugin>.data.x }}.
PLUGIN-GUIDE.yaml is the plugin's integration knowledge as schema-validated YAML: each operation's behaviour, business and logical rules, error semantics, and requirements with stable IDs. User-owned, scaffolded on generate, shipped with the plugin via syz install, and read by AI assistants authoring scenarios.
.syz/plugins/payments/├── plugin.yaml# plugin + version + syz_format, three fields├── PLUGIN-GUIDE.yaml# integration knowledge — schema'd YAML, ships via syz install├── CLAUDE.md# one-line pointer to PLUGIN-GUIDE.yaml├── procedures/│ ├── send_order.yaml# a send procedure (structured HTTP request)│ ├── validate_order_response.yaml│ ├── workflow_checkout.yaml# a workflow procedure that composes the two│ └── craft_order_payload.yaml# a craft procedure rendering an artefact via Nunjucks├── models/│ └── order_request.yaml# input contract for the send procedure├── templates/│ └── order_payload.njk# Nunjucks template for the craft procedure└── data.yaml# static test data, shared across environments
plugin.yaml
plugin: payments
version: 1.0.0
syz_format: 1 # the syz artefact contract this plugin is written against
PLUGIN-GUIDE.yaml
summary: >
Charges, captures, and refunds customer payments for orders.
requirements:
- id: REQ-PAYMENTS-PG-001 # stable traceability ID
behaviour:
given: a valid card and a supported currency
when: a payment is made for a positive amount
then: the payment is captured and a reference is returned
inputs:
- { field: amount, type: integer, min: 1, max: 1000000, required: true }
- { field: currency, type: string, values: [USD, EUR, GBP], required: true }
rules:
- amount must be between 1 and 1,000,000
- a declined card is refused and nothing is charged
errors:
- { code: DECLINED, meaning: the issuer refused the charge }
- { code: INVALID_CURRENCY, meaning: currency not in the supported set }
notes: >
The maximum is a business limit. A declined card is a normal,
expected outcome — the charge simply does not happen.
procedures/craft_order_payload.yaml
name: payments.craft_order_payload
type: craft
crafter: nunjucks
description: "Render an order payload for the downstream send procedure"
input_model: models/order_request.yaml
template: templates/order_payload.njk
output_to_file: "order_{{ inputs.userId }}.json" # lands in the run's crafted/ folder
procedures/send_order.yaml
name: payments.send_order
type: send
execution_library: http
description: "POST a new order to the payments service"
outputs:
orderId: { path: "$.fields.body.data.orderId", description: "Created order ID" }
httpStatus: { path: "$.fields.status_code", description: "HTTP status" }
http:
request:
fields:
base_url: "{{ env.payments.base_url }}"
method: POST
path: /orders
body:
type: raw
content: "{{ payments.data.send_order.body }}"
headers:
Authorization: "Bearer {{ env.payments.auth_token }}"
sensitive_fields:
- http.request.headers.Authorization
input_model: models/order_request.yaml
What ships today runs over HTTP. The architecture was built so new protocols, sources, and surfaces drop into the same registry without changing the runtime, the ledger, or how you author a scenario. The items below are planned, not yet shipped — everything else on the site is.
More executors. The executor registry takes new protocols behind one native-outcome contract: every executor captures its system's real response, and your validate steps decide pass or fail — so the way you author and assert a test never changes as the protocol does. Planned executors: Kafka (producer + consumer), message queues (publish + consume), SFTP (upload + download), SSH, SQL databases, NoSQL databases, key-value stores, gRPC, WebSocket, cloud services (object storage, queues, pub/sub), and a browser executor — with the priority order driven by what teams actually ask for. Stateful executors use named connection handles — open once, reuse across steps, torn down at scenario end; eventual-consistency systems are already served by the shipped poll: and syz.wait primitives.
More generation sources.syz generate reads OpenAPI and Postman today. The same --from interface and source registry are built to extend — each new format is just a new parser, with no change to the deterministic builder. On the list: AsyncAPI, WSDL, XSD, cURL, RAML, API Blueprint, GraphQL SDL, GraphQL Introspection Schema — and more on demand.
The point of the roadmap is the inverse of a feature list: the surfaces grow, the model doesn't. A team that learns to author one HTTP scenario today authors a Kafka or SQL scenario the same way tomorrow — same procedures, same resolvers, same ledger, same report.
07 Go hands-on
Learn to evaluate, not to operate.
You never have to learn how to code to use SYZYGY. The CLI ships all its knowledge directly to the AI code assistants you already use — they have everything they need to author, generate, and maintain your scenarios and plugins on your behalf. The technical detail on this page serves a different purpose: it equips you to read what runs, understand why a test failed, monitor what the runtime produces, and customise or improve generated output when the need arises. That context keeps you in control without requiring you to build from scratch.
The best next step is the step-by-step Tutorial — it walks through everything hands-on, from CLI setup to a working plugin, paired with a complete demo workspace (github.com/syzs-code/syz-cli-demo) you can clone and run by following its DEMO.md.