From first install to
Full Control.

00 Overview

What you'll build

By the end of this tutorial you'll have a working SYZYGY project that captures business requirements, generates a plugin, authors test scenarios, sends requests to a service, validates responses, and produces an auditable run report — all from plain YAML committed to git. No framework to stand up, no server to operate.

The path is linear. Each step produces an artifact the next step uses:

1. Install & init — get the syz CLI and lay down the .syz/ project structure.
2. Capture domain knowledge — document business rules and test strategy in DOMAIN.yaml.
3. Generate a plugin — create agents and templates from an OpenAPI spec with syz generate.
4. Write scenarios — compose agents into test flows in YAML.
5. Run & audit — execute tests and read the ledger, debug log, and interactive HTML report.
6. Share plugins — install shared plugins from git and use them like local ones.

01 Installation & Setup

Setting up your first SYZYGY project

SYZYGY is a single Node.js CLI — syz, published as syzs-cli. There's nothing else to operate. You install one binary, run syz init once, and everything else lives as plain text in your repo.

Prerequisites

Before installing SYZYGY CLI, ensure you have:

Operating Systems: macOS, Linux, Windows (via WSL or native)

1. 1

   Install the CLI

   Installs syz globally so it's available in any terminal. Requires Node.js 20+.

   npm install -g syzs-cli

   Verify Installation:

   node --version
   # Output should show: v20.x.x or higher
   
   npm --version
   # Output should show: 10.x or higher
   
   syz --version
   # Output: e.g., "v0.7.2"

2. 2

   Initialise the project

   Run this from your repo root (any git repo will do). It creates the .syz/ directory and the framework files inside it.

   syz init

   syz init generates:

   • .syz/SYZ-RULES.md — framework authoring rules (owned by SYZYGY, overwritten when you run syz init).
   • Pointer files for every AI assistant — CLAUDE.md, .cursor/rules/syzygy.md, .github/copilot-instructions.md, GEMINI.md, and more.
   • .syz/ai-api.yaml — config for AI-powered features (optional).
   • .syz/dependency.json — declares git-installed plugin dependencies.
   • Empty environments/ and scenarios/ folders, plus plugins/ for your work.

   Explicit initialization required. Run syz init once in your project root to create the .syz/ workspace. After that, all commands (syz run, syz generate, syz install) require the workspace — they fail fast if you're outside it. The one file in .syz/ that's yours to edit is .syz/README.md (optional).

3. 3

   Explore the structure

   Everything the framework manages lives under .syz/. Discovery is convention-based — there's no config file; the directory shape is the contract.

   <project-root>/
   ├── .gitignore                    # syz adds plugins.installed/ automatically
   └── .syz/
       ├── SYZ-RULES.md              # framework rules — owned by syz
       ├── CLAUDE.md                 # AI assistant pointers (+ others)
       ├── ai-api.yaml               # AI config — optional
       ├── dependency.json           # git plugin dependencies
       ├── dossier/
       │   ├── DOMAIN.yaml            # business rules (you write this)
       │   └── TEST-STRATEGY.md       # test approach (you write this)
       ├── plugins/                  # hand-authored plugins — committed
       ├── plugins.installed/        # git-installed plugins — gitignored
       ├── environments/             # base URLs + credentials per env
       ├── scenarios/                # component/ integration/ e2e/ uat/
       └── results/                  # one folder per run — gitignored

   • plugins/ — plugins you author by hand. Committed to git like any other code.
   • plugins.installed/ — plugins fetched by syz install. Auto-added to .gitignore.
   • environments/ — one YAML per environment, naming base URLs and credentials.
   • scenarios/ — your tests, organized free-form into level folders.
   • results/ — execution output, one folder per syz run.
   • dossier/ — knowledge documents: business domain, test strategy (you write these).
4. 4

   Write your first environment file

   An environment names the systems under test. Plugin config sits at the top level of the file, keyed by plugin name — base URL, credentials via environment variables, optional timeout.

   .syz/environments/local.yaml

   name: local
   production: false
   payments:
     base_url: "http://localhost:3000"
     token: "{{ os-env-var:PAYMENTS_TOKEN }}"  # resolved at load time; value auto-masked everywhere
     timeout_ms: 5000                           # optional — default 30000

   Then export the secret in your shell (or your CI's secret store) — never in the YAML:

   export PAYMENTS_TOKEN="dev-token"

   {{ os-env-var:NAME }} is inline-capable. It can appear anywhere inside a value — "https://{{ os-env-var:API_HOST }}/v1" works. Every resolved value is automatically masked in logs, debug files, HTML reports, and terminal output. If the variable is unset, syz lint catches it before execution, failing loudly.

Installation Troubleshooting

Problem: syz: command not found

Solution: Node.js or npm isn't installed, or SYZYGY wasn't installed globally.

# Reinstall with explicit global flag
npm install -g syzs-cli

# Verify the installation path
npm list -g syzs-cli

Problem: npm ERR! code EACCES (Permission Denied)

Solution: npm doesn't have permission to write to global directories.

Option 1 — Fix npm permissions (Recommended):

mkdir ~/.npm-global
npm config set prefix '~/.npm-global'
export PATH=~/.npm-global/bin:$PATH
npm install -g syzs-cli

Option 2 — Use sudo (Not Recommended):

sudo npm install -g syzs-cli

Problem: Node.js version too old

Solution: Upgrade to Node.js 20 LTS or later.

# On macOS (with Homebrew)
brew upgrade node

# On Windows/Linux
# Download from https://nodejs.org/

Problem: syz init fails or creates incomplete structure

Solution: Ensure you're running the latest version and have write permissions.

# Update to latest version
npm install -g syzs-cli@latest

# Verify you're in an empty directory or have write permissions
pwd
ls -la

# Try again
syz init

Alternative Installation Methods

Install Locally (Project-Specific)

If you want SYZYGY only for one project:

cd my-integration-tests
npm install --save-dev syzs-cli

# Run via npx
npx syz --version
npx syz init
npx syz run

Install from Source (For Development)

If you're contributing to SYZYGY:

git clone https://github.com/syzs-code/cli.git
cd cli
npm install
npm run build
npm link  # makes 'syz' command available globally during development

02 Introduction & Core Concepts

What SYZYGY is and how it works

What Is SYZYGY?

SYZYGY is a deterministic test execution engine for integration testing. Instead of writing test code by hand, you describe your API operations, environments, and test scenarios in simple YAML files. SYZYGY then runs those tests, captures detailed requests and responses, and lets you audit exactly what happened at every step — without frameworks, servers, or generated code complexity.

The Three-Layer Architecture

SYZYGY organizes knowledge into three layers:

Running Example: The Payments Plugin

Throughout this tutorial, we'll work with a fictional Payments API that manages financial transactions. Our plugin will support:

• Creating a payment (POST /payments)
• Retrieving payment status (GET /payments/{id})
• Refunding a payment (POST /payments/{id}/refund)

We'll start with an OpenAPI spec, generate a plugin, author test scenarios, run them, and review the results.

03 Part 1: Capture — Define Your Domain

Document business requirements before building tests

What Is "Capture"?

Before you generate a plugin or write test scenarios, you need to document your business domain. This means answering questions like:

• What is this API for? (e.g., processing customer transactions)
• What are the business rules? (e.g., refunds must be issued within 30 days)
• What integrations depend on this API? (e.g., order processing, invoicing)
• What are the key test scenarios? (e.g., happy path, error handling, edge cases)

This knowledge lives in two files under .syz/dossier/:

• DOMAIN.yaml — Cross-integration business knowledge and requirements
• TEST-STRATEGY.md — Your approach to testing this domain

Understanding DOMAIN.yaml

DOMAIN.yaml is a schema-validated YAML file that captures business and technical requirements shared across your entire integration suite.

.syz/dossier/DOMAIN.yaml

domain_name: "Payments Processing"
domain_description: |
  Manages financial transactions across the organization.
  Integrates with: Order Service, Invoicing, Accounting.

concepts:
  - name: "Transaction"
    definition: "A monetary exchange between customer and merchant"
    attributes:
      - "id: Unique identifier (UUID)"
      - "amount: Positive decimal in minor units (cents)"
      - "currency: ISO 4217 code (GBP, USD, EUR)"
      - "status: pending, processing, completed, failed, refunded"

  - name: "Refund"
    definition: "Reversal of a completed transaction"
    constraints:
      - "Must occur within 30 days of original transaction"
      - "Amount cannot exceed original transaction amount"
      - "Duplicate refund requests for same txn are rejected"

business_rules:
  - rule_id: "BR-PAYMENT-001"
    description: "Payments must be positive amounts"
    validation: "amount > 0"

  - rule_id: "BR-PAYMENT-002"
    description: "Only supported currencies are accepted"
    validation: "currency in [GBP, USD, EUR, AUD]"

integrations:
  - service: "Order Processing"
    interaction: "Calls payment API to charge customer"

test_data:
  currencies: ["GBP", "USD", "EUR", "AUD"]
  transaction_amounts:
    valid_range: "0.01 to 99999.99"
    examples: ["1.00", "99.99", "1000.00"]

critical_scenarios:
  - "Happy path: successful payment creation and retrieval"
  - "Error handling: invalid currency, missing fields"
  - "Edge cases: zero amount, negative amount, amount precision"

Key Points:

• Business-focused, not technical. A Product Owner should be able to read and update this file.
• Drives test design. Each business rule should have at least one test scenario.
• Lives in .syz/dossier/ — created by syz init, scaffolded if absent, never overwritten by the CLI.
• Shared across all plugins. If you have multiple plugins, they all reference the same DOMAIN.yaml.

04 Part 2: Generate — From API Spec to Runnable Plugin

Deterministic plugin generation from OpenAPI or Postman

What Is "Generate"?

After documenting your domain, the next step is to generate a plugin from your API specification. SYZYGY reads an OpenAPI or Postman collection and deterministically creates agents, templates, input models, and sample scenarios — all ready to customize.

The syz generate Command

Basic Syntax:

syz generate plugin --name <plugin-name> --from openapi:<path> [options]

Full Signature:

syz generate plugin \
  --name payments \
  --from openapi:./payments-api.openapi.yaml \
  --force \
  --no-install

Command Flags Explained

Step-by-Step: Generating the Payments Plugin

Step 1: Prepare Your OpenAPI Spec

Create payments-api.openapi.yaml:

openapi: 3.0.0
info:
  title: Payments API
  version: 1.0.0
servers:
  - url: https://api.payments.example.com/v1

paths:
  /payments:
    post:
      operationId: createPayment
      summary: Create a new payment
      requestBody:
        required: true
        content:
          application/json:
            schema:
              type: object
              required: [amount, currency, customer_id]
              properties:
                amount:
                  type: number
                  description: Amount in minor units (cents)
                  example: 1000
                currency:
                  type: string
                  example: GBP
                customer_id:
                  type: string
                  example: cust_123
      responses:
        '201':
          description: Payment created
          content:
            application/json:
              schema:
                type: object
                properties:
                  id:
                    type: string
                  status:
                    type: string
                  amount:
                    type: number

  /payments/{id}:
    get:
      operationId: getPaymentStatus
      summary: Retrieve payment details
      parameters:
        - name: id
          in: path
          required: true
          schema:
            type: string
      responses:
        '200':
          description: Payment details

Step 2: Run the Generator

syz generate plugin --name payments --from openapi:./payments-api.openapi.yaml

Step 3: Review Generated Files

The generator creates:

.syz/plugins/payments/
├── plugin.yaml                  # plugin identity
├── agents/
│   ├── step_send_order.yaml       # POST /payments
│   └── step_get_payment_status.yaml # GET /payments/{id}
├── models/
│   ├── create_payment_input.yaml
│   └── get_payment_status_input.yaml
├── templates/
│   ├── create_payment.njk
│   └── get_payment_status.njk
└── data.yaml                   # example test data per operation

Generator Workflow Scenarios

Scenario 1: Fresh Generation

syz generate plugin --name payments --from openapi:./api.yaml

The generator creates all files. Your custom modifications are safe — only the autogenerated agent/template files will be regenerated later if needed.

Scenario 2: API Updated — Regenerate Everything

syz generate plugin --name payments --from openapi:./api.yaml --force

Scenario 3: API Updated — Preserve Custom Work

syz generate plugin --name payments --from openapi:./api.yaml --overwrite

The generator prompts: "Which operations do you want to regenerate?" You select only the new ones. Your customized agents are untouched.

Scenario 4: Preview Before Writing

syz generate plugin --name payments --from openapi:./api.yaml --dry-run

Output shows file paths and operations found, but creates nothing.

05 Part 3: Author — Write Your Test Scenarios

Compose agents into test flows

What Is "Author"?

Once your plugin is generated, you write test scenarios — YAML files that describe a sequence of steps: send a request, validate the response, send another request, and so on. A scenario mirrors a real workflow in your system.

Understanding Step Types

Type 1: Send Steps

A send step makes a request to your API or system:

- type: send
  agent: payments.step_send_order
  inputs:
    amount: 5000
    currency: "GBP"
    customer_id: "cust_alice"

Key Points:

• The agent name (payments.step_send_order) maps to a YAML file in .syz/plugins/payments/agents/
• inputs are the concrete values passed to the request template
• The response is automatically captured in the ledger
• A send step never stops the run on its own — it just captures the response (success or error)

Referencing Previous Responses:

Use {{ ledger.<plugin>.<agent>.<field> }} to reference previous step outputs:

# Create a payment first
- type: send
  agent: payments.step_send_order
  inputs:
    amount: 5000

# Then get its status using the ID from the create response
- type: send
  agent: payments.step_get_payment_status
  inputs:
    id: "{{ ledger.payments.step_send_order.outputs.payment_id }}"

Type 2: Validate Steps

A validate step runs assertions on a previous response:

- type: validate
  agent: payments.validate_payment_created
  inputs:
    response: "{{ ledger.payments.step_send_order.response }}"
    expected_status: 201

Key Points:

• Assertions check the response against expectations
• If an assertion fails, the scenario stops (unless --continue-scenario-on-failure is set)
• The validate agent defines the checks in its YAML

Example Validate Agent:

name: payments.validate_payment_created
type: validate
description: "Validates a successful payment creation"

input_model: models/validate_payment_created_input.yaml
checks:
  - type: http
    description: "Response status is 201"
    path: "$.metadata.http_status"
    operator: equals
    value_from_input: expected_status

  - type: json
    description: "Payment ID is present"
    path: "$.body.data.id"
    operator: exists

  - type: json
    description: "Status is 'pending' or 'processing'"
    path: "$.body.data.status"
    operator: in
    value: ["pending", "processing"]

  - type: json
    description: "Amount matches request"
    path: "$.body.data.amount"
    operator: equals
    value_from_input: expected_amount

Assertion Operators:

This table covers the most common operators for beginners. For the complete list including not_contains, less_than, matches_schema, all, any, and others, see the Quick Reference section at the end of this tutorial.

Anatomy of a Basic Scenario

Scenarios live under .syz/scenarios/. The folder you choose is the level (component/, integration/, e2e/, uat/) — free-form, but those four are the convention.

.syz/scenarios/e2e/checkout.yaml

name: "Checkout — successful order"
level: e2e
environment: local
plugins:
  - name: payments
    version: "1.0.0"

flow:
  - type: send
    agent: payments.step_send_order
    inputs:
      userId:   "u-001"
      amount:   100
      currency: "GBP"

  - type: validate
    agent: payments.validate_order_response
    inputs:
      response:        "{{ ledger.payments.step_send_order.response }}"
      expected_status: 201

• name / level — human-readable metadata; level labels the test phase.
• environment — which environment file to load (local → .syz/environments/local.yaml).
• plugins — the plugins this scenario uses, pinned by version.
• flow — the ordered list of steps. The validate step reads the send step's response straight from the ledger.

Parameterising with an Outline

Add an examples table and the scenario becomes an outline — the same flow runs once per row, each row an isolated execution with its own ledger.

.syz/scenarios/e2e/checkout-parameterized.yaml

name: "Checkout — parameterised amounts"
level: e2e
environment: local
plugins:
  - name: payments
    version: "1.0.0"

flow:
  - type: send
    agent: payments.step_send_order
    inputs:
      userId:   "{{ outline.userId }}"
      amount:   "{{ outline.amount }}"
      currency: "{{ outline.currency }}"

  - type: validate
    agent: payments.validate_order_response
    inputs:
      response:        "{{ ledger.payments.step_send_order.response }}"
      expected_status: "{{ outline.expected_status }}"

examples:
  - name: standard-order
    userId: "u-001"
    amount: 100
    currency: "GBP"
    expected_status: 201

  - name: zero-amount-rejected
    userId: "u-001"
    amount: 0
    currency: "GBP"
    expected_status: 400

  - name: large-usd-order
    userId: "u-001"
    amount: 500000
    currency: "USD"
    expected_status: 201

  - name: invalid-currency
    userId: "u-001"
    amount: 100
    currency: "XXX"
    expected_status: 422

Each row runs the whole flow independently. A failing row doesn't halt the others — you see exactly which inputs pass and which fail in a single run.

Advanced: Context Variables

Use ctx.var to store and reference values across steps:

flow:
  # Capture the payment ID
  - type: send
    agent: payments.step_send_order
    inputs:
      amount: 5000
    set:
      payment_id: "{{ ledger.payments.step_send_order.outputs.payment_id }}"

  # Use the stored ID later
  - type: send
    agent: payments.step_get_payment_status
    inputs:
      id: "{{ ctx.var.payment_id }}"

06 Part 4: Execute — Run Your Tests

Execute scenarios and handle the results

Lint First (Pre-Flight Check)

Before running, pre-flight the scenario. syz lint validates schemas, resolves the environment, confirms every referenced agent exists, and runs a residual scan for any unresolved {{ os-env-var:... }} expressions — all without making any HTTP calls.

syz lint --env local

Additional flags: --plugin <name> (scope to specific plugin), --tag <label> (filter by tags), --example <name> (filter outline rows), --json (structured output).

Run It

syz run executes your scenarios. Required flags: --suite (scenario file or directory) and --env (environment to run against).

syz run --suite .syz/scenarios/e2e/checkout.yaml --env local

Point --suite at a directory to run everything under it, recursively:

# run every scenario in the e2e folder
syz run --suite .syz/scenarios/e2e/ --env local

# run the entire suite, every level
syz run --suite .syz/scenarios/ --env local

Useful Flags

Reading the Terminal Output

Each step streams its phases live, then a one-line verdict. Sensitive fields are masked everywhere:

─── STEP 1 of 2: send — payments.step_send_order ───────────────────

  [Input Resolution]
    url           "{{ env.payments.base_url }}/orders"  →  "http://localhost:3000/orders"
    headers.Auth  "Bearer {{ env.payments.token }}"     →  "Bearer [MASKED]"
    body.amount   100                                      (concrete)

  [Payload Construction]
    Template: payments/templates/create_order.njk
    Rendered: {"userId":"u-001","amount":100,"currency":"GBP"}

  [Executor]
    →  POST http://localhost:3000/orders
    ←  201 Created  145ms
       {"data":{"orderId":"ord_abc123"},"status":"success"}

  [Ledger Write]
    Outputs:   { orderId: "ord_abc123", httpStatus: 201 }
    Duration:  145ms

✓ PASS  145ms

The Results Folder

Every run writes one folder under .syz/results/, named with the execution ID (or your --run-name):

.syz/results/run-2026-06-07T09-12-34-567Z/
├── execution.json    # run-level index — all scenarios + status
├── index.html        # self-contained debug viewer — opens via file://
├── junit.xml         # only with --reporter junit
└── e2e/
    └── checkout/      # mirrors .syz/scenarios/e2e/checkout.yaml
        ├── debug.log  # phase-by-phase trace, plain text
        └── ledger.json  # the audit artefact — every request/response

Handling Failures

By Default (Stop on First Failure):

syz run
# Scenario 1: ✓ pass
# Scenario 2: ✗ fail — run stops here
# Scenario 3: not executed
# Exit code: 1

Continue on Failure:

syz run --continue-scenario-on-failure
# Scenario 1: ✓ pass
# Scenario 2: ✗ fail — continue anyway
# Scenario 3: ✓ pass
# Exit code: 1 (still fails overall)

Use this when you want to see all test results even if some fail, instead of stopping at the first problem.

07 Part 5: Audit & Review — Read Your Results

Understand what happened at every step

What Is "Audit & Review"?

After running tests, you review the results to understand:

• What passed and what failed? (execution summary)
• What was sent and what was received? (request/response capture)
• Which assertions passed and which failed? (validation details)
• How does this trace back to business requirements? (traceability)

Reading execution.json

.syz/results/run-2026-06-07T09-12-34-567Z/execution.json

The run-level summary of all scenarios and their outcomes:

{
  "execution_id": "run-2026-06-07T09-12-34-567Z",
  "timestamp": "2026-06-28T10:30:45.123Z",
  "continue_scenario_on_failure": false,
  "scenarios": [
    {
      "scenario_id": "e2e/payment_flow",
      "name": "Payment Flow — Happy Path",
      "level": "e2e",
      "verdict": "success",
      "timestamp": "2026-06-28T10:30:46.234Z",
      "duration_ms": 1234,
      "steps_total": 4,
      "steps_passed": 4,
      "steps_failed": 0,
      "step_outcomes": ["success", "success", "success", "success"]
    },
    {
      "scenario_id": "e2e/payment_refund",
      "verdict": "failure",
      "steps_total": 5,
      "steps_passed": 4,
      "steps_failed": 1,
      "failure_details": {
        "failed_step": 4,
        "step_name": "payments.validate_refund_applied",
        "reason": "Assertion failed: refund amount mismatch"
      }
    }
  ],
  "summary": {
    "total": 2,
    "passed": 1,
    "failed": 1,
    "error": 0
  }
}

Verdict Types:

• success — All steps passed
• failure — ≥1 validation failed (test logic mismatch)
• error — ≥1 step couldn't run (bad template, wrong syntax)
• exception — External system down (API unreachable, DB timeout)
• aborted — SYZYGY internal issue

Reading index.html

.syz/results/run-2026-06-07T09-12-34-567Z/index.html

The interactive debug viewer — self-contained, opens in a browser without any server. Double-click the file or run:

open .syz/results/run-2026-06-07T09-12-34-567Z/index.html

Sections you'll find:

1. Execution Summary — Top-level pass/fail, duration, scenario count
2. Scenario List — Each scenario with its status, duration, step count
3. Step Details — For each step:
   • Phase 1 — Input Resolution: What values were resolved (e.g., {{ ledger.x }} → 123)
   • Phase 2 — Payload Construction: The rendered request (what was sent)
   • Phase 3 — Executor: The raw response (what was received)
   • Phase 4 — Assertion Engine: Which assertions passed/failed and why
   • Phase 5 — Ledger Write: Masked sensitive fields, output shortcuts resolved
4. Timeline — Duration of each step
5. Issues — Problems found (assertion failures, execution errors)

Reading debug.log

.syz/results/run-2026-06-07T09-12-34-567Z/e2e/checkout/debug.log

A plain-text transcript of the entire scenario execution. Useful for CI logs or quick text search:

[Input Resolution]
  {{ env.payments.base_url }}  →  http://localhost:3000
[Payload Construction]
  Template 'create_order.njk' rendered successfully
[Executor]
  POST http://localhost:3000/orders  →  201 Created  145ms
[Validate]
  http  $.metadata.http_status  equals 201   →  PASS
  json  $.body.data.orderId     exists        →  PASS
  json  $.body.data.amount      greater_than 0 → PASS

Reading ledger.json

.syz/results/run-2026-06-07T09-12-34-567Z/e2e/checkout/ledger.json

The authoritative machine-readable log of the scenario execution. Every request, response, and assertion result is recorded:

{
  "scenario_id": "e2e/payment_flow",
  "timestamp": "2026-06-28T10:30:46.234Z",
  "entries": [
    {
      "agent": "payments.step_send_order",
      "type": "send",
      "request": {
        "method": "POST",
        "url": "https://api.payments.example.com/v1/payments",
        "headers": {
          "Content-Type": "application/json",
          "Authorization": "[MASKED]"
        },
        "body": {
          "amount": 5000,
          "currency": "GBP"
        }
      },
      "response": {
        "body": {
          "data": {
            "id": "pay_abc123xyz",
            "status": "pending",
            "amount": 5000
          }
        },
        "metadata": {
          "http_status": 201
        }
      },
      "outputs": {
        "payment_id": "pay_abc123xyz",
        "payment_status": "pending"
      },
      "timestamp": "2026-06-28T10:30:46.479Z",
      "duration_ms": 245
    }
  ]
}

Debugging Failed Tests

Scenario: Assertion Failed

Step 1: Check the execution summary

cat .syz/results/run-2026-06-07T09-12-34-567Z/execution.json | grep -A5 "verdict"

Step 2: Open index.html in browser, navigate to the failed step, look at "Assertion Engine" section to see which check failed and why

Step 3: Check the debug.log for context

cat .syz/results/run-2026-06-07T09-12-34-567Z/e2e/payment_flow/debug.log

Step 4: Fix and Re-run

Update the scenario or test data based on what the assertion showed, then:

syz run --suite .syz/scenarios/e2e/payment_flow.yaml --env local

08 Running Example: Complete Walkthrough

A full end-to-end example from scratch

Scenario: The Payments API Integration

Goal: Set up a plugin for a Payments API, write test scenarios, run them, and review results.

1. 1

   Initialize the Workspace

   mkdir payments-integration
   cd payments-integration
   syz init

2. 2

   Define Your Domain (DOMAIN.yaml)

   Edit .syz/dossier/DOMAIN.yaml with business rules and test strategy. This captures what you're testing and why.

3. 3

   Create OpenAPI Spec

   Create payments-api.openapi.yaml with your API endpoints (POST /payments, GET /payments/{id}, etc.)

4. 4

   Generate Plugin

   syz generate plugin --name payments --from openapi:./payments-api.openapi.yaml

5. 5

   Set Up Environment

   Create .syz/environments/local.yaml with base URL and credentials:

   name: local
   production: false
   payments:
     base_url: "https://api-staging.payments.example.com/v1"
     api_key: "{{ os-env-var:PAYMENTS_API_KEY }}"

   Export the environment variable:

   export PAYMENTS_API_KEY="sk_live_abc123xyz"

6. 6

   Write Test Scenarios

   Create .syz/scenarios/e2e/payment_happy_path.yaml and payment_parameterized.yaml with your test flows.

7. 7

   Run Tests

   syz run --suite .syz/scenarios/e2e/ --env local

8. 8

   Review Results

   # Interactive viewer
   open .syz/results/run-2026-06-28T10-30-45-123Z/index.html
   
   # Or check plain text log
   cat .syz/results/run-2026-06-28T10-30-45-123Z/e2e/payment_happy_path/debug.log
   
   # Or parse the ledger
   cat .syz/results/run-2026-06-28T10-30-45-123Z/e2e/payment_happy_path/ledger.json | jq

09 Common Workflows & Troubleshooting

How to handle common situations

Workflow 1: "I Found a Bug in My API"

Scenario: A test fails because the API returned an unexpected error.

Diagnosis:

1. Open index.html from the failed run
2. Find the failed step
3. Look at the Executor phase — what was the actual response?
4. Compare against the Input Resolution phase — what were the inputs?
5. Check the Payload Construction phase — was the request correct?

Solution: Usually one of:

• API bug: Report to dev team with ledger.json attached (proves what was sent)
• Test bug: Update scenario inputs or assertions
• Integration bug: Environment/credentials issue

Workflow 2: "I Want to Verify My Tests Are Actually Testing"

Scenario: You want proof that your test would catch a real bug.

Verification (Mutation Testing):

1. Temporarily break the assertion:

checks:
  - type: json
    path: "$.body.data.id"
    operator: equals
    value: "wrong-id-on-purpose"

2. Run: syz run --suite your-scenario.yaml --env local
3. Verify it fails (if it passes, your test isn't actually checking that field)
4. Revert the change

Workflow 3: "I Need to Test Against Multiple Environments"

Scenario: Same tests, different servers (local, staging, production).

Solution:

# Local
syz run --suite .syz/scenarios/ --env local

# Staging
syz run --suite .syz/scenarios/ --env staging

# Production (read-only, no destructive tests)
syz run --suite .syz/scenarios/e2e/ --env production

Each environment has its own .syz/environments/<name>.yaml with different base URLs and credentials.

Workflow 4: "My Test Passes Locally but Fails in CI"

Scenario: Flaky or environment-dependent test.

Diagnosis:

1. Get the CI run results: check the ledger.json from CI
2. Compare CI ledger vs local ledger
3. Look for differences in:
   • Response timestamps
   • Base URLs (is CI using the right environment?)
   • Credentials (are env vars set?)
   • Rate limiting (did CI hit a rate limit?)

Common Causes:

• Credentials not exported in CI
• Wrong environment flag missing
• Data cleanup between runs (CI doesn't clean up, next run conflicts)
• Timing issues (requests are too slow or fast)

Workflow 5: "I Need to Mask Sensitive Data"

Scenario: Logs contain API keys that shouldn't be visible.

Solution: Mark sensitive fields in the agent:

name: payments.step_send_order
type: send
sensitive_fields:
  - request.headers.Authorization
  - request.body.api_key
  - response.body.data.access_token

SYZYGY automatically masks these in:

• debug.log (shows [MASKED])
• index.html (shows [MASKED])
• ledger.json (shows [MASKED])
• Terminal output

The actual value is preserved internally and forwarded to later steps, but hidden from logs.

Workflow 6: "I Need to Generate a Report for Management"

Scenario: Convert test results to a format for stakeholders.

Solution:

# Generate JUnit XML for CI tools (Jenkins, GitLab, GitHub)
syz run --suite .syz/scenarios/ --env local --reporter junit --output ./results.xml

# Open the HTML report for visual walkthrough
open .syz/results/run-2026-06-28T10-30-45-123Z/index.html

10 Quick Reference & Glossary

Cheat sheet and terminology

Command Reference

File Locations Cheat Sheet

.syz/
├── dossier/
│   ├── DOMAIN.yaml                 ← Business rules (you write this)
│   └── TEST-STRATEGY.md            ← Test approach (you write this)
├── plugins/
│   └── payments/
│       ├── plugin.yaml
│       ├── agents/                 ← send, validate, craft, workflow agents
│       ├── models/                 ← input field contracts
│       ├── templates/              ← Nunjucks request templates
│       └── data.yaml               ← static test data
├── environments/
│   ├── local.yaml                  ← Local env config (you write this)
│   ├── staging.yaml                ← Staging env config
│   └── production.yaml             ← Production env config
├── scenarios/
│   ├── e2e/                        ← Test scenarios (you write these)
│   ├── integration/
│   ├── component/
│   └── uat/
└── results/
    └── run-2026-06-28T10-30-45-123Z/
        ├── execution.json          ← Execution summary
        ├── index.html              ← Interactive viewer
        └── e2e/checkout/
            ├── debug.log           ← Plain-text trace
            └── ledger.json         ← Full request/response log

Glossary

Reference: Expression Syntax

One expression syntax — {{ ... }} — across scenarios, agent inputs, and templates:

Reference: Assertion Operators

Complete list of operators available in validate agent checks:

Next Steps

Learn more:

• Read .syz/dossier/SYZ-TESTING.md for scenario authoring best practices
• Read .syz/dossier/SYZ-GUIDELINES.md for plugin design patterns
• Read ARCHITECTURE.md for technical internals and advanced features
• Set up CI/CD to run tests on every commit