Configuration Schema

servers[ ]

• id (string, required) — unique identifier used in scenario server lists.
• transport (string, required) — connection type. Use "http" for HTTP/SSE.
• url (string, required) — full URL of the MCP endpoint.
• token (string, optional) — bearer token for authentication. Use a literal string or $ENV_VAR to read from the environment.

agents[ ]

• id (string, required) — unique identifier referenced by scenarios.
• provider (string, required) — LLM provider. One of: anthropic, openai, azure.
• model (string, required) — model identifier as used by the provider API.
• temperature (number, optional) — sampling temperature. Defaults to 0.

scenarios[ ]

• Inline scenario: id, prompt, optional name, optional mcp_servers, optional eval and extract.
• Referenced scenario: ref with optional mcp_servers override.
• mcp_servers entries can be { ref: <server-id> } or inline server objects.
• Legacy top-level servers pool is deprecated; prefer scenario-owned mcp_servers.

eval.tool_constraints, eval.tool_sequence and eval.response_assertions

• tool_constraints.required_tools (string[], optional) — tool names the agent must call.
• tool_constraints.forbidden_tools (string[], optional) — tool names the agent must not call.
• tool_sequence (string[], optional) — ordered tool names that must appear in the run in that order, with other tools allowed in between.
• response_assertions supports: contains, not_contains, starts_with, ends_with, equals, regex, jsonpath, jsonpath_exists, jsonpath_not_exists.
• For detailed examples of each assertion, see Reference / Tool and Response Assertions.

Reference Syntax

Use ref inside agents and scenarios list items to pull from library ids.

agents:
  - ref: claude-sonnet-46

scenarios:
  - ref: add-calculations
    mcp_servers:
      - ref: kpi-api-prod