Libraries & Refs

What the Library System Is

MCPLab libraries let you define shared servers, agents, and scenarios once, then reference them from many eval configs.

This reduces duplication and keeps team-wide defaults consistent.

Recommended Library Structure

• servers.yaml contains reusable MCP server definitions.
• agents.yaml contains reusable LLM agent definitions.
• scenarios/ contains reusable scenario files.

mcplab/
├── servers.yaml
├── agents.yaml
└── scenarios/
    ├── scenario-a.yaml
    └── scenario-b.yaml

Suite Labels from Folder Paths

Eval configs placed in subfolders automatically derive a suite label from their folder path. The App groups and displays configs by suite in the Configurations page, making it easy to run or browse related configs together.

The suite label is the relative folder path from the evals root. A file at evals/search/basic.yaml gets the suite label "search".

evals/
├── search/
│   ├── basic.yaml       → suite: "search"
│   └── advanced.yaml    → suite: "search"
└── auth/
    └── login.yaml       → suite: "auth"

Reference Library Items in eval.yaml

Use ref entries to pull shared items into an eval config by id.

For referenced scenarios, you can override MCP targets with mcp_servers while keeping prompt/eval logic in the test-case.

agents:
  - ref: claude-sonnet

scenarios:
  - ref: add-calculations
    mcp_servers:
      - ref: kpi-api-stage

How Ref Resolution Works

• Server refs resolve from servers.yaml.
• Agent refs resolve from agents.yaml.
• Scenario refs resolve from files in test-cases/.
• When a referenced scenario includes mcp_servers overlay, that overlay is used for runtime MCP binding.
• Missing refs are reported and should be fixed before running.

Use Libraries in App and CLI

• App: start with --libraries-dir so shared library content is loaded.
• CLI run: refs resolve when library files are present in the configured layout.
• App UI: inspect and manage shared items in the Library section.

npx @inspectr/mcplab app --libraries-dir ./mcplab