EvalGate

Measured conclusion: Deep-tested. L2 outcome: tools_ok.

Create and run AI evaluations, record traces, and inspect results with scoped organization access.

Add to your agent

Copy this into your MCP client config:

{
  "mcpServers": {
    "EvalGate": {
      "url": "https://www.evalgate.com/api/mcp",
      "transport": "streamable-http"
    }
  }
}

Measured tools (12)

• project.inspect read-only

  Inspect the evaluations and project keys available in the authenticated organization.

  Parameters:

  • projectKey — string — Exact project key to filter the first 50 organization evaluations; omit to inspect that entire snapshot.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• project.plan read-only

  Return a non-mutating next-step plan for connecting a repository, evaluation, or trace source.

  Parameters:

  • target — string — Workflow to plan: local repository setup, hosted evaluation review, or trace ingestion. Defaults to repository; never executes the plan.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• evaluations.list read-only

  List evaluations with a stable cursor for the organization.

  Parameters:

  • limit — integer — Maximum results in this page, from 1 to 50; defaults to 20.
  • cursor — string — Opaque nextCursor from the preceding page of this tool. Omit for the first page.
  • status — string — Restrict evaluations to this lifecycle status; omit to include all statuses.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• runs.list read-only

  List evaluation runs with stable cursors and regression signals.

  Parameters:

  • evaluationId (required) — Evaluation ID returned by evaluations.list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").
  • limit — integer — Maximum results in this page, from 1 to 50; defaults to 20.
  • cursor — string — Opaque nextCursor from the preceding page of this tool. Omit for the first page.
  • status — string — Exact persisted run status to filter by; omit to include all statuses, including incomplete runs.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• run.explain read-only

  Explain a run using its status, failed-case metadata, and evidence-lineage summary.

  Parameters:

  • runId (required) — Run ID returned by runs.list; must belong to the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 3 or "3").

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• baseline.lineage read-only

  Show the published baseline anchor and its evidence lineage.

  Parameters:

  • evaluationId (required) — Evaluation ID returned by evaluations.list in the authenticated organization. Accepts a positive safe integer or its canonical decimal string (e.g. 8 or "8").

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• docs.search read-only

  Search the published OpenAPI operation catalog for relevant help.

  Parameters:

  • query (required) — string — Search terms for the public OpenAPI operation catalog (1–200 characters); all whitespace-separated terms must match.
  • limit — integer — Maximum matching operations, from 1 to 20; defaults to 10.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• connector.health read-only

  Show the latest connector-readiness reports for the organization.

  Parameters:

  • connectorKey — string — Exact report key or metadata connector key to filter the latest 50 persisted reports; omit for all reports in that snapshot. Requires admin:org.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• connections.list read-only

  List current workspace connection metadata, with source restrictions, freshness and bounded continuation. Does not reveal secrets or grant source access.

  Parameters:

  • q — string — Case-insensitive connection name or service search; up to 200 characters.
  • limit — integer — Maximum connections in one page; defaults to 25, capped at 100.
  • cursor — string — Opaque nextCursor from this caller and inventory revision; refresh if the inventory changes.
  • source — string — Filter by canonical store; direct metadata still requires admin role and scope.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• connections.catalog read-only

  Inspect all connector definitions and exact installed setup paths. Unavailable blueprints and live verification are separate facts.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• connections.roots read-only

  Page independent configured account roots using current administrator authority. Does not list secrets or imply verified connector availability.

  Parameters:

  • q — string — Literal case-insensitive root name/driver search; wildcard characters have no special meaning.
  • limit — integer — Bounded page size; defaults to 25 and is capped at 100.
  • cursor — string — Opaque nextCursor for the same caller, workspace, search, limit and observed metadata revision; refresh after changes.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

• connections.graph read-only

  Read canonical connection references and currently visible exact resource revisions. No source access grant or execution.

  Parameters:

  • id (required) — string — Stable graph UUID returned by connections.backfill, not a credential ID.

  Hints: read-only yes · destructive no · idempotent yes · open-world no

Publisher website

Measured by MCPYP probing. Untested fields are omitted, never estimated. Listing is always free; MCPYP never proxies calls or stores user tokens.