evalpair

io.github.Srikarmk/evalpair

Run LLM eval suites on local or hosted models; compare runs with intervals and a noise floor.

View repository →

Connect this server

Add this to your MCP client config — claude_desktop_config.json (Claude Desktop) or .cursor/mcp.json (Cursor).Suits: Claude Desktop · Cursor.

{
  "mcpServers": {
    "evalpair": {
      "command": "uvx",
      "args": [
        "evalpair"
      ],
      "env": {
        "EVALPAIR_API_KEY": "<your EVALPAIR_API_KEY>"
      }
    }
  }
}
Environment variables
  • EVALPAIR_API_KEYsecretBearer token for an OpenAI-compatible server, if it needs one (OPENAI_API_KEY also works).

A starting point — check the repository for exact arguments and setup. Restart your client after editing the config.

Category
AI & ML
Version
0.1.1
Registry
active
Published
11/5/2026

More AI & ML servers

Browse all AI & ML →

Hive Compute

srotzin

HiveCompute MCP Server — decentralized inference router for AI agents

AI & MLAdded 6 months ago

Hive Insurance Broker

srotzin

Insurance brokerage for AI agents — quote, bind, and settle in USDC

AI & MLAdded 6 months ago

io.github.stephenballot-ai/legacy-shield

stephenballot-ai

Zero-knowledge encrypted vault for AI agents. EU-hosted persistent storage.

AI & MLAdded 7 months ago

io.github.sudo-ai-git/mcp-benchmark-hygiene

sudo-ai-git

Detect pytest config-leakage that corrupts agent-benchmark grading. Deterministic, no-LLM.

AI & MLAdded 2 months ago

io.github.sudo-ai-git/mcp-secret-scrub

sudo-ai-git

Deterministic no-LLM MCP server that scrubs secrets before they reach an agent. Never leaks values.

AI & MLAdded 2 months ago

io.github.sudo-ai-git/mcp-skill-sec

sudo-ai-git

Audit agent skills/prompts against 8 supply-chain attack patterns. Deterministic, no-LLM scanner.

AI & MLAdded 2 months ago

Add the badge to your site

Show visitors that evalpair is listed here — it’s also the link we check for a free listing.

evalpair — listed on Serverdex

HTML — for your website

<a href="https://serverdex.io/server/io-github-srikarmk-evalpair?utm_source=badge"><img src="https://serverdex.io/brand/badge.svg" alt="evalpair — listed on Serverdex" width="123" height="40" loading="lazy"></a>

Recommended. A dofollow link from your own domain is the one that counts for search engines, and it's the link we check for a free listing.

Markdown — for your README

[![evalpair — listed on Serverdex](https://serverdex.io/brand/badge.svg)](https://serverdex.io/server/io-github-srikarmk-evalpair?utm_source=badge)

Nice for visibility, but GitHub marks README links nofollow, so it doesn't pass authority. Use the HTML badge on your site as well.