Conkurrence

Created By

AlligatorC0der2 months ago

Conkurrence measures whether multiple AI models produce consistent outputs on your evaluation tasks. It tells you which items your AI agrees on and which need human review — using Fleiss' κ, Kendall's W, and bootstrap confidence intervals, the same psychometric methods trusted in clinical research.

# ai-evaluation

# golden-dataset

Overview Content Tools Comments

Overview

ConKurrence

One command. Find out if your AI agrees with itself.

ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.

Install

npm install -g conkurrence

MCP Server

Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:

npx conkurrence mcp

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": ["-y", "conkurrence", "mcp"]
    }
  }
}

Claude Code Plugin

/plugin marketplace add AlligatorC0der/conkurrence

Features

Multi-model evaluation — Run your schema against Bedrock, OpenAI, and Gemini models simultaneously
Statistical rigor — Fleiss' kappa with bootstrap confidence intervals, Kendall's W for validity
Self-consistency mode — No API keys needed; uses the host model via MCP Sampling
Schema suggestion — AI-powered schema design from your data
Trend tracking — Compare runs over time, detect agreement degradation
Cost estimation — Know the cost before running

MCP Tools

Tool	Description
`conkurrence_run`	Execute an evaluation across multiple AI raters
`conkurrence_report`	Generate a detailed markdown report
`conkurrence_compare`	Side-by-side comparison of two runs
`conkurrence_trend`	Track agreement over multiple runs
`conkurrence_suggest`	AI-powered schema suggestion from your data
`conkurrence_validate_schema`	Validate a schema before running
`conkurrence_estimate`	Estimate cost and token usage

License

BUSL-1.1 — Business Source License 1.1

Try in Playground

Server Config

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": [
        "-y",
        "conkurrence",
        "mcp"
      ]
    }
  }
}

Project Info

Created At

2 months ago

Updated At

2 months ago

Author Name

AlligatorC0der

Star

Language

License

Recommend Servers

View All

Zhipu Web Search

@BigModel

Zhipu Web Search MCP Server is a search engine specifically designed for large models. It integrates four search engines, allowing users to flexibly compare and switch between them. Building upon the web crawling and ranking capabilities of traditional search engines, it enhances intent recognition capabilities, returning results more suitable for large model processing (such as webpage titles, URLs, summaries, site names, site icons, etc.). This helps AI applications achieve "dynamic knowledge acquisition" and "precise scenario adaptation" capabilities.

a year ago

MiniMax MCP

@MiniMax-AI

Official MiniMax Model Context Protocol (MCP) server that enables interaction with powerful Text to Speech, image generation and video generation APIs.

Python

a year ago

MostaORM MCP — generate schemas, lint & migrate from your AI tool

@apolocine

MCP server for @mostajs/orm: generate type-safe EntitySchemas, lint them (24 rules) and produce SQL migrations — for one ORM API across 13 databases.

2 days ago

Calypso Multimodal RAG

@Calypso

MCP server for Calypso’s multimodal RAG layer. Supports agent-store uploads, knowledge-store ingestion, and retrieval-backed responses workflows with file-aware rag_policy semantics.

a day ago

Gosce Structured Output Agent (fastmcp + Instructor)

GOSCE-generated MCP agent combining FastMCP and Instructor for structured LLM output. Provides agent protocol and structured output capabilities. Part of the VDA/GOSCE portfolio at getvda.ai. A2A-compliant, governed, scale-to-zero on Cloud Run.

21 hours ago

Firecrawl Mcp Server

@mendableai

Official Firecrawl MCP Server - Adds powerful web scraping to Cursor, Claude and any other LLM clients.

JavaScript

a year ago

Dc Hub — Data Center Intelligence Mcp Server

DC Hub is the neutral, real-time data layer for data-center infrastructure — built so AI agents can both query it (MCP + REST) and cite it (every full-data response carries Source: DC Hub, CC-BY-4.0). Coverage: Facilities — 21,000+ data centers worldwide: search, profiles, scoring, alternatives Markets — 232 markets with the DCPI (Data Center Power Index): BUILD/CAUTION/AVOID Gas — the DCGI (Data Center Gas Index): per-state natural-gas suitability for siting Grid / ISO — live fuel mix, carbon intensity, demand, prices & interconnection-queue depth across 10 North-American grid operators; one-call all-ISO scoreboard Capital — $324B+ M&A history + hyperscaler capex tracker + AI Compute Capacity Index Site factors — fiber routes, water-stress, tax incentives, nearby substations/transmission Why agents pick it: the only DC-intelligence source an LLM can query live AND cite — the MCP-native alternative to static PDF research. No NDAs, no quarterly reports, just live JSON.

2 hours ago

Gosce Authenticated Multi Llm Agent (anthropic + Google Auth + Mcp + Openai)

21 hours ago

Agentdevx Gateway

5 minutes ago

Dealboard

@Dealboard

Track every deal without the spreadsheet mess or CRM overhead. Dealboard gives salespeople, BD teams, founders, and small teams one clear sales pipeline with deal cards, notes, goals, files, reporting, and AI-ready MCP tools.

2 days ago

a day ago

AI-integrated tools for learning, writing, and publishing. ProvenanceAI builds free MCP tools that help authors develop and protect their voice, editors assess manuscript quality, instructors design humanities courses, and researchers produce defensible, well-sourced work. Current tools cover editorial analysis: voice profiling, authenticity scoring, structural review, style consistency, research grading, and manuscript readiness. Expanding into curriculum design, academic writing, and book production workflows. No email gate. Every tool returns a full structured result on the first call. Powered by ProvenanceAI × Exact Rush.

9 hours ago

cityparity: Cost-of-Living & Quality-of-Life Comparison

@Skyler Bissell

cityparity gives AI agents real cost-of-living and quality-of-life comparison tools. Point it at two cities and it returns take-home pay, a full cost breakdown (housing, healthcare, childcare, food, transit, travel home, property tax), and the equivalent salary you'd need in the target city to come out even. It also surfaces the non-cash deltas people actually move for: vacation days, parental leave, universal healthcare. Beyond two-city compares, it ranks cities by a composite quality score and looks up inbound-worker tax regimes (Italy impatriati, Portugal IFICI, and a few others). Over 165 cities across 69 countries. Free, no API key, hosted Streamable HTTP.

10 hours ago

Online Cyber Tools Mcp

@Jambozx

Exposes 280+ free dev/security tools from onlinecybertools.com as MCP tools in one server: Base64/URL/JWT encoders, MD5/SHA/HMAC/bcrypt/argon2 hashes, JSON/YAML/XML formatters, regex tester, network diagnostics (ping/traceroute/dig/whois/SSL/SPF/DMARC), and OSINT lookups. No auth, no API key.

13 hours ago

Mailtrap Email Sending MCP

@Mailtrap

An MCP server that provides a tool for sending transactional emails via Mailtrap

a year ago

Tavily Mcp

@tavily-ai

JavaScript

a year ago

Brave Search

@modelcontextprotocol

Web and local search using Brave's Search API

a year ago

Pdf Content Search Mcp

@juergenkoller-software

2 days ago

REintel Ontario Real Estate — AI-Powered Market Intelligence

@Jeff Chisholm — Chisholm Real Estate Team

Ontario real estate MCP server with 22 tools — listings search, CMA valuations, paired-sales adjustments, market snapshots, mortgage calculators, neighbourhood profiles, price trends, and investment analysis. Covers all of Ontario with live MLS data.

2 days ago

Proton Mail MCP

@sethbang

Unofficial MCP server for Proton Mail (not affiliated with Proton AG) — send, read, search & organize email over SMTP/IMAP

a day ago

Gelbooru

14 hours ago

Neon MCP Server

@neondatabase-labs

MCP server for interacting with Neon Management API and databases

TypeScript

a year ago

Perplexity Ask MCP Server

@ppl-ai

A Model Context Protocol Server connector for Perplexity API, to enable web search without leaving the MCP ecosystem.

JavaScript

a year ago

302_sandbox_mcp

@302ai

Create a remote sandbox that can execute code/run commands/upload and download files. 创建远程沙盒，可以执行代码/运行命令/上传下载文件

a year ago

Unfetch

@guangtoutong

unfetch — a download manager for humans and AI with a native MCP server: add_task, list_tasks, wait_for_task with progress notifications from any MCP host. Multi-threaded multi-mirror HTTP, BT/magnet, 1000+ video sites (yt-dlp), RSS, token-secured remote Web UI. ~8MB, open source, no ads, no telemetry.

a day ago

Github

@modelcontextprotocol

Repository management, file operations, and GitHub API integration

a year ago

Coderplan Api

@CoderPlan

Unified LLM API gateway with OpenAI-compatible protocol. Access Claude, GPT, Gemini, DeepSeek through a single endpoint. Pay-per-use at ~70% off official pricing. Supports Alipay/WeChat. Hong Kong/Singapore nodes.

15 hours ago

Puppeteer

@modelcontextprotocol

Browser automation and web scraping

a year ago

MCP Advisor

@istarwyh

MCP Advisor & Installation - Use the right MCP server for your needs

TypeScript

a year ago

GitLab

@modelcontextprotocol