Conkurrence

Created By

AlligatorC0der3 months ago

Conkurrence measures whether multiple AI models produce consistent outputs on your evaluation tasks. It tells you which items your AI agrees on and which need human review — using Fleiss' κ, Kendall's W, and bootstrap confidence intervals, the same psychometric methods trusted in clinical research.

# ai-evaluation

# golden-dataset

Overview Content Tools Comments

Overview

ConKurrence

One command. Find out if your AI agrees with itself.

ConKurrence is a statistically validated consensus measurement toolkit for AI evaluation pipelines. It uses multiple AI models as independent raters, measures inter-rater reliability with Fleiss' kappa and bootstrap confidence intervals, and routes contested items to human experts.

Install

npm install -g conkurrence

MCP Server

Use ConKurrence as an MCP server in Claude Desktop or any MCP-compatible client:

npx conkurrence mcp

Claude Desktop Configuration

Add to your claude_desktop_config.json:

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": ["-y", "conkurrence", "mcp"]
    }
  }
}

Claude Code Plugin

/plugin marketplace add AlligatorC0der/conkurrence

Features

Multi-model evaluation — Run your schema against Bedrock, OpenAI, and Gemini models simultaneously
Statistical rigor — Fleiss' kappa with bootstrap confidence intervals, Kendall's W for validity
Self-consistency mode — No API keys needed; uses the host model via MCP Sampling
Schema suggestion — AI-powered schema design from your data
Trend tracking — Compare runs over time, detect agreement degradation
Cost estimation — Know the cost before running

MCP Tools

Tool	Description
`conkurrence_run`	Execute an evaluation across multiple AI raters
`conkurrence_report`	Generate a detailed markdown report
`conkurrence_compare`	Side-by-side comparison of two runs
`conkurrence_trend`	Track agreement over multiple runs
`conkurrence_suggest`	AI-powered schema suggestion from your data
`conkurrence_validate_schema`	Validate a schema before running
`conkurrence_estimate`	Estimate cost and token usage

Links

Homepage: conkurrence.com
npm: npmjs.com/package/conkurrence
Terms of Service: app.conkurrence.com/terms
Privacy Policy: app.conkurrence.com/privacy

License

BUSL-1.1 — Business Source License 1.1

Try in Playground

Server Config

{
  "mcpServers": {
    "conkurrence": {
      "command": "npx",
      "args": [
        "-y",
        "conkurrence",
        "mcp"
      ]
    }
  }
}

Project Info

Created At

3 months ago

Updated At

3 months ago

Author Name

AlligatorC0der

Star

-

Language

-

License

-

Category

Tags

Homepage

https://github.com/AlligatorC0der/conkurrence

Recommend Servers

Belgium Payments Mcp

@junter1989k-ai

12 days ago

Uruguay Payments Mcp

@junter1989k-ai

11 days ago

Switzerland Payments Mcp

@junter1989k-ai

12 days ago

Uganda Payments Mcp

@junter1989k-ai

11 days ago

Cyprus Payments Mcp

@junter1989k-ai

11 days ago

Serper MCP Server

A Serper MCP Server

Python

a year ago

Realdentalcosts Mcp

Query verified 2026 US dental cost data from Real Dental Costs: average implant, veneer, braces and procedure prices for all 50 states and 206 cities, state-vs-state comparisons, and the composite US Dental Cost Index. Open data (CC BY 4.0, Zenodo DOIs). Every response includes the source and a reference URL. Pricing/market research, not medical advice. Data: https://realdentalcosts.com

11 days ago

Ghana Payments Mcp

@junter1989k-ai

12 days ago

Portugal Payments Mcp

@junter1989k-ai

12 days ago

Denmark Payments Mcp

@junter1989k-ai

12 days ago

Romania Payments Mcp

@junter1989k-ai

12 days ago

Luxembourg Payments Mcp

@junter1989k-ai

11 days ago

基于Anduin2017 / HowToCook （程序员在家做饭指南）的mcp server，帮你推荐菜谱、规划膳食，解决“今天吃什么“的世纪难题； Based on Anduin2017/HowToCook (Programmer's Guide to Cooking at Home), MCP Server helps you recommend recipes, plan meals, and solve the century old problem of "what to eat today"

a year ago

@Henning Witzel-Acikgöz

Host the HTML or Markdown pages your AI generates and share each as a link with comments, versioning, and access control. Create, update, and organize pages and read reviewer comments over MCP.

11 days ago

@modelcontextprotocol

Retrieving and analyzing issues from Sentry.io

a year ago

mcp-server-flomo MCP Server

Write notes to Flomo

JavaScript

a year ago

Neon MCP Server

@neondatabase-labs

MCP server for interacting with Neon Management API and databases

TypeScript

a year ago

Secure file operations with configurable access controls

a year ago

One API for Search, Crawling, and Sitemaps

a year ago

Guatemala Payments Mcp

@junter1989k-ai

11 days ago

Bulgaria Payments Mcp

@junter1989k-ai

11 days ago

Arcgis Portal Mcp

MCP server for ArcGIS Portal and ArcGIS Online. Lets AI assistants search content, query feature layers, manage features, handle content operations, and administer users and groups. Built on the Model Context Protocol for integration with Claude Desktop, Cursor, VS Code Copilot, and other MCP clients. Disclaimer: This is an independent open-source project. It is not affiliated with, endorsed by, or sponsored by Esri Inc. "ArcGIS" is a registered trademark of Esri.

11 days ago

Kuwait Payments Mcp

@junter1989k-ai

11 days ago

@modelcontextprotocol

AI image generation using various models

a year ago

Kenya Payments Mcp

@junter1989k-ai

12 days ago

Web content fetching and conversion for efficient LLM usage

9 months ago

302_sandbox_mcp

Create a remote sandbox that can execute code/run commands/upload and download files. 创建远程沙盒，可以执行代码/运行命令/上传下载文件

a year ago

Lithuania Payments Mcp

@junter1989k-ai

11 days ago

Perplexity Ask MCP Server

A Model Context Protocol Server connector for Perplexity API, to enable web search without leaving the MCP ecosystem.

JavaScript

a year ago

Canada Payments Mcp

@junter1989k-ai

12 days ago