openreview-mcp: A Model Context Protocol Server for Querying Peer Review at Scale

Abstract

The Model Context Protocol (MCP) ecosystem now covers arXiv, Semantic Scholar, Hugging Face, and Crossref, but not peer review: the reasoning of expert reviewers, hosted publicly on OpenReview, is unreachable by any MCP-connected LLM. This report introduces openreview-mcp, an open-source MCP server that closes the gap. It exposes eleven tools across four entity families (venues, submissions, reviews, profiles) plus a signature analysis tool, openreview_aggregate_weaknesses, which clusters recurrent reviewer complaints across a venue's rejections and returns raw evidence (top TF-IDF terms, exemplar quotes, contributing submission identifiers) for the consuming agent to label. We document the system architecture, the analysis pipeline (TF-IDF, Truncated SVD, KMeans), and a reproducible case study on 100 rejected ICLR 2024 submissions. The case study yields 1,361 individual weakness statements across 14 coherent clusters and surfaces a non-obvious finding: evaluation concerns drive 42% of reviewer complaints, while generic novelty complaints account for less than 8%, contradicting the folklore that novelty is the primary battleground in machine-learning peer review. Source code: https://github.com/OpenCodice-Research/openreview-mcp

Keywords