MCP / agent infra
Hardening MCP servers for production
MCP demos work on day one. They break on day 30. These are the field notes from running them for real.
the problem
Most Model Context Protocol servers are written for a demo: one caller, one happy path, no untrusted input, no cost ceiling.
In production the failures cluster in three places: concurrency, cost, and content you did not write. None of them show up on day one.
architecture
caller (an LLM)
|
v
MCP server --- tools ---> external work
|
v
SQLite (WAL + busy_timeout)A per-item character cap with a [truncated] marker cut input cost materially on a real workload. If the model has to ignore most of the input, you are paying for nothing.
Set on every connection, not just the first. The bug bites on day 30.
def connect(path: str) -> sqlite3.Connection:
conn = sqlite3.connect(path)
# WAL lets readers and one writer coexist
conn.execute("PRAGMA journal_mode = WAL")
# without this, overlapping writers throw 'database is locked'
conn.execute("PRAGMA busy_timeout = 5000")
return conn
def enrich(urls: list[str], cap: int = 25) -> list[str]:
# clamp at the entry, before the first append, or cap=0 still does one
urls = urls[: max(0, min(cap, 25))]
return [fetch(u) for u in urls]what is still broken
- ▸Cost and retry thresholds are heuristics, not yet tuned against a long window of real traffic.
- ▸No central telemetry across servers. Each one logs locally; there is no single pane.