# mizan-ledger — a local calibration ledger for agents

> Calibration ledger for AI agents: commit confidence before a task, score it against real outcomes (Brier, ECE). Offline, hash-only, with a pytest plugin.

- Package: https://pypi.org/project/mizan-ledger/
- Source: https://pypi.org/project/mizan-ledger/#files
- Human page: https://qants-site.qants.workers.dev/library
- Publisher: QANTS (https://qants-site.qants.workers.dev)
- Licence: MIT
- Status: v0.1.3, early alpha. Python 3.9 or later. The core has no required dependencies, no network access, and no telemetry.
- Optional local MCP server (stdio, offline, Python 3.10 or later): `pip install 'mizan-ledger[mcp]'`, then run `mizan-ledger-mcp`. Not yet listed in the official MCP Registry.

mizan-ledger scores an agent's stated confidence against real outcomes. Before the task, you declare a probability. The library commits a salted SHA-256 hash of that declaration. After the outcome, it scores the ledger with Brier, ECE and log loss (and also accuracy and coverage of declared, resolved and pending records).

This library does **not** verify that a claimed success is true. It scores the number you stated against the outcome you (or pytest) supply.

## Install

```bash
pip install mizan-ledger
```

## Quickstart in pytest

The cheapest way to try it is inside pytest. Mark only the tests that should be scored. The declaration is committed after fixture setup, immediately before the test body:

```python
# test_demo.py
import pytest

@pytest.mark.confidence(0.9)
def test_demo():
    assert True
```

```bash
# pip install mizan-ledger
pytest test_demo.py --mizan-ledger-dir .mizan --mizan-agent-id my-agent
```

Keep `.mizan/private/` out of version control. Only tests with the marker are recorded. If you omit `--mizan-agent-id`, the default agent id is `pytest-agent`.

## Python Ledger API

The README API remains valid for workflows outside pytest, with the honesty limit below on self-supplied outcomes:

```python
from mizan_ledger import Ledger

led = Ledger("ledger/", private_path="private/")
led.declare("fix-bug-421", 0.80, agent_id="my-agent", version="1.2.0")
# ... run the task, run the tests ...
led.resolve("fix-bug-421", outcome=True)
print(led.summary("my-agent"))
```

```bash
mizan-ledger verify ledger/
```

## Honest limits

- The library cannot tell whether an outcome you supply yourself is honest. Outside the pytest path, use an outcome source the agent does not control. The pytest plugin uses the test result (pass or fail).
- Timestamps come from the local clock. They prove order only if the chain head is published somewhere independent. Without that, a ledger owner could rebuild a chain after the fact.
- Small samples give wide intervals. ECE is biased upward for small n. These scores measure calibration, not whether an agent is good, safe or secure.

## Demonstration seeds

QANTS runs three demonstration seed agents, labelled QANTS-operated and never counted as external. They are simple rule-based programs, not language-model agents, and they are demonstrations, not endorsements.

Figures published on https://qants-site.qants.workers.dev/seed-agents (checked 17 Nov 2026, 07:44 GMT+2):

- qants-seed-codetask v0.1.0: 60 declared, 60 resolved. Brier 0.173, 95% interval [0.118, 0.231]. ECE 0.069. Best constant guess (hindsight) 0.170.
- qants-seed-apifetch v0.1.0: 50 declared, 50 resolved. Brier 0.084, 95% interval [0.061, 0.111]. ECE 0.209. Best constant guess 0.160.
- qants-seed-countrypairs v0.1.0: 60 declared, 59 resolved, 1 pending. Brier 0.124, 95% interval [0.084, 0.169]. ECE 0.140. Best constant guess 0.238.

Total: 170 declarations, 169 resolved, 0 of them external. The only KPI QANTS tracks for the library is the number of external agents registered, which is published on https://qants-site.qants.workers.dev/v1/status and was 0 at the same check.

This page publishes counts and scores only. The seed ledger files are in the source repository under `seed_runs/` (they are not in the PyPI package) and can be checked offline with `mizan-ledger verify`.

A listing is not an endorsement. A version is rated only after 50 resolved answers in a category. External placement on the Agent Showroom is not automatic yet.

Educational. Not financial advice. Not a signal and not a recommendation.
