Issued each quarter with your marking
A score is only worth having if it isn't one person's opinion. Your marking is done by a calibrated panel of assessors; one call in five is independently marked a second time by another of them, working blind. This statement reports how closely those pairs agreed — the measure of whether the marking is consistent, and something you can place in front of your own clients.
Each double-marked call is scored by two assessors independently; the second never sees the first's marks, the AI's score, the source report, or the participant's real name. Two figures are reported, and both are published rather than the flattering one alone:
| Double-marked call | Assessor pair | Criteria | Exact | Within one band |
|---|---|---|---|---|
| [call] | [#0__ · #0__] | [n] | [_]% | [_]% |
| [call] | [#0__ · #0__] | [n] | [_]% | [_]% |
| [call] | [#0__ · #0__] | [n] | [_]% | [_]% |
| Combined | [panel] | [N] | [_]% | [_]% |
Calls are identified by number and participants by code in all reliability reporting; assessors by an anonymous reference. Marks are 0–4 with an explicit not-applicable option, against [client]'s [N]-criterion [programme] rubric.
Where a selected second-marking pass yields no comparable score — for example a
not-applicable pass, or a recording too compromised to score fairly — it is left out of the figures
above rather than counted as disagreement, and noted here, so that a data-quality finding is never
misreported as an assessor one.
This quarter: [detail any exclusions, or “none”].
These figures are drawn from [N] double-marked criteria across [n] calls this quarter. They are measured against the baseline at [baseline ref], and re-established as marking volume grows. Movement away from it, in either direction, is the signal worth watching: a falling figure asks whether the rubric or the calibration needs attention; a rising one confirms the panel is converging.