Concepts
Transparency & Evals
A verdict you can verify is only half the promise. The other half is whether the verdict is any good. Fidacy publishes that too: accuracy, the direction of its errors, and how much of the decision runs on the deterministic hot path are exposed on a public endpoint anyone can read, and every verdict carries the policy_version it was produced under. You do not have to take our word for the quality of the engine, the same way you do not have to take our word for a signature.
The public transparency endpoint
GET /v1/transparencyreturns the engine's current evaluation metrics, computed from a labeled, human-reviewed ground-truth set. No authentication, no account.
{
"labeled_cases": 6,
"accuracy": 1,
"cost_weighted_error": 0,
"error_favors_fail_safe": true,
"hot_path_share": 1,
"model_version": "fidacy-risk-0.1.0",
"generated_at": "2026-06-28T09:46:52Z"
}| Field | What it tells you |
|---|---|
labeled_cases | Size of the human-reviewed ground-truth set the metrics are computed against. |
accuracy | Share of cases where the engine's decision matched the human label. |
cost_weighted_error | Error weighted by how costly each mistake is, so a wrongful approve counts far more than a wrongful review. |
error_favors_fail_safe | Whether the remaining errors lean toward the safe side (review/deny) rather than wrongful approve. The invariant the whole engine is built around. |
hot_path_share | Fraction of the decision made by the deterministic, replayable hot path (versus the optional reasoning layer). |
model_version | The version label that also appears as policy_version in every signed verdict, so you can tie a payload to a published metric set. |
cost_weighted_error and error_favors_fail_safe make that direction public, not just the headline number.How the engine improves: the eval loop
The deterministic core does not drift on its own. It improves through a human-in-the-loop cycle, and the result is published, not asserted:
- ·Capture. Real assessments are captured as cases, structured signals only, no payment credentials or PII.
- ·Label. A human reviews a case and assigns the correct outcome, building a ground-truth set.
- ·Evaluate. A pure, deterministic eval engine replays the cases and computes the metrics above, including the cost-weighted error and the fail-safe direction.
- ·Propose. A change to the rules is proposed against the ground-truth, with its expected effect on the metrics. A human approves or rejects it; nothing ships itself.
- ·Version. An accepted change bumps the
policy_version, which is stamped into every signed verdict from that point on.
Tie a verdict to a published version
Every signed Risk Payload carries the policy_version it was produced under. Combined with the public metrics, an auditor or counterparty can take a verdict from months ago, read which model version produced it, and see the accuracy and error profile of that version, without contacting Fidacy.