Concepts
Transparency & Evals
A verdict you can verify is only half the promise. The other half is whether the verdict is any good. Fidacy publishes that too: accuracy, the direction of its errors, and how much of the decision runs on the deterministic hot path are exposed on a public endpoint anyone can read. These are calibration signals, not a substitute for a customer's own validation of the controls it deploys.
The public transparency endpoint
GET /v1/transparencyreturns the engine's current evaluation metrics, computed from a labeled, human-reviewed ground-truth set. No authentication, no account.
{
"labeled_cases": 0,
"accuracy": null,
"cost_weighted_error": null,
"error_favors_fail_safe": null,
"hot_path_share": 0,
"model_version": "…",
"generated_at": "…"
}| Field | What it tells you |
|---|---|
labeled_cases | Size of the human-reviewed ground-truth set the metrics are computed against. |
accuracy | Share of cases where the engine's decision matched the human label. |
cost_weighted_error | Error weighted by how costly each mistake is, so a wrongful approve counts far more than a wrongful review. |
error_favors_fail_safe | Whether the remaining errors lean toward the safe side (review/deny) rather than wrongful approve. The invariant the whole engine is built around. |
hot_path_share | Fraction of the decision made by the deterministic, replayable hot path (versus the optional reasoning layer). |
model_version | The engine model version represented by the aggregate report. Check the live endpoint rather than treating this example as current performance. |
cost_weighted_error and error_favors_fail_safe make that direction public, not just the headline number.How the engine improves: the eval loop
The deterministic core does not drift on its own. It improves through a human-in-the-loop cycle, and the result is published, not asserted:
- ·Capture. Real assessments are captured as cases, structured signals only, no payment credentials or PII.
- ·Label. A human reviews a case and assigns the correct outcome, building a ground-truth set.
- ·Evaluate. A pure, deterministic eval engine replays the cases and computes the metrics above, including the cost-weighted error and the fail-safe direction.
- ·Propose. A change to the rules is proposed against the ground-truth, with its expected effect on the metrics. A human approves or rejects it; nothing ships itself.
- ·Version. An accepted change bumps the
policy_version, which is stamped into every signed verdict from that point on.
Tie a verdict to a published version
Every signed Risk Payload carries the policy_version it was produced under. Combined with the public metrics, an auditor or counterparty can take a verdict from months ago, read which model version produced it, and see the accuracy and error profile of that version, without contacting Fidacy.