Score APIs in CI

Use a quality threshold to detect documentation regressions while checking that the report contains enough evidence to grade.

evlio insights checks --category Security
evlio insights review openapi.yaml
cat openapi.yaml | evlio --json insights review - --fail-under 80
evlio insights show "Orders API" --fail-under 80

File and stdin reviews submit the specification to Evlio without requiring an account; the CLI review limit is 2 MiB. Reviewing a stored collection requires workspace authentication. Remove secrets and private example data before submitting a spec.

set -euo pipefail
evlio --json insights review openapi.yaml --fail-under 80 > review.json
python3 - <<'PY'
import json
with open("review.json") as stream:
    report = json.load(stream)
if not report["scorable"]:
    raise SystemExit("Specification is not scorable")
print("Quality gate passed:", report["overallScore"])
PY

Pin the CLI version used by your build and keep the spec file in the checked-out workspace. Store review.json as a build artifact. With --fail-under, a score below the threshold or an unscorable report exits with code 4. Without a threshold, a completed unscorable report can exit successfully, so check scorable explicitly.

Inspect stderr on failure. A transport error is not a passing quality gate. For unscorable reports, inspect unscorableReason, operationCount, rubricCoverage, and check comments; resolve missing operations or references before rerunning.