Run Reference Census from your own code
Both metered lanes are available over HTTP and each returns one JSON object. The division of labour: the page reads the OpenAPI spec in the browser (it is never sent), builds the census both ways against the operations the published reference lists, grades every operation with a severity per finding, measures coverage and ranks the remediation tier and the stage reached, then hands the census to the run as facts. The run writes the audit or the plan; the page then holds it to its own census (rcreply.js): every blocker named at its census severity, every discrepancy covered both ways, no percentage the census does not show, every published figure labelled; the ranked tier, exactly the stages in range, the four gates scoped to the tier.
Base URL and the response shape
Every endpoint lives under https://api.skillsafe.ai/v1/app-api, and every response has one shape:
{ "ok": true, "data": { ... } }
{ "ok": false, "error": { "code": "...", "message": "..." } }
The token is minted for this app (the guest call takes {"slug":"reference-census"}), so no slug header is needed afterwards. Send it as Authorization: Bearer YOUR_TOKEN. The run body is the input object, sent bare.
| Status | Code | Meaning |
|---|---|---|
| 400 | validation_error | The body is not a JSON object, or a required key (task, team, census, facts) is missing. The page always sends all five of task, team, census, carry, facts; carry is an empty string for audit. |
| 401 | unauthorized | No token, or a stale one. Mint a guest token or sign in again. |
| 402 | insufficient_credits | The balance is under min_credits. Price with /estimate first. |
| 403 | forbidden | A guest token tried to run: both lanes are metered and need a personal sign-in. |
| 404 | not_found | Unknown job id. |
| 429 | rate_limited | Back off and retry. |
| 5xx | server_error | Transient. Retry with the same Idempotency-Key so a retry never double-bills. |
The task field comes first
Two lanes, chosen by task, in the order the work runs - the audit, then keeping the reference complete:
audit- the audit report: whether findings recur, what to fix first with severities from the census, every census discrepancy in both directions, the Stripe and Twilio benchmarks and the reference quadrant.remediate- the remediation plan: the tier and which answer moved it, the stages in order from where the team stands, the four gates scoped to the tier, the CI steps, the re-audit and the trends to watch.
A missing or unknown task is answered as audit. Both lanes send the same five fields; the spec itself is never one of them. The facts are the page's checks, one row per line, and a line starting "must:" is a rule the reply has to hold. The page's handoff carries the audit's decisions into carry for the remediation lane.
The input
| Field | Required | What |
|---|---|---|
task | yes | audit or remediate. If it is missing or neither, do audit and set "task": "audit". |
team | yes | the API's name and purpose and the interview: the spec version, how the reference is rendered, the shipped SDK languages, how snippets are produced, what CI already gates, whether a safely-callable test environment exists, the readers, the timing, a docs-platform team, a scheduled platform change, a try-it console, docs in review; "not given" means not given. |
census | yes | every operation in the spec with its worst severity, whether the published reference lists it, and the page's findings; the documented-but-removed endpoints; the coverage the page measured and its completeness score. |
carry | send it (empty for audit) | remediate task only: the audit's decisions, when the audit task ran first; may be empty. |
facts | yes | the page's checks, one line each as area | id | status | evidence, the tier, the stage reached and what each task must hold. A line starting "must:" applies to your reply. |
retry_note | no | optional; if present, your previous reply was unusable - follow it. |
The rules the model applies
From the source skill, @samber/api-reference-quality - the only standard the run applies:
- Reference documentation is a projection of the API contract; written by hand, it becomes a second copy kept in sync only by human effort. Every finding is either a completeness gap or a drift mechanism, and the fix for drift is structural. A hand-maintained reference means every finding will recur - say so first.
- The census runs both ways: live-but-undocumented surface makes integrators guess; documented-but-removed endpoints destroy trust. The census table is operation, in spec, in the published reference, discrepancy. An unlinted, contract-untested spec is not ground truth.
- Every operation needs a description, every parameter with type, required-ness, constraints and an example, every response code including at least one 4xx, every error code with its cause and its fix (its own rubric row, never folded into examples), request and response examples, a snippet per shipped SDK language (a curl block relabelled per tab is not parity), a version statement and an idempotency-safety note. Also check disallowed values and invocation-ordering requirements, and flag tutorial prose padding a lookup page.
- Severity per finding: blocker, major, minor. Benchmark with checkable mechanisms, not reputation: Stripe renders the reference from the spec, embeds personal test keys in samples and uses typed resource-ID prefixes; Twilio publishes per-endpoint error tables and a CI-tested snippet repository whose samples run against a fake API server before merge.
- Remediation tiers, ranked: lint gate plus contract tests by default; lint gate only when no safely-callable test environment exists (contract tests leave the menu until one does); the full generated-docs migration with SDK-generated snippets when the reference is hand-maintained, a docs-platform or SDK-generation change is already scheduled, or an in-house docs-platform team makes it nearly free. A hard near-term deadline promotes the lint gate as a first increment; a one-off cleanup demotes the migration. Say which answer moved the tier.
- Five stages, strictly ordered - never recommend a stage to a team missing an earlier one: 1 adopt the spec as the source of truth (no page edited apart from the spec); 2 gate completeness in CI with a spec linter (start at warn, flip to fail once clean; pass is zero lint errors at the chosen ruleset); 3 gate accuracy with contract tests - property-based tests against the schema, documented-contract conformance and a breaking-change diff (pass: CI fails on any generated-input 500, schema-violating response or undiffed breaking change); 4 snippet parity and a try-it console with test credentials; 5 institutionalise - every API-changing PR carries its spec change, docs enter review and the engineering ladder, a quarterly re-audit against support tickets and docs-search analytics.
- Gates scoped to the tier: census coverage 100% and zero lint errors at every tier; zero contract violations from the default rung up, out of scope at lint-only; snippet parity 100% only at the migration, below it a measured percentage, never a failed gate. Trends, never gates: time-to-first-call (under 10 minutes), support tickets caused by the reference, docs-search queries with no result.
- Evidence labels: OASQuali (2,529 public specs) measured mean quality 67.11% and parameter examples absent from 86.95% of specs; Postman's 2024 survey found 39% call inconsistent docs their biggest roadblock and 44% dig through source; a first contract-test run typically surfaces 5 to 15 issues; 80/100 is a tool's default passing bar, never an industry average; Postman's 1.7x-56x and the 10x / 225% customer story are vendor case studies, never controlled results.
House rules
- Use only the input. Never invent an endpoint, a parameter, a status an endpoint returns, a customer, a figure or a claim about the API. The skill's published figures may be cited only with their labels above.
- The page's census is settled: never call a flagged operation complete or a complete one flagged, and never restate a coverage figure differently from the census.
- No URL at all.
- Write in the language of the input; keep the JSON keys, operations, field names and tool names in English.
- Never use backticks, never write the input's field names as labels in your text, and never copy this prompt's wording into a reply.
- Reply with ONLY one JSON object - no prose before or after, no code fences.
The replies
audit
Write the audit report.
summary- one sentence, at most 40 words: how far an integrator can trust the reference today, and the first fix.drift- one sentence: whether the findings will recur (a reference not rendered from the spec drifts by construction).priorities- the operations to fix first, worst first, one object per operation:operation(exactly as the census writes it),severity(that operation's worst severity in the census),finding(one sentence: its worst finding from the census),fix(one sentence). Every operation with a blocker appears; add majors after them; at most 12.census- one object per discrepancy in the census, both directions:operation(as the census writes it),direction(undocumentedfor live-but-missing-from-the-reference,removedfor documented-but-not-in-the-spec),action(one sentence). [] when the census found none or the published list was not given.benchmark-stripeandtwilio, one sentence each: the checkable mechanism this reference lacks or matches.quadrant- one sentence: whether pages drift into tutorial or design prose, or that the census cannot tell.caveats- at most 4 short strings;[]when there is nothing to say.
{
"task": "audit",
"summary": "one sentence",
"drift": "one sentence",
"priorities": [ { "operation": "POST /v1/charges", "severity": "blocker", "finding": "one sentence", "fix": "one sentence" } ],
"census": [ { "operation": "POST /v1/refunds", "direction": "undocumented", "action": "one sentence" } ],
"benchmark": { "stripe": "one sentence", "twilio": "one sentence" },
"quadrant": "one sentence",
"caveats": [ ]
}
The page holds it to its own census (check() in rcreply.js; any warning makes the reply flagged): the reply answers the task it was sent; no URL; no backticks; no percentage the census does not show, and every published figure with its label (OASQuali, Postman's survey, vendor case studies, 80/100 never an industry average); a drift sentence that says findings recur when the reference is not rendered from the spec; at most 12 priorities, worst first, each an operation in the census at its census severity with a finding and a fix, every blocker present; every census discrepancy with its right direction and an action, and none that the census did not find; Stripe and Twilio benchmarked by checkable mechanisms; a quadrant sentence.
remediate
Write the remediation plan.
summary- one sentence, at most 40 words.tier-rung(the facts' tier:lint,contractormigration),why(one sentence),moved_by(one sentence: which interview answer moved it, or that it is the default).stages- the stages to run, in order, exactly the facts' range: objects withstage(a number 1 to 5),action(one sentence),pass(one sentence: the pass condition).gates- four objects, one per gate:gate(census,lint,contractorparity),scope(in-scope,out-of-scopeormeasured, as the facts settle it),threshold(one sentence).ci- two to five strings, in order: the concrete CI steps for this tier.reaudit- one sentence: when and against what the reference is audited again (at every tier - the re-audit is a standing practice, not a stage to run).trends- two or three strings: what to watch, never gate.caveats- at most 4 short strings;[]when there is nothing to say.
{
"task": "remediate",
"summary": "one sentence",
"tier": { "rung": "contract", "why": "one sentence", "moved_by": "one sentence" },
"stages": [ { "stage": 2, "action": "one sentence", "pass": "one sentence" } ],
"gates": [ { "gate": "census", "scope": "in-scope", "threshold": "one sentence" } ],
"ci": [ "a step" ],
"reaudit": "one sentence",
"trends": [ "time-to-first-call" ],
"caveats": [ ]
}
The page requires: the reply answers the task it was sent; no URL; no backticks; the same figure rules; the ranked tier and which answer moved it; exactly the stages in range, in order, each with an action and a pass condition; the four gates once each, scoped to the tier (census and lint in scope, contract out of scope at lint-only, parity measured below the migration), each with a threshold; two to five CI steps with no contract tests at the lint-only tier; a re-audit cadence; two or three trends, never gated.
Worked examples
Three runs from the page's free samples (invented APIs): the audit for a payments API with a hand-written reference, the remediation plan for the same API carrying that audit, and the audit for a logistics API rendered from a JSON spec. Each input is exactly what the page sends - the census, never the spec; each reply is the saved reply the sample replays, shipped unedited - but the facts and long strings are shortened in this display only, and every cut is listed under the example. The last line is the page's own check of the full reply.
audit - A payments API with a hand-written reference
The page read it as: integrators will guess - 4 operations, 2 blocker, 5 major, 6 minor findings, completeness 63/100; tier by rank: migration; stage reached: 0; failing checks: undocumented, removed, params, fourxx, errors, examples, snippets, source.
Input (facts is 2066 characters; the first 1392 are shown):
{
"task": "audit",
"team": "API: Ledgerline\nWhat it does: Card charges, refunds and customers for online merchants; integrators call it from their servers.\nSpec: OpenAPI 3.0.3, info.version 2026-09-01\nPublished reference: Hand-maintained prose\nShipped SDK languages: python, node, ruby\nSnippets today: One curl block, relabelled per language tab\nCI today: spec lint no, contract tests no, breaking-change diff no; a safely-callable test environment: yes\nReaders: Third-party integrators\nTiming: A standing gate kept for years\nIn-house docs-platform team: no; docs-platform or SDK-generation change scheduled: no\nTry-it console: no; docs in PR review and the ladder: no",
"census": "Operations in the spec (operation | worst severity | reference | findings):\n- GET /v1/charges | major | in the reference | major: parameter limit: no description; minor: parameter limit: no example; major: the 400 response has no cause and fix, only \"Bad Request\"; major: no response example; minor: no snippet in python, node, ruby\n- POST /v1/charges | blocker | in the reference | major: no request example; blocker: no 4xx response; major: the 500 response has no cause and fix, only \"Internal Server Error\"; minor: no snippet in python, node, ruby\n- GET /v1/charges/{charge} | minor | in the reference | minor: no snippet in python, node, ruby\n- POST /v1/refunds | blocker | NOT in the reference | minor: no snippet in python, node, ruby; minor: no idempotency-safety note; blocker: live in the spec but missing from the published reference\nDocumented but not in the spec: POST /v1/charges/{id}/capture\nCoverage (the page's census):\n- operations in the published reference: 3 of 4 (75%)\n- operations with a description: 4 of 4 (100%)\n- parameters fully documented (type, required-ness, description, example): 2 of 3 (66.7%)\n- operations with at least one 4xx: 3 of 4 (75%)\n- error responses with cause and fix: 2 of 4 (50%)\n- operations with request and response examples: 2 of 4 (50%)\n- snippets per operation and shipped language: 0 of 12 (0%)\n- the page's completeness score: 63/100",
"carry": "",
"facts": "The page's checks (8 fail, 5 warn, 3 pass, 0 n/a); findings by severity: 2 blocker, 5 major, 6 minor:\n- The census | spec | pass | 4 operations read from YAML, OpenAPI 3.0.3\n- The census | undocumented | fail | missing from the reference: POST /v1/refunds\n- The census | removed | fail | documented but not in the spec: POST /v1/charges/{id}/capture\n- The census | version | pass | info.version 2026-09-01\n- Every operation | descriptions | pass | 4 of 4 (100%)\n- Every operation | params | fail | 2 of 3 (66.7%); missing on GET /v1/charges\n- Every operation | fourxx | fail | 3 of 4 (75%); missing on POST /v1/charges\n- Every operation | errors | fail | 2 of 4 (50%); missing on GET /v1/charges, POST /v1/charges\n- Every operation | examples | fail | 2 of 4 (50%); missing on GET /v1/charges, POST /v1/charges\n- Every operation | snippets | fail | a curl block relabelled per tab is not per-language parity\n- Every operation | idempotency | warn | no note on POST /v1/refunds\n- The gates | source | fail | Hand-maintained prose - every finding will recur\n- The gates | lint | warn | no lint gate - the page wrote a ruleset to start from\n- The gates | contract | warn | no contract tests, no breaking-change diff\n- The gates | tryit | warn | none\n- The gates | review | warn | not required\nRemediation tier by the skill's ranking: migration (a full generated-docs migration with SDK-generated\n\n[... 674 more characters cut from this display ...]"
}
Reply (long strings shortened for display - see the cuts below):
{
"task": "audit",
"summary": "Integrators cannot fully trust the Ledgerline reference: it omits the live refunds endpoint, still documents a capture endpoint absent from the spec, and gives charge creation no 4xx; first, document POST /v1/refunds and settle the capture [... 6 more characters cut from this display]",
"drift": "Every finding will recur, because the reference is hand-maintained prose rather than rendered from the spec, so it is a second copy kept in sync only by human effort and drifts by construction.",
"priorities": [
{
"operation": "POST /v1/charges",
"severity": "blocker",
"finding": "The operation declares no 4xx response, its 500 says only Internal Server Error with no cause or fix, and it has no request example.",
"fix": "Add to the spec the 4xx responses the endpoint actually returns, each with its cause and fix, give the 500 a description of what triggers it and what the caller should do, and add a request example."
},
{
"operation": "POST /v1/refunds",
"severity": "blocker",
"finding": "The operation is live in the spec but missing from the published reference, and it carries no idempotency-safety note.",
"fix": "Publish a reference entry for it from the spec, including a note on whether a retried refund request is safe to repeat."
},
{
"operation": "GET /v1/charges",
"severity": "major",
"finding": "The limit parameter has no description or example, the 400 says only Bad Request with no cause or fix, and there is no response example.",
"fix": "Describe limit and give it an example value, replace the bare Bad Request on the 400 with the cause that triggers it and how the caller corrects the request, and add a response example."
}
],
"census": [
{
"operation": "POST /v1/refunds",
"direction": "undocumented",
"action": "Add the operation to the published reference so integrators stop guessing how refunds work."
},
{
"operation": "POST /v1/charges/{id}/capture",
"direction": "removed",
"action": "Confirm whether capture still exists: if it is live, add it to the spec; if it was retired, take the page down and tell integrators what replaced it."
}
],
"benchmark": {
"stripe": "Stripe renders its reference from the spec, while Ledgerline's pages are written by hand, so nothing ties them to the contract.",
"twilio": "Twilio publishes per-endpoint error tables and runs its snippets in CI against a fake API server, while Ledgerline leaves error causes and fixes undocumented and relabels one curl block across its python, node and ruby tabs."
},
"quadrant": "The census cannot tell whether pages drift into tutorial or design prose, because it graded contract completeness and recorded no prose finding.",
"caveats": [
"The spec is neither linted nor contract-tested, so it is not yet proven ground truth.",
"No operation has a real snippet in python, node or ruby: 0 of 12."
]
}
Cut from this display only (the saved reply and the check use the full text): summary (245 characters, 239 shown).
What step 5 prints for it: blocker: POST /v1/charges | blocker: POST /v1/refunds | major: GET /v1/charges
The page's check: kept - nothing to flag
remediate - A payments API with a hand-written reference
The input carries the saved audit's decisions (R.laneForm fills carry from the audit reply), as the page's handoff does.
Input (facts is 1767 characters; the first 1392 are shown):
{
"task": "remediate",
"team": "API: Ledgerline\nWhat it does: Card charges, refunds and customers for online merchants; integrators call it from their servers.\nSpec: OpenAPI 3.0.3, info.version 2026-09-01\nPublished reference: Hand-maintained prose\nShipped SDK languages: python, node, ruby\nSnippets today: One curl block, relabelled per language tab\nCI today: spec lint no, contract tests no, breaking-change diff no; a safely-callable test environment: yes\nReaders: Third-party integrators\nTiming: A standing gate kept for years\nIn-house docs-platform team: no; docs-platform or SDK-generation change scheduled: no\nTry-it console: no; docs in PR review and the ladder: no",
"census": "Operations in the spec (operation | worst severity | reference | findings):\n- GET /v1/charges | major | in the reference | major: parameter limit: no description; minor: parameter limit: no example; major: the 400 response has no cause and fix, only \"Bad Request\"; major: no response example; minor: no snippet in python, node, ruby\n- POST /v1/charges | blocker | in the reference | major: no request example; blocker: no 4xx response; major: the 500 response has no cause and fix, only \"Internal Server Error\"; minor: no snippet in python, node, ruby\n- GET /v1/charges/{charge} | minor | in the reference | minor: no snippet in python, node, ruby\n- POST /v1/refunds | blocker | NOT in the reference | minor: no snippet in python, node, ruby; minor: no idempotency-safety note; blocker: live in the spec but missing from the published reference\nDocumented but not in the spec: POST /v1/charges/{id}/capture\nCoverage (the page's census):\n- operations in the published reference: 3 of 4 (75%)\n- operations with a description: 4 of 4 (100%)\n- parameters fully documented (type, required-ness, description, example): 2 of 3 (66.7%)\n- operations with at least one 4xx: 3 of 4 (75%)\n- error responses with cause and fix: 2 of 4 (50%)\n- operations with request and response examples: 2 of 4 (50%)\n- snippets per operation and shipped language: 0 of 12 (0%)\n- the page's completeness score: 63/100",
"carry": "Drift: Every finding will recur, because the reference is hand-maintained prose rather than rendered from the spec, so it is a second copy kept in sync only by human effort and drifts by construction.\nBlockers: POST /v1/charges, POST /v1/refunds\nCensus discrepancies: POST /v1/refunds (undocumented), POST /v1/charges/{id}/capture (removed)\nBenchmark gaps: Stripe - Stripe renders its reference from the spec, while Ledgerline's pages are written by hand, so nothing ties them to the contract. Twilio - Twilio publishes per-endpoint error tables and runs its snippets in CI against a fake API server, while Ledgerline leaves error causes and fixes undocumented and relabels one curl block across its python, node and ruby tabs.",
"facts": "The page's checks (8 fail, 5 warn, 3 pass, 0 n/a); findings by severity: 2 blocker, 5 major, 6 minor:\n- The census | spec | pass | 4 operations read from YAML, OpenAPI 3.0.3\n- The census | undocumented | fail | missing from the reference: POST /v1/refunds\n- The census | removed | fail | documented but not in the spec: POST /v1/charges/{id}/capture\n- The census | version | pass | info.version 2026-09-01\n- Every operation | descriptions | pass | 4 of 4 (100%)\n- Every operation | params | fail | 2 of 3 (66.7%); missing on GET /v1/charges\n- Every operation | fourxx | fail | 3 of 4 (75%); missing on POST /v1/charges\n- Every operation | errors | fail | 2 of 4 (50%); missing on GET /v1/charges, POST /v1/charges\n- Every operation | examples | fail | 2 of 4 (50%); missing on GET /v1/charges, POST /v1/charges\n- Every operation | snippets | fail | a curl block relabelled per tab is not per-language parity\n- Every operation | idempotency | warn | no note on POST /v1/refunds\n- The gates | source | fail | Hand-maintained prose - every finding will recur\n- The gates | lint | warn | no lint gate - the page wrote a ruleset to start from\n- The gates | contract | warn | no contract tests, no breaking-change diff\n- The gates | tryit | warn | none\n- The gates | review | warn | not required\nRemediation tier by the skill's ranking: migration (a full generated-docs migration with SDK-generated\n\n[... 375 more characters cut from this display ...]"
}
Reply:
{
"task": "remediate",
"summary": "Move Ledgerline to a reference generated from the spec with SDK-generated snippets, then gate it with a spec linter and contract tests so the refunds gap, the stale capture page and the missing errors cannot return.",
"tier": {
"rung": "migration",
"why": "The reference is hand-maintained prose, so drift is structural and no gate on the spec can protect pages that are written separately from it.",
"moved_by": "The hand-maintained reference moved the tier up from the default to the full generated-docs migration."
},
"stages": [
{
"stage": 1,
"action": "Adopt the OpenAPI spec as the single source of truth, add POST /v1/refunds and settle POST /v1/charges/{id}/capture there, and render the reference from it in place of the prose pages.",
"pass": "No reference page is edited anywhere except through the spec."
},
{
"stage": 2,
"action": "Run a spec linter on every PR from the ruleset the page wrote, starting at warn while the missing 4xx, error causes, parameter details and examples are fixed, then flip it to fail.",
"pass": "Zero lint errors at the chosen ruleset, with the linter failing the build."
},
{
"stage": 3,
"action": "Add property-based tests against the schema, documented-contract conformance checks and a breaking-change diff, all run against the safely-callable test environment.",
"pass": "CI fails on any generated-input 500, any response that violates the schema and any breaking change that was not diffed."
},
{
"stage": 4,
"action": "Replace the relabelled curl block with snippets generated from the python, node and ruby SDKs, and add a try-it console that uses test credentials.",
"pass": "Every operation has a generated snippet in python, node and ruby, and the console calls the test environment with test credentials."
}
],
"gates": [
{
"gate": "census",
"scope": "in-scope",
"threshold": "Coverage 100%: every operation in the spec appears in the reference and no documented operation is missing from the spec."
},
{
"gate": "lint",
"scope": "in-scope",
"threshold": "Zero lint errors at the chosen ruleset on every PR."
},
{
"gate": "contract",
"scope": "in-scope",
"threshold": "Zero contract violations: no generated-input 500, no schema-violating response and no undiffed breaking change."
},
{
"gate": "parity",
"scope": "in-scope",
"threshold": "Snippet parity 100%: every operation has an SDK-generated snippet in python, node and ruby."
}
],
"ci": [
"Lint the spec on every PR with the page's starter ruleset, failing the build once it is clean.",
"Run property-based and conformance tests against the test environment and diff the spec against the last release for breaking changes.",
"Generate the reference and the python, node and ruby snippets from the spec and the SDKs in the same pipeline.",
"Compare the operations in the spec with those in the generated reference and fail on any mismatch or any operation lacking a snippet in a shipped language."
],
"reaudit": "Re-audit the reference every quarter against the support tickets it caused and docs-search queries that returned nothing.",
"trends": [
"Time-to-first-call, aiming under 10 minutes",
"Support tickets caused by the reference",
"Docs-search queries with no result"
],
"caveats": [
"Expect the first contract-test run to surface issues; the skill notes a first run typically finds 5 to 15.",
"Stage 5, institutionalising spec changes in every API-changing PR and docs review, lies beyond this tier's reach."
]
}
What step 5 prints for it: migration | stage 1: Adopt the OpenAPI spec as the single sou | stage 2: Run a spec linter on every PR from the r | stage 3: Add property-based tests against the sch | stage 4: Replace the relabelled curl block with s
The page's check: kept - nothing to flag
audit - A logistics API rendered from a JSON spec
The page read it as: integrators will guess - 4 operations, 2 blocker, 3 major, 9 minor findings, completeness 60/100; tier by rank: lint; stage reached: 1; failing checks: refs, params, fourxx, errors, examples.
Input (facts is 2004 characters; the first 1392 are shown):
{
"task": "audit",
"team": "API: Quaybrook\nWhat it does: Shipment booking and tracking for freight forwarders.\nSpec: OpenAPI 3.0.1, info.version 1.4.0\nPublished reference: Rendered from the spec\nShipped SDK languages: python, java\nSnippets today: Generated from the spec or an SDK\nCI today: spec lint no, contract tests no, breaking-change diff no; a safely-callable test environment: no\nReaders: Third-party integrators\nTiming: A standing gate kept for years\nIn-house docs-platform team: no; docs-platform or SDK-generation change scheduled: no\nTry-it console: no; docs in PR review and the ladder: no",
"census": "Operations in the spec (operation | worst severity | reference | findings):\n- POST /shipments | minor | minor: a summary but no description; minor: no idempotency-safety note\n- GET /shipments/{id} | blocker | minor: parameter id: no example; major: no response example; blocker: unresolved $ref #/components/responses/Shipment\n- DELETE /shipments/{id} | major | minor: parameter id: no example; major: the 409 response has no cause and fix, only \"Conflict\"; minor: no snippet in java; minor: no idempotency-safety note\n- GET /shipments/{id}/events | blocker | major: parameter id: no description; minor: parameter id: no example; minor: parameter since: no example; blocker: no 4xx response; minor: no snippet in python, java\nDocumented but not in the spec: the published list was not given\nCoverage (the page's census):\n- operations in the published reference: n/a\n- operations with a description: 4 of 4 (100%)\n- parameters fully documented (type, required-ness, description, example): 0 of 4 (0%)\n- operations with at least one 4xx: 3 of 4 (75%)\n- error responses with cause and fix: 2 of 3 (66.7%)\n- operations with request and response examples: 3 of 4 (75%)\n- snippets per operation and shipped language: 5 of 8 (62.5%)\n- the page's completeness score: 60/100",
"carry": "",
"facts": "The page's checks (5 fail, 6 warn, 4 pass, 1 n/a); findings by severity: 2 blocker, 3 major, 9 minor:\n- The census | spec | pass | 4 operations read from JSON, OpenAPI 3.0.1\n- The census | undocumented | warn | list the operations the published reference shows to diff both ways\n- The census | refs | fail | GET /shipments/{id}\n- The census | version | pass | info.version 1.4.0\n- Every operation | descriptions | pass | 4 of 4 (100%)\n- Every operation | params | fail | 0 of 4 (0%); missing on GET /shipments/{id}, DELETE /shipments/{id}, GET /shipments/{id}/events\n- Every operation | fourxx | fail | 3 of 4 (75%); missing on GET /shipments/{id}/events\n- Every operation | errors | fail | 2 of 3 (66.7%); missing on DELETE /shipments/{id}\n- Every operation | examples | fail | 3 of 4 (75%); missing on GET /shipments/{id}\n- Every operation | snippets | warn | 5 of 8 (62.5%) across python, java\n- Every operation | idempotency | warn | no note on POST /shipments, DELETE /shipments/{id}\n- The gates | source | pass | Rendered from the spec\n- The gates | lint | warn | no lint gate - the page wrote a ruleset to start from\n- The gates | contract | na | out of scope - no safely-callable environment\n- The gates | tryit | warn | none\n- The gates | review | warn | not required\nRemediation tier by the skill's ranking: lint (a lint gate only) - there is no safely-callable test environment, so\n\n[... 612 more characters cut from this display ...]"
}
Reply:
{
"task": "audit",
"summary": "Integrators can trust the Quaybrook reference to match the spec but not to be complete: one response reference never resolves and the events endpoint has no 4xx; first, resolve the broken Shipment response reference.",
"drift": "The pages are rendered from the spec, so they do not drift from it; these are completeness gaps in the spec that persist until fixed there, and new ones can enter freely until a linter gates the spec.",
"priorities": [
{
"operation": "GET /shipments/{id}",
"severity": "blocker",
"finding": "The response points at the unresolved reference #/components/responses/Shipment, there is no response example, and the id parameter has no example.",
"fix": "Define the Shipment response under components/responses or repoint the reference at the component that exists, then add a response example and an example id."
},
{
"operation": "GET /shipments/{id}/events",
"severity": "blocker",
"finding": "The operation declares no 4xx response, id has no description or example, since has no example, and it has no python or java snippet.",
"fix": "Add to the spec the 4xx responses the endpoint actually returns with cause and fix, describe id, and give id and since example values."
},
{
"operation": "DELETE /shipments/{id}",
"severity": "major",
"finding": "The 409 says only Conflict with no cause or fix, id has no example, there is no java snippet and no idempotency-safety note.",
"fix": "Rewrite the 409 description to state what makes the delete conflict and what the caller does to resolve it, and add a note on whether repeating the delete is safe."
}
],
"census": [],
"benchmark": {
"stripe": "Quaybrook matches Stripe's main checkable mechanism, a reference rendered from the spec, though the census cannot show whether its samples carry test keys or typed resource-ID prefixes.",
"twilio": "Twilio publishes a per-endpoint error table, while Quaybrook leaves the DELETE /shipments/{id} 409 without cause or fix and gives the events endpoint no 4xx at all, and its generated snippets miss python and java on some operations."
},
"quadrant": "The census cannot tell whether pages drift into tutorial or design prose, since it graded contract completeness and recorded no prose finding.",
"caveats": [
"The published operation list was not given, so neither undocumented nor removed endpoints could be checked.",
"The spec is neither linted nor contract-tested, so it is not yet proven ground truth."
]
}
What step 5 prints for it: blocker: GET /shipments/{id} | blocker: GET /shipments/{id}/events | major: DELETE /shipments/{id}
The page's check: kept - nothing to flag
Build the input yourself
Load the engine in node (global.window = global; require("./rcyaml.js"); require("./rckit.js"); require("./rcreply.js")), then:
const K = RefKit, R = RefReply;
const f = K.form(Object.assign(K.blank(), { name: "Acme", rendered: "generated", languages: "python", sandbox: "yes",
spec: "openapi: 3.1.0\npaths:\n /v1/things:\n get:\n description: Lists things.\n responses:\n '200':\n description: The things.\n '401':\n description: The key is missing or revoked - send a live key.",
published: "GET /v1/things" }));
// f.census.ops (each operation with its findings and worst severity), f.cov (coverage), f.tier, f.stage, K.ruleset(f)
const b = K.buildInput(f, "audit"); // or "remediate" (R.laneForm(form, "remediate", audit) fills carry)
const wire = K.toWire(b.input); // POST this to /estimate and /run - it never carries the spec
// then: const reply = R.normalize(R.parseReply(job.output.output), "audit");
// R.check(reply, b) // {status: kept or flagged, notes}
Get a token for the steps below from /tokens.html.
Step 1 — a tiny client helper
One function that sends JSON and unwraps the response. Paste a personal token from /tokens.html, or leave it empty and mint a guest token in step 2.
BASE="https://api.skillsafe.ai/v1/app-api"
TOKEN="YOUR_TOKEN" # from https://reference-census.skillsafe.ai/tokens.html
api() { # api METHOD PATH [JSON-BODY]
curl -s -X "$1" "$BASE$2" -H "Content-Type: application/json" \
${TOKEN:+-H "Authorization: Bearer $TOKEN"} ${3:+--data "$3"}
}
import json, urllib.request
BASE = "https://api.skillsafe.ai/v1/app-api"
TOKEN = "YOUR_TOKEN" # from https://reference-census.skillsafe.ai/tokens.html
def api(method, path, body=None, headers=None):
req = urllib.request.Request(BASE + path, method=method,
data=None if body is None else json.dumps(body).encode(),
headers={"Content-Type": "application/json", **({"Authorization": "Bearer " + TOKEN} if TOKEN else {}), **(headers or {})})
with urllib.request.urlopen(req) as r:
env = json.load(r)
if env.get("error"):
raise RuntimeError(env["error"])
return env["data"]
// reference-census.mjs -- Node 18+ (global fetch)
const BASE = "https://api.skillsafe.ai/v1/app-api";
let TOKEN = "YOUR_TOKEN"; // from https://reference-census.skillsafe.ai/tokens.html
async function api(method, path, body, headers = {}) {
const r = await fetch(BASE + path, { method, headers: { "Content-Type": "application/json",
...(TOKEN ? { Authorization: "Bearer " + TOKEN } : {}), ...headers },
body: body === undefined ? undefined : JSON.stringify(body) });
const env = await r.json();
if (env.error) throw new Error(env.error.code + ": " + env.error.message);
return env.data;
}
package main
import ("bytes"; "encoding/json"; "fmt"; "net/http")
const base = "https://api.skillsafe.ai/v1/app-api"
var token = "YOUR_TOKEN" // from https://reference-census.skillsafe.ai/tokens.html
func api(method, path string, body any, headers map[string]string) (map[string]any, error) {
var buf bytes.Buffer
if body != nil { json.NewEncoder(&buf).Encode(body) }
req, _ := http.NewRequest(method, base+path, &buf)
req.Header.Set("Content-Type", "application/json")
if token != "" { req.Header.Set("Authorization", "Bearer "+token) }
for k, v := range headers { req.Header.Set(k, v) }
res, err := http.DefaultClient.Do(req)
if err != nil { return nil, err }
defer res.Body.Close()
var env struct { Data map[string]any; Error map[string]any }
json.NewDecoder(res.Body).Decode(&env)
if env.Error != nil { return nil, fmt.Errorf("%v", env.Error) }
return env.Data, nil
}
// Reference Census.java -- Java 17+, with com.fasterxml.jackson.core:jackson-databind
import com.fasterxml.jackson.databind.*;
import java.net.URI; import java.net.http.*; import java.util.*;
public class Reference Census {
static final String BASE = "https://api.skillsafe.ai/v1/app-api";
static String token = "YOUR_TOKEN"; // from https://reference-census.skillsafe.ai/tokens.html
static final ObjectMapper M = new ObjectMapper();
static final HttpClient C = HttpClient.newHttpClient();
static JsonNode api(String method, String path, Object body, Map<String, String> headers) throws Exception {
var b = HttpRequest.newBuilder(URI.create(BASE + path)).header("Content-Type", "application/json");
if (!token.isEmpty()) b.header("Authorization", "Bearer " + token);
if (headers != null) headers.forEach(b::header);
b.method(method, body == null ? HttpRequest.BodyPublishers.noBody() : HttpRequest.BodyPublishers.ofString(M.writeValueAsString(body)));
JsonNode env = M.readTree(C.send(b.build(), HttpResponse.BodyHandlers.ofString()).body());
if (env.hasNonNull("error")) throw new RuntimeException(env.get("error").toString());
return env.get("data");
}
}
require "json"
require "net/http"
BASE = "https://api.skillsafe.ai/v1/app-api"
TOKEN = "YOUR_TOKEN" # from https://reference-census.skillsafe.ai/tokens.html
def api(method, path, body = nil, headers = {})
uri = URI(BASE + path)
req = Net::HTTPGenericRequest.new(method, !body.nil?, true, uri.request_uri,
{ "Content-Type" => "application/json" }.merge(TOKEN.empty? ? {} : { "Authorization" => "Bearer #{TOKEN}" }).merge(headers))
req.body = JSON.generate(body) unless body.nil?
env = JSON.parse(Net::HTTP.start(uri.host, uri.port, use_ssl: true) { |h| h.request(req) }.body)
raise env["error"].to_s if env["error"]
env["data"]
end
<?php
const BASE = "https://api.skillsafe.ai/v1/app-api";
$TOKEN = "YOUR_TOKEN"; // from https://reference-census.skillsafe.ai/tokens.html
function api(string $method, string $path, $body = null, array $headers = []) {
global $TOKEN;
$h = ["Content-Type: application/json"];
if ($TOKEN) $h[] = "Authorization: Bearer $TOKEN";
foreach ($headers as $k => $v) $h[] = "$k: $v";
$ch = curl_init(BASE . $path);
curl_setopt_array($ch, [CURLOPT_CUSTOMREQUEST => $method, CURLOPT_HTTPHEADER => $h, CURLOPT_RETURNTRANSFER => true]);
if ($body !== null) curl_setopt($ch, CURLOPT_POSTFIELDS, json_encode($body));
$env = json_decode(curl_exec($ch), true);
if (!empty($env["error"])) throw new Exception(json_encode($env["error"]));
return $env["data"];
}
// Program.cs -- .NET 6+ console app
using System.Net.Http.Json; using System.Text.Json; using System.Text.Json.Nodes;
static class Api {
public const string Base = "https://api.skillsafe.ai/v1/app-api";
public static string Token = "YOUR_TOKEN"; // from https://reference-census.skillsafe.ai/tokens.html
static readonly HttpClient C = new();
public static async Task<JsonNode?> Call(string method, string path, object? body = null, Dictionary<string, string>? headers = null) {
var req = new HttpRequestMessage(new HttpMethod(method), Base + path);
if (Token != "") req.Headers.Add("Authorization", "Bearer " + Token);
if (headers != null) foreach (var h in headers) req.Headers.Add(h.Key, h.Value);
if (body != null) req.Content = JsonContent.Create(body);
var env = JsonNode.Parse(await (await C.SendAsync(req)).Content.ReadAsStringAsync())!;
if (env["error"] is JsonNode e && e.GetValueKind() != JsonValueKind.Null) throw new Exception(e.ToJsonString());
return env["data"];
}
}
Step 2 — a token
A guest token is enough for /me and /estimate. A run is metered, so it needs a personal token: sign in on /tokens.html and copy it from there.
# a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
TOKEN=$(curl -s -X POST "$BASE/guest" -H "Content-Type: application/json" -d '{"slug":"reference-census"}' | jq -r '.data.token')
# a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
TOKEN = ""
TOKEN = api("POST", "/guest", {"slug": "reference-census"})["token"]
// a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
TOKEN = "";
TOKEN = (await api("POST", "/guest", { slug: "reference-census" })).token;
// inside main: a guest token (enough for /me and /estimate); a run needs your personal token
token = ""
g, err := api("POST", "/guest", map[string]string{"slug": "reference-census"}, nil)
if err != nil { panic(err) }
token = g["token"].(string)
// inside main: a guest token (enough for /me and /estimate); a run needs your personal token
token = "";
token = api("POST", "/guest", Map.of("slug", "reference-census"), null).get("token").asText();
# a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
TOKEN.replace("")
TOKEN.replace(api("POST", "/guest", { slug: "reference-census" })["token"])
// a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
$TOKEN = "";
$TOKEN = api("POST", "/guest", ["slug" => "reference-census"])["token"];
// a guest token (enough for /me and /estimate); a run needs your personal token from /tokens.html
Api.Token = "";
Api.Token = (await Api.Call("POST", "/guest", new { slug = "reference-census" }))!["token"]!.GetValue<string>();
Step 3 — GET /me
Who the token is (subject_type: user or guest) and the credit balance.
api GET /me | jq '.data | {subject_type, credits}'
me = api("GET", "/me")
print(me["subject_type"], me.get("credits"))
const me = await api("GET", "/me");
console.log(me.subject_type, me.credits);
me, _ := api("GET", "/me", nil, nil)
fmt.Println(me["subject_type"], me["credits"])
JsonNode me = api("GET", "/me", null, null);
System.out.println(me.get("subject_type") + " " + me.get("credits"));
me = api("GET", "/me")
puts [me["subject_type"], me["credits"]].inspect
$me = api("GET", "/me");
echo $me["subject_type"], " ", $me["credits"] ?? "", "\n";
var me = await Api.Call("GET", "/me");
Console.WriteLine($"{me!["subject_type"]} {me["credits"]}");
Step 4 — POST /estimate
Free. The body is the run input itself, the bare object (never wrapped in {"input": ...}). hold_credits is a reservation, not the price; min_credits is what the balance must cover; model_alias reads gpt-terra.
# input.json: the run input object - {"task": "audit", "team": "API: Ledgerline\nSpec: OpenAPI 3.0.3 ...", "census": "Operations in the spec (operation | worst severity | reference | findings):\n- GET /v1/charges | major | ...", "carry": "", "facts": "The page's checks (9 fail, ...):\n- The census | spec | pass | ..."}
# - see the input table
api POST /estimate "$(cat input.json)" | jq '.data | {hold_credits, min_credits, model, model_alias, markup_bps, warnings}'
inp = json.load(open("input.json")) # {"task": "audit", "team": "API: Ledgerline\nSpec: OpenAPI 3.0.3 ...", "census": "Operations in the spec (operation | worst severity | reference | findings):\n- GET /v1/charges | major | ...", "carry": "", "facts": "The page's checks (9 fail, ...):\n- The census | spec | pass | ..."}
# - see the input table
est = api("POST", "/estimate", inp)
print(est["hold_credits"], est["min_credits"], est["model"], est.get("warnings"))
import { readFileSync } from "node:fs";
const input = JSON.parse(readFileSync("input.json", "utf8")); // {task: "audit" or "remediate", team, census, carry, facts} (see the input table)
const est = await api("POST", "/estimate", input);
console.log(est.hold_credits, est.min_credits, est.model, est.warnings);
raw, _ := os.ReadFile("input.json") // add "os" to the imports
var input map[string]any
json.Unmarshal(raw, &input)
est, _ := api("POST", "/estimate", input, nil)
fmt.Println(est["hold_credits"], est["min_credits"], est["model"])
Map<String, Object> input = M.readValue(new java.io.File("input.json"), Map.class);
JsonNode est = api("POST", "/estimate", input, null);
System.out.println(est.get("hold_credits") + " " + est.get("min_credits") + " " + est.get("model"));
input = JSON.parse(File.read("input.json"))
est = api("POST", "/estimate", input)
puts [est["hold_credits"], est["min_credits"], est["model"]].inspect
$input = json_decode(file_get_contents("input.json"), true);
$est = api("POST", "/estimate", $input);
echo $est["hold_credits"], " ", $est["min_credits"], " ", $est["model"], "\n";
var input = JsonNode.Parse(File.ReadAllText("input.json"))!;
var est = await Api.Call("POST", "/estimate", input);
Console.WriteLine($"{est!["hold_credits"]} {est["min_credits"]} {est["model"]}");
Step 5 — POST /run, then poll the job
Send an Idempotency-Key built from the lane and the input: a retry with the same key never bills twice. The page uses reference-census:<task>:<inputHash>:a<attempt> - the lane is part of the key, retry_note is left out of the hash - and its one automatic reformat retry is a new attempt number, named in the page as one extra run. An audit reply carries summary, drift, priorities, census, benchmark, quadrant and caveats; a remediate reply carries summary, tier, stages, gates, ci, reaudit, trends and caveats. The code below prints the gist of either: each priority with its severity, or the tier and each stage.
KEY="reference-census:$(jq -r .task input.json):$(shasum -a 256 input.json | cut -c1-16):a1"
JOB=$(curl -s -X POST "$BASE/run" -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" \
-H "Idempotency-Key: $KEY" --data @input.json | jq -r '.data.job_id')
while :; do
J=$(api GET "/jobs/$JOB"); S=$(echo "$J" | jq -r '.data.status')
case "$S" in succeeded|failed|cancelled) break;; esac; sleep 2
done
echo "$J" | jq -r '.data.output.output' > reply.json # one JSON object, the reply contract
echo "$J" | jq '.data | {charged_credits, truncated}'
jq -r 'if .task == "remediate" then (.tier.rung, (.stages[] | "stage \(.stage): \(.action)"))
else (.priorities[] | "\(.severity): \(.operation)") end' reply.json
import hashlib, time
key = "reference-census:" + inp["task"] + ":" + hashlib.sha256(json.dumps(inp).encode()).hexdigest()[:16] + ":a1"
job = api("POST", "/run", inp, {"Idempotency-Key": key})
while True:
j = api("GET", "/jobs/" + job["job_id"])
if j["status"] in ("succeeded", "failed", "cancelled"):
break
time.sleep(2)
reply = json.loads(j["output"]["output"]) # the reply as one JSON object
print(j["charged_credits"], j.get("truncated"), reply["task"])
if reply["task"] == "remediate":
print(reply["tier"]["rung"])
for x in reply["stages"]:
print(x["stage"], x["action"])
else:
for x in reply["priorities"]:
print(x["severity"], x["operation"])
import { createHash } from "node:crypto";
const key = `reference-census:${input.task}:${createHash("sha256").update(JSON.stringify(input)).digest("hex").slice(0, 16)}:a1`;
const job = await api("POST", "/run", input, { "Idempotency-Key": key });
let j;
do { await new Promise((r) => setTimeout(r, 2000)); j = await api("GET", "/jobs/" + job.job_id); }
while (!["succeeded", "failed", "cancelled"].includes(j.status));
const reply = JSON.parse(j.output.output);
console.log(j.charged_credits, j.truncated, reply.task);
if (reply.task === "remediate") { console.log(reply.tier.rung); for (const x of reply.stages) console.log(x.stage, x.action); }
else for (const x of reply.priorities) console.log(x.severity, x.operation);
sum := sha256.Sum256(raw) // add "crypto/sha256", "encoding/hex", "time" to the imports
key := "reference-census:" + input["task"].(string) + ":" + hex.EncodeToString(sum[:8]) + ":a1"
job, _ := api("POST", "/run", input, map[string]string{"Idempotency-Key": key})
var j map[string]any
for {
time.Sleep(2 * time.Second)
j, _ = api("GET", "/jobs/"+job["job_id"].(string), nil, nil)
if s := j["status"]; s == "succeeded" || s == "failed" || s == "cancelled" { break }
}
var reply map[string]any
json.Unmarshal([]byte(j["output"].(map[string]any)["output"].(string)), &reply)
fmt.Println(j["charged_credits"], j["truncated"], reply["task"])
key := "priorities"
if reply["task"] == "remediate" {
key = "stages"
}
ls, _ := reply[key].([]any)
for _, x := range ls {
xx := x.(map[string]any)
fmt.Println(xx["severity"], xx["operation"], xx["stage"], xx["action"])
}
var digest = java.security.MessageDigest.getInstance("SHA-256").digest(M.writeValueAsBytes(input));
String key = "reference-census:" + input.get("task") + ":" + HexFormat.of().formatHex(digest, 0, 8) + ":a1";
JsonNode job = api("POST", "/run", input, Map.of("Idempotency-Key", key)), j;
do { Thread.sleep(2000); j = api("GET", "/jobs/" + job.get("job_id").asText(), null, null); }
while (!Set.of("succeeded", "failed", "cancelled").contains(j.get("status").asText()));
JsonNode reply = M.readTree(j.get("output").get("output").asText());
System.out.println(j.get("charged_credits") + " " + reply.path("task").asText());
if (reply.path("task").asText().equals("remediate")) {
for (JsonNode x : reply.path("stages")) System.out.println(x.path("stage").asText() + " " + x.path("action").asText());
} else {
for (JsonNode x : reply.path("priorities")) System.out.println(x.path("severity").asText() + " " + x.path("operation").asText());
}
require "digest"
key = "reference-census:#{input["task"]}:#{Digest::SHA256.hexdigest(JSON.generate(input))[0, 16]}:a1"
job = api("POST", "/run", input, { "Idempotency-Key" => key })
j = nil
loop do
sleep 2
j = api("GET", "/jobs/#{job["job_id"]}")
break if %w[succeeded failed cancelled].include?(j["status"])
end
reply = JSON.parse(j["output"]["output"])
puts [j["charged_credits"], j["truncated"], reply["task"]].inspect
if reply["task"] == "remediate"
reply["stages"].each { |x| puts "#{x["stage"]} #{x["action"]}" }
else
reply["priorities"].each { |x| puts "#{x["severity"]} #{x["operation"]}" }
end
$key = "reference-census:" . $input["task"] . ":" . substr(hash("sha256", json_encode($input)), 0, 16) . ":a1";
$job = api("POST", "/run", $input, ["Idempotency-Key" => $key]);
do { sleep(2); $j = api("GET", "/jobs/" . $job["job_id"]); }
while (!in_array($j["status"], ["succeeded", "failed", "cancelled"]));
$reply = json_decode($j["output"]["output"], true);
echo $j["charged_credits"], " ", $reply["task"], "\n";
if ($reply["task"] === "remediate") { foreach ($reply["stages"] as $x) echo $x["stage"], " ", $x["action"], "\n"; }
else { foreach ($reply["priorities"] as $x) echo $x["severity"], " ", $x["operation"], "\n"; }
var digest = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(input.ToJsonString()));
var key = $"reference-census:{input["task"]}:{Convert.ToHexString(digest, 0, 8).ToLowerInvariant()}:a1";
var job = await Api.Call("POST", "/run", input, new() { ["Idempotency-Key"] = key });
JsonNode j;
do { await Task.Delay(2000); j = (await Api.Call("GET", "/jobs/" + job!["job_id"]))!; }
while (!new[] { "succeeded", "failed", "cancelled" }.Contains(j["status"]!.GetValue<string>()));
var reply = JsonNode.Parse(j["output"]!["output"]!.GetValue<string>())!;
var task = (string?)reply["task"];
Console.WriteLine($"{j["charged_credits"]} {task}");
if (task == "remediate") { foreach (var x in reply["stages"]!.AsArray()) Console.WriteLine($"{x!["stage"]} {x["action"]}"); }
else { foreach (var x in reply["priorities"]!.AsArray()) Console.WriteLine($"{x!["severity"]} {x["operation"]}"); }
Step 6 — POST /run-stream
The same run as Server-Sent Events, with the same input.json. Browsers receive progress ticks rather than text deltas; the final done event carries the reply.
# Server-Sent Events: tick/delta events while the job runs, then a final event with the job
curl -N -s -X POST "$BASE/run-stream" -H "Content-Type: application/json" -H "Authorization: Bearer $TOKEN" \
-H "Idempotency-Key: reference-census:$(jq -r .task input.json):$(shasum -a 256 input.json | cut -c1-16):a1" --data @input.json
rev = json.load(open("input.json")) # the same run input as input.json: {"task": "audit", "team": "API: Ledgerline\nSpec: OpenAPI 3.0.3 ...", "census": "Operations in the spec (operation | worst severity | reference | findings):\n- GET /v1/charges | major | ...", "carry": "", "facts": "The page's checks (9 fail, ...):\n- The census | spec | pass | ..."}
# - see the input table
req = urllib.request.Request(BASE + "/run-stream", method="POST", data=json.dumps(rev).encode(),
headers={"Content-Type": "application/json", "Authorization": "Bearer " + TOKEN,
"Idempotency-Key": "reference-census:" + rev["task"] + ":" + hashlib.sha256(json.dumps(rev).encode()).hexdigest()[:16] + ":a1"})
with urllib.request.urlopen(req) as r:
for line in r:
line = line.decode().strip()
if line.startswith("data:"):
print(line[5:].strip()[:200])
const rev = JSON.parse(readFileSync("input.json", "utf8"));
const r = await fetch(BASE + "/run-stream", { method: "POST", body: JSON.stringify(rev), headers: {
"Content-Type": "application/json", Authorization: "Bearer " + TOKEN,
"Idempotency-Key": `reference-census:${rev.task}:${createHash("sha256").update(JSON.stringify(rev)).digest("hex").slice(0, 16)}:a1` } });
const dec = new TextDecoder();
for await (const chunk of r.body) for (const line of dec.decode(chunk).split("\n"))
if (line.startsWith("data:")) console.log(line.slice(5).trim().slice(0, 200));
rraw, _ := os.ReadFile("input.json") // add "bufio", "strings" to the imports
rs := sha256.Sum256(rraw)
req, _ := http.NewRequest("POST", base+"/run-stream", bytes.NewReader(rraw))
req.Header.Set("Content-Type", "application/json")
req.Header.Set("Authorization", "Bearer "+token)
req.Header.Set("Idempotency-Key", "reference-census:"+input["task"].(string)+":"+hex.EncodeToString(rs[:8])+":a1")
res, _ := http.DefaultClient.Do(req)
sc := bufio.NewScanner(res.Body)
for sc.Scan() { if l := sc.Text(); strings.HasPrefix(l, "data:") { fmt.Println(l[5:]) } }
String rev = java.nio.file.Files.readString(java.nio.file.Path.of("input.json"));
var rd = java.security.MessageDigest.getInstance("SHA-256").digest(rev.getBytes());
var sreq = HttpRequest.newBuilder(URI.create(BASE + "/run-stream")).header("Content-Type", "application/json")
.header("Authorization", "Bearer " + token).header("Idempotency-Key", "reference-census:" + input.get("task") + ":" + HexFormat.of().formatHex(rd, 0, 8) + ":a1")
.POST(HttpRequest.BodyPublishers.ofString(rev)).build();
C.send(sreq, HttpResponse.BodyHandlers.ofLines()).body().filter(l -> l.startsWith("data:")).forEach(System.out::println);
rev = File.read("input.json")
uri = URI(BASE + "/run-stream")
req = Net::HTTP::Post.new(uri, "Content-Type" => "application/json", "Authorization" => "Bearer #{TOKEN}",
"Idempotency-Key" => "reference-census:#{JSON.parse(rev)["task"]}:#{Digest::SHA256.hexdigest(rev)[0, 16]}:a1")
req.body = rev
Net::HTTP.start(uri.host, uri.port, use_ssl: true) do |h|
h.request(req) { |res| res.read_body { |chunk| chunk.each_line { |l| puts l if l.start_with?("data:") } } }
end
$rev = file_get_contents("input.json");
$ch = curl_init(BASE . "/run-stream");
curl_setopt_array($ch, [CURLOPT_POST => true, CURLOPT_POSTFIELDS => $rev, CURLOPT_HTTPHEADER => [
"Content-Type: application/json", "Authorization: Bearer $TOKEN",
"Idempotency-Key: reference-census:" . json_decode($rev, true)["task"] . ":" . substr(hash("sha256", $rev), 0, 16) . ":a1"],
CURLOPT_WRITEFUNCTION => function ($ch, $chunk) { foreach (explode("\n", $chunk) as $l) if (str_starts_with($l, "data:")) echo $l, "\n"; return strlen($chunk); }]);
curl_exec($ch);
var rev = File.ReadAllText("input.json");
var rd = System.Security.Cryptography.SHA256.HashData(System.Text.Encoding.UTF8.GetBytes(rev));
var sreq = new HttpRequestMessage(HttpMethod.Post, Api.Base + "/run-stream") { Content = new StringContent(rev, System.Text.Encoding.UTF8, "application/json") };
sreq.Headers.Add("Authorization", "Bearer " + Api.Token);
sreq.Headers.Add("Idempotency-Key", $"reference-census:{input["task"]}:{Convert.ToHexString(rd, 0, 8).ToLowerInvariant()}:a1");
using var http = new HttpClient();
using var res = await http.SendAsync(sreq, HttpCompletionOption.ResponseHeadersRead);
using var rdr = new StreamReader(await res.Content.ReadAsStreamAsync());
for (string? l; (l = await rdr.ReadLineAsync()) != null;) if (l.StartsWith("data:")) Console.WriteLine(l);