Pin a hypothesis and bounded retrospective comparison protocol
POST
/v1/quality/checks/benchmark/{evaluationId}/experimentsRequires native benchmark result-read and case-management permissions. Baseline and candidate must be distinct retained outputs for identical immutable cases and source bytes. Locks measurement, reference and grader identities. Creation is idempotent for the same actor and payload. No target executes, no cloud call occurs, and imported evidence never becomes release authority.
How to call this endpoint
Every ACP API request uses bearer authentication. The examples here show the actual request path, auth header, and body shape that the platform expects.
Path, query, and header parameters
These parameters control which ACP object the endpoint acts on and how the request is processed.
Path parameters
| Name | Location | Type | Required | Description |
|---|---|---|---|---|
| evaluationId | path | string | Yes | — |
Query parameters
None.
Body schema
Content type: application/json · Required
| Field | Type | Required | Description |
|---|---|---|---|
| id | string | Yes | Unique identifier. |
| name | string | Yes | Human-readable name. |
| hypothesis | string | Yes | — |
| allowedChanges | string | Yes | Describes the change being investigated; this diagnostic protocol never executes or grants permission to modify a target |
| baselineMeasurementId | string | Yes | — |
| candidateObservationRunId | string | Yes | — |
| primaryMetricId | string | Yes | — |
| minimumImprovementPercentagePoints | number | Yes | — |
| minimumPairedExamples | integer | Yes | Declared descriptive cohort threshold |
| guardrails | object[] | Yes | — |
| guardrails[].metricId | string | Yes | — |
| guardrails[].maximumRegressionPercentagePoints | number | Yes | — |
What the API returns
Each response code below includes the documented payload shape for the ACP API.
200Saved protocol or identical retry receiptapplication/json
| Field | Type | Required | Description |
|---|---|---|---|
| experiment | object | Yes | — |
| experiment.schemaVersion | computer_agents_quality_experiment_v1 | Yes | — |
| experiment.id | string | Yes | Unique identifier. |
| experiment.name | string | Yes | Human-readable name. |
| experiment.hypothesis | string | Yes | — |
| experiment.allowedChanges | string | Yes | — |
| experiment.baselineMeasurementId | string | Yes | — |
| experiment.candidateObservationRunId | string | Yes | — |
| experiment.primaryMetricId | string | Yes | — |
| experiment.minimumImprovementPercentagePoints | number | Yes | — |
| experiment.minimumPairedExamples | integer | Yes | — |
| experiment.guardrails | object[] | Yes | — |
| experiment.guardrails[].metricId | string | Yes | — |
| experiment.guardrails[].maximumRegressionPercentagePoints | number | Yes | — |
| experiment.evaluationId | string | Yes | — |
| experiment.versionId | string | Yes | — |
| experiment.purpose | diagnostic | Yes | — |
| experiment.design | retained_output_comparison | Yes | — |
| experiment.baselineMeasurementFingerprint | string | Yes | — |
| experiment.baselineObservationRunId | string | Yes | — |
| experiment.baselineObservationFingerprint | string | Yes | — |
| experiment.candidateObservationFingerprint | string | Yes | — |
| experiment.populationFingerprint | string | Yes | — |
| experiment.populationCount | integer | Yes | — |
| experiment.baselineSystemLabel | string | Yes | Caller-declared import label |
| experiment.candidateSystemLabel | string | Yes | Caller-declared import label |
| experiment.limits | object | Yes | — |
| experiment.limits.maximumComparisons | 1 | Yes | — |
| experiment.limits.maximumCases | 100 | Yes | — |
| experiment.limits.externalCalls | 0 | Yes | — |
| experiment.createdByUserId | string | Yes | — |
| experiment.createdAt | string | Yes | ISO 8601 timestamp. |
| experiment.fingerprint | string | Yes | — |
400Invalid protocol
None.
401Authentication required
None.
403Organization or entitlement unavailable; execution credentials cannot author protocols
None.
404Benchmark evidence or case-management permission unavailable
None.
409Identity
None.
503Durable storage unavailable
None.
Pin a hypothesis and bounded retrospective comparison protocol
Loading...
Response 200
Loading...