> ## Documentation Index
> Fetch the complete documentation index at: https://docs.kiteml.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Create Evaluation

> **Beta.** Usable today, but this endpoint can change before it is stable, without a new `Kite-Version`. Changes are announced before they ship.

Run an SDK spec on Kite. The spec is checked first (the same checks as /v1/evaluations/validate), so a spec
that cannot run is refused here, for free. Returns 202 with a `queued` run; poll GET /v1/evaluations/{id}, or
subscribe to evaluation.succeeded / evaluation.failed / evaluation.canceled. Send an `Idempotency-Key` to make retries safe: a retry
with the same key and body returns the same run, even while the first request is still in flight.



## OpenAPI

````yaml /api-reference/openapi.json post /v1/evaluations
openapi: 3.1.0
info:
  title: Kite Platform API
  description: >-
    Public API for Kite. Authenticate with `Authorization: Bearer kite_...`, and
    send `Kite-Version: 2026-10-12` to pin the dated contract your code was
    written against (without it you get 2026-09-27). Every response carries the
    version that served it.
  version: '2026-10-12'
servers:
  - url: https://api.kiteml.com
    description: Kite Platform API
security:
  - bearerAuth: []
tags:
  - name: Augmentations
    description: Beta. Generate new episodes from a LeRobot dataset.
  - name: Training runs
    description: Fine-tune a policy on a dataset, one run per policy.
  - name: Twins
    description: >-
      Rebuild a robot episode as an interactable MuJoCo scene (beta, enabled per
      account).
  - name: RL runs
    description: >-
      Train a robot policy in simulation from a task spec; get the policy, its
      scene and a verdict.
  - name: Evaluations
    description: >-
      Evaluate a policy on a Kite SDK spec on Kite's machines: episodes,
      evaluators and a scorecard (beta).
  - name: Objects and environments
    description: Register your own objects and background scenes (beta).
  - name: Augmentation environments
    description: >-
      What an augmentation puts behind the work cell: from Kite's library, a
      photo, a prompt or a panorama (beta, enabled per account).
  - name: Catalog
    description: Policies, hardware tiers, and dataset inspection.
  - name: Operations
    description: Poll any run by its id.
  - name: Webhooks
    description: Endpoints that receive events, and the event log.
  - name: API keys
    description: The key making the request.
  - name: Usage
    description: Token usage and credit balance.
  - name: Requests
    description: >-
      The request log: every call of the last 30 days, who made it and how it
      ended.
  - name: Status
    description: Whether Kite is up, part by part. Needs no key.
paths:
  /v1/evaluations:
    post:
      tags:
        - Evaluations
      summary: Create Evaluation
      description: >-
        **Beta.** Usable today, but this endpoint can change before it is
        stable, without a new `Kite-Version`. Changes are announced before they
        ship.


        Run an SDK spec on Kite. The spec is checked first (the same checks as
        /v1/evaluations/validate), so a spec

        that cannot run is refused here, for free. Returns 202 with a `queued`
        run; poll GET /v1/evaluations/{id}, or

        subscribe to evaluation.succeeded / evaluation.failed /
        evaluation.canceled. Send an `Idempotency-Key` to make retries safe: a
        retry

        with the same key and body returns the same run, even while the first
        request is still in flight.
      operationId: create_evaluation_v1_evaluations_post
      parameters:
        - name: Idempotency-Key
          in: header
          required: false
          schema:
            anyOf:
              - type: string
                maxLength: 255
              - type: 'null'
            title: Idempotency-Key
        - $ref: '#/components/parameters/KiteVersion'
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: '#/components/schemas/EvaluationCreate'
            example:
              files:
                ../evals:
                  kind: directory
                  shipped: evals
              spec_path: specs/smoke.yaml
              spec_yaml: |-
                kite_spec_version: "2"
                name: smoke
                policy: {kind: scripted_pick, ref: scripted_pick}
                ...
      responses:
        '202':
          description: The run, queued
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/Evaluation-Input'
        '400':
          description: >-
            Invalid request (`parameter_invalid`, `unsupported_version`, ...);
            `param` names the field
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
        '401':
          description: Missing or invalid API key (`unauthenticated`)
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
        '402':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
          description: >-
            Not enough credits for this run (`insufficient_tokens`), or the
            account's monthly budget is spent and its hard limit is on
            (`budget_exceeded`)
        '403':
          description: >-
            The key lacks a scope this route needs (`missing_scope`), or the
            feature is not enabled for the account
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
        '409':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
          description: >-
            Conflicts with the resource's state or an earlier request (e.g.
            `not_cancelable`, `idempotency_key_reused`)
        '429':
          description: >-
            Rate or concurrency limited (`rate_limit_exceeded`,
            `concurrency_limit_exceeded`, `capacity_exceeded`); honor
            `Retry-After`
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
        '502':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
          description: >-
            An upstream service failed (e.g. `launch_failed`); the request is
            safe to retry
        '503':
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/ApiErrorResponse'
          description: >-
            Temporarily unavailable (`service_unavailable`); retry after
            `Retry-After`
components:
  parameters:
    KiteVersion:
      name: Kite-Version
      in: header
      required: false
      description: >-
        The dated contract your code was written against. Without it you get
        2026-09-27; every response says which version served it.
      schema:
        type: string
        enum:
          - '2026-09-27'
          - '2026-10-12'
        default: '2026-09-27'
  schemas:
    EvaluationCreate:
      properties:
        spec_yaml:
          type: string
          maxLength: 2000000
          minLength: 1
          title: Spec Yaml
          description: The spec file's text, as written
        spec_path:
          type: string
          maxLength: 500
          title: Spec Path
          description: 'Its path in your workspace: what errors cite'
          default: spec.yaml
        files:
          additionalProperties:
            $ref: '#/components/schemas/EvaluationInputFile'
          type: object
          title: Files
          description: Each path the spec names, as it names it
        overrides:
          items:
            type: string
          type: array
          maxItems: 100
          title: Overrides
          description: '`FIELD=VALUE`, as `kite --set`'
        kite_version:
          anyOf:
            - type: string
              maxLength: 40
            - type: 'null'
          title: Kite Version
          description: The Kite release to run; default the latest
        options:
          $ref: '#/components/schemas/EvaluationOptions'
        label:
          anyOf:
            - type: string
              maxLength: 120
            - type: 'null'
          title: Label
        webhook_metadata:
          anyOf:
            - type: object
            - type: 'null'
          title: Webhook Metadata
          description: Opaque metadata echoed on the resource and its webhook events
      additionalProperties: false
      type: object
      required:
        - spec_yaml
      title: EvaluationCreate
      description: >-
        Run a spec. `spec_yaml` and the files it names are required; everything
        else has a default.
      examples:
        - files:
            ../evals:
              kind: directory
              shipped: evals
          spec_path: specs/smoke.yaml
          spec_yaml: |-
            kite_spec_version: "2"
            name: smoke
            policy: {kind: scripted_pick, ref: scripted_pick}
            ...
    Evaluation-Input:
      properties:
        id:
          type: string
          title: Id
          description: >-
            `eval_` + a time-ordered ULID (`srun_` for one made before
            2026-10-18)
        object:
          type: string
          enum:
            - evaluation
          const: evaluation
          title: Object
          default: evaluation
        status:
          type: string
          enum:
            - queued
            - processing
            - canceling
            - succeeded
            - failed
            - canceled
          title: Status
        phase:
          anyOf:
            - type: string
            - type: 'null'
          title: Phase
          description: materializing, planning, fetching, running, verifying or packaging
        progress:
          type: number
          maximum: 1
          minimum: 0
          title: Progress
        status_message:
          anyOf:
            - type: string
            - type: 'null'
          title: Status Message
        episodes:
          anyOf:
            - $ref: '#/components/schemas/EvaluationEpisodes'
            - type: 'null'
        error:
          anyOf:
            - $ref: '#/components/schemas/EvaluationError'
            - type: 'null'
        spec:
          type: object
          title: Spec
          description: '`path` and `name`'
        kite_version:
          anyOf:
            - type: string
            - type: 'null'
          title: Kite Version
        options:
          type: object
          title: Options
        label:
          anyOf:
            - type: string
            - type: 'null'
          title: Label
        result:
          anyOf:
            - $ref: '#/components/schemas/EvaluationResult'
            - type: 'null'
        download:
          anyOf:
            - $ref: '#/components/schemas/EvaluationDownload'
            - type: 'null'
        tokens:
          $ref: '#/components/schemas/EvaluationTokens'
        webhook_metadata:
          anyOf:
            - type: object
            - type: 'null'
          title: Webhook Metadata
        created_at:
          anyOf:
            - type: string
            - type: 'null'
          title: Created At
        started_at:
          anyOf:
            - type: string
            - type: 'null'
          title: Started At
        completed_at:
          anyOf:
            - type: string
            - type: 'null'
          title: Completed At
        links:
          additionalProperties:
            type: string
          type: object
          title: Links
      type: object
      required:
        - id
        - status
        - progress
        - tokens
      title: Evaluation
      description: One run of an SDK spec.
    ApiErrorResponse:
      properties:
        error:
          $ref: '#/components/schemas/ApiErrorBody'
      type: object
      required:
        - error
      title: ApiErrorResponse
      description: Every /v1 error has this shape.
    EvaluationInputFile:
      properties:
        kind:
          type: string
          enum:
            - file
            - directory
          title: Kind
          default: file
        shipped:
          anyOf:
            - type: string
              maxLength: 500
            - type: 'null'
          title: Shipped
          description: >-
            A path Kite ships, e.g. `evals` or
            `profiles/orin-agx-64gb.estimated.yaml`
        sha256:
          anyOf:
            - type: string
              pattern: ^[0-9a-f]{64}$
            - type: 'null'
          title: Sha256
          description: >-
            Its hash: alone, a file or directory Kite ships; with content, a
            check
        text:
          anyOf:
            - type: string
              maxLength: 2000000
            - type: 'null'
          title: Text
          description: A text file's content
        base64:
          anyOf:
            - type: string
              maxLength: 20000000
            - type: 'null'
          title: Base64
          description: A file's bytes, or a directory as a .tar
        hf:
          anyOf:
            - $ref: '#/components/schemas/EvaluationHfFile'
            - type: string
            - type: 'null'
          title: Hf
          description: >-
            A file of a public Hugging Face repository, at a commit; or the
            file's URL (`https://huggingface.co/OWNER/NAME/blob/REVISION/PATH`,
            `…/resolve/…` too), whose branch or tag Kite pins to the commit it
            is at when the run is created
        s3:
          anyOf:
            - $ref: '#/components/schemas/EvaluationS3File'
            - type: string
            - type: 'null'
          title: S3
          description: >-
            An object in a public S3 bucket: `{bucket, key}`, or its link
            (`s3://BUCKET/KEY`, `https://BUCKET.s3.amazonaws.com/KEY`), pinned
            to the object it is when the run is created
      additionalProperties: false
      type: object
      title: EvaluationInputFile
      description: >-
        Where the bytes of one path the spec names come from. Exactly one of
        `shipped`, `text`, `base64`, `hf`, `s3`,

        or a `sha256` the catalog lists. (Uploads by URL come with /v1/files.)
    EvaluationOptions:
      properties:
        verify_determinism:
          type: boolean
          title: Verify Determinism
          description: Re-run episode 0 and refuse a differing hash
          default: false
        no_eval:
          type: boolean
          title: No Eval
          description: Skip the evaluators (no scorecard)
          default: false
        max_minutes:
          anyOf:
            - type: integer
              maximum: 720
              minimum: 1
            - type: 'null'
          title: Max Minutes
          description: >-
            The run's cap: what it reserves, at its tier's rate, and the most it
            can be charged (default 60 on CPU, 120 on GPU)
      additionalProperties: false
      type: object
      title: EvaluationOptions
    EvaluationEpisodes:
      properties:
        done:
          type: integer
          title: Done
        total:
          type: integer
          title: Total
      type: object
      required:
        - done
        - total
      title: EvaluationEpisodes
    EvaluationError:
      properties:
        code:
          type: string
          title: Code
          description: >-
            What failed, e.g. `spec_invalid`, `policy_failed`,
            `verification_failed`, `interrupted`
        message:
          type: string
          title: Message
        param:
          anyOf:
            - type: string
            - type: 'null'
          title: Param
          description: The spec field it is about (the SDK's field path)
        details:
          anyOf:
            - type: object
            - type: 'null'
          title: Details
          description: '`kite_code`, `remediation`, `location`'
        resumable:
          type: boolean
          title: Resumable
          description: POST /v1/evaluations/{id}/resume runs it again
      type: object
      required:
        - code
        - message
        - resumable
      title: EvaluationError
    EvaluationResult:
      properties:
        run_id:
          anyOf:
            - type: string
            - type: 'null'
          title: Run Id
          description: 'The SDK''s run id: the manifest hash''s first 12 characters'
        manifest_hash:
          anyOf:
            - type: string
            - type: 'null'
          title: Manifest Hash
          description: >-
            What identifies the run: the same spec, files and versions give the
            same hash, locally too
        run_hash:
          anyOf:
            - type: string
            - type: 'null'
          title: Run Hash
        determinism_verified:
          anyOf:
            - type: boolean
            - type: 'null'
          title: Determinism Verified
        verified:
          type: boolean
          title: Verified
          description: '`kite verify` passed on the run before it was reported'
        metrics:
          items:
            $ref: '#/components/schemas/EvaluationMetric'
          type: array
          title: Metrics
      type: object
      required:
        - verified
      title: EvaluationResult
    EvaluationDownload:
      properties:
        url:
          type: string
          title: Url
          description: >-
            This API's archive path, which redirects to a fresh signed link
            (send your key: `requires_auth`). Before Kite-Version 2026-10-12,
            the signed link itself (no credentials; valid until `expires_at`)
            when one could be made
        requires_auth:
          type: boolean
          title: Requires Auth
        expires_at:
          anyOf:
            - type: string
            - type: 'null'
          title: Expires At
        filename:
          type: string
          title: Filename
          description: '`<id>.tar.gz`; it extracts to `<id>/`, the SDK''s run directory'
        bytes:
          type: integer
          title: Bytes
        sha256:
          type: string
          title: Sha256
      type: object
      required:
        - url
        - requires_auth
        - filename
        - bytes
        - sha256
      title: EvaluationDownload
    EvaluationTokens:
      properties:
        charged:
          type: integer
          title: Charged
        refunded:
          type: integer
          title: Refunded
      type: object
      required:
        - charged
        - refunded
      title: EvaluationTokens
    ApiErrorBody:
      properties:
        type:
          type: string
          title: Type
          description: >-
            invalid_request_error, authentication_error, permission_error,
            not_found_error, conflict_error, rate_limit_error, or api_error
        code:
          type: string
          title: Code
          description: 'Machine-readable: branch on this, not on the message'
        message:
          type: string
          title: Message
        param:
          anyOf:
            - type: string
            - type: 'null'
          title: Param
          description: The request field the error is about, when there is one
        request_id:
          anyOf:
            - type: string
            - type: 'null'
          title: Request Id
          description: Also in the X-Request-Id header; quote it to support
        details:
          anyOf:
            - type: object
            - type: 'null'
          title: Details
          description: >-
            Extra data for some codes (e.g. the cameras a dataset does have, for
            camera_not_found)
      type: object
      required:
        - type
        - code
        - message
      title: ApiErrorBody
    EvaluationHfFile:
      properties:
        repo:
          type: string
          pattern: ^[A-Za-z0-9][A-Za-z0-9._-]{0,95}/[A-Za-z0-9][A-Za-z0-9._-]{0,95}$
          title: Repo
          description: '`OWNER/NAME`'
        revision:
          type: string
          pattern: ^[0-9a-f]{40}$
          title: Revision
          description: 'A full commit hash: a branch or a tag moves'
        path:
          type: string
          maxLength: 500
          minLength: 1
          title: Path
          description: The file's path in the repository
        repo_type:
          type: string
          enum:
            - model
            - dataset
          title: Repo Type
          default: model
      additionalProperties: false
      type: object
      required:
        - repo
        - revision
        - path
      title: EvaluationHfFile
      description: >-
        One file of a public Hugging Face repository at a commit: Kite checks it
        exists and pins its sha256 when the

        run is created, and the run's job downloads it.
    EvaluationS3File:
      properties:
        bucket:
          type: string
          pattern: ^[a-z0-9][a-z0-9-]{1,61}[a-z0-9]$
          title: Bucket
          description: The bucket (a name without dots)
        key:
          type: string
          maxLength: 1024
          minLength: 1
          title: Key
          description: The object's key
      additionalProperties: false
      type: object
      required:
        - bucket
        - key
      title: EvaluationS3File
      description: >-
        One object in a public S3 bucket: Kite checks it and pins its ETag (and
        its sha256 when S3 keeps one) when the

        run is created, and the run's job downloads it, refusing it if it has
        changed since.
    EvaluationMetric:
      properties:
        id:
          type: string
          title: Id
          description: The evaluator's id
        version:
          anyOf:
            - type: string
            - type: 'null'
          title: Version
        value:
          anyOf:
            - type: number
            - type: 'null'
          title: Value
        interval:
          anyOf:
            - items:
                anyOf:
                  - type: number
                  - type: 'null'
              type: array
            - type: 'null'
          title: Interval
          description: Its 95% interval, [low, high]
        'n':
          anyOf:
            - type: integer
            - type: 'null'
          title: 'N'
      type: object
      required:
        - id
      title: EvaluationMetric
  securitySchemes:
    bearerAuth:
      type: http
      scheme: bearer
      description: >-
        A Kite API key (`kite_...`), from **Platform API → Keys** in the
        dashboard.

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.