> ## Documentation Index > Fetch the complete documentation index at: https://app.miyon.ai/docs/llms.txt > Use this file to discover all available pages before exploring further. # Create an eval scenario > Create a scenario with at least one case. `slug` is generated from `name`. ## OpenAPI ````yaml /api-reference/openapi.yaml post /api/v1/evals openapi: 3.1.0 info: title: Customer API version: 1.0.0-beta description: > **Beta.** This API is in beta. Contracts can still change while we complete it, including breaking changes; we announce changes in the release notes and keep them as rare as we can. Pin your integration to the field names and status codes documented here and treat unknown fields as optional. REST API to create and configure voice agents, add tools, test them with evals, read conversations, and place AI outbound calls. **Authentication.** Voice agents, settings, tools, evals and conversations use an API token created on the **Keys** page of your dashboard. Each operation lists the scope it needs under `x-required-scopes`; `:*` grants every action of a resource. Outbound calls use a separate outbound API token from the **Outbound Calls** page. Send either token as `Authorization: Bearer `. **Responses.** A single resource is returned as a flat JSON object; a collection is wrapped in a plural key (`voice_agents`, `tools`, `evals`, `conversations`) next to its paging fields. Deleting returns `204 No Content`. Everything is addressed by the voice agent's `id`; prompts and numbers are internal. **Errors.** Authentication failures return `{ "error", "message" }`. Request quotas return `429` with a `Retry-After` header and `{ "error", "message", "details" }`, where `error` is a stable machine-readable code. Every API call must finish within 30 seconds; longer requests return `503` with `mf-core:api-request-timeout`. Validation errors list the problems per field; the exact shape is shown on each operation. servers: - url: https://app.miyon.ai security: [] tags: - name: Voice Agents description: >- Create and manage voice agents. Configure them through the voice, prompt and knowledge-base settings endpoints. - name: Voice Agent Settings description: >- Read and change a voice agent's voice, prompt and knowledge-base settings. A voice agent has one prompt (its instructions, greeting and conversation settings). Read the schema of a settings group, then update one `key`/`value` pair per request. - name: Tools description: >- Create, configure, enable and execute agent tools. Tools are enabled per voice agent with `agent_id`. These endpoints accept scoped API tokens only; every error uses the `{ "error", "message", "details" }` envelope with a stable `mf-core:tool-*` code. - name: Outbound Calls description: >- Place AI calls and review their outcome. Authenticate with the outbound API token from the **Outbound Calls** page. - name: Conversations description: >- List and inspect the conversations of your account, including transcripts when your retention settings allow. - name: Evals description: >- Evals are automated test conversations that check how a voice agent responds. A scenario contains cases; each case contains the caller's turns and what the agent must say or do. Run a scenario to get pass/fail results. paths: /api/v1/evals: post: tags: - Evals summary: Create an eval scenario description: >- Create a scenario with at least one case. `slug` is generated from `name`. operationId: createEvalScenario requestBody: required: true content: application/json: schema: allOf: - $ref: '#/components/schemas/EvalScenarioInput' - required: - name - agent_id - cases properties: cases: minItems: 1 example: name: Opening hours cases: - name: asks for hours turns: - user_input: When are you open? assistant_contains_any: - '9' - nine agent_id: 42 responses: '201': description: The created scenario. content: application/json: schema: $ref: '#/components/schemas/EvalScenario' '400': $ref: '#/components/responses/EvalBadRequest' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '422': $ref: '#/components/responses/EvalValidationError' '429': $ref: '#/components/responses/RateLimited' '503': $ref: '#/components/responses/ServiceUnavailable' security: - scopedJwt: [] components: schemas: EvalScenarioInput: type: object description: >- Request body for create and replace. `id`, `slug`, `active` and `position` are ignored if sent. properties: name: type: string description: >- Scenario name. `slug` is generated from it (lowercase letters, digits and underscores). number_id: type: integer description: Deprecated alias of `agent_id`. deprecated: true description: type: string description: Free-text description. meta: $ref: '#/components/schemas/EvalScenarioMeta' description: Models and overrides used when running the scenario. cases: type: array items: $ref: '#/components/schemas/EvalCaseInput' description: Test cases of the scenario. agent_id: type: integer description: >- The voice agent to evaluate (its `id` from `GET /api/v1/voice_agents`). EvalScenario: type: object description: Full scenario as returned by get, create, replace and duplicate. required: - id - name - slug - active - cases - recent_runs - runs_pagination - agent_id properties: id: type: integer description: ID of the scenario. name: type: string description: Scenario name. slug: type: string description: URL-safe identifier generated from `name`. description: type: - string - 'null' description: Free-text description. number_id: type: integer description: Deprecated alias of `agent_id`. deprecated: true number_name: type: - string - 'null' description: Deprecated alias of `agent_name`. deprecated: true active: type: boolean description: Whether the scenario is active in the dashboard. meta: oneOf: - $ref: '#/components/schemas/EvalScenarioMeta' title: Eval Scenario Meta - type: 'null' description: Models and overrides used when running the scenario. cases: type: array items: $ref: '#/components/schemas/EvalCase' description: Test cases in order. latest_run: oneOf: - $ref: '#/components/schemas/EvalRunSummary' title: Eval Run Summary - type: 'null' description: Summary of the most recent run. recent_runs: type: array items: $ref: '#/components/schemas/EvalRunListItem' description: A page of recent runs, newest first. runs_pagination: $ref: '#/components/schemas/EvalRunsPagination' description: Paging information for `recent_runs`. agent_id: type: integer description: The voice agent the scenario tests. agent_name: type: - string - 'null' description: Name of that voice agent. EvalScenarioMeta: type: object properties: caller_model: type: string maxLength: 100 description: Language model that plays the caller. judge_model: type: string maxLength: 100 description: Language model that judges the agent's replies. mock_responses: type: object description: >- Fixed tool results to use instead of calling the real tool, keyed by tool name. additionalProperties: true tool_variants: type: array items: $ref: '#/components/schemas/EvalToolVariant' description: Alternative agent configurations to run the scenario with. EvalCaseInput: type: object properties: name: type: string description: Defaults to `case_`. context: type: string description: Background given to the simulated caller. mode: type: string default: scripted description: >- `scripted` replays `turns`; `goal_driven` uses `user_persona`, `opening_message`, `max_turns` and the goals. Any other value behaves as `scripted`. examples: - scripted - goal_driven user_persona: type: string description: Who the simulated caller is and how they behave (goal-driven cases). opening_message: type: string description: First thing the simulated caller says (goal-driven cases). max_turns: type: integer description: Maximum caller turns before the case stops (goal-driven cases). goal_intent: type: string description: Outcome a goal-driven case must reach. goal_tool_calls: type: array items: type: object required: - name properties: name: type: string description: Tools the agent must call for the case to pass (goal-driven cases). turns: type: array items: $ref: '#/components/schemas/EvalTurnInput' description: Scripted caller turns and the expectations for each reply. source: $ref: '#/components/schemas/EvalSource' description: The conversation this case was imported from. EvalCase: type: object required: - id - name - position - mode - turns properties: id: type: integer description: ID of the case. name: type: string description: Defaults to `case_`. context: type: - string - 'null' description: Background given to the simulated caller. position: type: integer description: Order of the case within the scenario. source: oneOf: - $ref: '#/components/schemas/EvalSource' title: Eval Source - type: 'null' description: The conversation this case was imported from. mode: type: string examples: - scripted - goal_driven description: >- `scripted` replays `turns`; `goal_driven` uses `user_persona`, `opening_message`, `max_turns` and the goals. Any other value behaves as `scripted`. user_persona: type: - string - 'null' description: Who the simulated caller is and how they behave (goal-driven cases). opening_message: type: - string - 'null' description: First thing the simulated caller says (goal-driven cases). max_turns: type: - integer - 'null' description: Maximum caller turns before the case stops (goal-driven cases). goal_intent: type: - string - 'null' description: Outcome a goal-driven case must reach. goal_tool_calls: type: - array - 'null' items: type: object required: - name properties: name: type: string description: Tools the agent must call for the case to pass (goal-driven cases). turns: type: array items: $ref: '#/components/schemas/EvalTurn' description: Scripted caller turns and the expectations for each reply. EvalRunSummary: type: object description: Summary of the scenario's most recent run. required: - id - status properties: id: type: integer description: ID of the run. status: $ref: '#/components/schemas/EvalRunStatus' description: Current state of the run. total_cases: type: - integer - 'null' description: Cases in the run, once started. passed_cases: type: - integer - 'null' description: Cases that passed, once finished. failed_cases: type: - integer - 'null' description: Cases that failed, once finished. completed_at: type: - string - 'null' format: date-time description: When the run finished. EvalRunListItem: type: object description: Summary of one run in `recent_runs`. required: - id - status properties: id: type: integer description: ID of the run. sequence_number: type: - integer - 'null' description: Position of the run within its scenario, starting at 1. status: $ref: '#/components/schemas/EvalRunStatus' description: Current state of the run. total_cases: type: - integer - 'null' description: Cases in the run, once started. passed_cases: type: - integer - 'null' description: Cases that passed, once finished. failed_cases: type: - integer - 'null' description: Cases that failed, once finished. started_at: type: - string - 'null' format: date-time description: When the run started. completed_at: type: - string - 'null' format: date-time description: When the run finished. duration: type: - integer - 'null' description: Seconds between `started_at` and `completed_at`. results: description: Detailed per-case results. Available once the run finishes. error_message: type: - string - 'null' description: Why the run ended in `error`. EvalRunsPagination: type: object required: - current_page - total_pages - total_entries properties: current_page: type: integer description: Page returned. total_pages: type: integer description: Number of pages. total_entries: type: integer description: Total runs of the scenario. EvalRequestError: type: object required: - error properties: error: type: string description: Why the request failed, in English. AuthError: type: object description: >- Returned when the API token is missing, invalid, expired or lacks the required scope. properties: error: type: string description: Short error category. examples: - Authentication failed - Forbidden message: type: string description: Human-readable explanation. Absent on some 403 responses. examples: - 'Missing required scope: voice_agents:write' required: - error EvalValidationError: type: object required: - errors properties: errors: type: array items: type: string description: Human-readable problems. field_errors: type: object description: >- Problems per field. Keys are ``, `cases..` or `turns..`. additionalProperties: type: array items: type: string ApiError: type: object description: >- Error format with a stable machine-readable `error` code, a human-readable `message` and request-specific `details`. properties: error: type: string description: >- Stable machine-readable error code, for example `mf-core:rate-limit-exceeded`. message: type: string description: Human-readable explanation. details: type: object additionalProperties: true description: >- Request-specific data, for example the offending parameter or the quota policy. required: - error - message - details EvalToolVariant: type: object required: - name properties: name: type: string description: Variant name. prompt_override: type: string description: Replaces the agent's instructions for this variant. system_override: type: string description: Replaces the agent's system prompt for this variant. model_override: type: string maxLength: 100 description: Language model to use for this variant. temperature_override: type: string maxLength: 20 description: Model temperature for this variant. tool_overrides: type: object description: Tool schema overrides, keyed by tool name. additionalProperties: true EvalTurnInput: type: object required: - user_input properties: name: type: string description: Defaults to `turn_`. user_input: type: string description: What the caller says in this turn. original_assistant_response: type: string description: The agent's reply in the imported conversation, for reference. assistant_intent: type: string description: What the reply should accomplish, judged by the eval model. assistant_contains: type: array items: type: string maxLength: 1000 description: Phrases the agent's reply must all contain. assistant_contains_any: type: array items: type: string maxLength: 1000 description: The reply must contain at least one of these phrases. assistant_not_contains: type: array items: type: string maxLength: 1000 description: Phrases the reply must not contain. tool_calls: type: array items: $ref: '#/components/schemas/EvalToolCallExpectation' description: Tool calls the agent must make in this turn. no_more_events: type: boolean default: false description: If true, the agent must do nothing further after this turn. source: $ref: '#/components/schemas/EvalSource' description: The conversation this turn was imported from. EvalSource: type: object description: >- The conversation this case or turn was imported from. Absent keys are omitted. properties: conversation_id: type: integer description: ID of the conversation. conversation_name: type: string description: Name of the conversation at import time. imported_at: type: string format: date-time description: When it was imported. EvalTurn: type: object required: - id - name - position - user_input - no_more_events properties: id: type: integer description: ID of the turn. name: type: string description: Defaults to `turn_`. position: type: integer description: Order of the turn within the case. user_input: type: string description: What the caller says in this turn. original_assistant_response: type: - string - 'null' description: The agent's reply in the imported conversation, for reference. assistant_intent: type: - string - 'null' description: What the reply should accomplish, judged by the eval model. assistant_contains: type: - array - 'null' items: type: string description: Phrases the agent's reply must all contain. assistant_contains_any: type: - array - 'null' items: type: string description: The reply must contain at least one of these phrases. assistant_not_contains: type: - array - 'null' items: type: string description: Phrases the reply must not contain. tool_calls: type: - array - 'null' items: $ref: '#/components/schemas/EvalToolCallExpectation' description: Tool calls the agent must make in this turn. no_more_events: type: boolean description: If true, the agent must do nothing further after this turn. source: oneOf: - $ref: '#/components/schemas/EvalSource' title: Eval Source - type: 'null' description: The conversation this turn was imported from. EvalRunStatus: type: string enum: - pending - running - passed - failed - error - cancelled description: >- `pending` and `running` while in progress; `passed`, `failed`, `error` or `cancelled` when finished. EvalToolCallExpectation: type: object required: - name properties: name: type: string description: Name of the tool the agent must call. arguments_contains: type: object additionalProperties: true description: Argument values the call must include. description: A tool call the agent is expected to make. responses: EvalBadRequest: description: >- The body is not a JSON object, or `agent_id` does not belong to your account. content: application/json: schema: $ref: '#/components/schemas/EvalRequestError' examples: invalidBody: value: error: Invalid request invalidAgent: value: error: Invalid voice agent Unauthorized: description: Missing, expired or invalid bearer token. content: application/json: schema: $ref: '#/components/schemas/AuthError' example: error: Authentication failed message: Missing bearer token Forbidden: description: >- The token does not carry the scope this operation requires (see `x-required-scopes`). content: application/json: schema: $ref: '#/components/schemas/AuthError' example: error: Forbidden message: 'Missing required scope: voice_agents:write' EvalValidationError: description: Validation failed. content: application/json: schema: $ref: '#/components/schemas/EvalValidationError' examples: noCases: value: errors: - cases are required fieldErrors: value: errors: - Name can't be blank field_errors: name: - can't be blank RateLimited: description: >- Too many requests. Wait the number of seconds given in `Retry-After` before retrying. headers: Retry-After: description: Seconds to wait before retrying. schema: type: integer minimum: 1 content: application/json: schema: $ref: '#/components/schemas/ApiError' example: error: mf-core:rate-limit-exceeded message: Rate limit exceeded details: policy: api_write retry_after: 3 ServiceUnavailable: description: >- The request was not completed. Either it exceeded the 30-second request time limit that applies to every API call (narrow the request and retry), or rate limiting is temporarily unavailable. Retry shortly. content: application/json: schema: $ref: '#/components/schemas/ApiError' examples: timeout: value: error: mf-core:api-request-timeout message: The API request exceeded the 30-second time limit details: {} limiter: value: error: mf-core:rate-limiter-unavailable message: API rate limiting is temporarily unavailable details: policy: api_write securitySchemes: scopedJwt: type: http scheme: bearer bearerFormat: JWT description: >- Scoped API token. Required scopes are listed on each operation in x-required-scopes. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.