canonical: https://jentic.com/apis/roark.ai/roark

# Roark Analytics API

The Roark Analytics API tests and monitors voice and chat AI agents: it runs simulated conversations against an agent and scores real calls and chats with metrics. Simulations can start from a built-in template, an inline plan, or a saved run plan, and use personas, environments, customer flows with edge cases, and tool fixtures that mock the agent's tool calls. Recorded calls and chats are ingested with their transcripts, then scored by metric definitions, thresholds, and policies through metric collection jobs. The same API manages agents and their prompts, webhooks, issues, managed agent configs with staging and production revisions, projects and members, and the benchmark leaderboard.

## For AI agents

Run simulated test conversations against voice and chat AI agents, ingest real calls and transcripts, and score them with custom metrics, thresholds, and policies.

## Scope

Does not place production phone calls, host telephony numbers, or generate speech for your live agent. Use for testing, simulating, and scoring voice and chat agent conversations only.

## Capabilities

- Start a simulation against a voice or chat agent from a built-in template, an inline plan, or a saved planId
- Ingest a call with its recordingUrl, startedAt, interfaceType, callDirection, and transcript for analysis
- Score selected calls or chats by creating a metric collection job with callIds or chatIds and metrics
- Define custom metric definitions with thresholds and variants, and apply them through metric policies
- Model test callers with personas that set language, accent, gender, speech pace, and background noise
- Author customer flows with a happy path and edge cases, and mock tool calls with tool fixtures
- Subscribe a webhook to Roark events and open issues with call or chat evidence attached

## Use cases

### AI coding agent that tests a voice agent before release

A coding agent editing a voice agent's prompt can ask Roark to run a test before the change ships. Through Jentic the agent searches for 'run a simulation against my voice agent', loads the Roark simulation-run operation with its input schema, and starts a run from a template or a saved plan with the bearer token added at call time. It then reads the plan job status and the resulting metrics to decide whether the prompt change is ready.

Example prompt: POST `/v1/simulation/run` with an existing planId, then poll GET `/v1/simulation/plan/job/{jobId}` until the job finishes and report the outcome

### Post-call analytics for production calls

Teams running voice agents in production send each finished call to Roark with its recording URL, start time, interface type, direction, and transcript. A metric collection job then scores a batch of calls against the metrics the team cares about, and per-call metrics and sentiment runs can be read back by call id. This turns raw call logs into measurable quality signals.

Example prompt: POST `/v1/call` with recordingUrl, startedAt, interfaceType, and callDirection, then POST `/v1/metric/collection-jobs` with the new call id in callIds and the chosen metrics

### Realistic test callers and flows

A simulation is only as good as its callers. Roark personas describe a caller's language, accent, gender, speech pace, speech clarity, background noise, and interruption behaviour, while customer flows define the happy path and edge cases a conversation should cover. Tool fixtures and tool mocks let the test agent's tool calls return controlled responses.

Example prompt: POST `/v1/persona` with name, language, accent, and gender for a fast-talking caller, then POST `/v1/customer-flow/{flowId}/edge-case` to add an interruption scenario to the refund flow

### Quality issue tracking

When a metric drops or a reviewer spots a bad conversation, an issue can be opened through the API with a title, summary, severity, and the calls or chats as evidence in one transaction. Listing issues by status, source, or severity gives a running view of open quality problems across the project.

Example prompt: POST `/v1/issue` with the title 'Agent skips identity check', severity, and the two failing call ids as evidence

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/v1/simulation/run` | Run a simulation |
| GET | `/v1/simulation/plan/job/{jobId}` | Get a simulation plan job |
| POST | `/v1/call` | Create a call for analysis |
| GET | `/v1/call/{callId}/transcript` | Get call transcript |
| GET | `/v1/call/{callId}/metrics` | List call metrics |
| POST | `/v1/metric/collection-jobs` | Create a metric collection job |
| POST | `/v1/persona` | Create a persona |
| POST | `/v1/webhook` | Create a webhook |

## Key resources

- **Simulations** — Simulation runs, run plans, plan jobs, environments, templates, and tool fixtures for testing agents
- **Calls and chats** — Ingested conversations with transcripts, metrics, sentiment runs, and tool invocations
- **Metrics** — Metric definitions, thresholds, variants, policies, and collection jobs that score conversations
- **Agents and configs** — Agents, agent endpoints, prompts and prompt versions, and managed configs with staging and production revisions
- **Personas and customer flows** — Simulated caller profiles and the scripted flows, happy paths, and edge cases they follow

## AI readiness

This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.

- **Score:** 65 / 100
- **Maturity:** AI-Aware
- **Dimensions:**
  - Foundational Compliance: 73 / 100
  - Developer Experience & Jentic Compatibility: 73 / 100
  - AI-Readiness & Agent Experience: 48 / 100
  - Agent Usability: 69 / 100
  - Security: 75 / 100
  - AI Discoverability: 83 / 100
- **View full report:** https://jentic.com/apis/roark.ai/roark/scorecard
- **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/
- **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview

### Score it yourself

Every API in the directory is allowlisted, so you can re-score it with no key required.

- **Score your own API:** https://jentic.com/scorecard.md
- **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md

```sh
npx @jentic/api-scorecard-cli score <openapi-url>
```

## Why Jentic

- **Setup:** Wiring Roark by hand means handling bearer tokens that differ between personal credentials and project API keys, and learning how templates, run plans, plan jobs, and metric collection jobs relate. Through Jentic you install once, import Roark from the API Directory, store the token once, and your agent calls it.
- **Permission scoping:** Roark puts the agent id in the URL path for agent and prompt calls (`/v1/agent/{agentId}/prompts`), so a rule can pin your agent to one Roark agent's prompts. Simulation runs carry their plan or template in the request body, so for those, rules bound which operations are allowed. You choose the operations it may call, so destructive ones like deleting an agent or revoking a project API key are not included unless you add them.
- **Credential handling:** Your Roark token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'run a simulation against my voice agent' or 'get a call transcript', and Jentic returns the matching Roark operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Langfuse** — LLM tracing, evaluation, and prompt management for text-based AI applications
- **Braintrust API** — Evaluation datasets, experiments, and scoring for LLM applications
- **Retell AI API** — Platform for building and running voice AI phone agents
- **Bland AI API** — Voice AI calling platform for automated inbound and outbound phone calls

## FAQ

### What authentication does the Roark Analytics API use?

Per its OpenAPI spec, the Roark Analytics API uses HTTP bearer authentication with a JWT-format token in the `Authorization` header. The spec also distinguishes personal credentials from project API keys, and some credential endpoints refuse a project API key. Through Jentic the token is held by your own instance and injected at call time, so the agent never sees it.

### Can I run a test simulation against my agent with the Roark Analytics API?

Yes. POST `/v1/simulation/run` starts a simulation from a built-in template, an inline plan, or a saved planId, and returns the run. Plan jobs can be listed, read, and cancelled under `/v1/simulation/plan/job/{jobId}.` The older POST `/v1/simulation/plan/{planId}/job` call is deprecated in favour of the run endpoint.

### What are the rate limits for the Roark Analytics API?

The OpenAPI spec does not specify rate limits. Check the Roark website at https://roark.ai or your Roark dashboard for current limits before scheduling large metric collection jobs.

### Is there a Roark MCP server?

You do not need an MCP server to give your agent Roark. Jentic connects it directly from the API Directory: import it, store your token once, and your agent calls the simulation, call, and metric operations it needs on demand without loading another server's tool definitions into its context.

### How do I score calls with my own metrics through Jentic?

Search Jentic for 'score calls with a metric', load the POST `/v1/metric/collection-jobs` operation, and execute it with callIds and the metrics to compute. Then read GET `/v1/metric/collection-jobs/{jobId}/metric-values` for the results. To run it on your own infrastructure, install Jentic One from its GitHub repo.

### Can I limit what my agent is allowed to do with the Roark Analytics API?

Yes. For a release-check agent, write a rule that allows only POST `/v1/simulation/run` and GET `/v1/simulation/plan/job/{jobId}`, so it can start tests and read their results but cannot delete agents, revoke API keys, or remove project members. Every call it makes is logged.
