canonical: https://jentic.com/apis/arize.com/arize

# Arize REST API

Jentic publishes the only available OpenAPI specification for Arize REST API, keeping it validated and agent-ready. The Arize REST API is the programmatic interface to Arize's AI observability and evaluation platform, covering datasets, prompts, projects, spaces, evaluators, experiments, annotation queues, traces (spans), and AI integration management. Seventy-three endpoints let teams script the full lifecycle of evaluating LLM and ML systems - creating datasets, running experiments, annotating outputs, and managing API keys, roles, and integrations across regions or self-hosted deployments.

## For AI agents

Manage AI observability resources on Arize: datasets, prompts, projects, evaluators, experiments, annotation queues, and traces for LLM and ML evaluation.

## Scope

Does not host model inference, train models, or store raw production application logs - use for AI evaluation, observability, and annotation workflows only.

## Capabilities

- Create and manage datasets and dataset examples used to evaluate LLM and ML systems
- Configure evaluators and run experiments against datasets
- Operate annotation queues and configurations for human-in-the-loop labelling
- Manage prompts, projects, and spaces that scope evaluation work
- Issue, refresh, and revoke API keys and assign roles to team members
- Read trace spans and tasks generated by instrumented applications
- Wire up AI integrations (model providers, vector stores) inside Arize

## Use cases

### LLM Evaluation Pipeline

Build a continuous LLM evaluation pipeline that creates a dataset of test inputs, registers evaluators, runs experiments against new prompt versions, and reads back metrics - all driven from CI. The `/v2/datasets`, `/v2/evaluators`, and `/v2/experiments` endpoints provide the primitives so a release pipeline can block deploys when an evaluator regresses on the test dataset.

Example prompt: Create a dataset called 'qa-regression-2026', add 50 examples via `/v2/datasets/{dataset_id}/examples`, then trigger an experiment that runs evaluator id 7 against prompt id 21

### Human-in-the-Loop Annotation

Stand up a human review workflow by creating annotation configs, queues, and records via `/v2/annotation-configs`, `/v2/annotation-queues`, and `/v2/annotation-queues/{annotation_queue_id}/records.` Reviewers label model outputs and the resulting labels feed back into evaluators, closing the loop between production traces and improved evaluation criteria.

Example prompt: Create an annotation config for 'response-quality' with three labels, attach it to a new annotation queue, and push 200 trace spans into the queue's records endpoint

### Trace and Span Observability

Pull production trace spans from `/v2/spans` into a downstream warehouse or alerting system. The endpoint exposes telemetry from instrumented LLM applications, so a monitoring agent can surface latency spikes, tool-call failures, or unusual token usage without standing up a separate tracing backend.

Example prompt: Query `/v2/spans` for the last hour of spans on project id 5 and write them to S3 as Parquet for downstream analysis

### Workspace and Access Management

Provision spaces, assign roles, and rotate API keys programmatically when onboarding teams onto Arize. The `/v2/spaces`, `/v2/roles`, and `/v2/api-keys` endpoints let an internal IT or platform agent provision new evaluation workspaces without console clicks.

Example prompt: Create a new space, add three users with the 'editor' role, and refresh the API key for the platform service account via `/v2/api-keys/{api_key_id}/refresh`

### Agent-Driven Eval Loop via Jentic

Let an evaluation agent create datasets and trigger experiments on Arize whenever a new prompt is committed. Through Jentic the agent searches once for 'run an Arize experiment', loads the schema for `/v2/experiments`, and executes - reusing one credential vault entry across runs.

Example prompt: Search Jentic for 'run an Arize experiment', load the schema, and execute against dataset id 12 with evaluator id 4

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| GET | `/v2/datasets` | List datasets in a space |
| POST | `/v2/datasets` | Create a new dataset |
| POST | `/v2/datasets/{dataset_id}/examples` | Add examples to a dataset |
| GET | `/v2/evaluators` | List configured evaluators |
| GET | `/v2/experiments` | List experiment runs |
| GET | `/v2/annotation-queues` | List annotation queues |
| GET | `/v2/spans` | Query trace spans |
| POST | `/v2/api-keys/{api_key_id}/refresh` | Refresh an API key |

## Key resources

- **Datasets** — Create datasets and manage examples used for LLM and ML evaluation
- **Evaluators** — Define and manage automated evaluators that score model outputs
- **Experiments** — Run experiments tying datasets, prompts, and evaluators together
- **Annotation Queues** — Create annotation configs, queues, and records for human review
- **Prompts** — Store, version, and search prompt templates
- **Projects and Spaces** — Organise evaluation work into projects and tenant spaces
- **Spans** — Read trace spans emitted by instrumented LLM and ML applications
- **API Keys and Roles** — Issue, refresh, revoke API keys and assign roles to users
- **AI Integrations** — Configure model providers and vector store integrations

## Why Jentic

- **Setup:** Wiring the Arize REST API by hand means learning its bearer auth and mapping its datasets, evaluators, experiments, annotation-queues, and spans surface yourself. Through Jentic you install once, import the Arize REST API from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** Arize puts the dataset and api-key ids in the URL path (`/v2/datasets/{dataset_id}/examples`, `/v2/api-keys/{api_key_id}/refresh`), so a rule can pin your agent to a specific dataset for reads and example writes. You choose the operations it may call, so key rotation via the refresh endpoint is not included unless you add it.
- **Credential handling:** Your Arize API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'run an Arize experiment' or 'add examples to a dataset', and Jentic returns the matching Arize operation with its input schema so the agent calls the right endpoint without mapping paths and parameters by hand.

## Related APIs

- **Langfuse API** — Langfuse covers similar LLM tracing, prompt management, and evaluation territory as Arize
- **OpenAI API** — OpenAI generates the responses that Arize evaluates and traces
- **Anthropic Messages API** — Anthropic produces model completions; Arize captures them as spans and feeds them into evaluators
- **New Relic API** — New Relic monitors infrastructure and app metrics; Arize monitors AI/ML model behaviour

## FAQ

### Why is there no official OpenAPI spec for Arize REST API?

Arize does not publish a unified OpenAPI specification. Jentic generates and maintains this spec so that AI agents and developers can call Arize REST API via structured tooling. It is validated against the live API and kept up to date. Get started with Jentic One, the self-hosted execution layer.

### What authentication does the Arize REST API use?

Arize uses HTTP Bearer authentication. You pass a personal or service API key in the Authorization header. Keys are managed under `/v2/api-keys` and can be refreshed via `/v2/api-keys/{api_key_id}/refresh.` Through Jentic, the key is encrypted in the vault and never enters the agent context.

### Can I run experiments against a dataset with the Arize REST API?

Yes. Create a dataset under `/v2/datasets`, add examples via `/v2/datasets/{dataset_id}/examples`, then trigger experiments through `/v2/experiments` referencing your dataset, prompt, and evaluator ids.

### What are the rate limits for the Arize REST API?

Arize does not publish a single global rate limit in the spec; in practice limits are tied to the workspace plan. For high-volume span ingestion use the streaming/SDK ingest path; the REST API is intended for control-plane operations rather than per-event traces.

### How do I create a new dataset on Arize through Jentic?

Run pip install jentic, then search Jentic with 'create an Arize dataset', load the schema for POST `/v2/datasets`, and execute with the dataset name and space id. Add examples in a follow-up call to `/v2/datasets/{dataset_id}/examples.`

### Can I export trace spans from Arize?

Yes. GET `/v2/spans` returns spans for a given project and time window, which you can page through and write to a downstream store like S3 or BigQuery for offline analysis.

### Can the Arize REST API manage roles and access?

Yes. `/v2/roles` and `/v2/spaces` let you assign roles to users at the space level. Combined with `/v2/api-keys`, this is enough to script onboarding and offboarding of evaluation team members.

### Can I limit what my agent is allowed to do with the Arize REST API?

Yes. Because you run Jentic One yourself, your own rules decide which Arize operations and credentials the agent may use. Since Arize puts dataset and API-key ids in the URL path (for example `/v2/datasets/{dataset_id}/examples` and `/v2/api-keys/{api_key_id}/refresh`), you can pin the agent to a specific dataset for reads and example writes. You choose the exact operations it may call, so key rotation through the refresh endpoint stays off unless you explicitly grant it.