canonical: https://jentic.com/apis/algolia.com/algolia-ab-testing

# Algolia A/B Testing API

Jentic publishes the only available OpenAPI specification for the Algolia A/B Testing API, keeping it validated and agent-ready. The A/B Testing API lets teams run controlled search relevance experiments on Algolia indices, comparing two index variants against live user traffic. Endpoints cover creating tests, listing them, retrieving variant performance, stopping tests, and estimating sample size and duration before launch. Authentication uses two headers: x-algolia-application-id and x-algolia-api-key.

## For AI agents

Create, monitor, and stop search relevance A/B tests on Algolia indices through 11 dedicated endpoints with sample size estimation.

## Scope

Does not run search queries, ingest events, or manage indices - use for creating, monitoring, stopping, and estimating Algolia A/B tests only.

## Capabilities

- Create a search relevance A/B test between two index variants with POST /2/abtests
- List every active and historical A/B test on the application via GET /2/abtests
- Retrieve detailed performance metrics for a specific A/B test by id
- Stop a running A/B test before its scheduled end with POST /2/abtests/{id}/stop
- Delete an A/B test record once analysis is finished
- Estimate the required sample size and duration for a planned A/B test before launch

## Use cases

### Search relevance experiment for a marketplace

A search engineering team tests a new ranking formula by creating an A/B test in Algolia: variant A is the production index, variant B uses an updated custom ranking. They call POST /2/abtests with the variants and traffic split, then poll GET /2/abtests/{id} for click-through and conversion deltas. The team avoids shipping a regression to all users by gating the change on the test result.

Example prompt: POST /2/abtests with variant A=production, variant B=updated, and 50/50 traffic; poll GET /2/abtests/{id} every 24 hours for the result delta

### Pre-launch sample size estimation

Before launching an experiment, a product analyst calls POST /2/abtests/estimate with the expected effect size and confidence level so the team can budget the test correctly. The endpoint returns the required sample size and duration; the analyst then schedules the actual A/B test only if the projected runtime fits the release plan.

Example prompt: POST /2/abtests/estimate with the effect size and traffic, then only call POST /2/abtests if the projected duration fits the release window

### Automatic shut-off for underperforming experiments

An ops automation polls /2/abtests/{id} for each running test and calls POST /2/abtests/{id}/stop when the leading variant crosses a confidence threshold or when negative conversion impact exceeds a guardrail. This keeps experiments from running longer than necessary and protects revenue when a variant clearly hurts performance.

Example prompt: GET /2/abtests/{id} on a schedule, evaluate guardrail metrics, and call POST /2/abtests/{id}/stop when the threshold is breached

### Agent-driven experiment lifecycle through Jentic

A Jentic agent receives an intent like 'check on my running search experiments' and calls GET /2/abtests followed by GET /2/abtests/{id} for any tests that are still active. The agent summarises the results without exposing the application id and API key, which stay in your Jentic One instance.

Example prompt: Search Jentic for 'list Algolia A/B tests', execute GET /2/abtests, then GET /2/abtests/{id} for any running tests and return a summary

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | /2/abtests | Create an A/B test |
| GET | /2/abtests | List all A/B tests |
| GET | /2/abtests/{id} | Retrieve A/B test details |
| DELETE | /2/abtests/{id} | Delete an A/B test |
| POST | /2/abtests/{id}/stop | Stop a running A/B test |
| POST | /2/abtests/estimate | Estimate sample size and duration |

## Key resources

- **abtests** — Create, list, retrieve, stop, delete, and estimate A/B tests on Algolia indices

## Why Jentic

- **Setup:** Wiring the Algolia A/B Testing API by hand means sending its API key, resolving the analytics.{region}.algolia.com host for your app, and managing the test lifecycle calls yourself. Through Jentic you install once, import the A/B Testing API from the API Directory, store the API key once, and your agent calls it.
- **Permission scoping:** The A/B Testing API puts the test id in the URL path (/2/abtests/{id}, /2/abtests/{id}/stop), so a rule can pin your agent to one A/B test. You choose the operations it may call, so deleting a test is not included unless you add it.
- **Credential handling:** Your Algolia API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'create an A/B test' or 'stop a running A/B test', and Jentic returns the matching Algolia operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Algolia Search API** — The Search API runs the queries that the A/B Testing API splits between two index variants
- **Algolia Insights API** — Send click and conversion events that feed the metrics shown for each A/B test variant
- **Algolia Personalization API** — Personalisation strategies are a common subject of A/B tests run via this API

## FAQ

### Why is there no official OpenAPI spec for the Algolia A/B Testing API?

Algolia does not publish a downstream-stable OpenAPI specification for direct agent use. Jentic generates and maintains this spec from Algolia's source bundles so AI agents and developers can call the A/B Testing API via structured tooling. It is validated against the live API and kept up to date. Get started with Jentic One, the self-hosted execution layer.

### What authentication does the Algolia A/B Testing API use?

The API requires two headers: x-algolia-application-id and x-algolia-api-key. The API key must have ACL permissions matching the requested endpoint. Jentic stores both values encrypted and injects the headers at execution time so neither key enters the agent's prompt context.

### Can I create a new search A/B test through this API?

Yes. POST /2/abtests with the two variant index references, the traffic split, and the test name. The API returns the test id which you can use with GET /2/abtests/{id} to track results or POST /2/abtests/{id}/stop to end the test early.

### What are the rate limits for the A/B Testing API?

Algolia documents a limit of 100 requests per minute per application for the A/B Testing API. Response headers include current rate-limit information; pace polling jobs to stay under that ceiling, especially when monitoring multiple tests at once.

### How do I estimate the duration of a planned A/B test through Jentic?

Search Jentic for 'estimate algolia ab test duration', load the schema for POST /2/abtests/estimate, and execute it with the expected effect size and traffic share. Jentic returns the recommended sample size so an agent can decide whether to schedule the experiment.

### Is the A/B Testing API available on every Algolia plan?

Algolia restricts A/B Testing to its Premium and Elevate plans. Confirm your plan in the Algolia dashboard before scripting against this API; calls from accounts without access return an authorisation error rather than a permission scope error.

### Can I limit what my agent is allowed to do with the Algolia A/B Testing API?

Yes. Because you run your own self-hosted Jentic One instance, your rules decide which A/B Testing operations the agent may call, so it can create and read tests with POST /2/abtests and GET /2/abtests/{id} while an operation like DELETE /2/abtests/{id} stays off unless you add it. Since the test id sits in the URL path (/2/abtests/{id}, /2/abtests/{id}/stop), a rule can also pin the agent to a single A/B test. Your Algolia application id and API key are held by your own instance and injected at execution time, so the agent never sees them.