canonical: https://jentic.com/apis/promptlayer.com/promptlayer

# PromptLayer API

Jentic publishes the only available OpenAPI specification for PromptLayer API, keeping it validated and agent-ready. The PromptLayer API versions and serves prompt templates and records what happens to LLM requests after they run. It publishes new prompt template versions, moves release labels between versions, and fetches a template by version or label. It attaches metadata, scores, group ids, and the prompt used to a logged request, reads back requests and traces, and builds evaluation pipelines whose columns, runs, and scores are all reachable over the API, alongside read access to datasets and their rows.

## For AI agents

Fetch and publish versioned prompt templates, move release labels, attach metadata and scores to logged LLM requests, read traces, and run evaluation pipelines in PromptLayer.

## Scope

Does not run model completions, host models, or fine-tune them. Use for prompt template versioning, request tracking, and evaluation in PromptLayer only.

## Capabilities

- Publish a new prompt template version with a prompt_name, prompt_template body, and tags
- Fetch a prompt template by identifier at a specific version or release label
- Create a release label on a prompt version and move it to another version
- Attach metadata, a score with an optional score_name, a group_id, or prompt input variables to a logged request
- Read a logged request or a full trace by id
- Create an evaluation pipeline, add columns, run it, and read its score and rows
- List datasets in the workspace and read their rows

## Use cases

### Agent-Managed Prompt Releases

An AI agent connected through Jentic pulls the prompt template it should use by label, so it always runs the version tagged for production. When a prompt engineer approves a change, the agent publishes the new template version and moves the release label onto it, keeping prompt rollout in PromptLayer instead of in application code.

Example prompt: Fetch the 'support-reply' prompt template with label 'prod', publish a new version with an updated prompt_template, then move the prod label to that version

### Request Scoring and Tagging

After each LLM call is logged, a pipeline attaches context so the request can be filtered and reviewed later. It records metadata such as customer tier, links related calls with a group id, and writes a numeric score from a user rating or automated grader. Scores and metadata both require only the request_id plus the value, so the step is cheap to add to an existing logging flow.

Example prompt: Attach metadata {"customer_tier": "pro"} and a score of 0.9 with score_name 'helpfulness' to request_id 12345

### Prompt Regression Evaluation

Before a prompt change ships, an evaluation pipeline runs it against a dataset and scores the output. The PromptLayer API creates the pipeline, adds columns that define each evaluation step, triggers a full run, and returns the overall score plus per-row results, so a CI job or an agent can block a release whose score drops.

Example prompt: Create an evaluation pipeline named 'classifier-v2-check', add a column to it, run the full evaluation, and report the score

### Trace Debugging

When an agent run goes wrong, a developer or a debugging assistant reads the full trace by its trace_id and the individual logged request by its request_id to see the inputs, outputs, and steps involved. This keeps the investigation in one place without exporting logs by hand.

Example prompt: Retrieve trace abc-123 and the logged request 12345, and summarize where the run diverged from the expected output

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/prompt-templates/{identifier}` | Get a prompt template by version or label |
| POST | `/rest/prompt-templates` | Publish a prompt template version |
| POST | `/prompts/{prompt_id}/label` | Create a release label on a prompt |
| POST | `/rest/track-score` | Attach a score to a logged request |
| POST | `/rest/track-metadata` | Attach metadata to a logged request |
| GET | `/api/public/v2/traces/{trace_id}` | Get a trace |
| POST | `/reports/{report_id}/run` | Run a full evaluation |
| GET | `/reports/{report_id}/score` | Get an evaluation score |

## Key resources

- **Prompt templates** — Versioned prompt templates with release labels
- **Request tracking** — Metadata, scores, groups, and prompt links attached to logged requests
- **Evaluations** — Evaluation pipelines with columns, runs, scores, and rows
- **Datasets** — Workspace datasets and their rows
- **Traces** — Logged requests and full traces by id

## AI readiness

This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.

- **Score:** 60 / 100
- **Maturity:** AI-Aware
- **Dimensions:**
  - Foundational Compliance: 74 / 100
  - Developer Experience & Jentic Compatibility: 56 / 100
  - AI-Readiness & Agent Experience: 47 / 100
  - Agent Usability: 94 / 100
  - Security: 50 / 100
  - AI Discoverability: 64 / 100
- **View full report:** https://jentic.com/apis/promptlayer.com/promptlayer/scorecard
- **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/
- **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview

### Score it yourself

Every API in the directory is allowlisted, so you can re-score it with no key required.

- **Score your own API:** https://jentic.com/scorecard.md
- **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md

```sh
npx @jentic/api-scorecard-cli score <openapi-url>
```

## Why Jentic

- **Setup:** Wiring PromptLayer by hand means sending the `X-API-KEY` header on every call, keeping the integer request_id from each logged call so you can attach metadata and scores, and mapping prompt identifiers to labels and versions. Through Jentic you install once, import PromptLayer from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** PromptLayer puts the template identifier in the URL path for template reads (`/prompt-templates/{identifier}`), so a rule can limit your agent to reading one prompt template. You choose the operations it may call, so destructive ones like deleting an evaluation pipeline or a prompt label are not included unless you add them.
- **Credential handling:** Your PromptLayer API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'get a prompt template by label' or 'score a logged request', and Jentic returns the matching PromptLayer operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Langfuse** — Langfuse traces LLM applications and manages prompts and evaluations, much like PromptLayer.
- **Helicone** — Helicone logs and monitors LLM requests through a proxy, overlapping with PromptLayer request tracking.
- **LangSmith** — LangSmith traces, evaluates, and manages datasets for LangChain and other LLM apps.
- **OpenAI** — OpenAI runs the model calls whose prompts and requests PromptLayer versions and tracks.

## FAQ

### Why is there no official OpenAPI spec for PromptLayer API?

PromptLayer does not publish an OpenAPI specification. Jentic generates and maintains this spec so that AI agents and developers can call PromptLayer API via structured tooling. It is validated against the live API and kept up to date. To run it on your own infrastructure, install Jentic One from its GitHub repo.

### What authentication does the PromptLayer API use?

The PromptLayer API authenticates with an API key sent in the `X-API-KEY` header, per its OpenAPI spec. Through Jentic the key is stored encrypted by your own Jentic One instance and injected at call time, so it never reaches the agent's prompt, logs, or context.

### Can I fetch a specific prompt version with the PromptLayer API?

Yes. The get prompt template operation takes the template identifier in the path and accepts an optional version number or label in the body, so you can pin a call to version 7 or to whatever version currently carries the prod label. A separate operation returns the raw template data, and another lists the labels on a template.

### What are the rate limits for the PromptLayer API?

The OpenAPI spec does not specify rate limits. Check https://docs.promptlayer.com for the current limits that apply to your plan.

### Is there a PromptLayer MCP server?

You don't need an MCP server to give your agent PromptLayer. Jentic connects it directly from the API Directory: import it, store your credential once, and your agent calls it, with operations found on demand rather than loaded into the agent's context up front.

### Can I limit what my agent is allowed to do with the PromptLayer API?

Yes. For example, write a rule that allows only the get prompt template, track-score, and track-metadata operations, so the agent can read its prompts and annotate its own requests, while publishing templates, moving labels, and deleting evaluation pipelines stay out of reach. Every call it makes is logged.