canonical: https://jentic.com/apis/decimal.ai/decimal

# Decimal DecimalAI Platform

The DecimalAI Platform API runs the evaluation and dataset lifecycle for AI agents: it ingests agent traces, scores them with configurable evaluators, and promotes annotated traces into versioned eval datasets. It gates changes with regression checks against a baseline manifest, raises and resolves regression alerts, and tracks per-agent cost and delegation analytics. It also hosts a public skills registry where an agent's skills are authored, benchmarked, versioned, and published, and it can launch fine-tuning jobs through configured providers.

## For AI agents

Ingest AI agent traces, score them with evaluators, and build eval datasets. Runs regression checks, manages a skills registry, and launches fine-tuning jobs across 333 endpoints.

## Scope

Does not run your agent in production, serve live end-user traffic, or manage hosting infrastructure. Use for tracing, evaluating, and regression-testing AI agents and their skills only.

## Capabilities

- Ingest agent traces individually or in batches and run configurable evaluators against them
- Promote annotated traces into versioned eval datasets and validate their quality for CI gates
- Run regression checks for a candidate manifest against a baseline and raise or resolve regression alerts
- Author, version, benchmark, and publish agent skills to a public registry
- Register agent manifests, diff them, and read compatibility reports across versions
- Launch fine-tuning jobs through configured providers and attribute multi-agent failures to sub-agents
- Track workspace LLM cost analytics and per-agent cost and delegation trends

## Use cases

### AI Agent Self-Observability

An AI agent connected to DecimalAI through Jentic can ingest its own run traces and immediately request evaluator scores on them, turning each run into measured feedback. It can list the evaluators configured for the agent, push external eval scores, and read a full eval breakdown with provenance, so the agent observes its own behaviour rather than running blind.

Example prompt: Ingest a trace for the current run, run the agent's configured evaluators on it, and read the eval breakdown with provenance

### Regression Gating in CI

DecimalAI gates agent changes by running a regression check for a candidate manifest against the baseline and surfacing a verdict a CI pipeline can act on. Teams can validate a dataset version's quality for CI gates, record an engineer's human decision on a check, and roll back to a prior manifest in one call when a regression is confirmed.

Example prompt: Run a regression check for a candidate manifest against the baseline and, if it fails, roll back to the prior manifest

### Eval Dataset Lifecycle

DecimalAI builds versioned eval datasets from production traces: annotated traces are promoted into a dataset, a new dataset version is built, and quality statistics and side-by-side version comparisons show whether the set is fit for evaluation. Exported versions and DPO preference pairs feed downstream fine-tuning and benchmarking.

Example prompt: Promote annotated traces into an eval dataset, build a new version, and read its quality statistics

### Skill Registry and Benchmarking

The platform hosts a public skills registry where an agent's skills are authored, benchmarked, versioned, forked, and published. Benchmark runs produce per-case results, novelty checks flag near-duplicate skills before publishing, and verified-lift badges summarise a skill's measured effect, so skills are promoted on evidence rather than assertion.

Example prompt: Create a skill, run its benchmark suite on the latest version, and publish it to the registry once the results clear

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/api/v1/traces` | Ingest a trace |
| POST | `/api/v1/traces/{trace_id}/evaluate` | Run evaluators on a trace |
| POST | `/api/v1/datasets/{dataset_id}/build` | Build a dataset version from traces |
| POST | `/api/v1/regression-check` | Run a regression check against the baseline |
| GET | `/api/v1/evaluators` | List evaluators configured for an agent |
| POST | `/api/v1/skills` | Create a skill |
| POST | `/api/v1/training/jobs` | Launch a fine-tuning job |
| GET | `/api/v1/manifests/{manifest_id}/compatibility` | Get compatibility reports for a manifest |

## Key resources

- **Traces** — Ingest, list, group, evaluate, and export agent run traces and their violations
- **Evaluators** — Create evaluators from templates, run them on traces, and track pass-rate trends and staleness
- **Datasets** — Build versioned eval datasets from annotated traces and validate quality for CI
- **Manifests** — Register manifests, diff them, read compatibility reports, and roll back transitions
- **Regression checks** — Run checks against a baseline, record human decisions, and manage regression alerts
- **Skills** — Author, version, benchmark, fork, and publish agent skills to the public registry
- **Training** — Launch and monitor fine-tuning jobs through configured providers

## AI readiness

This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.

- **Score:** 53 / 100
- **Maturity:** Foundational
- **Dimensions:**
  - Foundational Compliance: 60 / 100
  - Developer Experience & Jentic Compatibility: 63 / 100
  - AI-Readiness & Agent Experience: 44 / 100
  - Agent Usability: 43 / 100
  - Security: 60 / 100
  - AI Discoverability: 75 / 100
- **View full report:** https://jentic.com/apis/decimal.ai/decimal/scorecard
- **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/
- **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview

### Score it yourself

Every API in the directory is allowlisted, so you can re-score it with no key required.

- **Score your own API:** https://jentic.com/scorecard.md
- **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md

```sh
npx @jentic/api-scorecard-cli score <openapi-url>
```

## Why Jentic

- **Setup:** Wiring DecimalAI by hand means handling its bearer API-key auth and its many surfaces: trace ingestion, evaluators, datasets, manifests, and regression checks. Through Jentic you install once, import DecimalAI from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** DecimalAI is scoped per operation, and its destructive paths (cascade-deleting an agent, rolling back a manifest, removing org members, starting billing checkout) are separate from its read and ingest operations. A rule can let your agent ingest traces and read evaluations while those account-level operations stay out of its reach. You choose the operations it may call, so nothing destructive runs unless you add it.
- **Credential handling:** Your DecimalAI API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'ingest an agent trace' or 'run a regression check', and Jentic returns the matching DecimalAI operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Braintrust API** — LLM evaluation and dataset platform with experiments and scoring, versus DecimalAI's agent-centric trace and manifest lifecycle.
- **Langfuse API** — Open-source LLM tracing and evaluation, versus DecimalAI's broader skill registry and training surface.
- **Arize API** — Production ML and LLM observability and drift monitoring, complementing DecimalAI's eval and regression gating.
- **Helicone API** — Lightweight LLM request logging and gateway, complementing DecimalAI's deeper evaluation lifecycle.

## FAQ

### Is there a DecimalAI MCP server?

You don't need an MCP server to give your agent DecimalAI. Jentic connects it directly from the API Directory: import it, store your API key once, and your agent calls the trace, evaluator, dataset, and regression operations. Operations are discovered on demand, so no extra server's tool definitions sit in the agent's context.

### Can I limit what my agent is allowed to do with DecimalAI?

Yes. Write a rule that allows trace ingestion and evaluator reads but not agent deletion, manifest rollback, or billing changes, so the agent can report and score its runs while account-level changes stay with you, and every call it makes is logged.

### What authentication does the DecimalAI Platform use?

Per its OpenAPI spec, the DecimalAI Platform authenticates with a bearer API key in the dai_sk_ format, sent in the Authorization header; the whoami operation confirms the key's scope and permissions. Through Jentic the key is stored encrypted by your own Jentic One instance and injected only at call time.

### Can I run regression checks for my agent with the DecimalAI Platform?

Yes. The regression-check operation runs a candidate manifest against the baseline and returns a verdict, you can record an engineer's human decision on the result, and a call-replay can test a model change behaviourally. Regression alerts can then be listed, acknowledged, and resolved from the on-call surface.

### What are the rate limits for the DecimalAI Platform?

The OpenAPI spec does not specify request-rate limits; usage is instead governed by plan limits exposed through the billing plan and usage operations, which also project current-period usage. Check docs.decimal.ai for current plan limits and quotas.

### How do I evaluate an agent trace with the DecimalAI Platform through Jentic?

Ingest the trace, then call the evaluate operation to run the agent's configured evaluators and read the resulting scores and breakdown. In Jentic, search for 'ingest and evaluate an agent trace', store your API key once, and the agent runs both steps. To run it on your own infrastructure, install Jentic One from its GitHub repo.