canonical: https://jentic.com/apis/futureagi.com/futureagi

# Future AGI API

The Future AGI API evaluates, annotates, and tests the outputs of AI applications so a team can measure quality and catch regressions before they ship. It runs evaluation groups that score traced outputs, records those scores against a source, and bulk-annotates spans pulled from your traces. It also drives human review through annotation queues, where items are assigned, annotated, and tracked for inter-annotator agreement, and it lets you generate test scenarios and execute test runs against a saved agent definition.

## For AI agents

Run evaluation groups and record scores on your AI application's traced outputs, manage human annotation queues, and execute agent test runs in Future AGI. Authenticates with a bearer token.

## Scope

Does not host language models, serve production inference, or manage billing. Use for evaluating, annotating, and testing AI application outputs only.

## Capabilities

- Create and apply evaluation groups that score your AI application's traced outputs
- Record evaluation scores against a source and read them back by source
- Bulk-annotate spans pulled from your traces for later review
- Route items through annotation queues, assigning and submitting human annotations
- Track inter-annotator agreement and queue progress for a review project
- Generate test scenarios and execute test runs against a saved agent definition
- Manage reusable annotation labels, including restoring a deleted label

## Use cases

### Agent-Scored Output Evaluation

An AI agent connected through Jentic creates an evaluation group, applies it to a batch of traced outputs, and reads back the resulting scores, so quality measurement runs as a step in the agent's own workflow rather than a separate dashboard task. Because scores are stored against their source, the agent can later pull the scores for that source to decide whether an output passed.

Example prompt: Create an evaluation group, apply it to the latest traced outputs, and return the resulting scores for that source

### Human-in-the-Loop Annotation

A review team builds an annotation queue, adds items to it, and assigns them to annotators who submit labels on each item, while the API tracks queue progress and inter-annotator agreement. This turns ad hoc labeling into a managed review project with a clear record of who annotated what and how much reviewers agreed.

Example prompt: Create an annotation queue, add these items, assign them to a reviewer, and report the queue progress

### Agent Simulation and Regression Testing

A developer generates a test scenario for a saved agent definition, then creates and executes a test run to check the agent against that scenario before a release. Running tests through the API means regression checks can sit inside a CI step rather than a manual review, with each run tied to a specific agent definition version.

Example prompt: Generate a scenario for this agent definition, create a test run, and execute it

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/model-hub/eval-groups/` | Create an evaluation group |
| GET | `/model-hub/eval-groups/` | List evaluation groups |
| POST | `/model-hub/eval-groups/apply-eval-group/` | Apply an evaluation group to outputs |
| POST | `/model-hub/scores/` | Create an evaluation score |
| POST | `/tracer/bulk-annotation/` | Bulk-annotate trace spans |
| POST | `/model-hub/annotation-queues/` | Create an annotation queue |
| POST | `/simulate/run-tests/create/` | Create a test run |
| POST | `/simulate/run-tests/{run_test_id}/execute/` | Execute a test run |

## Key resources

- **Evaluation Groups** — Collections of evals that are applied to traced outputs and produce scores
- **Scores** — Evaluation results recorded against a source, retrievable per source and in bulk
- **Annotation Queues** — Human-review queues with item assignment, submission, progress, and agreement tracking
- **Labels** — Reusable annotation labels that can be created, updated, deleted, and restored
- **Simulations** — Agent definitions, generated scenarios, and test runs for agent testing

## AI readiness

This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.

- **Score:** 67 / 100
- **Maturity:** AI-Aware
- **Dimensions:**
  - Foundational Compliance: 65 / 100
  - Developer Experience & Jentic Compatibility: 77 / 100
  - AI-Readiness & Agent Experience: 52 / 100
  - Agent Usability: 93 / 100
  - Security: 60 / 100
  - AI Discoverability: 70 / 100
- **View full report:** https://jentic.com/apis/futureagi.com/futureagi/scorecard
- **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/
- **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview

### Score it yourself

Every API in the directory is allowlisted, so you can re-score it with no key required.

- **Score your own API:** https://jentic.com/scorecard.md
- **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md

```sh
npx @jentic/api-scorecard-cli score <openapi-url>
```

## Why Jentic

- **Setup:** Wiring the Future AGI API by hand means handling its bearer-token auth, learning the model-hub and tracer route groups, and threading evaluation groups, scores, and annotation queues together yourself. Through Jentic you install once, import Future AGI from the API Directory, store the token once, and your agent calls it.
- **Permission scoping:** Future AGI carries most ids in the URL path, such as a score id or an annotation queue id, so a rule can bound your agent to the operations it needs. You choose which operations it may call, so destructive ones like deleting a score or a queue are not included unless you add them.
- **Credential handling:** Your Future AGI token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'score my AI outputs' or 'annotate trace spans', and Jentic returns the matching Future AGI operation with its input schema so the agent calls the right endpoint without reading the reference docs.

## Related APIs

- **Langfuse API** — Open-source LLM observability and evaluation platform with tracing and scoring
- **Humanloop API** — LLM evaluation and prompt management platform with human feedback workflows
- **Braintrust API** — Evaluation platform for scoring and comparing AI application outputs across versions
- **Helicone API** — LLM request logging and observability that captures the traces you later evaluate

## FAQ

### Is there a Future AGI MCP server?

You don't need an MCP server to give your agent the Future AGI API. Jentic connects it directly from the API Directory: import it, store your credential once, and your agent can create evaluation groups, record scores, and run agent tests straight away, with no extra server to run and no tool definitions to load into the agent's context.

### Can I limit what my agent is allowed to do with the Future AGI API?

Yes. Write a rule that allows only the operations the agent needs, such as creating scores and reading evaluation logs, so it can measure output quality without touching review queues or labels. Destructive operations like deleting a score or a queue are not included unless you add them, and every call the agent makes is logged.

### What authentication does the Future AGI API use?

The Future AGI API authenticates with a bearer token, where your API key is sent as a Bearer token in the Authorization header, per its OpenAPI spec. Through Jentic the token is stored encrypted by your own instance and added to each request at call time, so it never appears in the agent's prompt or logs.

### Can I score my AI application's outputs with the Future AGI API?

Yes. You create an evaluation group, apply it to traced outputs, and the API records a score against each source, which you can then read back per source or list in bulk. This lets an agent or a CI step measure output quality without a human opening a dashboard.

### What are the rate limits for the Future AGI API?

The OpenAPI spec does not specify rate limits for the Future AGI API. For current limits and fair-use guidance, see the Future AGI documentation at https://docs.futureagi.com.

### How do I run an agent evaluation through Jentic?

Import the Future AGI API from the Jentic API Directory, then have your agent issue a request such as 'create an evaluation group and apply it to my traced outputs'. Jentic matches the intent to the create and apply operations and returns their input schema so the agent builds the correct request, with your stored credential injected at call time. To run it on your own infrastructure, install Jentic One from its GitHub repo.