canonical: https://jentic.com/apis/daishi.ai/daishi # Daishi Studio API The Daishi Studio API authors evaluation scenarios and launches runs against them on a Daishi Studio account. It validates a scenario against the Studio schema, estimates a run before launching, runs single evaluations and multi-condition experiments, and returns run logs, seats, and reports. Every write goes through the same validation as the Studio run builder, and runs, experiments, and series can be published or cancelled. ## For AI agents Author and validate evaluation scenarios, estimate and launch runs and experiments, and read their reports and logs on a Daishi Studio account. ## Scope Does not handle model training, production serving, or prompt storage. Use for authoring scenarios and running evaluations only. ## Capabilities - Author and validate evaluation scenarios against the Studio schema - Estimate a run before launching it and then launch the run - Create experiments and read their reports - List and cancel runs, experiments, and series - Read usage and allowances, available models, and live worlds - Publish or unpublish a run, experiment, or series ## Use cases ### Agent-driven evaluation runs An AI agent iterating on a design can validate a scenario, estimate the run, and launch it against a Daishi Studio account, then poll for the result. Because the Daishi Studio API estimates a run before it starts, the agent can confirm the cost is acceptable before committing to the evaluation. Example prompt: Validate a scenario, estimate the run, launch it, then poll the run until it completes and read the report ### Scenario authoring and validation Building reliable evaluations starts with well-formed scenarios. The Daishi Studio API exposes the scenario schema and a validate operation, so an author can check a scenario before saving it and catch errors with the same validation the run builder uses. Library scenarios, anchor bots, and templates provide starting points. Example prompt: Read the scenario schema, validate a draft scenario against it, then create the scenario once it passes ### Experiment reporting Comparing conditions needs structured results. The Daishi Studio API creates experiments, estimates them, and returns experiment reports and designs, so a workflow can launch a multi-condition experiment and read back the report for analysis. Experiments can be published so others can view the results. Example prompt: Create an experiment, launch it, then read the experiment report once it finishes ## Key endpoints | Method | Path | Description | | --- | --- | --- | | POST | `/scenarios/validate` | Validate a scenario against the Studio schema | | POST | `/scenarios` | Create a scenario | | POST | `/runs/estimate` | Estimate a run before launching | | POST | `/runs` | Launch a run | | GET | `/runs/{id}` | Get a run | | POST | `/experiments` | Create an experiment | | GET | `/experiments/{id}` | Get an experiment report | | GET | `/usage` | Read usage and allowances | ## Key resources - **Scenarios** — Author, validate, list, and read evaluation scenarios against the Studio schema - **Runs** — Estimate, launch, read, cancel, and publish evaluation runs, plus their logs and seats - **Experiments** — Create, estimate, read, and publish multi-condition experiments and their designs - **Account** — Read usage and allowances, available models, and live worlds ## AI readiness This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use. - **Score:** 62 / 100 - **Maturity:** AI-Aware - **Dimensions:** - Foundational Compliance: 74 / 100 - Developer Experience & Jentic Compatibility: 61 / 100 - AI-Readiness & Agent Experience: 44 / 100 - Agent Usability: 94 / 100 - Security: 60 / 100 - AI Discoverability: 69 / 100 - **View full report:** https://jentic.com/apis/daishi.ai/daishi/scorecard - **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/ - **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview ### Score it yourself Every API in the directory is allowlisted, so you can re-score it with no key required. - **Score your own API:** https://jentic.com/scorecard.md - **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md ```sh npx @jentic/api-scorecard-cli score ``` ## Why Jentic - **Setup:** Wiring the Daishi Studio API by hand means handling its bearer-token auth, the validate-then-estimate-then-launch run flow, and the distinct scenario, run, and experiment shapes yourself. Through Jentic you install once, import Daishi Studio from the API Directory, store the token once, and your agent calls it. - **Permission scoping:** Runs and experiments are addressed by an id in the URL path, but a rule bounds which operations your agent may call rather than which record it touches. You choose the operations it may call, such as validating scenarios and launching runs, so cancelling a run or publishing a report is not included unless you add it. - **Credential handling:** Your Daishi Studio token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context. - **Discovery method:** Agents search Jentic by intent such as 'launch an evaluation run' or 'validate a scenario', and Jentic returns the matching Daishi Studio operation with its input schema so the agent calls the right endpoint without reading the reference docs. ## Related APIs - **Braintrust** — Braintrust evaluates and monitors LLM applications, where Daishi Studio runs scenario-based agent evaluations. - **Humanloop** — Humanloop manages prompts and evaluations for LLM features, where Daishi Studio focuses on scenario runs and experiments. - **Arize** — Arize observes model and agent behavior in production alongside Daishi's pre-launch evaluations. ## FAQ ### What authentication does the Daishi Studio API use? The Daishi Studio API authenticates with a Studio access token sent as a bearer token, prefixed dsk_, as declared in its OpenAPI spec. The API index and the OpenAPI document need no token. Through Jentic the token is stored once and injected at call time, so it never appears in your agent's prompt or logs. ### Can I estimate a run before launching it with the Daishi Studio API? Yes. The API exposes estimate operations for both runs and experiments, so an agent can read the estimated cost before launching and only commit to the evaluation once the figure is acceptable. ### What are the rate limits for the Daishi Studio API? The OpenAPI spec exposes a usage and allowances endpoint but does not state numeric rate limits. Check the vendor documentation at https://daishi.ai for current limits and allowances before launching many runs. ### How do I launch an evaluation run through Jentic with the Daishi Studio API? Search Jentic for 'launch an evaluation run', which returns the POST /runs operation with its input schema, then validate the scenario and launch the run. To run it on your own infrastructure, install Jentic One from its GitHub repo. ### Can I limit what my agent is allowed to do with the Daishi Studio API? Yes. Write a rule that allows only the scenario and run read-and-create operations, so the agent can author scenarios, launch runs, and read reports but cannot cancel a run or publish a report, and every call it makes is logged. You decide which operations are in the rule, so nothing destructive runs unless you add it.