canonical: https://jentic.com/apis/humanloop.com/humanloop # Humanloop API The Humanloop API manages the prompts, agents, and evaluation workflows behind an LLM application, and logs every call so teams can measure and improve quality. Prompts, agents, tools, and flows are versioned files you create and update through the API, while logs capture each generation with its inputs, outputs, and feedback. Datasets and evaluators let you score versions against fixed inputs, so a change to a prompt can be compared before it ships. Product and ML teams use it to move prompt engineering, logging, and evaluation out of scattered notebooks and into a versioned, testable system. ## For AI agents Create and version prompts, agents, tools, and flows on Humanloop, call and log LLM generations, and run evaluations against datasets and evaluators. ## Scope Does not host the underlying models or handle billing. Use it to manage prompts, agents, tools, and flows, call and log generations, and run evaluations only. ## Capabilities - Create, list, and version prompt files - Call a managed prompt and receive the model generation - Log a generation with its inputs, outputs, and feedback - Manage agents, tools, and flows as versioned files - Run evaluations of a version against a dataset and evaluators - List and delete logs for analysis and cleanup ## Use cases ### Managed Prompt Execution Rather than hard-coding prompt text in application code, an agent calls POST `/prompts/call` to run a prompt that is versioned in Humanloop. The prompt template, model settings, and version all live server-side, so a prompt change ships without a code deploy. Example prompt: Call the checkout-support prompt with the customer message and return the generated reply ### Generation Logging and Feedback After an LLM call, an agent records the interaction with POST `/prompts/log` so inputs, outputs, and later user feedback are captured against the right prompt version. This builds the dataset that quality metrics and regressions are measured from. Example prompt: Log the latest support generation with the user's thumbs-up feedback against its prompt ### Pre-Ship Prompt Evaluation Before promoting a new prompt version, an agent starts an evaluation with POST /evaluations to score the candidate against a fixed dataset and a set of evaluators. The scores show whether the change improved quality before any users are exposed to it. Example prompt: Run an evaluation of the new summarization prompt against the review dataset and report the scores ### Workspace Inventory and Cleanup An operations agent lists the prompt files with GET /prompts to inventory what exists, and prunes stale records with DELETE on /logs. This keeps the workspace tidy and the evaluation corpus focused on relevant traffic. Example prompt: List all prompts, then delete logs older than the retention window ## Key endpoints | Method | Path | Description | | --- | --- | --- | | GET | `/prompts` | List prompt files in the workspace | | POST | `/prompts/call` | Call a managed prompt and return the generation | | POST | `/prompts/log` | Log a generation against a prompt | | POST | `/evaluations` | Start an evaluation run | | GET | `/logs` | List logged generations | ## Key resources - **Prompts** — Versioned prompt files that can be called and logged against - **Agents, Tools, and Flows** — Versioned files for multi-step and tool-using LLM applications - **Logs** — Recorded generations with inputs, outputs, and feedback - **Evaluations** — Scoring runs of a version against datasets and evaluators ## Why Jentic - **Setup:** Wiring the Humanloop API by hand means setting the `X-API-KEY` header, learning which of its file, log, and evaluation endpoints you need, and handling errors yourself. Through Jentic you install once, import the API from the API Directory, store the key once, and your agent calls prompts, logs, and evaluations. - **Permission scoping:** Humanloop puts the file id in the URL path, so you can scope by both operation and resource: allow an agent to call and log a specific prompt while withholding file creation, deletion, and evaluation until you add them to the allowed set. - **Credential handling:** Your Humanloop API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context. - **Discovery method:** Agents search Jentic by intent such as 'call a managed prompt' or 'run a prompt evaluation', and Jentic returns the matching Humanloop operation with its input schema so the agent calls the right endpoint. ## Related APIs - **OpenAI API** — OpenAI generates the completions; Humanloop versions the prompts and logs and evaluates those generations. - **Langfuse API** — Both trace and evaluate LLM calls; Langfuse leans to open-source tracing, Humanloop to prompt management and evaluation. - **Pinecone API** — Pinecone retrieves the context; Humanloop manages the prompt that consumes it and logs the result. ## FAQ ### What does the Humanloop API do? It manages the prompts, agents, tools, and flows behind an LLM application as versioned files, calls them to generate output, logs every generation with its inputs and outputs, and runs evaluations that score a version against a dataset and evaluators. ### What authentication does the Humanloop API use? Requests send an API key in the `X-API-KEY` header. Through Jentic the key is held by your own instance and supplied at execution time, so the agent calls the API without ever reading the raw key. ### Can I run a managed prompt through the API? Yes. POST `/prompts/call` runs a prompt that is versioned in Humanloop, so the template and model settings live server-side. You then record the result with POST `/prompts/log` to build up logging and feedback data. ### How do I evaluate a prompt before shipping it? Start an evaluation with POST /evaluations to score a candidate version against a fixed dataset and a set of evaluators. Compare the scores against the current version before you promote the change. ### What are the rate limits for the Humanloop API? The OpenAPI specification does not declare rate limits. Batch evaluation and logging work sensibly and back off on error responses; confirm current limits in the Humanloop documentation for your plan. ### Do I need an MCP server to use the Humanloop API with an agent? You do not. Jentic connects the API to your agent directly from the API Directory: import it, store your key once, and the agent calls prompts, logging, and evaluation operations without a separate server to run. ### Can I control which Humanloop operations my agent is allowed to call? Yes. Because you run Jentic One yourself, you decide the allowed set: an agent can be limited to calling and logging prompts while file creation, deletion, and evaluation stay excluded, and because Humanloop puts the file id in the URL path you can keep the agent to specific prompts.