canonical: https://jentic.com/apis/langfuse.com/langfuse # langfuse Langfuse is an open-source LLM observability and evaluation platform. The public API records traces, observations, generations, scores, and sessions for LLM-powered applications, and exposes endpoints to run evaluations, manage prompts, datasets, annotation queues, and project-level resources. Teams use it to debug agent runs, measure quality across model versions, and track cost and latency for production LLM workloads. The spec covers 86 operations across observability, prompt management, datasets, evaluations, and project administration. ## For AI agents Record LLM traces, manage prompts and datasets, and run evaluations on Langfuse to observe and debug agent and model behaviour. ## Scope Does not generate model responses, host models, or vectorise text - use for LLM tracing, evaluation, and prompt management only. ## Capabilities - Record traces, observations, generations, and scores for an LLM-powered application - Manage versioned prompts and pull the active prompt by name from the prompt registry - Create and update datasets and dataset items used to evaluate model outputs - Run evaluations and read evaluator results against dataset items - Manage annotation queues and queue items for human-in-the-loop review - Query project, comment, and integration resources for organisation-level admin ## Use cases ### LLM Observability for Production Agents An engineering team logs every LLM call from a production agent into Langfuse, capturing the prompt, model, latency, cost, and final response so they can replay and debug failures. The API accepts traces, observations, and generations under a single trace_id so a multi-step agent run is reconstructable end to end. Teams use this to triage incidents and spot regressions after model swaps. Example prompt: Create a Langfuse trace for the current agent run, log a generation observation with the model, prompt, output, and token usage, then attach a quality score before closing the trace. ### Versioned Prompt Management Product and ML teams keep prompts in Langfuse rather than scattered across code, fetching the active version of a named prompt at runtime. The API supports versioning, labels, and rollback so a prompt can be changed without a redeploy and historical versions remain auditable. This decouples prompt iteration from application releases. Example prompt: Fetch the active 'support-reply' prompt from Langfuse, render it with the current ticket context, and submit the call to the model - log the full exchange back to Langfuse. ### Dataset-Driven Evaluation ML engineers maintain an evaluation dataset in Langfuse, run model outputs against it, and compare scores between candidate prompts or models. The API exposes datasets, dataset items, and evaluation runs so a CI pipeline can grade a release before it ships. Useful for catching regressions before production rollout. Example prompt: Run the candidate prompt against the 'tickets-eval' dataset in Langfuse, collect scores per item, and report whether the new prompt beats the current production prompt. ### AI Agent Self-Observation An AI agent built on Jentic logs its own runs to Langfuse for later inspection by engineers and other agents. The agent searches Jentic for trace logging, loads the operation schema, and writes traces directly without manually wiring Langfuse client setup. This makes agent runs auditable and replayable across the team. Example prompt: Use Jentic to call Langfuse and write a trace for the current agent run, including each tool call as a span, then attach a final score after the user confirms the result. ## Key endpoints | Method | Path | Description | | --- | --- | --- | | GET | `/api/public/annotation-queues` | List annotation queues. | | GET | `/api/public/annotation-queues/{queueId}/items` | List items waiting in an annotation queue. | | GET | `/api/public/comments` | List comments attached to traces or observations. | | GET | `/api/public/dataset-items` | List dataset items used in evaluation runs. | ## Key resources - **Traces** — Top-level run records grouping observations and generations. - **Observations** — Individual steps inside a trace, including spans and events. - **Generations** — LLM call records with prompt, model, output, and token usage. - **Prompts** — Versioned prompt registry with labels and rollback. - **Datasets** — Evaluation datasets and items used for scored runs. - **Scores** — Quality and evaluator scores attached to traces or generations. - **Annotation Queues** — Queues of items routed for human-in-the-loop review. - **Projects** — Project-level admin and configuration. ## Why Jentic - **Setup:** Wiring Langfuse by hand means setting up its basic auth from a public and secret key pair, targeting your Langfuse host, and pointing the SDK at the right endpoints yourself. Through Jentic you install once, import Langfuse from the API Directory, store the key pair once, and your agent calls it. - **Permission scoping:** You choose which Langfuse operations the agent may call, so you can limit it to the ones it needs, such as listing annotation queues, reading comments, or fetching dataset items. Granting read operations lets the agent inspect traces and evaluations without gaining the ability to write scores or change prompts. - **Credential handling:** Your Langfuse public and secret keys are stored once, encrypted, by your own Jentic One instance and injected at execution time. They never enter the agent's prompt, logs, or context. - **Discovery method:** Agents search Jentic by intent such as 'log an LLM trace' or 'list annotation queue items', and Jentic returns the matching Langfuse operation with its input schema so the agent calls the right endpoint without wiring the Langfuse SDK. ## Related APIs - **Arize** — Arize provides ML and LLM observability with a focus on production monitoring and drift. - **OpenAI** — OpenAI generates the model calls that Langfuse records and evaluates. - **Anthropic Messages** — Anthropic Messages produces Claude responses that Langfuse traces alongside other model providers. ## FAQ ### What authentication does the Langfuse API use? Langfuse uses HTTP Basic auth with a public key as the username and a secret key as the password, both issued per project. Through Jentic, both keys live in the encrypted vault and are injected at execution time so they never appear in the agent context. ### Can I log LLM traces with the Langfuse API? Yes. Create a trace, attach observations and generations under the same trace_id, then post scores to grade the run. The endpoints under `/api/public` cover the full ingest path used by the official SDKs. ### Can I manage prompts with the Langfuse API? Yes. The API exposes versioned prompts with labels so you can fetch the active 'production' version of a named prompt at runtime, push a new version, or roll back to a prior one without a code redeploy. ### What are the rate limits for the Langfuse API? Langfuse Cloud applies plan-based rate limits that the OpenAPI spec does not enumerate. Self-hosted deployments have no fixed limits beyond the underlying database. Check the Langfuse plan dashboard for current Cloud limits. ### How do I run an evaluation through Jentic? Search Jentic for 'run llm evaluation', load the dataset and run operations, then POST dataset run items pointing at your candidate prompt. Scores attached to each run are returned for comparison. ### Is the Langfuse API free? Langfuse is open source and self-hostable for free. Langfuse Cloud has a free tier with paid plans for higher trace volume and longer retention. ### Can I limit what my agent is allowed to do with the Langfuse API? Yes. Because you run Jentic One yourself, your own rules decide which Langfuse operations and credentials the agent can use, so you can restrict it to just the endpoints it needs, such as listing annotation queues, reading comments, or fetching dataset items. Granting only read operations lets the agent inspect traces and evaluations while withholding the ability to write scores or change prompts. The public and secret key pair is stored by your own instance and injected at execution time, so the agent never handles the raw credentials.