canonical: https://jentic.com/apis/litellm.ai/litellm

# LiteLLM API

The LiteLLM API is a proxy server that exposes a wide range of language model providers through a single OpenAI-compatible interface, covering chat completions, embeddings, moderations, audio, and key management over a REST interface. It lets you send a request in the OpenAI format and have the proxy route it to the configured model, generate embeddings, moderate content, and manage virtual keys. Requests carry an API key in a header and return structured JSON your application can act on.

## For AI agents

Run chat completions, text completions, embeddings, and moderations against configured models through a LiteLLM proxy, and manage virtual keys. Covers 703 endpoints behind an OpenAI-compatible interface.

## Scope

Does not host the underlying models itself; it routes to whichever providers the proxy is configured for. Use for calling models and managing proxy keys only.

## Capabilities

- Run chat completions against any model the proxy is configured for
- Run text completions and generate embeddings
- Moderate content through the moderations endpoint
- Generate speech audio from text
- List the models available on the proxy
- Generate and manage virtual API keys for the proxy

## Use cases

### Model-Agnostic AI Agent via Jentic

An AI agent can call many model providers through one LiteLLM proxy without a developer wiring each provider by hand. Through Jentic the agent searches for the chat completion operation by intent, receives the endpoint and its input schema, and calls it with the proxy key injected at execution time. Swapping the underlying model becomes a proxy configuration change rather than a code change.

Example prompt: Search Jentic for 'run a chat completion', load the operation schema, and call POST `/chat/completions` with the messages and target model

### Embeddings Pipeline

Retrieval systems need embeddings for the text they index and query. The LiteLLM proxy generates embeddings through an OpenAI-compatible endpoint, so a pipeline can embed documents and queries against whichever provider the proxy routes to. This keeps the embedding call stable even when the backing model changes.

Example prompt: Call POST /embeddings for each batch of documents, then store the returned vectors for retrieval

### Governed Key Management

Platform teams that share a proxy across many users need to issue and revoke keys per team. The LiteLLM API generates virtual keys through the proxy, so an onboarding flow can mint a scoped key for each team automatically. This centralizes model access behind keys the platform controls.

Example prompt: Call POST `/key/generate` to mint a virtual key for a team, then store it for that team's requests

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/chat/completions` | Run a chat completion |
| POST | `/completions` | Run a text completion |
| POST | `/embeddings` | Generate embeddings |
| POST | `/moderations` | Moderate content |
| POST | `/audio/speech` | Generate speech audio from text |
| GET | `/models` | List available models |
| POST | `/key/generate` | Generate a virtual API key |

## Key resources

- **Chat Completions** — Run chat completions against configured models
- **Embeddings** — Generate vector embeddings for text
- **Moderations** — Screen content through the moderations endpoint
- **Audio** — Generate speech audio from text
- **Keys** — Generate and manage virtual proxy keys

## Why Jentic

- **Setup:** Wiring a LiteLLM proxy by hand means setting the x-litellm-api-key header on every request and pointing your code at the right proxy host yourself. Through Jentic you install once, import LiteLLM from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** LiteLLM exposes hundreds of operations, so scoping is by operation: a rule can allow only inference such as POST `/chat/completions` and POST /embeddings while withholding key and model management. You choose which operations the agent may call, so an inference-only agent never mints or revokes keys.
- **Credential handling:** Your LiteLLM proxy key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'run a chat completion' or 'generate embeddings', and Jentic returns the matching LiteLLM operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **OpenRouter** — Hosted router that exposes many model providers through one API
- **OpenAI** — A model provider the proxy can route requests to
- **Anthropic** — A model provider the proxy can route requests to

## FAQ

### What authentication does the LiteLLM API use?

The LiteLLM proxy authenticates with an API key sent in the x-litellm-api-key header, as declared in its OpenAPI spec. Through Jentic the key is stored encrypted by your own Jentic One instance and injected at execution time, so it never enters the agent's prompt, logs, or context.

### Can I call different models through the LiteLLM API?

Yes. Send a request in the OpenAI format to POST `/chat/completions` and name the target model, and the proxy routes it to whichever provider is configured for that model. Call GET /models to see which models the proxy currently exposes.

### What are the rate limits for the LiteLLM API?

The OpenAPI spec does not specify rate limits, since they depend on how each proxy is configured. Check the LiteLLM documentation at https://docs.litellm.ai for how limits are set before running high-volume workloads.

### How do I run a chat completion through Jentic?

Search Jentic for 'run a chat completion', which resolves to the POST `/chat/completions` operation, and Jentic returns its input schema so your agent can submit messages and a model. Credentials are injected at call time from your own instance. To run it on your own infrastructure, install Jentic One from its GitHub repo.

### Can I restrict what my agent is allowed to do with the LiteLLM API?

Yes. Write a rule that allows only the inference operations your agent needs, such as POST `/chat/completions` and POST /embeddings, so it can run models but cannot generate or revoke keys, and every call it makes is logged by your own instance. You add key-management operations to the allowed set only when you decide to.

### Is there a LiteLLM MCP server?

You don't need an MCP server to give your agent a LiteLLM proxy. Jentic connects it directly from the API Directory: import LiteLLM, store your key once, and your agent calls the completion, embedding, and key operations on demand without loading another server's tool definitions into its context.