LiteLLM API
Official vendor OpenAPI document · agent-readyAI/MLLanguage ModelsapiKey703 EndpointsREST
For Agents
Run chat completions, text completions, embeddings, and moderations against configured models through a LiteLLM proxy, and manage virtual keys. Covers 703 endpoints behind an OpenAI-compatible interface.
Use for: Run a chat completion against a configured model, Generate embeddings for a batch of text, Moderate a piece of user-supplied content, List the models available on the proxy
Not supported: Does not host the underlying models itself; it routes to whichever providers the proxy is configured for. Use for calling models and managing proxy keys only.
The LiteLLM API is a proxy server that exposes a wide range of language model providers through a single OpenAI-compatible interface, covering chat completions, embeddings, moderations, audio, and key management over a REST interface. It lets you send a request in the OpenAI format and have the proxy route it to the configured model, generate embeddings, moderate content, and manage virtual keys. Requests carry an API key in a header and return structured JSON your application can act on.
Install Jentic One Beta
Connect the LiteLLM API to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the LiteLLM API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Flitellm.ai%2Flitellm" | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Flitellm.ai%2Flitellm" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with LiteLLM API.
Run chat completions against any model the proxy is configured for
Run text completions and generate embeddings
Moderate content through the moderations endpoint
Generate speech audio from text
List the models available on the proxy
Generate and manage virtual API keys for the proxy
Use Cases
Patterns agents use LiteLLM API for, with concrete tasks.
★ Model-Agnostic AI Agent via Jentic
An AI agent can call many model providers through one LiteLLM proxy without a developer wiring each provider by hand. Through Jentic the agent searches for the chat completion operation by intent, receives the endpoint and its input schema, and calls it with the proxy key injected at execution time. Swapping the underlying model becomes a proxy configuration change rather than a code change.
Search Jentic for 'run a chat completion', load the operation schema, and call POST /chat/completions with the messages and target model
Embeddings Pipeline
Retrieval systems need embeddings for the text they index and query. The LiteLLM proxy generates embeddings through an OpenAI-compatible endpoint, so a pipeline can embed documents and queries against whichever provider the proxy routes to. This keeps the embedding call stable even when the backing model changes.
Call POST /embeddings for each batch of documents, then store the returned vectors for retrieval
Governed Key Management
Platform teams that share a proxy across many users need to issue and revoke keys per team. The LiteLLM API generates virtual keys through the proxy, so an onboarding flow can mint a scoped key for each team automatically. This centralizes model access behind keys the platform controls.
Call POST /key/generate to mint a virtual key for a team, then store it for that team's requests
Key Endpoints
703 endpoints — the litellm api is a proxy server that exposes a wide range of language model providers through a single openai-compatible interface, covering chat completions, embeddings, moderations, audio, and key management over a rest interface.
METHOD
PATH
DESCRIPTION
POST
/chat/completions
Run a chat completion
POST
/completions
Run a text completion
POST
/embeddings
Generate embeddings
POST
/moderations
Moderate content
POST
/audio/speech
Generate speech audio from text
GET
/models
List available models
POST
/key/generate
Generate a virtual API key
POST
/chat/completions
Run a chat completion
POST
/completions
Run a text completion
POST
/embeddings
Generate embeddings
POST
/moderations
Moderate content
POST
/audio/speech
Generate speech audio from text
GET
/models
List available models
POST
/key/generate
Generate a virtual API key
Why Jentic?
What agents get from Jentic-routed access to this vendor.
Setup
Wiring a LiteLLM proxy by hand means setting the x-litellm-api-key header on every request and pointing your code at the right proxy host yourself. Through Jentic you install once, import LiteLLM from the API Directory, store the key once, and your agent calls it.
Permission scoping
LiteLLM exposes hundreds of operations, so scoping is by operation: a rule can allow only inference such as POST /chat/completions and POST /embeddings while withholding key and model management. You choose which operations the agent may call, so an inference-only agent never mints or revokes keys.
Credential isolation
Your LiteLLM proxy key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'run a chat completion' or 'generate embeddings', and Jentic returns the matching LiteLLM operation with its input schema so the agent calls the right endpoint without browsing the reference docs.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using LiteLLM API through Jentic.
What authentication does the LiteLLM API use?
The LiteLLM proxy authenticates with an API key sent in the x-litellm-api-key header, as declared in its OpenAPI spec. Through Jentic the key is stored encrypted by your own Jentic One instance and injected at execution time, so it never enters the agent's prompt, logs, or context.
Can I call different models through the LiteLLM API?
Yes. Send a request in the OpenAI format to POST /chat/completions and name the target model, and the proxy routes it to whichever provider is configured for that model. Call GET /models to see which models the proxy currently exposes.
What are the rate limits for the LiteLLM API?
The OpenAPI spec does not specify rate limits, since they depend on how each proxy is configured. Check the LiteLLM documentation at https://docs.litellm.ai for how limits are set before running high-volume workloads.
How do I run a chat completion through Jentic?
Search Jentic for 'run a chat completion', which resolves to the POST /chat/completions operation, and Jentic returns its input schema so your agent can submit messages and a model. Credentials are injected at call time from your own instance. To run it on your own infrastructure, install Jentic One from its GitHub repo.
Can I restrict what my agent is allowed to do with the LiteLLM API?
Yes. Write a rule that allows only the inference operations your agent needs, such as POST /chat/completions and POST /embeddings, so it can run models but cannot generate or revoke keys, and every call it makes is logged by your own instance. You add key-management operations to the allowed set only when you decide to.
Is there a LiteLLM MCP server?
You don't need an MCP server to give your agent a LiteLLM proxy. Jentic connects it directly from the API Directory: import LiteLLM, store your key once, and your agent calls the completion, embedding, and key operations on demand without loading another server's tool definitions into its context.
GET STARTED