canonical: https://jentic.com/apis/groq.com/groq

# Groq API

Groq runs open-weight large language models on its custom LPU inference hardware, exposing an OpenAI-compatible API surface. Agents can run chat completions, list available models, transcribe and translate audio, and create text embeddings - all through the same OpenAI v1 paths so existing OpenAI client code can target Groq with a base URL change. The API authenticates with a bearer token issued from the Groq console.

## For AI agents

Run chat completions, audio transcription, translation, and embeddings on Groq's low-latency LPU inference using OpenAI-compatible endpoints.

## Scope

Does not handle image generation, fine-tuning, or assistants/threads - use for chat completions, audio transcription/translation, and embeddings only.

## Capabilities

- Run chat completions with POST /openai/v1/chat/completions
- List the models available on Groq via GET /openai/v1/models
- Inspect a single model with GET /openai/v1/models/{model}
- Transcribe audio to text with POST /openai/v1/audio/transcriptions
- Translate audio into English text with POST /openai/v1/audio/translations
- Create text embeddings with POST /openai/v1/embeddings

## Use cases

### Low-Latency Chat Inference

Run high-throughput chat completions when latency matters more than the largest possible context window. Agents call POST /openai/v1/chat/completions with the same payload shape they would send to OpenAI; Groq's LPU hardware typically returns tokens at far higher tokens-per-second than GPU inference, useful for live agent loops.

Example prompt: Call POST /openai/v1/chat/completions with model=llama-3.3-70b-versatile and the user's messages, then return the assistant message content.

### Audio Transcription Pipelines

Transcribe meeting recordings or voice notes with Groq-hosted Whisper. POST /openai/v1/audio/transcriptions accepts an audio file and returns the transcript, while /audio/translations returns an English translation. Useful for batch transcription where speed and cost both matter.

Example prompt: Upload the audio file to POST /openai/v1/audio/transcriptions with model=whisper-large-v3 and persist the returned transcript to the case record.

### Embeddings for Semantic Search

Generate vector embeddings for documents and queries using Groq's embeddings endpoint, then store them in a vector database for retrieval. POST /openai/v1/embeddings returns the same response shape as OpenAI's embeddings endpoint, so the consuming code is largely identical.

Example prompt: Call POST /openai/v1/embeddings for each chunk of a document with the chosen embedding model, then upsert the vectors into the project's vector store.

### AI Agent Model Routing

An agent that routes tasks to the cheapest viable model uses Jentic to call Groq for low-latency steps and falls back to other providers when needed. Jentic's intent search returns the right Groq operation by description, so the agent does not need to hard-code OpenAI-compatible paths in multiple code paths.

Example prompt: Search Jentic for 'run a chat completion on Groq', load the POST /openai/v1/chat/completions schema, and execute it with the chosen Groq model name.

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | /openai/v1/chat/completions | Run a chat completion |
| GET | /openai/v1/models | List available models |
| GET | /openai/v1/models/{model} | Get a single model's metadata |
| POST | /openai/v1/audio/transcriptions | Transcribe audio to text |
| POST | /openai/v1/audio/translations | Translate audio to English text |
| POST | /openai/v1/embeddings | Create text embeddings |

## Key resources

- **chat/completions** — Run streaming or non-streaming chat completions
- **models** — List and inspect models hosted on Groq
- **audio/transcriptions** — Transcribe audio files to text
- **audio/translations** — Translate non-English audio files to English text
- **embeddings** — Create vector embeddings from text

## Why Jentic

- **Setup:** Wiring the Groq API by hand means handling its bearer auth against api.groq.com and hard-coding the OpenAI-compatible paths for chat, audio, and embeddings yourself, along with retries. Through Jentic you install once, import Groq from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** Groq sends the target model in the request body rather than the URL path, so scope it by operations: limit the agent to the operations it needs, such as chat completions or audio transcription, and leave translation or embeddings out of the allowed set. That way the agent can only run the calls you selected.
- **Credential handling:** Your Groq API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context, which matters when many agent operations share the same key.
- **Discovery method:** Agents search Jentic by intent such as 'run a chat completion' or 'transcribe audio', and Jentic returns the matching Groq operation with its input schema so the agent calls the right endpoint without hard-coding URLs.

## Related APIs

- **OpenAI API** — Closed-weight frontier models on the same OpenAI-compatible surface Groq mirrors.
- **Anthropic Messages API** — Claude family of models with a different request shape than OpenAI's.
- **Mistral API** — Mistral's hosted models on its own API surface.

## FAQ

### What authentication does the Groq API use?

Groq uses an HTTP bearer token issued from the Groq console. Through Jentic, the key is stored encrypted in your Jentic One instance and the executor injects the Authorization: Bearer header at call time, so the raw key never enters the agent's context window.

### Is the Groq API OpenAI-compatible?

Yes. The paths are mounted under /openai/v1 (e.g. /openai/v1/chat/completions, /openai/v1/embeddings) and the request and response bodies match OpenAI's. Existing OpenAI client code can usually target Groq with only a base URL change.

### What are the rate limits for the Groq API?

The OpenAPI spec does not publish explicit rate limits - Groq enforces them per API key based on the account tier. Production agents should retry on 429 with exponential backoff and respect any Retry-After header.

### How do I run a chat completion on Groq through Jentic?

Search Jentic for 'run a chat completion on Groq', load the POST /openai/v1/chat/completions schema, and execute it with a model name and messages array. Jentic injects the bearer token from the vault.

### Can I transcribe audio with the Groq API?

Yes. POST /openai/v1/audio/transcriptions accepts an audio file and a model name (Groq hosts Whisper variants) and returns the transcript. POST /openai/v1/audio/translations returns an English translation of non-English audio.

### Which models can I call on Groq?

Call GET /openai/v1/models to retrieve the current list - Groq updates the available models periodically and the spec does not pin specific names. GET /openai/v1/models/{model} returns the metadata for a single model.

### Can I limit what my agent is allowed to do with the Groq API?

Yes. Because you self-host Jentic One, your own rules decide which Groq operations and credentials the agent may use. Groq sends the target model in the request body rather than the URL, so you scope by operation: allow only the calls the agent needs, such as chat completions on POST /openai/v1/chat/completions or audio transcription on POST /openai/v1/audio/transcriptions, and leave translation or embeddings out of the allowed set. The agent can then only run the operations you selected.