canonical: https://jentic.com/apis/amazonaws.com/rekognition

# AWS Amazon Rekognition

Jentic publishes the only available OpenAPI specification for Amazon Rekognition, keeping it validated and agent-ready. Rekognition is a managed computer-vision service for image and video understanding: face detection, comparison, and search; object, scene, and label detection; text-in-image (OCR); content moderation; celebrity recognition; PPE detection; and stream-based video analysis. The API surfaces 65 operations across synchronous image analysis, asynchronous video jobs, face-collection management, custom Rekognition Custom Labels project lifecycle, and live-stream processors. It is the canonical AWS API for adding vision intelligence to applications.

## For AI agents

Detect labels, faces, text, celebrities, and unsafe content in images and video, and search face collections through Amazon Rekognition.

## Scope

Does not generate or edit images, run text-only NLP, or transcribe audio - use for image and video analysis (labels, faces, text, moderation, celebrities, PPE) only.

## Capabilities

- Detect labels, scenes, and objects in an image with bounding boxes and confidence via DetectLabels
- Compare two faces and return a similarity score using CompareFaces for identity verification
- Search a face collection for matches against a probe image with SearchFacesByImage
- Run text-in-image OCR with DetectText, returning words, lines, and bounding geometry
- Detect unsafe or moderated content with DetectModerationLabels for trust-and-safety pipelines
- Start asynchronous video analysis jobs (StartLabelDetection, StartFaceSearch, StartContentModeration) and pull results when SNS signals completion

## Use cases

### Identity Verification with Face Match

Verify that a selfie matches a stored ID photo by calling CompareFaces with both image bytes (or S3 object references) and reading the returned Similarity score. Pair with CreateFaceLivenessSession and GetFaceLivenessSessionResults to defend against spoofing using printed photos or replay video. Used in onboarding flows for fintech, ride-share, and marketplaces.

Example prompt: Call CompareFaces with SourceImage of the selfie bytes and TargetImage of the stored ID photo, then return the FaceMatch with the highest Similarity

### Image Content Moderation

Filter user-generated content by calling DetectModerationLabels on each upload. Rekognition returns a hierarchy of moderation labels (e.g., 'Suggestive' > 'Female Swimwear or Underwear') with confidence scores so policy engines can block, age-gate, or human-review accordingly. Combine with DetectLabels and DetectText for richer signals.

Example prompt: Call DetectModerationLabels with Image.S3Object pointing to the uploaded image and MinConfidence 80, then route based on returned ModerationLabels

### Video Label Detection at Scale

Tag long-form video assets with timestamped labels by starting an async job. StartLabelDetection takes an S3 video object and an SNS topic; when the job completes, GetLabelDetection returns labels with timestamps for every detected object or scene. Useful for media archives, ad insertion, and content discovery.

Example prompt: Call StartLabelDetection with Video.S3Object pointing to the source video and NotificationChannel.SNSTopicArn, then call GetLabelDetection with the returned JobId once the SNS message arrives

### AI Agent Vision Tool via Jentic

A multimodal agent uses Jentic to discover Rekognition operations such as DetectLabels and DetectText, load their schemas, and call them when a tool route requires image understanding. Jentic stores AWS credentials in its vault and signs each call with SigV4, so the agent only handles the image bytes (or S3 reference) and the structured response.

Example prompt: Use Jentic to search 'detect labels in an image with rekognition', load DetectLabels, and execute it with the user's image bytes

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | /#X-Amz-Target=RekognitionService.DetectLabels | Detect labels, scenes, and objects in an image |
| POST | /#X-Amz-Target=RekognitionService.CompareFaces | Compare two faces and return a similarity score |
| POST | /#X-Amz-Target=RekognitionService.DetectText | Run OCR and extract words and lines from an image |
| POST | /#X-Amz-Target=RekognitionService.DetectModerationLabels | Identify unsafe or moderated content categories in an image |
| POST | /#X-Amz-Target=RekognitionService.IndexFaces | Add faces from an image to a face collection |
| POST | /#X-Amz-Target=RekognitionService.SearchFacesByImage | Search a face collection for matches to a probe image |
| POST | /#X-Amz-Target=RekognitionService.StartLabelDetection | Start an asynchronous video label-detection job |
| POST | /#X-Amz-Target=RekognitionService.GetLabelDetection | Retrieve completed video label-detection results by JobId |

## Key resources

- **Image** — Synchronous image analysis: labels, faces, text, moderation, celebrities, PPE
- **Video** — Asynchronous video analysis jobs and result retrieval
- **FaceCollection** — Indexed face collections for search and identity matching
- **Project** — Rekognition Custom Labels project, dataset, and model lifecycle
- **StreamProcessor** — Live video stream processors backed by Kinesis Video

## Why Jentic

- **Setup:** Wiring Rekognition by hand means computing SigV4 HMAC signatures, setting the X-Amz-Target header per action, routing to the regional rekognition host, handling S3 references for image and video inputs, and polling asynchronous video jobs yourself. Through Jentic you install once, import Amazon Rekognition from the API Directory, store your AWS access keys once, and your agent calls it with signing and targeting handled per request.
- **Permission scoping:** The image, video, and collection targets travel in the request body rather than the URL path, so you limit the agent to the operations it needs, such as DetectLabels, DetectText, and DetectModerationLabels for analysis. Operations that mutate a face collection, like IndexFaces or deleting a collection, are included only when you grant them.
- **Credential handling:** Your AWS access keys are stored once, encrypted, by your own Jentic One instance and used to produce a SigV4 signature per request. The raw secret access key never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'detect labels in an image', and Jentic returns DetectLabels with its input schema for Image, MinConfidence, and Features so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Amazon Textract** — Textract specializes in document OCR, forms, and tables; Rekognition handles general scene text and broader visual analysis
- **Amazon Comprehend** — Apply NLP (entities, sentiment, classification) to text extracted from images by Rekognition
- **Amazon Polly** — Narrate Rekognition's image descriptions for accessibility and audio-described media

## FAQ

### Why is there no official OpenAPI spec for Amazon Rekognition?

AWS does not publish an OpenAPI specification. Jentic generates and maintains this spec so that AI agents and developers can call Amazon Rekognition via structured tooling. It is validated against the live API and kept up to date. Get started with Jentic One, the self-hosted execution layer.

### What authentication does the Rekognition API use?

Rekognition uses AWS SigV4 HMAC request signing with an AWS access key ID and secret. Through Jentic, AWS credentials are stored in the vault and SigV4 signatures are produced per request, so the agent never sees raw secrets.

### Can I run Rekognition on video as well as images?

Yes. Synchronous image operations (DetectLabels, DetectFaces, DetectText, etc.) accept Image bytes or an S3 object. Video operations are asynchronous: call StartLabelDetection, StartFaceSearch, StartContentModeration, or StartCelebrityRecognition with a Video.S3Object and NotificationChannel, then call the matching Get* operation with the returned JobId.

### What image formats and sizes are supported?

Rekognition accepts JPEG and PNG. Synchronous calls cap image bytes at 5 MB when passed inline and 15 MB when referenced via S3. Minimum face size for detection is roughly 40x40 pixels. Videos must be H.264 in MP4 or MOV containers and live in S3.

### What are the rate limits for Rekognition?

Rekognition enforces per-region transactions-per-second limits per operation; defaults vary (DetectLabels often starts around 50 TPS) and quota increases can be requested. Throttled calls return ProvisionedThroughputExceededException - back off and retry.

### How do I detect labels in an image through Jentic?

Search Jentic for 'detect labels in an image with rekognition', load DetectLabels, and execute it with Image bytes (or Image.S3Object) and MinConfidence. Install with pip install jentic; AWS credentials are pulled from your Jentic One instance.

### Can I limit what my agent is allowed to do with the Amazon Rekognition API?

Yes. Because you run Jentic One yourself, your own rules decide which Rekognition operations and which AWS credentials the agent may use. You can grant read-only analysis calls such as DetectLabels, DetectText, and DetectModerationLabels while withholding operations that mutate a face collection, like IndexFaces or deleting a collection. The image, video, and collection targets travel in the request body rather than the URL path, so the operator scopes access at the operation level and the agent can only call what you allow.