canonical: https://jentic.com/apis/googleapis.com/vision # Google Cloud Vision API The Cloud Vision API performs image and PDF analysis including label detection, OCR, face detection, landmark and logo recognition, explicit content (SafeSearch) detection, object localization, and product search. Requests can analyze images inline or stored in Cloud Storage and can be batched synchronously or run asynchronously for large PDF and TIFF documents. It powers content moderation, document digitization, and visual search workloads. ## For AI agents Run OCR, label detection, face detection, landmark recognition, and explicit content checks on images and PDFs so an agent can extract structured data from visual content. ## Scope Does not handle video annotation, image generation, or image editing - use for image and PDF analysis (OCR, labels, faces, logos, SafeSearch, product search) only. ## Capabilities - Extract typed and handwritten text from images and PDFs with full OCR layout - Detect object labels and bounding boxes for inventory and content tagging - Recognize landmarks, logos, and well-known products in user-supplied images - Score images for adult, violent, racy, medical, and spoof content via SafeSearch - Run async batch annotation on large PDF and TIFF documents in Cloud Storage - Match a query image against a custom product catalog for visual search ## Use cases ### Document Digitization and OCR Operations and back-office teams convert scanned invoices, contracts, and forms into searchable text using the Vision API's DOCUMENT_TEXT_DETECTION feature. The API returns the full hierarchical layout of pages, blocks, paragraphs, words, and symbols with confidence scores, which the digitization pipeline maps into structured records. Asynchronous batch endpoints handle large multi-page PDFs without blocking caller threads. Example prompt: Submit a files:asyncBatchAnnotate with DOCUMENT_TEXT_DETECTION on a 200-page PDF in gs://invoices/q3.pdf ### User-Generated Image Moderation Marketplaces and social platforms screen every user-uploaded photo with SafeSearch and label detection before publishing. The images:annotate endpoint returns likelihood ratings for adult, violent, racy, medical, and spoof content alongside detected labels, letting the moderation pipeline auto-block clear violations and route ambiguous cases to human review. Synchronous mode keeps response times suitable for upload flows. Example prompt: Annotate an image with SAFE_SEARCH_DETECTION and reject if adult or violent likelihood is LIKELY or VERY_LIKELY ### Retail Visual Search Retailers index their product catalog into a Vision Product Search corpus, then accept query images at runtime to return visually similar SKUs ranked by similarity. The productSearch annotate path matches against the configured product set and returns matching product IDs with bounding boxes for each detected object in the query image. This powers in-app 'find similar products' features without training a custom model. Example prompt: Annotate a customer photo with PRODUCT_SEARCH and return the top three matching SKUs from the home goods product set ### AI Agent Image Understanding An AI agent integrated through Jentic answers prompts like 'what is in this photo?' or 'is this image safe to publish?' by discovering the Vision API by intent search, calling images:annotate with the relevant feature set, and returning the structured response. Because the API uses OAuth 2.0 with the cloud-platform scope, Jentic isolates the token in your Jentic One instance and exposes only a scoped reference. Example prompt: Search Jentic for analyze an image, load the schema, and call images:annotate with LABEL_DETECTION and SAFE_SEARCH_DETECTION ## Key endpoints | Method | Path | Description | | --- | --- | --- | | POST | /v1/images:annotate | Synchronous batch annotation of one or more images | | POST | /v1/images:asyncBatchAnnotate | Asynchronous batch annotation for large image sets | | POST | /v1/files:annotate | Synchronous annotation of multi-page PDF and TIFF files | | POST | /v1/files:asyncBatchAnnotate | Asynchronous annotation of large PDF and TIFF documents in Cloud Storage | ## Key resources - **Images** — Synchronous and asynchronous annotation of images for labels, OCR, faces, logos, landmarks, and SafeSearch - **Files** — Synchronous and asynchronous annotation of multi-page PDF and TIFF documents in Cloud Storage - **Operations** — Track long-running async annotation operations from submission through completion - **Projects** — Project-scoped product search resources for visual catalog matching ## Why Jentic - **Setup:** Wiring the Cloud Vision API by hand means setting up a Google service account, granting the cloud-platform or cloud-vision scope, assembling the feature list and image source, and handling async batch operations yourself. Through Jentic you install once, import the Cloud Vision API from the API Directory, store the service-account credential once, and your agent calls it. - **Permission scoping:** The image or file and requested features travel in the request body, so scope by operation: limit the agent to the operations it needs, such as images:annotate and files:annotate. You choose the operations it may call, so the async batch operations are not included unless you add them. - **Credential handling:** Your Cloud Vision service-account credential is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context. - **Discovery method:** Agents search Jentic by intent such as 'extract text from an image' or 'detect labels', and Jentic returns the images:annotate operation with its input schema, including the feature enum and image source options, so the agent calls the right endpoint without browsing the reference docs. ## Related APIs - **Cloud Video Intelligence API** — Per-frame and per-shot annotation for video; Vision is the still-image equivalent - **Cloud Translation API** — Translate text extracted by Vision OCR into other languages - **Cloud Storage API** — Stage images and PDFs in a bucket so Vision can read them by URI - **Sensitive Data Protection (DLP) API** — Scan OCR output for PII and PHI before storing or surfacing it ## FAQ ### What authentication does the Cloud Vision API use? The Cloud Vision API uses OAuth 2.0 with the https://www.googleapis.com/auth/cloud-platform or cloud-vision scope. Through Jentic, the OAuth token is stored encrypted in your Jentic One instance and only a scoped reference is exposed to the agent at execution time. ### Can I run OCR on a multi-page PDF with the Cloud Vision API? Yes. Use POST /v1/files:asyncBatchAnnotate with DOCUMENT_TEXT_DETECTION to OCR a PDF or TIFF stored in Cloud Storage. The async endpoint returns an operation name; the final annotation result is written to a Cloud Storage destination you specify in the request. ### What are the rate limits for the Cloud Vision API? Default project quotas allow 1,800 requests per minute and 16 images per request, with feature-specific image-size and PDF-page caps. Higher quotas can be requested in the Google Cloud Console; pricing is per feature per image. ### How do I detect explicit content in an image through Jentic with the Cloud Vision API? Install Jentic with pip install jentic, search for detect explicit content in image, load the schema for POST /v1/images:annotate, then call it with features set to SAFE_SEARCH_DETECTION and the image source as either a Cloud Storage URI or base64 content. The response includes adult, violent, racy, medical, and spoof likelihood values. ### Does the Cloud Vision API support handwriting recognition? Yes. Use DOCUMENT_TEXT_DETECTION rather than the simpler TEXT_DETECTION feature; DOCUMENT_TEXT_DETECTION is tuned for dense text and handwriting and returns full document layout. Accuracy depends on legibility, contrast, and language. ### Why does my Vision API request return INVALID_ARGUMENT for an inline image? Inline image content must be valid base64-encoded bytes under the 10 MB request limit, and the image format must be one of JPEG, PNG, GIF, BMP, WEBP, RAW, ICO, PDF, or TIFF. For larger files, upload to Cloud Storage and pass the gs:// URI in image.source.imageUri instead. ### Can I limit what my agent is allowed to do with the Cloud Vision API? Yes. Because Jentic One is self-hosted by you, your own rules decide which Cloud Vision operations and credentials the agent may use. Since the image or file and the requested features travel in the request body, you scope by operation and can grant the agent only what it needs, such as images:annotate and files:annotate for synchronous OCR, label, and SafeSearch checks. The async batch operations like images:asyncBatchAnnotate and files:asyncBatchAnnotate are excluded unless you explicitly add them, and the service-account credential is injected at execution time rather than exposed to the agent.