Google Datastream API
Browse all Google APIs43
AI ReadinessFoundational (D-)43/100
Official vendor OpenAPI document · agent-readyAnalyticsData Pipelinesoauth221 EndpointsREST
For Agents
Set up and operate change-data-capture streams from Oracle, MySQL, PostgreSQL, and SQL Server into BigQuery and Cloud Storage. Lets agents drive CDC pipelines without standing up custom replication.
Use for: I need to stream MySQL changes to BigQuery in near real time, Create a source connection profile for an Oracle database, Start a backfill job on a Datastream stream, Pause a Datastream stream during a maintenance window
Not supported: Does not run database queries, transform data in flight beyond per-row mapping, or replicate to non-Google destinations - use for serverless CDC from operational databases into BigQuery and Cloud Storage only.
Google Cloud Datastream is a serverless change-data-capture (CDC) and replication service that streams changes from operational databases (Oracle, MySQL, PostgreSQL, SQL Server) into BigQuery, Cloud Storage, and other destinations. The API exposes 21 endpoints covering connection profiles for source and destination, streams, private connectivity for VPC peering, and routes. It supports continuous replication with backfill, schema discovery, and per-stream pause/resume.
Install Jentic One Beta
Connect the Datastream API to your agent
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Datastream API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
1
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fgoogleapis.com%2Fdatastream" | sh2
Step 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fgoogleapis.com%2Fdatastream" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
Capabilities
What an agent can do with Datastream API.
Create source connection profiles for Oracle, MySQL, PostgreSQL, and SQL Server
Create destination connection profiles for BigQuery and Cloud Storage
Run CDC streams with optional historical backfill on tables
Pause and resume streams to control downstream load
Discover source schema and tables before starting a stream
Configure private connectivity and routes for VPC-peered sources
Use Cases
Patterns agents use Datastream API for, with concrete tasks.
★ Operational-Database to BigQuery CDC
Stream changes from Oracle, MySQL, PostgreSQL, or SQL Server into BigQuery with minutes of latency, plus an initial backfill of historical rows. The API creates source and destination connection profiles, then a stream that reads source binlogs/redo logs and lands change events in BigQuery tables. Setup is typically hours rather than the weeks needed for a custom CDC pipeline.
Create source profile 'mysql-prod', destination profile 'bq-warehouse', and stream 'orders-cdc' with backfillAll on; start the stream and report rowsBackfilled per object
Schema Discovery Before Stream Setup
Before launching a stream, call discoverConnectionProfile on the source profile to enumerate available schemas, tables, and columns. The response surfaces primary keys and unsupported types, so the operator can decide which tables to include and whether to add type-mapping overrides. This avoids stream creation failures from invalid include lists.
Call discoverConnectionProfile on 'mysql-prod' with hierarchyDepth 2, then return the full list of schemas and tables to the operator for selection
Backfill and Pause-Resume Operations
Trigger a backfill on individual stream objects after a schema change, and pause and resume streams to coordinate with downstream maintenance. POST /v1/{+object}:startBackfillJob and :stopBackfillJob give per-table control without restarting the whole stream. Pause/resume on the parent stream is exposed via the patch endpoint with the state field.
Call startBackfillJob on stream object 'orders-cdc/orders' to replay history after adding a column, then resume CDC on the parent stream
AI Agent CDC Operator
An AI agent can stand up and operate Datastream pipelines through Jentic without operator-written replication code. Jentic search returns the matching connection-profile, stream, or backfill operation, the agent loads the schema, and Jentic executes against datastream.googleapis.com using vault-stored credentials. This compresses the multi-day setup of OAuth and Datastream resources into one agent run.
Use Jentic to search 'create a datastream stream', load the createStream schema, and execute it with the configured source and destination profiles plus an includeObjects table list
Key Endpoints
21 endpoints — google cloud datastream is a serverless change-data-capture (cdc) and replication service that streams changes from operational databases (oracle, mysql, postgresql, sql server) into bigquery, cloud storage, and other destinations.
METHOD
PATH
DESCRIPTION
POST
/v1/{+parent}/connectionProfiles
Create a source or destination connection profile
POST
/v1/{+parent}/streams
Create a Datastream stream
POST
/v1/{+name}:run
Start a stream
POST
/v1/{+object}:startBackfillJob
Start backfill on a stream object
POST
/v1/{+object}:stopBackfillJob
Stop backfill on a stream object
GET
/v1/{+name}:fetchStaticIps
Fetch Datastream egress IPs for source allowlisting
POST
/v1/{+parent}/connectionProfiles
Create a source or destination connection profile
POST
/v1/{+parent}/streams
Create a Datastream stream
POST
/v1/{+name}:run
Start a stream
POST
/v1/{+object}:startBackfillJob
Start backfill on a stream object
POST
/v1/{+object}:stopBackfillJob
Stop backfill on a stream object
GET
/v1/{+name}:fetchStaticIps
Fetch Datastream egress IPs for source allowlisting
Jentic AI Readiness Score
This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.
D
Datastream API
- Foundational (D-)43/100
58
Foundational Compliance56
Developer Experience & Jentic Compatibility20
AI-Readiness & Agent Experience94
Agent Usability63
Security64
AI DiscoverabilityPowered by JenticScoring Framework 1.0.0 | Scoring Engine 0.4.0
Show dimension breakdown
58
Foundational Compliance
Base layer of spec validity and structural soundness.
Grade: C+Signals: 4
59%
Lint Results
Aggregated quality score from linter diagnostics, weighted by severity.
100%
Resolution Completeness
Percentage of `$ref` references that resolve successfully.
0%
Specification Validity
Checks whether the API description parses successfully and conforms to its declared specification (e.g., OpenAPI).
75%
Structural Integrity
Structural correctness score based on schema issues using logarithmic dampening.
56
Developer Experience & Jentic Compatibility
Clarity, completeness, and ingestion readiness for developers and tooling.
Grade: CSignals: 4
0%
Example Density
How richly the API is illustrated with examples.
100%
Example Validity
Percentage of examples that conform to their schemas.
25%
Response Coverage
Percentage of operations with complete response definitions (success, client error, server error).
100%
Tooling Readiness
Health of API ingestion, bundling, and resolution within Jentic pipelines.
20
AI-Readiness & Agent Experience
Semantic breadth, depth, and agent comprehension for AI systems.
Grade: FSignals: 4
81%
Description Coverage
Coverage of descriptions across API elements.
0%
Error Standardization
Coverage of RFC 9457 Problem Details for error responses.
0%
OperationId Quality
Coverage, uniqueness, and casing consistency of operationIds for AI inference.
0%
Summary Coverage
Coverage of summaries across operations/tags/info.
94
Agent Usability
Functional utility, complexity comfort, and AI orchestration readiness.
Grade: A+Signals: 1
94%
Complexity Comfort
Agent comfort level based on API operational and structural complexity.
63
Security
Trust, risk posture, and security compliance.
Grade: B-Signals: 1
62%
Authentication Strength
Average quality of security schemes based on authentication method strength (weakest link for OAuth2).
64
AI Discoverability
Findability, semantic richness, and reasoning readiness.
Grade: BSignals: 1
64%
Descriptive Richness
Clarity and depth of descriptions across API elements.
Score it yourself
Every API in the directory is allowlisted, so you can re-score it with no key required.
npx @jentic/api-scorecard-cli score <openapi-url>Why Jentic?
What agents get from Jentic-routed access to this vendor.
Setup
Wiring the Datastream API by hand means setting up Google OAuth, refreshing short-lived scoped tokens instead of holding client secrets in code, and addressing connection profile and stream resources on datastream.googleapis.com. Through Jentic you install once, import the Datastream API from the API Directory, store the Google credential once, and your agent calls it.
Permission scoping
The API carries the parent and resource name in the URL path (/v1/{parent}/streams and /v1/{name}:run), so a rule can pin your agent to one stream or connection profile. You choose the operations it may call, so it can create and run a stream and start a backfill while stopping a backfill is not included unless you add it.
Credential isolation
Your Google OAuth credential is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'create a Datastream stream', and Jentic returns the matching Datastream operation with its input schema so the agent calls the right endpoint without browsing the reference docs.
Related APIs
Alternatives and complements available in the Jentic catalogue.
FAQs
Specific to using Datastream API through Jentic.
What authentication does the Datastream API use?
The Datastream API uses OAuth 2.0 with the cloud-platform scope. Through Jentic the OAuth client and refresh tokens are stored in your Jentic One instance and the agent receives short-lived scoped access tokens, so raw Google credentials never enter the agent context.
Can I stream Oracle changes to BigQuery with Datastream?
Yes. Create a source connection profile of type ORACLE, a destination connection profile of type BIGQUERY, then a stream that references both. Datastream uses Oracle LogMiner to capture changes and writes per-table change events to BigQuery with seconds-to-minutes lag.
What are the rate limits for the Datastream API?
Google enforces standard Cloud quotas on datastream.googleapis.com: per-project rate limits on read/write calls and per-region quotas on concurrent streams and connection profiles. Quotas are visible in the Cloud Console under IAM and admin, quotas, filtered to datastream.googleapis.com.
How do I allowlist Datastream's source IPs?
Call GET /v1/{+name}:fetchStaticIps on the location resource. The response returns the egress IP ranges Datastream uses for IP-allowlist source connectivity in that region; add those to the source database's network allowlist before creating the stream.
Is the Datastream API free?
API calls are free. Datastream is billed per GiB of changed data captured plus a small per-stream operating fee. Underlying BigQuery storage and Cloud Storage destinations are billed at standard rates.
How do I backfill a single table after a schema change?
Call POST /v1/{+object}:startBackfillJob on the stream object that represents the table. Datastream replays historical rows for that object only, leaving CDC running on the rest of the stream. Stop a runaway backfill with :stopBackfillJob.
Can I limit what my agent is allowed to do with the Datastream API?
Yes. Because you run Jentic One yourself, your own rules decide which Datastream operations and which Google credential the agent may use. Since the parent and resource name travel in the URL path, such as /v1/{parent}/streams and /v1/{name}:run, you can pin the agent to a single stream or connection profile. You might allow it to create and run a stream and start a backfill while withholding stopBackfillJob unless you explicitly add it.
GET STARTED