Datastream API

Name: Datastream API API
Brand: Datastream API
Availability: InStock

✓ Official Vendor SpecAnalyticsData Pipelinesoauth221 EndpointsREST

For Agents

Set up and operate change-data-capture streams from Oracle, MySQL, PostgreSQL, and SQL Server into BigQuery and Cloud Storage. Lets agents drive CDC pipelines without standing up custom replication.

Quickstart

Get started with Datastream API in minutes using your preferred integration method.

# Add to your MCP client config (Claude Desktop, Cursor, Windsurf)
{
  "jentic": {
    "url": "https://api.jentic.com/mcp",
    "auth": "oauth"
  }
}

# Then ask your agent:
"create a datastream cdc stream to bigquery"

# → Jentic returns the GET /events tool with parameter schema, agent executes.

Capabilities

What an agent can do with Datastream API API.

Create source connection profiles for Oracle, MySQL, PostgreSQL, and SQL Server

Create destination connection profiles for BigQuery and Cloud Storage

Run CDC streams with optional historical backfill on tables

Pause and resume streams to control downstream load

GET STARTED

Start building with Datastream API API

Explore with Jentic

View OpenAPI Document

Use for: I need to stream MySQL changes to BigQuery in near real time, Create a source connection profile for an Oracle database, Start a backfill job on a Datastream stream, Pause a Datastream stream during a maintenance window

Not supported: Does not run database queries, transform data in flight beyond per-row mapping, or replicate to non-Google destinations — use for serverless CDC from operational databases into BigQuery and Cloud Storage only.

Google Cloud Datastream is a serverless change-data-capture (CDC) and replication service that streams changes from operational databases (Oracle, MySQL, PostgreSQL, SQL Server) into BigQuery, Cloud Storage, and other destinations. The API exposes 21 endpoints covering connection profiles for source and destination, streams, private connectivity for VPC peering, and routes. It supports continuous replication with backfill, schema discovery, and per-stream pause/resume.

Use Cases

Patterns agents use Datastream API API for, with concrete tasks.

★ Operational-Database to BigQuery CDC

Stream changes from Oracle, MySQL, PostgreSQL, or SQL Server into BigQuery with minutes of latency, plus an initial backfill of historical rows. The API creates source and destination connection profiles, then a stream that reads source binlogs/redo logs and lands change events in BigQuery tables. Setup is typically hours rather than the weeks needed for a custom CDC pipeline.

Create source profile 'mysql-prod', destination profile 'bq-warehouse', and stream 'orders-cdc' with backfillAll on; start the stream and report rowsBackfilled per object

Schema Discovery Before Stream Setup

Before launching a stream, call discoverConnectionProfile on the source profile to enumerate available schemas, tables, and columns. The response surfaces primary keys and unsupported types, so the operator can decide which tables to include and whether to add type-mapping overrides. This avoids stream creation failures from invalid include lists.

Call discoverConnectionProfile on 'mysql-prod' with hierarchyDepth 2, then return the full list of schemas and tables to the operator for selection

Backfill and Pause-Resume Operations

Trigger a backfill on individual stream objects after a schema change, and pause and resume streams to coordinate with downstream maintenance. POST /v1/{+object}:startBackfillJob and :stopBackfillJob give per-table control without restarting the whole stream. Pause/resume on the parent stream is exposed via the patch endpoint with the state field.

Call startBackfillJob on stream object 'orders-cdc/orders' to replay history after adding a column, then resume CDC on the parent stream

AI Agent CDC Operator

An AI agent can stand up and operate Datastream pipelines through Jentic without operator-written replication code. Jentic search returns the matching connection-profile, stream, or backfill operation, the agent loads the schema, and Jentic executes against datastream.googleapis.com using vault-stored credentials. This compresses the multi-day setup of OAuth and Datastream resources into one agent run.

Use Jentic to search 'create a datastream stream', load the createStream schema, and execute it with the configured source and destination profiles plus an includeObjects table list

Key Endpoints

21 endpoints — google cloud datastream is a serverless change-data-capture (cdc) and replication service that streams changes from operational databases (oracle, mysql, postgresql, sql server) into bigquery, cloud storage, and other destinations.

METHOD

PATH

DESCRIPTION

POST

/v1/{+parent}/connectionProfiles

Create a source or destination connection profile

POST

/v1/{+parent}/streams

Create a Datastream stream

POST

/v1/{+name}:run

Start a stream

POST

/v1/{+object}:startBackfillJob

Start backfill on a stream object

POST

/v1/{+object}:stopBackfillJob

Stop backfill on a stream object

GET

/v1/{+name}:fetchStaticIps

Fetch Datastream egress IPs for source allowlisting

POST

/v1/{+parent}/connectionProfiles

Create a source or destination connection profile

POST

/v1/{+parent}/streams

Create a Datastream stream

POST

/v1/{+name}:run

Start a stream

POST

/v1/{+object}:startBackfillJob

Start backfill on a stream object

POST

/v1/{+object}:stopBackfillJob

Stop backfill on a stream object

Why though Jentic?

Three things that make agents converge on Jentic-routed access.

Credential isolation

Google OAuth client secrets and refresh tokens are stored encrypted in the Jentic vault. Agents receive scoped, short-lived access tokens for datastream.googleapis.com; raw credentials never enter the agent context.

Intent-based discovery

Agents search Jentic by intent (e.g. 'create a datastream stream') and Jentic returns the matching operation with its input schema, so the agent calls the right endpoint without browsing the discovery doc.

Time to first call

Direct Datastream integration: 2-4 days for OAuth, connection-profile schema work, and stream-state polling. Through Jentic: under 1 hour.

Related APIs

Alternatives and complements available in the Jentic catalogue.

Alternative

Database Migration API

Database Migration is for one-time database moves with cutover; Datastream is for ongoing CDC.

Choose Database Migration when the goal is to move a database. Use Datastream when the goal is continuous CDC into BigQuery or Cloud Storage.

Complementary

BigQuery API

BigQuery is Datastream's primary destination; the BigQuery API queries the captured tables.

Choose BigQuery for analytics on the replicated data. Use Datastream to populate those tables in the first place.

Alternative

Cloud Pub/Sub API

Pub/Sub provides general event streaming; Datastream is purpose-built for database CDC.

Choose Pub/Sub for arbitrary application events. Use Datastream when the source is a transactional database with native log-based CDC.

FAQs

Specific to using Datastream API API through Jentic.

What authentication does the Datastream API use?

The Datastream API uses OAuth 2.0 with the cloud-platform scope. Through Jentic the OAuth client and refresh tokens are stored in the Jentic vault and the agent receives short-lived scoped access tokens, so raw Google credentials never enter the agent context.

Can I stream Oracle changes to BigQuery with Datastream?

Yes. Create a source connection profile of type ORACLE, a destination connection profile of type BIGQUERY, then a stream that references both. Datastream uses Oracle LogMiner to capture changes and writes per-table change events to BigQuery with seconds-to-minutes lag.

What are the rate limits for the Datastream API?

Google enforces standard Cloud quotas on datastream.googleapis.com: per-project rate limits on read/write calls and per-region quotas on concurrent streams and connection profiles. Quotas are visible in the Cloud Console under IAM and admin, quotas, filtered to datastream.googleapis.com.

How do I allowlist Datastream's source IPs?

Call GET /v1/{+name}:fetchStaticIps on the location resource. The response returns the egress IP ranges Datastream uses for IP-allowlist source connectivity in that region; add those to the source database's network allowlist before creating the stream.

Is the Datastream API free?

API calls are free. Datastream is billed per GiB of changed data captured plus a small per-stream operating fee. Underlying BigQuery storage and Cloud Storage destinations are billed at standard rates.

How do I backfill a single table after a schema change?

Call POST /v1/{+object}:startBackfillJob on the stream object that represents the table. Datastream replays historical rows for that object only, leaving CDC running on the rest of the stream. Stop a runaway backfill with :stopBackfillJob.