BigQuery Data Transfer API

Name: BigQuery Data Transfer API API
Brand: BigQuery Data Transfer API
Availability: InStock

✓ Official Vendor SpecAnalyticsData Pipelinesoauth214 EndpointsREST

For Agents

Schedule and monitor recurring data loads into BigQuery from SaaS apps, AWS, and Cloud Storage. Agents can create transferConfigs, trigger backfills, and read run history.

Quickstart

Get started with BigQuery Data Transfer API in minutes using your preferred integration method.

# Add to your MCP client config (Claude Desktop, Cursor, Windsurf)
{
  "jentic": {
    "url": "https://api.jentic.com/mcp",
    "auth": "oauth"
  }
}

# Then ask your agent:
"schedule a recurring data transfer into BigQuery"

# → Jentic returns the GET /events tool with parameter schema, agent executes.

Capabilities

What an agent can do with BigQuery Data Transfer API API.

Enroll a project into the available data sources catalog before creating transfers

Create transferConfigs that pull from Google Ads, YouTube, S3, or Cloud Storage on a schedule

Trigger on-demand backfills and ad-hoc runs for a transferConfig

List transfer runs and inspect transferLogs to debug failed loads

GET STARTED

Start building with BigQuery Data Transfer API API

Explore with Jentic

View OpenAPI Document

The BigQuery Data Transfer Service automates recurring data loads into BigQuery from SaaS applications (Google Ads, Campaign Manager, YouTube, Search Ads), other clouds (S3, Azure Blob), and Cloud Storage. The API exposes data sources, transfer configurations, runs, and transfer logs so you can enroll sources, schedule transfers, backfill historical windows, and monitor failures. Each transferConfig holds a schedule, destination dataset, and source-specific parameters. This is the right plane when the use case is 'land this source in BigQuery on a cadence' rather than ad-hoc loads.

Use Cases

Patterns agents use BigQuery Data Transfer API API for, with concrete tasks.

★ Marketing Data Lake Population

Schedule daily transfers from Google Ads, Campaign Manager, and YouTube into a BigQuery marketing dataset so the warehouse always has yesterday's spend and impression data. transferConfigs are managed at /v1/{parent}/transferConfigs with schedule, destinationDatasetId, and a per-source params map. Failed runs surface in the runs and transferLogs collections for alerting.

Create a daily Google Ads transferConfig that lands account 123-456-7890 into dataset marketing_raw at 04:00 UTC.

Cross-Cloud Ingestion

Pull files from Amazon S3 or Azure Blob Storage into BigQuery on a schedule using the s3 and azure_blob_storage data sources. The transfer service handles auth, file pattern matching, and write disposition. This avoids running custom Cloud Functions for every cross-cloud pipeline.

Create an hourly S3 transferConfig that copies s3://logs/year=*/month=*/day=*/*.json into dataset raw_logs in project myproj.

Backfill and Replay

Trigger backfills via /v1/{parent}:scheduleRuns when source data is corrected or a transfer was paused. The runs collection records each execution with start/end time and state, and transferLogs explains why a run failed. Operators can replay arbitrary windows without manual SQL.

Schedule backfill runs for transferConfig projects/myproj/locations/US/transferConfigs/abc123 between 2026-06-01 and 2026-06-09.

Failure Monitoring

List runs filtered to states like FAILED and pull /v1/{name}/transferLogs for each, then forward summaries to a paging system. The transferConfig itself records lastModifiedTime and emailPreferences so on-call can be notified directly. This replaces hand-rolled monitoring of cron-driven loads.

List all transfer runs in project myproj location US in state FAILED in the last 24h and return their config name and error message.

Agent-Driven Pipeline Setup

Let an AI agent take a request like 'load our HubSpot contacts into BigQuery every morning' and create the matching transferConfig through Jentic. The agent picks the data source, configures parameters, and confirms credentials with checkValidCreds before scheduling. OAuth tokens stay in Jentic's vault.

From the request 'pull S3 logs into BigQuery hourly', create the transferConfig with the supplied bucket and credentials, run checkValidCreds, then schedule a backfill for the past 24h.

Key Endpoints

14 endpoints — the bigquery data transfer service automates recurring data loads into bigquery from saas applications (google ads, campaign manager, youtube, search ads), other clouds (s3, azure blob), and cloud storage.

METHOD

PATH

DESCRIPTION

POST

/v1/{+parent}/transferConfigs

Create a new transferConfig

GET

/v1/{+parent}/transferConfigs

List transferConfigs in a project and location

PATCH

/v1/{+name}

Update a transferConfig

DELETE

/v1/{+name}

Delete a transferConfig

POST

/v1/{+parent}:scheduleRuns

Schedule backfill runs for a transferConfig

POST

/v1/{+name}:checkValidCreds

Validate source credentials before saving

GET

/v1/{+parent}/runs

List runs of a transferConfig

GET

/v1/{+parent}/transferLogs

List transfer log lines for a run

POST

/v1/{+parent}/transferConfigs

Create a new transferConfig

GET

/v1/{+parent}/transferConfigs

List transferConfigs in a project and location

PATCH

/v1/{+name}

Update a transferConfig

DELETE

/v1/{+name}

Delete a transferConfig

POST

/v1/{+parent}:scheduleRuns

Schedule backfill runs for a transferConfig

Why though Jentic?

Three things that make agents converge on Jentic-routed access.

Credential isolation

Google OAuth 2.0 client credentials and refresh tokens are stored encrypted in the Jentic vault (MAXsystem). Source credentials embedded in a transferConfig (S3 keys, Ads tokens) flow through Google's API and are not persisted by Jentic.

Intent-based discovery

Agents search Jentic with intents like 'schedule a BigQuery transfer from Google Ads' and Jentic returns the create transferConfig operation with the data source's specific parameter schema.

Time to first call

Direct integration: 1-2 days for OAuth, source enrollment, and per-source params. Through Jentic: under an hour — search, load schema, execute.

Related APIs

Alternatives and complements available in the Jentic catalogue.

Complementary

BigQuery API

Stores the data the transfer service lands and runs queries over it.

Choose BigQuery when the agent needs to query data; choose Data Transfer when it needs to schedule the load.

Alternative

BigQuery Connection API

Federates external sources without copying the data into BigQuery.

Use Connection to query data in place; use Data Transfer when the data must land in BigQuery storage.

Alternative

Cloud Scheduler API

General-purpose cron service for triggering arbitrary HTTP jobs.

Use Cloud Scheduler when transfers need custom logic; use Data Transfer for built-in connectors with state tracking.

FAQs

Specific to using BigQuery Data Transfer API API through Jentic.

What authentication does the BigQuery Data Transfer API use?

The API uses Google OAuth 2.0 with the cloud-platform and bigquery scopes. Source-side credentials (Ads accounts, S3 keys) are saved on each transferConfig. Through Jentic, the OAuth refresh token is stored encrypted; only short-lived access tokens reach the agent.

Can I create scheduled transfers from Google Ads with this API?

Yes. POST /v1/{parent}/transferConfigs with dataSourceId set to google_ads, the destinationDatasetId, schedule (e.g., 'every 24 hours'), and params containing the Ads customer_id creates a recurring load. The first run kicks off at the next scheduled slot.

What are the rate limits for the BigQuery Data Transfer API?

Quotas are project-level: typically a few thousand transferConfigs per project and modest write QPS on the same config. Run history is retained for 90 days. Check the Google Cloud quotas page for the values applied to your project.

How do I trigger a backfill through Jentic?

Search Jentic for 'backfill a BigQuery transfer', load the schema for POST /v1/{parent}:scheduleRuns, and execute with startTime and endTime in RFC3339. Jentic forwards the OAuth token, and the runs collection will show the new BACKFILL runs.

Is the BigQuery Data Transfer API free?

The API itself has no surcharge, but the data it loads is billed at the standard BigQuery storage and (for some sources) ingestion rate. Some SaaS connectors are also subject to per-source quotas. Storage in the destination dataset is the main ongoing cost.

Can I pause a transfer without deleting it?

Yes. PATCH /v1/{name} with disabled=true (and updateMask=disabled) pauses the schedule. The transferConfig stays in place; resume by patching back to disabled=false.