canonical: https://jentic.com/apis/googleapis.com/datafusion

# Google Cloud Data Fusion API

Cloud Data Fusion is a fully-managed, cloud-native data integration service for building and operating ETL/ELT pipelines through a graphical pipeline designer. The API exposes 15 endpoints that provision and lifecycle Data Fusion instances, manage DNS peerings, and inspect available platform versions. It is the control-plane API: pipeline design and execution happen inside the instance UI and the per-instance CDAP REST API.

## For AI agents

Provision and lifecycle Cloud Data Fusion instances, manage DNS peerings, and list available versions. Lets agents stand up the data integration platform without going through the Cloud Console.

## Scope

Does not design pipelines, run pipeline executions, or expose CDAP plugins - use for Data Fusion instance provisioning, lifecycle, and DNS peering only.

## Capabilities

- Provision Cloud Data Fusion instances in a chosen region and edition
- Restart, upgrade, and delete Data Fusion instances through the API
- Configure DNS peerings so instances can resolve private network services
- List available Data Fusion versions before creating an instance
- Track long-running operations for instance create and upgrade calls
- Inspect instance state, edition, and connectivity before pipelines are deployed

## Use cases

### Programmatic Data Fusion Instance Provisioning

Stand up Cloud Data Fusion instances from infrastructure-as-code or an agent workflow rather than the Cloud Console. The API accepts edition, version, and network settings on create and exposes long-running operations to wait for the instance to reach the RUNNING state. End-to-end provisioning typically completes in 15-25 minutes once the create call returns.

Example prompt: Create a Cloud Data Fusion instance named 'analytics-prod' in us-central1 with edition ENTERPRISE and the latest available version, then poll the operation until state is RUNNING

### Private Network DNS Peering Setup

Configure DNS peerings on a Data Fusion instance so pipelines running inside the tenant project can resolve private hostnames in the customer VPC. Without the peering, pipelines fail to reach on-prem databases, internal Cloud SQL endpoints, or private DNS zones. The API handles create, list, get, update, and delete on the dnsPeerings collection.

Example prompt: Create a DNS peering on instance 'analytics-prod' for domain 'corp.internal.' targeting the customer VPC's private DNS zone

### Instance Lifecycle Automation in CI/CD

Spin up ephemeral Data Fusion instances for staging tests, run pipeline validation jobs, and tear them down to control cost. The API's restart, upgrade, and delete operations let CI pipelines drive the full lifecycle from a build job. Combined with version listing, this supports automated upgrade-test-promote flows.

Example prompt: Upgrade Data Fusion instance 'staging' to version 6.10.0 and wait for the operation to complete, then restart the instance and confirm state is RUNNING

### AI Agent Provisioning Operator

An AI agent can provision and lifecycle Data Fusion instances through Jentic without the operator writing OAuth code. Jentic's intent search returns the matching create, restart, upgrade, or delete operation, the agent loads the schema, and Jentic executes against the Data Fusion control plane with vault-stored credentials. This compresses platform-engineering setup work that normally takes a developer multiple days.

Example prompt: Use Jentic to search 'create a data fusion instance', load the create-instance schema, and execute it for project 'acme-data', region 'us-central1', edition 'BASIC'

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | /v1/{+parent}/instances | Create a Data Fusion instance |
| GET | /v1/{+parent}/instances | List Data Fusion instances in a location |
| POST | /v1/{+name}:restart | Restart a Data Fusion instance |
| POST | /v1/{+parent}/dnsPeerings | Create a DNS peering on an instance |
| GET | /v1/{+parent}/versions | List available Data Fusion versions |
| POST | /v1/{+name}:cancel | Cancel a long-running operation |

## Key resources

- **Instances** — Provision, restart, upgrade, and delete Cloud Data Fusion instances
- **Versions** — List Data Fusion versions available in a given location
- **DnsPeerings** — Configure DNS peerings between a Data Fusion instance and customer VPC
- **Operations** — Track long-running operations for create, upgrade, and delete calls

## Why Jentic

- **Setup:** Wiring the Cloud Data Fusion API by hand means setting up Google OAuth, refreshing short-lived scoped tokens instead of embedding client secrets, and addressing instance resources on datafusion.googleapis.com. Through Jentic you install once, import the Cloud Data Fusion API from the API Directory, store the Google credential once, and your agent calls it.
- **Permission scoping:** The API carries the parent and instance name in the URL path (/v1/{parent}/instances and /v1/{name}:restart), so a rule can pin your agent to one instance. You choose the operations it may call, so it can list and restart instances while instance creation is not included unless you add it.
- **Credential handling:** Your Google OAuth credential is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'provision a Data Fusion instance', and Jentic returns the matching Cloud Data Fusion operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Dataflow API** — Data Fusion pipelines compile to Dataflow jobs at runtime; Dataflow exposes the underlying job API.
- **Cloud Dataproc API** — Dataproc runs managed Hadoop and Spark clusters; Data Fusion provides a graphical pipeline layer over comparable engines.
- **Cloud Composer API** — Composer (managed Airflow) often orchestrates Data Fusion instances and pipeline runs as part of a wider DAG.

## FAQ

### What authentication does the Cloud Data Fusion API use?

The Cloud Data Fusion API uses OAuth 2.0 with the cloud-platform scope. Through Jentic the OAuth client and refresh tokens live in your Jentic One instance and the agent receives short-lived scoped access tokens, so raw Google credentials never enter the agent context.

### Can I run pipelines through the Cloud Data Fusion API?

No - the Data Fusion control-plane API at datafusion.googleapis.com only provisions and lifecycles instances. Pipeline create, deploy, and run operations are exposed by the per-instance CDAP REST API, served from the running instance's URL once it is provisioned through this API.

### What are the rate limits for the Cloud Data Fusion API?

Google enforces standard Cloud quotas on Data Fusion: per-project quotas on number of instances per edition, plus general read/write rate limits on the operations and dnsPeerings collections. Quotas are visible in the Cloud Console under IAM and admin, quotas, filtered to datafusion.googleapis.com.

### How do I provision a Data Fusion instance through Jentic?

Search Jentic for 'create a data fusion instance', load the schema for POST /v1/{+parent}/instances, and execute with project, location, edition, and version. Jentic returns the long-running operation name; poll GET /v1/{+name} on it until done is true.

### Is the Cloud Data Fusion API free?

API calls themselves are free, but Data Fusion instances are billed by edition (Developer, Basic, Enterprise) and runtime hours, plus underlying Dataproc and BigQuery costs incurred by pipelines. There is no per-API-call charge.

### Which versions can I create an instance with?

Call GET /v1/{+parent}/versions on the location to list available Data Fusion versions before creating an instance. The response includes version IDs and default flag, so an agent can pick the latest stable version programmatically rather than hard-coding it.

### Can I limit what my agent is allowed to do with the Cloud Data Fusion API?

Yes. Because you run Jentic One yourself, your own rules decide which Cloud Data Fusion operations and credentials the agent may use. Since each call carries the parent and instance name in the URL path, such as /v1/{parent}/instances and /v1/{name}:restart, a rule can pin the agent to a single instance. You choose the operations it may call, so it can list and restart instances while instance creation stays excluded unless you add it.