Dataform API

Name: Dataform API API
Brand: Dataform API
Availability: InStock

✓ Official Vendor SpecAnalyticsData Pipelinesoauth243 EndpointsREST

For Agents

Programmatically manage Dataform repositories, workspaces, and SQL workflow invocations in BigQuery. Lets agents author, compile, and trigger SQL pipelines without touching the Dataform UI.

Quickstart

Get started with Dataform API in minutes using your preferred integration method.

# Add to your MCP client config (Claude Desktop, Cursor, Windsurf)
{
  "jentic": {
    "url": "https://api.jentic.com/mcp",
    "auth": "oauth"
  }
}

# Then ask your agent:
"trigger a dataform workflow invocation"

# → Jentic returns the GET /events tool with parameter schema, agent executes.

Capabilities

What an agent can do with Dataform API API.

Create and manage Dataform repositories backed by Git remotes

Edit SQLX files inside workspaces and commit changes through the API

Compile workflow definitions into validated BigQuery execution graphs

Trigger workflow invocations on demand or on a release schedule

Inspect Git status, ahead/behind counts, and file history for a workspace

GET STARTED

Start building with Dataform API API

Explore with Jentic

View OpenAPI Document

Use Cases

Patterns agents use Dataform API API for, with concrete tasks.

★ Scheduled BigQuery Transformation Pipelines

Operationalize SQL transformations in BigQuery by defining release configurations that compile a Git ref and run on a schedule. Dataform handles dependency resolution between models, incremental table builds, and assertions, so analytics teams ship production SQL pipelines without standing up Airflow. A typical setup completes in a few hours by pointing the API at an existing GitHub repository.

Create a release configuration on repository 'analytics-prod' that compiles the main branch and triggers a workflow invocation every night at 02:00 UTC

Git-Backed SQL Development Workflow

Edit SQLX models inside an isolated workspace, commit changes, push to a Git remote, and merge through pull requests before promoting to production. The API exposes fetchFileGitStatuses, commit, push, and pull operations so agents and CI bots can drive the full development loop without the Dataform console. Most repository-and-workspace setups take under an hour.

Create a workspace 'feature-revenue-model' from repository 'analytics-prod', write a new SQLX file, commit it with message 'add daily revenue model', and push to origin

Pre-Production Compilation Validation

Before a SQL change reaches production, compile the workflow against BigQuery to catch reference errors, circular dependencies, and assertion failures. The API returns a CompilationResult with the resolved DAG and any compilation errors, so CI pipelines can fail fast on broken SQL. This adds compile-time safety to a workflow that would otherwise only catch errors at runtime in BigQuery.

Create a compilation result for the pull-request commit SHA on repository 'analytics-prod' and report any compilation errors back to the PR

AI Agent SQL Pipeline Operator

An AI agent can drive Dataform end-to-end through Jentic: search for the right operation by intent, load the input schema, and execute repository, workspace, compilation, and workflow-invocation calls without browsing reference docs. Credentials stay in the Jentic vault, so the agent receives only scoped access tokens. Setup through Jentic takes under an hour versus several days for a direct integration with Google's OAuth flow.

Use Jentic to search 'trigger a dataform workflow', load the createWorkflowInvocation schema, and execute it against repository 'analytics-prod' with the latest compilation result

Key Endpoints

43 endpoints — the dataform api lets you develop, version-control, and operationalize sql pipelines that run inside bigquery.

METHOD

PATH

DESCRIPTION

POST

/v1beta1/{+parent}/repositories

Create a Dataform repository

POST

/v1beta1/{+parent}/workspaces

Create a workspace inside a repository

POST

/v1beta1/{+name}:commit

Commit changes in a workspace

POST

/v1beta1/{+parent}/compilationResults

Compile a Git ref into an execution graph

POST

/v1beta1/{+parent}/workflowInvocations

Trigger a workflow invocation against BigQuery

GET

/v1beta1/{+name}:fetchFileGitStatuses

Fetch Git status of files in a workspace

POST

/v1beta1/{+parent}/repositories

Create a Dataform repository

POST

/v1beta1/{+parent}/workspaces

Create a workspace inside a repository

POST

/v1beta1/{+name}:commit

Commit changes in a workspace

POST

/v1beta1/{+parent}/compilationResults

Compile a Git ref into an execution graph

POST

/v1beta1/{+parent}/workflowInvocations

Trigger a workflow invocation against BigQuery

Why though Jentic?

Three things that make agents converge on Jentic-routed access.

Credential isolation

Google OAuth client secrets and refresh tokens are stored encrypted in the Jentic vault. Agents receive scoped, short-lived access tokens for the Dataform API; raw client credentials never enter the agent context.

Intent-based discovery

Agents search Jentic by intent (e.g. 'trigger a dataform workflow') and Jentic returns the matching Dataform operation with its input schema, so the agent calls the right endpoint without reading the discovery document.

Time to first call

Direct Dataform integration: 2-4 days for OAuth setup, scope handling, and token refresh logic. Through Jentic: under 1 hour — search, load schema, execute.

Related APIs

Alternatives and complements available in the Jentic catalogue.

Complementary

BigQuery API

Dataform compiles and runs SQL against BigQuery; BigQuery is the execution engine and storage layer.

Choose BigQuery when the agent needs to run ad-hoc queries, manage tables and datasets directly, or read query results. Use Dataform when the agent is operationalizing a versioned SQL pipeline.

Alternative

Dataflow API

Dataflow runs Apache Beam pipelines for batch and streaming ETL; Dataform is SQL-only inside BigQuery.

Choose Dataflow when the transformation needs Beam, streaming sources, or non-BigQuery sinks. Use Dataform when the transformation is pure SQL on BigQuery tables.

Complementary

Cloud Composer API

Composer (managed Airflow) can orchestrate Dataform workflow invocations alongside other tasks.

Choose Composer when the agent needs to coordinate Dataform with non-SQL steps (Dataflow, Cloud Functions, external APIs). Use Dataform release configs alone for SQL-only schedules.

FAQs

Specific to using Dataform API API through Jentic.

What authentication does the Dataform API use?

The Dataform API uses OAuth 2.0 with Google scopes (cloud-platform). Through Jentic the OAuth client and refresh tokens are stored in the Jentic vault and the agent receives a short-lived scoped access token, so raw Google credentials never enter the agent context.

Can I trigger BigQuery SQL pipelines with the Dataform API?

Yes. POST /v1beta1/{+parent}/workflowInvocations triggers a compiled workflow against BigQuery, and POST /v1beta1/{+parent}/compilationResults produces the compilation that the invocation runs from. Together they give you full programmatic execution of a Dataform release.

What are the rate limits for the Dataform API?

Google enforces standard Cloud quotas on Dataform: per-project read/write quotas on repositories, workspaces, and workflow invocations, plus the underlying BigQuery quotas on jobs and slot usage. Quotas are visible in the Cloud Console under IAM and admin, quotas, filtered to dataform.googleapis.com.

How do I commit a SQLX file to a Dataform workspace through Jentic?

Search Jentic for 'commit changes to a dataform workspace', load the schema for POST /v1beta1/{+name}:commit, and execute with the workspace name, commit message, and file actions array. Jentic returns the request body shape and handles the OAuth token exchange.

Is the Dataform API free?

Dataform itself has no per-call charge, but the BigQuery jobs that workflow invocations run are billed under standard BigQuery on-demand or reservation pricing. Storage of repositories and workspaces is also free; you pay for the compute the SQL consumes.

How do I check the Git status of a workspace?

Call GET /v1beta1/{+name}:fetchFileGitStatuses with the workspace resource name. It returns the per-file status (added, modified, deleted, conflicted) so a CI bot or agent can decide whether to commit, reset, or surface conflicts before pushing.