canonical: https://jentic.com/apis/googleapis.com/storagetransfer

# Google Storage Transfer API

The Storage Transfer API moves data into Google Cloud Storage on a schedule or on demand. It supports transfers from Amazon S3, HTTP/HTTPS sources, on-premises POSIX file systems via agent pools, and another Cloud Storage bucket. A transfer job describes source, destination, schedule, and filtering rules; transfer operations are individual runs of those jobs that can be paused, resumed, or cancelled. The API also exposes Google service accounts used to grant the transfer service access to source data.

## For AI agents

Schedule and run bulk data transfers into Google Cloud Storage from S3, HTTP, on-prem, or another GCS bucket. Pause, resume, or cancel running transfer operations.

## Scope

Does not run per-object reads or writes, transform records in-flight, or move data out of Cloud Storage to external sinks - use for scheduled bulk ingestion into Cloud Storage only.

## Capabilities

- Create scheduled or one-shot transfer jobs from S3, HTTP, on-premises, or another Cloud Storage bucket
- Run a transfer job immediately and track the resulting transfer operation
- Pause, resume, or cancel a long-running transfer operation
- Manage agent pools used for on-premises POSIX-source transfers
- List Google service accounts that the transfer service uses to access source storage
- Filter source data by prefix or last-modified time before transferring

## Use cases

### S3 to GCS Migration

Migrate a multi-terabyte S3 bucket into Cloud Storage as part of a cloud move. The Storage Transfer API accepts an awsS3DataSource with the bucket name and access credentials and a gcsDataSink with the destination bucket. The service handles parallelism, retries, and integrity checks; progress is reported through the transfer operation.

Example prompt: POST /v1/transferJobs with awsS3DataSource.bucketName='legacy-data', gcsDataSink.bucketName='migrated-data', schedule.scheduleStartDate=today.

### Daily Vendor Drop Ingestion

Ingest a daily file drop from a partner's HTTPS endpoint into a Cloud Storage landing bucket. The transfer job runs on a schedule, fetches the file list, and copies new objects into the sink. Filtering rules skip already-transferred objects so each run handles only the deltas.

Example prompt: POST /v1/transferJobs with httpDataSource.listUrl=https://partner.example/manifest.tsv, gcsDataSink.bucketName='vendor-drops', schedule.scheduleStartDate=tomorrow.

### On-Premises NAS Backup

Back up files from an on-premises NAS into a Cloud Storage archive bucket using a transfer agent pool. The Storage Transfer API exposes agent pool management for hosting the on-prem agents, and a posixDataSource on the transfer job that points the agent at a local path.

Example prompt: POST /v1/projects/myproj/agentPools to register the pool, then POST /v1/transferJobs with posixDataSource.rootDirectory='/mnt/nas/share' and the pool name.

### AI Agent Data Sync

An AI agent setting up a new analytics environment needs to copy a snapshot of an existing GCS bucket into the new project's bucket. Through Jentic, the agent searches for the create-transfer-job operation, loads the schema, and submits the job - Jentic handles auth so the agent never sees the underlying access token.

Example prompt: Search Jentic for 'transfer data into Cloud Storage', execute POST /v1/transferJobs with gcsDataSource.bucketName='source' and gcsDataSink.bucketName='dest'.

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | /v1/transferJobs | Create a transfer job |
| GET | /v1/transferJobs | List transfer jobs in a project |
| POST | /v1/{+jobName}:run | Run a transfer job immediately |
| POST | /v1/{+name}:pause | Pause a running transfer operation |
| POST | /v1/{+name}:resume | Resume a paused transfer operation |
| POST | /v1/{+name}:cancel | Cancel a running transfer operation |
| GET | /v1/googleServiceAccounts/{projectId} | Get the Google service account used by Storage Transfer for a project |
| GET | /v1/projects/{+projectId}/agentPools | List on-premises agent pools in a project |

## Key resources

- **transferJobs** — Create, list, get, update, and run transfer jobs that describe source, sink, schedule, and filters.
- **transferOperations** — List, get, pause, resume, and cancel individual runs of a transfer job.
- **agentPools** — Manage pools of on-premises agents used for POSIX-source transfers.
- **googleServiceAccounts** — Get the Google service account that the transfer service uses to access source storage.

## Why Jentic

- **Setup:** Wiring the Storage Transfer API by hand means configuring a service account, minting OAuth access tokens against storagetransfer.googleapis.com, and looking up the project's transfer service account yourself. Through Jentic you install once, import the Storage Transfer API from the API Directory, store the OAuth credential once, and your agent calls it.
- **Permission scoping:** Storage Transfer puts the job name in the URL path (/v1/{jobName}:run, /v1/{name}:pause), so a rule can pin your agent to the jobs you name: it can create and run those transfer jobs and nothing else. You choose the operations it may call, so pause, resume, or cancel are not included unless you add them.
- **Credential handling:** Your Storage Transfer OAuth credential is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'transfer data into Cloud Storage' or 'migrate from S3 to GCS', and Jentic returns the matching Storage Transfer operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Cloud Storage JSON API** — Storage manages individual objects; Storage Transfer schedules bulk movement into Storage.
- **Backblaze B2** — Backblaze B2 is an S3-compatible source bucket that Storage Transfer can read via S3-compatible source.
- **Dataflow API** — Dataflow runs custom transformation pipelines; Storage Transfer copies data without transformation.
- **Cloud Pub/Sub API** — Pub/Sub can carry per-object notifications produced by Cloud Storage after Storage Transfer lands data.

## FAQ

### What authentication does the Storage Transfer API use?

The Storage Transfer API uses OAuth 2.0 with the cloud-platform scope. Source access is delegated to a project-specific Google service account that must be granted read access on the S3, HTTP, or source GCS bucket. Through Jentic, OAuth credentials are stored in your Jentic One instance and exchanged for short-lived access tokens.

### Can I migrate from Amazon S3 with the Storage Transfer API?

Yes. Set awsS3DataSource on the transferSpec with the source bucket name and either an AWS access key pair or, preferably, role federation. The transfer service handles parallel object copy, retries, and integrity checks against the destination Cloud Storage bucket.

### What are the rate limits for the Storage Transfer API?

Storage Transfer scales transfer throughput based on the source and sink rather than per-call quotas, but admin calls (job creation, listing) are subject to the standard project per-minute quotas - typically a few hundred requests per minute. Source-side limits (e.g. S3 LIST quotas) often dominate large migrations.

### How do I run a transfer job through Jentic?

Search Jentic for 'transfer data into Cloud Storage', load the transferJobs.create schema, and execute POST /v1/transferJobs. To run an existing job immediately, call POST /v1/{+jobName}:run, which Jentic surfaces as a separate operation.

### Is the Storage Transfer API free?

The Storage Transfer Service is free for transfers from another Cloud Storage bucket and for on-premises agent transfers. Transfers from Amazon S3 or HTTP sources are billed per GB. Standard Cloud Storage operation, storage, and egress charges still apply on the destination side.

### Can I pause and resume a running transfer?

Yes. POST /v1/{+name}:pause halts a running transferOperation and POST /v1/{+name}:resume continues it. POST /v1/{+name}:cancel terminates the operation entirely without resuming.

### Can I limit what my agent is allowed to do with the Storage Transfer API?

Yes. Because you run Jentic One yourself, your own rules decide which Storage Transfer operations and credentials the agent may use. Since the job name sits in the URL path (POST /v1/{jobName}:run, POST /v1/{name}:pause), you can pin the agent to specific jobs and let it only create and run those, while excluding pause, resume, and cancel unless you explicitly grant them. The OAuth credential stays with your Jentic One instance and is injected at execution, so the agent calls only the endpoints you allow.