IBM DataStage API

Name: IBM DataStage API API
Brand: IBM DataStage API
Availability: InStock

✓ Official Vendor SpecAnalyticsData Pipelinesbearer17 EndpointsREST

For Agents

List, compile, run, and monitor IBM DataStage flows and jobs on IBM Cloud — automate ETL execution and inspect run logs without using the DataStage UI.

Connect the IBM DataStage API to your agent

Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the IBM DataStage API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.

Install Jentic One Beta (Self-Hosted)

Jentic One Host machine

# On the machine that will host your Jentic One instance:
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | sh

Agent machine

# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | sh
jentic register       # connects your agent to your Jentic One instance

Jentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.

Capabilities

What an agent can do with IBM DataStage API API.

List and inspect data integration flows via /v3/data_intg_flows

Clone an existing flow into a new flow via /v3/data_intg_flows/{flow_id}/clone

Compile a flow before execution via /v3/data_intg_flows/{flow_id}/compile

Trigger a DataStage job run via /v3/ds_jobs/{job_id}/run

GET STARTED

Start building with IBM DataStage API API

Explore with Jentic

View OpenAPI Document

Use Cases

Patterns agents use IBM DataStage API API for, with concrete tasks.

★ Pipeline Orchestration

Trigger DataStage jobs from an external orchestrator such as Airflow or a Jentic agent so that ETL execution sits alongside the rest of the data pipeline rather than running on a DataStage-only schedule. The API provides /v3/ds_jobs/{job_id}/run to start a job and /v3/ds_jobs/{job_id}/runs/{run_id} to poll for completion, which is enough to wire DataStage into a dependency-aware DAG.

Call /v3/ds_jobs/{job_id}/run for the daily-customer-load job, then poll /v3/ds_jobs/{job_id}/runs/{run_id} every 30 seconds until status is Completed.

Run Diagnostics

Pull logs and run metadata for failed DataStage executions so an on-call engineer or agent can diagnose without opening the DataStage UI. The /v3/ds_jobs/{job_id}/runs endpoint returns the recent run history, and /v3/ds_jobs/{job_id}/runs/{run_id}/logs returns the per-run log payload that contains row counts, warnings, and stack traces.

When a DataStage job run ends in Failed, fetch /v3/ds_jobs/{job_id}/runs/{run_id}/logs and post the last 50 log lines to the on-call channel.

Flow Cloning for Environments

Clone a production data integration flow into a new flow ID so it can be edited safely in a non-production project. The /v3/data_intg_flows/{flow_id}/clone endpoint copies the flow definition without copying credentials, which makes it suitable for promoting changes through dev, test, and prod environments.

Clone the data integration flow with id daily-orders-prod into a new flow named daily-orders-staging in the staging project.

AI Agent Data Pipeline Operator

An AI agent monitors DataStage runs, retries transient failures, and surfaces persistent ones to a human owner. Through Jentic, the agent searches by intent, loads the run and logs schemas, and executes calls without holding the raw IBM Cloud IAM token. Typical agent integration is under one hour because only a handful of operations are needed to cover the run-and-monitor surface.

Search Jentic for 'run a DataStage job', load the /v3/ds_jobs/{job_id}/run schema, run the job, and on Failed status fetch the logs and summarise the error.

Key Endpoints

17 endpoints — ibm datastage on ibm cloud is a managed etl and data integration service used to design, compile, and run data flows that move and transform data between sources and targets.

METHOD

PATH

DESCRIPTION

GET

/v3/data_intg_flows

List data integration flows

POST

/v3/data_intg_flows/{flow_id}/clone

Clone a flow

POST

/v3/data_intg_flows/{flow_id}/compile

Compile a flow

POST

/v3/ds_jobs/{job_id}/run

Run a DataStage job

GET

/v3/ds_jobs/{job_id}/runs

List runs for a job

GET

/v3/ds_jobs/{job_id}/runs/{run_id}

Get a specific run

GET

/v3/ds_jobs/{job_id}/runs/{run_id}/logs

Fetch run logs

GET

/v3/data_intg_flows

List data integration flows

POST

/v3/data_intg_flows/{flow_id}/clone

Clone a flow

POST

/v3/data_intg_flows/{flow_id}/compile

Compile a flow

POST

/v3/ds_jobs/{job_id}/run

Run a DataStage job

GET

/v3/ds_jobs/{job_id}/runs

List runs for a job

Why though Jentic?

Three things that make agents converge on Jentic-routed access.

Credential isolation

IBM Cloud IAM bearer tokens are stored encrypted in the Jentic vault. Agents call DataStage operations via scoped Jentic credentials and never see the raw IAM token in their context.

Intent-based discovery

Agents search by intent (for example 'run a DataStage job') and Jentic returns the matching DataStage operation with its parameter schema, so the agent does not have to read the full DataStage REST reference.

Time to first call

Direct DataStage integration: 2-5 days for IAM token exchange, run polling, and log handling. Through Jentic: under one hour — search, load schema, execute.

Related APIs

Alternatives and complements available in the Jentic catalogue.

Complementary

IBM webMethods Integration API

webMethods covers iPaaS and SaaS connectors; DataStage covers heavy ETL and data warehouse loads.

Use webMethods to move data between SaaS systems via prebuilt connectors. Use DataStage when the workload is bulk ETL into a data warehouse or lake.

Alternative

Azure Logic Apps Management API

Azure Logic Apps is a managed integration and workflow service; DataStage is a managed ETL service.

Choose Azure Logic Apps for event-driven SaaS integration on Azure. Choose DataStage for batch ETL into IBM data platforms.

Complementary

IBM API Connect Management API

API Connect manages the API lifecycle in front of services like DataStage.

Use API Connect when the agent needs to publish or govern an API that wraps a DataStage flow. Use DataStage directly to operate the flow itself.

FAQs

Specific to using IBM DataStage API API through Jentic.

What authentication does the IBM DataStage API use?

The DataStage REST API uses HTTP bearer authentication with an IBM Cloud IAM token. Through Jentic, the IAM credentials are stored in the Jentic vault and the agent calls DataStage via a scoped Jentic credential, never seeing the raw IAM token.

Can I run a DataStage job from this API?

Yes. POST to /v3/ds_jobs/{job_id}/run to start a run, then poll /v3/ds_jobs/{job_id}/runs/{run_id} for status. The run record returns success, failure, and timing fields that an external scheduler can use to gate downstream steps.

How do I retrieve logs for a failed DataStage run through Jentic?

Search Jentic for 'fetch DataStage run logs', load the /v3/ds_jobs/{job_id}/runs/{run_id}/logs schema, and execute with the job ID and run ID. Install the SDK with pip install jentic and use the async search, load, execute pattern.

Does the DataStage API support cloning flows between projects?

Yes. POST to /v3/data_intg_flows/{flow_id}/clone with the target project ID to copy a flow definition. The clone does not carry credentials, which is what makes it safe for promoting flows from production to a non-production project.

What are the rate limits for the IBM DataStage API?

IBM Cloud applies account-level rate limits across IAM-protected services rather than a fixed DataStage limit. Treat 429 responses as backpressure, retry with exponential backoff, and avoid polling run status more often than every few seconds for the same run.

How do I compile a flow before running it?

POST to /v3/data_intg_flows/{flow_id}/compile to compile a flow into an executable job. A flow must compile successfully before /v3/ds_jobs/{job_id}/run will produce a useful run, so this endpoint is part of the standard deploy-then-run sequence.