canonical: https://jentic.com/apis/brightdata.com/brightdata

# Bright Data Web Scraper API

The Bright Data Web Scraper API triggers asynchronous web data collections, monitors their progress, and downloads the resulting snapshots. It also scrapes individual URLs on demand and returns their structured content, and supports webhook delivery for completed batch jobs. Agents can run large scraping jobs against Bright Data's managed datasets without operating proxy rotation or scraper infrastructure themselves.

## For AI agents

Trigger web scraping collections, monitor snapshots, and download structured datasets, or scrape a single URL on demand. Returns collected web data for an agent to process.

## Scope

Does not handle proxy network configuration, browser automation, or long-term data storage. Use for web data collection and scraping only.

## Capabilities

- Trigger asynchronous data collection jobs against Bright Data scraper datasets
- Monitor collection progress for a running snapshot
- Download completed snapshot data once a collection finishes
- Scrape a single URL on demand and return its structured content
- List existing scraping snapshots and cancel collections that are still running

## Use cases

### Agent-Driven Web Data Collection

An AI agent gathers structured web data on demand by triggering a Bright Data collection, polling the snapshot until it completes, and downloading the results. Because collection is asynchronous with webhook delivery, the agent can start a large job and pick up the output later without holding a connection open. This turns open-ended research and monitoring tasks into a repeatable data pipeline the agent drives itself.

Example prompt: Trigger a data collection for a set of product page URLs, poll the snapshot until it reports complete, then download the structured results as JSON

### Bulk Dataset Collection

Teams scrape large batches of pages by submitting a collection request and letting Bright Data run it asynchronously, with a webhook signalling when the snapshot is ready to download. Progress can be monitored per snapshot and a running job can be cancelled if the inputs were wrong. This suits recurring catalogue, pricing, or listing refreshes where the input set changes but the extraction shape stays the same.

Example prompt: Submit a batch collection for a list of category URLs, wait for the completion webhook, and download the snapshot once progress reports finished

### On-Demand Single Page Scraping

For one-off lookups an agent scrapes a single URL synchronously and receives the page's structured content directly, without setting up a dataset collection. This is the fast path for answering a question about one page: a product detail, an article, or a listing. It keeps quick fetches separate from the heavier asynchronous batch flow.

Example prompt: Scrape a single product URL and return the extracted structured content for the agent to summarise

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/datasets/v3/trigger` | Trigger a data collection |
| GET | `/datasets/v3/progress/{snapshot_id}` | Monitor collection progress |
| GET | `/datasets/v3/snapshot/{snapshot_id}` | Download snapshot data |
| GET | `/datasets/v3/snapshots` | List snapshots |
| POST | `/datasets/v3/snapshot/{snapshot_id}/cancel` | Cancel a running collection |
| POST | `/request` | Scrape a single URL |

## Key resources

- **Collections** — Trigger asynchronous data collection jobs and receive results via webhook or download
- **Snapshots** — List, monitor progress of, download, and cancel scraping snapshots
- **Direct request** — Scrape a single URL synchronously and return its structured content

## Why Jentic

- **Setup:** Wiring the Bright Data Web Scraper API by hand means handling its bearer token, kicking off asynchronous collections, and polling snapshot progress before you can download anything. Through Jentic you install once, import Bright Data from the API Directory, store the token once, and your agent triggers and retrieves collections.
- **Permission scoping:** Bright Data puts the snapshot id in the URL path (`/datasets/v3/snapshot/{snapshot_id}`), so a rule can bound your agent to reading and downloading snapshots. You choose the operations it may call, so triggering new collections or cancelling a run are not included unless you add them.
- **Credential handling:** Your Bright Data token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'scrape a web page' or 'download a dataset snapshot', and Jentic returns the matching Bright Data operation with its input schema so the agent calls the right endpoint without reading the reference docs.

## Related APIs

- **ScraperAPI** — Proxy-based scraping API that returns page HTML for a requested URL
- **ZenRows** — Scraping API with proxy rotation and headless browser rendering
- **Apify** — Actor platform for running and scheduling custom scraping and automation jobs

## FAQ

### What authentication does the Bright Data Web Scraper API use?

The Bright Data Web Scraper API authenticates with an HTTP bearer token per its OpenAPI spec: you send your API key from your Bright Data account settings as an Authorization Bearer header on every request. Through Jentic the token is held encrypted by your own Jentic One instance and attached at call time, so it never reaches the agent's context.

### Can I scrape a single URL with the Bright Data Web Scraper API?

Yes. The API exposes a single-URL scrape operation that returns a page's structured content directly, alongside the dataset collection flow for larger asynchronous jobs. An agent can call it for one-off pages without setting up a full collection.

### What are the rate limits for the Bright Data Web Scraper API?

The OpenAPI spec does not specify rate limits. Check the Bright Data documentation at https://docs.brightdata.com/scraping-automation/web-scraper-api/overview for current limits and concurrency guidance.

### Can I limit what my agent is allowed to do with the Bright Data Web Scraper API?

Yes. Write a rule that allows only the trigger, progress, and snapshot download operations, so the agent can run and retrieve collections but cannot cancel jobs or reach any operation you did not grant, and every call it makes is logged.

### How do I trigger and download a Bright Data collection through Jentic?

Search Jentic for 'scrape a web page' or 'trigger a data collection', add the Bright Data Web Scraper API from the directory, and store your token once. Your agent then triggers a collection, polls the snapshot until it is ready, and downloads the results. To run it on your own infrastructure, install Jentic One from its GitHub repo.

### Is there a Bright Data MCP server?

You don't need an MCP server to give your agent the Bright Data Web Scraper API. Jentic connects it directly from the API Directory: import it, store your token once, and your agent triggers and downloads collections. Operations are discovered on demand, so no extra server's tool definitions sit in the agent's context.