canonical: https://jentic.com/apis/hydrafetch.com/hydrafetch

# Hydrafetch API

The Hydrafetch API turns web pages into data an agent can use: send a URL and get back clean Markdown, cleaned or raw HTML, links, the page's own JSON-LD, microdata, OpenGraph, and RDFa, images, or a full-page PNG screenshot. It maps a site's URLs, crawls a whole site or scrapes an explicit URL list as asynchronous jobs with webhook callbacks, runs web searches with optional scraping of each result, and extracts schema-shaped JSON from one or many pages with per-field confidence and the source passage behind each value. Brand endpoints resolve a company's logos, colours, and social presence by domain, and the styleguide call reads the colours and type scale a site actually renders. Calls are billed in credits and charged only on success.

## For AI agents

Scrape a URL to Markdown, HTML, links, structured data, or a screenshot, crawl or batch-scrape sites, search the web, extract schema-shaped JSON with source passages, and resolve company brands and logos with Hydrafetch.

## Scope

Does not host or edit websites, bypass logins you do not hold, or store pages as a document archive. Use for fetching, crawling, searching, and extracting public web content and brand data only.

## Capabilities

- Scrape one URL to clean Markdown, cleaned or raw HTML, links, or the page's own structured data
- Crawl a whole site or scrape an explicit URL list as one asynchronous job, with webhook callbacks and delivery history
- Map a site's URLs from links and sitemaps without scraping them
- Search the web for a query and optionally scrape every result in the same call
- Extract schema-shaped JSON from pages with per-field confidence and the source passage for each value
- Resolve a company's brand by domain, including logos for light and dark backgrounds and its colours, for up to 50 domains at once
- Read a site's rendered styleguide and capture a full-page screenshot

## Use cases

### Web reading for agents through Jentic

A developer's agent needs to read pages without wading through navigation, banners, and boilerplate. Through Jentic it finds the Markdown or extract operation by intent and gets clean content back; the X-API-Key credential is injected by the developer's own Jentic One instance, and a rule can keep the agent to single-page reads so it cannot start credit-heavy crawls.

Example prompt: Search Jentic for 'get a web page as markdown', call POST `/v1/web/markdown` with url https://acme.com/pricing and onlyMainContent true, and summarise the plan names and prices

### Documentation site ingestion

Load a documentation site into a retrieval index. The crawl call takes a start url with limit, maxDepth, includePaths, and excludePaths, returns a crawl id immediately, and either takes a webhook or is polled for per-page results; the deliveries call shows each webhook attempt and its last response if a callback fails.

Example prompt: Call POST `/v1/web/crawl` with url https://docs.example.dev, limit 200, and includePaths ['/guides/*'], then poll GET `/v1/web/crawl/{id}` until it completes and store each page's Markdown

### Schema-shaped extraction with sources

Pull the same fields from many pages without writing per-site parsers. The extract call takes urls plus a JSON schema and or a prompt, can add web search to find more sources, and with showConfidence and showSources returns per-field confidence and the passage each value came from, so low-confidence fields can be checked.

Example prompt: Call POST `/v1/web/extract` with five product URLs, a schema with name, price, and inStock, showConfidence true, and showSources true, and flag any field with low confidence

### Brand assets for a company picker

Render company logos and colours in an app without maintaining an asset library. Brand search turns what a user typed into a domain without billing, the brand call resolves logos for light and dark backgrounds, colours, and social presence, and the batch form resolves up to 50 domains, billing only the brands it resolves.

Example prompt: Call GET `/v1/web/brand/search` with query Stripe to get the domain, then GET `/v1/web/brand` with that domain and return the dark-background logo URL and primary colour

## Key endpoints

| Method | Path | Description |
| --- | --- | --- |
| POST | `/v1/web/scrape` | Scrape a URL |
| POST | `/v1/web/markdown` | Scrape to Markdown |
| POST | `/v1/web/crawl` | Start a crawl |
| GET | `/v1/web/crawl/{id}` | Get crawl status |
| POST | `/v1/web/search` | Search the web |
| POST | `/v1/web/extract` | Extract structured data from URLs |
| GET | `/v1/web/brand` | Resolve a brand |
| POST | `/v1/web/map` | Map a site |

## Key resources

- **Scrapes** — Single-URL fetches in Markdown, HTML, raw HTML, links, structured data, images, or screenshot form, synchronous or async
- **Crawls and batches** — Asynchronous multi-page jobs with status polling and webhook delivery history
- **Site maps** — URL enumeration from links and sitemaps without scraping
- **Search and extract** — Ranked web search results and schema-shaped extraction with confidence and sources
- **Brands and styleguides** — Company logos, colours, social presence, and rendered design tokens by domain

## AI readiness

This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use.

- **Score:** 54 / 100
- **Maturity:** Foundational
- **Dimensions:**
  - Foundational Compliance: 48 / 100
  - Developer Experience & Jentic Compatibility: 82 / 100
  - AI-Readiness & Agent Experience: 33 / 100
  - Agent Usability: 94 / 100
  - Security: 50 / 100
  - AI Discoverability: 80 / 100
- **View full report:** https://jentic.com/apis/hydrafetch.com/hydrafetch/scorecard
- **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/
- **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview

### Score it yourself

Every API in the directory is allowlisted, so you can re-score it with no key required.

- **Score your own API:** https://jentic.com/scorecard.md
- **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md

```sh
npx @jentic/api-scorecard-cli score <openapi-url>
```

## Why Jentic

- **Setup:** Wiring the Hydrafetch API by hand means sending the `X-API-Key` header, choosing between synchronous scrapes and async jobs you poll or receive by webhook, and throttling to the RateLimit headers. Through Jentic you install once, import Hydrafetch from the API Directory, store the key once, and your agent calls it.
- **Permission scoping:** Hydrafetch carries the target URL inside the request body, so rules bound which operations your agent may call, not which sites it fetches. You choose the operations it may call, so credit-heavy crawls, batches, and extraction are not included unless you add them.
- **Credential handling:** Your Hydrafetch API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
- **Discovery method:** Agents search Jentic by intent such as 'scrape a page to markdown' or 'crawl a website', and Jentic returns the matching Hydrafetch operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

## Related APIs

- **Firecrawl API** — Scrape and crawl websites into LLM-ready Markdown
- **Zyte API** — Web data extraction with browser rendering and automatic extraction
- **Brandfetch API** — Company logos, colours, and fonts by domain
- **OpenAI API** — Language models to summarise or reason over scraped content

## FAQ

### What authentication does the Hydrafetch API use?

Per its OpenAPI spec, the Hydrafetch API takes an API key in the `X-API-Key` header, scoped to a workspace and carrying its credit balance. The logo-serving call, GET `/v1/web/logo/{domain}`, also requires a publishable key in the `token` query parameter so it can be used from a browser. Through Jentic the API key is stored encrypted by your own Jentic One instance and injected per call.

### How are Hydrafetch calls billed?

The spec says calls are billed in credits and charged only on success. A standard scrape is one credit, a site map is one credit, a crawl or batch is one credit per page or URL scraped, and a search is one credit plus one more per result when scrapeResults is true. Richer formats and the extraction tier cost more, and each response reports what it consumed.

### What are the rate limits for the Hydrafetch API?

Hydrafetch's spec states that limits are per workspace, not per key, over a 60 second window, with the ceiling set by your plan. Every response carries RateLimit-Policy and RateLimit headers giving the quota, remaining requests, and seconds until reset, and a 429 includes Retry-After in seconds.

### Can Hydrafetch crawl a site asynchronously?

Yes. POST `/v1/web/crawl` discovers and scrapes a whole site as one asynchronous job and returns a crawl id immediately. Poll GET `/v1/web/crawl/{id}` for progress and per-page results, or register a webhook and check GET `/v1/web/crawl/{id}/deliveries` to see each callback attempt and its last response.

### How do I scrape a page to Markdown with the Hydrafetch API through Jentic?

Search Jentic for 'get a web page as markdown' to get POST `/v1/web/markdown`, then send the url, optionally with onlyMainContent and blockAds. To run it on your own infrastructure, install Jentic One from its GitHub repo and store your Hydrafetch API key once.

### Is there a Hydrafetch MCP server?

You don't need an MCP server to give your agent Hydrafetch. Jentic connects it directly from the API Directory: import it, store your API key once, and your agent calls the operations you allow, discovered when needed instead of preloaded into its context.

### Can I limit what my agent is allowed to do with the Hydrafetch API?

Yes. For example, allow only POST `/v1/web/markdown` and POST `/v1/web/search`, so the agent can read single pages and search but cannot start crawls or batches that spend credits per page. Your Jentic One instance logs the calls it makes.