For Agents
Programmatically get an answer to a question about a given web page, extract structured data fields from a web page. Covers 7 operations with apiKey authentication.
Install Jentic One Beta
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the WebScraping.AI, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fwebscraping.ai%2Fwebscraping" | shStep 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fwebscraping.ai%2Fwebscraping" | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
What an agent can do with WebScraping.AI API.
Get an answer to a question about a given web page
Extract structured data fields from a web page
Page HTML by URL
HTML of a selected page area by URL and CSS selector
GET STARTED
Use for: I need to an answer to a question about a given web page, I want to extract structured data fields from a web page, Search for page html by url, Find all page text by url
Not supported: Does not handle payments, communications, or crm - use for ai and machine learning only.
WebScraping.AI scraping API provides LLM-powered tools with Chromium JavaScript rendering, rotating proxies, and built-in HTML parsing. The API exposes 7 endpoints secured with apiKey authentication.
Information about your account calls quota
Patterns agents use WebScraping.AI API for, with concrete tasks.
★ AI and Machine Learning Operations
Use the WebScraping.AI to perform ai ml operations programmatically. The API provides 7 endpoints covering core functionality including get an answer to a question about a given web page, extract structured data fields from a web page, page html by url.
Call GET /ai/question to get an answer to a question about a given web page
Automated AI Management
Automate ai operations by combining multiple WebScraping.AI endpoints. Agents can extract structured data fields from a web page and then page html by url in a single workflow.
Call GET /ai/fields to extract structured data fields from a web page, then verify the result
AI Agent Integration via Jentic
AI agents discover and call WebScraping.AI endpoints through Jentic without managing credentials directly. An agent searches for the required operation by intent, receives the matching endpoint schema, and executes the call with Jentic-managed authentication. This eliminates the need to read API documentation or handle apiKey tokens manually.
Search Jentic for 'get an answer to a question about a given web page', load the operation schema, and execute with Jentic-managed credentials
7 endpoints — webscraping.
METHOD
PATH
DESCRIPTION
/ai/question
Get an answer to a question about a given web page
/ai/fields
Extract structured data fields from a web page
/html
Page HTML by URL
/text
Page text by URL
/selected
HTML of a selected page area by URL and CSS selector
/selected-multiple
HTML of multiple page areas by URL and CSS selectors
/account
Information about your account calls quota
/ai/question
Get an answer to a question about a given web page
/ai/fields
Extract structured data fields from a web page
/html
Page HTML by URL
/text
Page text by URL
/selected
HTML of a selected page area by URL and CSS selector
What agents get from Jentic-routed access to this vendor.
Setup
Wiring WebScraping.AI by hand means passing its api_key on every query, managing proxy and rendering options across the extraction endpoints, and handling retries yourself. Through Jentic you install once, import WebScraping.AI from the API Directory, store the API key once, and your agent calls it.
Permission scoping
WebScraping.AI carries the target URL and query in each request rather than in the path, so scope the agent to the operations it needs, such as fetching page text or asking a question about a page. You choose the operations it may call, so account or field-extraction calls are not included unless you add them.
Credential isolation
Your WebScraping.AI API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.
Intent-based discovery
Agents search Jentic by intent such as 'get an answer to a question about a web page' or 'extract the text of a page', and Jentic returns the matching WebScraping.AI operation with its input schema so the agent calls the right endpoint without browsing the reference docs.
Alternatives and complements available in the Jentic catalogue.
Specific to using WebScraping.AI API through Jentic.
What authentication does the WebScraping.AI use?
The WebScraping.AI uses an API key passed in the `api_key` query. Through Jentic, these credentials are stored encrypted in your Jentic One instance and injected at execution time, so raw secrets never enter the agent context.
Can I get an answer to a question about a given web page with the WebScraping.AI?
Yes. Use the GET /ai/question endpoint. The API returns structured JSON responses that agents can parse and act on directly.
What are the rate limits for the WebScraping.AI?
Rate limits are not specified in the OpenAPI spec. Check the vendor documentation for current limits. Through Jentic, rate limiting is handled automatically with retry logic built into the execution layer.
How do I get an answer to a question about a given web page through Jentic?
Install the Jentic SDK with pip install jentic, authenticate through Jentic One, the self-hosted execution layer, then search for 'get an answer to a question about a given web page'. Jentic returns the matching WebScraping.AI operation with its input schema. Load the schema and execute the call - credentials are injected automatically.
How many endpoints does the WebScraping.AI have?
The WebScraping.AI exposes 7 endpoints covering ai, html, text operations.
/selected-multiple
HTML of multiple page areas by URL and CSS selectors
/account
Information about your account calls quota