canonical: https://jentic.com/apis/webscraping.ai/webscraping # WebScraping.AI WebScraping.AI scraping API provides LLM-powered tools with Chromium JavaScript rendering, rotating proxies, and built-in HTML parsing. The API exposes 7 endpoints secured with apiKey authentication. ## For AI agents Programmatically get an answer to a question about a given web page, extract structured data fields from a web page. Covers 7 operations with apiKey authentication. ## Scope Does not handle payments, communications, or crm - use for ai and machine learning only. ## Capabilities - Get an answer to a question about a given web page - Extract structured data fields from a web page - Page HTML by URL - HTML of a selected page area by URL and CSS selector - Information about your account calls quota ## Use cases ### AI and Machine Learning Operations Use the WebScraping.AI to perform ai ml operations programmatically. The API provides 7 endpoints covering core functionality including get an answer to a question about a given web page, extract structured data fields from a web page, page html by url. Example prompt: Call GET `/ai/question` to get an answer to a question about a given web page ### Automated AI Management Automate ai operations by combining multiple WebScraping.AI endpoints. Agents can extract structured data fields from a web page and then page html by url in a single workflow. Example prompt: Call GET `/ai/fields` to extract structured data fields from a web page, then verify the result ### AI Agent Integration via Jentic AI agents discover and call WebScraping.AI endpoints through Jentic without managing credentials directly. An agent searches for the required operation by intent, receives the matching endpoint schema, and executes the call with Jentic-managed authentication. This eliminates the need to read API documentation or handle apiKey tokens manually. Example prompt: Search Jentic for 'get an answer to a question about a given web page', load the operation schema, and execute with Jentic-managed credentials ## Key endpoints | Method | Path | Description | | --- | --- | --- | | GET | `/ai/question` | Get an answer to a question about a given web page | | GET | `/ai/fields` | Extract structured data fields from a web page | | GET | `/html` | Page HTML by URL | | GET | `/text` | Page text by URL | | GET | `/selected` | HTML of a selected page area by URL and CSS selector | | GET | `/selected-multiple` | HTML of multiple page areas by URL and CSS selectors | | GET | `/account` | Information about your account calls quota | ## Key resources - **AI** — Analyze web pages using LLMs - **HTML** — Get full HTML content of pages using proxies and Chromium JS rendering - **Text** — Get visible text of pages using proxies and Chromium JS rendering - **Selected HTML** — Get HTML content of selected page areas (like price, search results, page title, etc.) - **Account** — Information about your account API credits quota ## AI readiness This API is usable in Jentic One now. Its AI-readiness score against Jentic's framework shows where it stands today and where improvements would make it even easier for agents to use. - **Score:** 42 / 100 - **Maturity:** Foundational - **Dimensions:** - Foundational Compliance: 100 / 100 - Developer Experience & Jentic Compatibility: 75 / 100 - AI-Readiness & Agent Experience: 29 / 100 - Agent Usability: 94 / 100 - Security: 15 / 100 - AI Discoverability: 84 / 100 - **View full report:** https://jentic.com/apis/webscraping.ai/webscraping/scorecard - **How the score is calculated:** https://docs.jentic.com/reference/api-readiness-framework/overview/ - **More about the dimensions:** https://docs.jentic.com/reference/api-readiness-framework/specification/#dimensional-model-overview ### Score it yourself Every API in the directory is allowlisted, so you can re-score it with no key required. - **Score your own API:** https://jentic.com/scorecard.md - **Scoring CLI agent skill:** https://github.com/jentic/jentic-api-scorecard/blob/main/skills/jentic-api-scorecard/SKILL.md ```sh npx @jentic/api-scorecard-cli score ``` ## Why Jentic - **Setup:** Wiring WebScraping.AI by hand means passing its api_key on every query, managing proxy and rendering options across the extraction endpoints, and handling retries yourself. Through Jentic you install once, import WebScraping.AI from the API Directory, store the API key once, and your agent calls it. - **Permission scoping:** WebScraping.AI carries the target URL and query in each request rather than in the path, so scope the agent to the operations it needs, such as fetching page text or asking a question about a page. You choose the operations it may call, so account or field-extraction calls are not included unless you add them. - **Credential handling:** Your WebScraping.AI API key is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context. - **Discovery method:** Agents search Jentic by intent such as 'get an answer to a question about a web page' or 'extract the text of a page', and Jentic returns the matching WebScraping.AI operation with its input schema so the agent calls the right endpoint without browsing the reference docs. ## Related APIs - **Openai** — Alternative ai ml API - **Anthropic** — Alternative ai ml API - **Cohere** — Complementary ai ml API - **Huggingface** — Complementary ai ml API ## FAQ ### What authentication does the WebScraping.AI use? The WebScraping.AI uses an API key passed in the `api_key` query. Through Jentic, these credentials are stored encrypted in your Jentic One instance and injected at execution time, so raw secrets never enter the agent context. ### Can I get an answer to a question about a given web page with the WebScraping.AI? Yes. Use the GET `/ai/question` endpoint. The API returns structured JSON responses that agents can parse and act on directly. ### What are the rate limits for the WebScraping.AI? Rate limits are not specified in the OpenAPI spec. Check the vendor documentation for current limits. Through Jentic, rate limiting is handled automatically with retry logic built into the execution layer. ### How do I get an answer to a question about a given web page through Jentic? Install the Jentic SDK with pip install jentic, authenticate through Jentic One, the self-hosted execution layer, then search for 'get an answer to a question about a given web page'. Jentic returns the matching WebScraping.AI operation with its input schema. Load the schema and execute the call - credentials are injected automatically. ### How many endpoints does the WebScraping.AI have? The WebScraping.AI exposes 7 endpoints covering ai, html, text operations. ### Can I limit what my agent is allowed to do with the WebScraping.AI API? Yes. Because you run Jentic One yourself, your own rules decide which WebScraping.AI operations and credentials the agent may use, and this API carries the target URL and query in each request rather than in the path, so you scope by operation. You can allow the agent to fetch page text (GET /text) or ask a question about a page (GET `/ai/question`) while leaving field extraction (GET `/ai/fields`) and account quota lookups (GET /account) out unless you explicitly add them. Only the operations you approve are callable, so the agent cannot reach beyond the pages and actions you permit.