Install Jentic One Beta

Connect the Spider API to your agent

Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Spider API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.

Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.

Step 1: Jentic One Host machine

# On the machine that will host your Jentic One instance:
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fspider.cloud%2Fspider-cloud" | sh

Step 2: Agent machine

# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL "https://jentic.com/install.sh?src=apis&api=%2Fapis%2Fspider.cloud%2Fspider-cloud" | sh
jentic register       # connects your agent to your Jentic One instance

Jentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.

Capabilities

What an agent can do with Spider API.

Crawl website

Scrape page

Bypass anti-bot protection

Search web

Collect links

GET STARTED

Start building with Spider API

Explore with Jentic One

View OpenAPI Document

Use Cases

Patterns agents use Spider API for, with concrete tasks.

★ Developer Tools Operations

Use the Spider API to perform developer tools operations programmatically. The API provides 11 endpoints covering core functionality including crawl website, scrape page, bypass anti-bot protection.

Call POST /crawl to crawl website

Automated Crawl Management

Automate crawl operations by combining multiple Spider API endpoints. Agents can scrape page and then bypass anti-bot protection in a single workflow.

Call POST /scrape to scrape page, then verify the result

AI Agent Integration via Jentic

AI agents discover and call Spider API endpoints through Jentic without managing credentials directly. An agent searches for the required operation by intent, receives the matching endpoint schema, and executes the call with Jentic-managed authentication. This eliminates the need to read API documentation or handle bearer tokens manually.

Search Jentic for 'crawl website', load the operation schema, and execute with Jentic-managed credentials

Key Endpoints

11 endpoints — web crawling, scraping, and data extraction api with advanced features like screenshot capture, anti-bot bypass, and content transformation.

METHOD

PATH

DESCRIPTION

POST

/crawl

Crawl website

POST

/scrape

Scrape page

POST

/unblocker

Bypass anti-bot protection

POST

/search

Search web

POST

/links

Collect links

POST

/screenshot

Capture screenshot

POST

/transform

Transform content

POST

/fetch/{domain}/{path}

Fetch with AI config

POST

/crawl

Crawl website

POST

/scrape

Scrape page

POST

/unblocker

Bypass anti-bot protection

POST

/search

Search web

POST

/links

Collect links

POST

/screenshot

Capture screenshot

POST

/transform

Transform content

POST

/fetch/{domain}/{path}

Fetch with AI config

Why Jentic?

What agents get from Jentic-routed access to this vendor.

Setup

Wiring the Spider API by hand means carrying its bearer token on every request and wiring the crawl, scrape, search, and screenshot operations yourself. Through Jentic you install once, import the Spider API from the API Directory, store the token once, and your agent calls it.

Permission scoping

Spider passes its targets in the request body, so limit the agent to the operations it needs, such as scraping a page or running a crawl. You choose the operations it may call, so it only runs the crawl, scrape, or search actions you allow and nothing more.

Credential isolation

Your Spider bearer token is stored once, encrypted, by your own Jentic One instance and injected at execution time. It never enters the agent's prompt, logs, or context.

Intent-based discovery

Agents search Jentic by intent such as 'crawl a website' or 'take a screenshot of a page', and Jentic returns the matching Spider operation with its input schema so the agent calls the right endpoint without browsing the reference docs.

FAQs

Specific to using Spider API through Jentic.

What authentication does the Spider API use?

The Spider API uses a Bearer token in the Authorization header. Through Jentic, these credentials are stored encrypted in your Jentic One instance and injected at execution time, so raw secrets never enter the agent context.

Can I crawl website with the Spider API?

Yes. Use the POST /crawl endpoint. The API returns structured JSON responses that agents can parse and act on directly.

What are the rate limits for the Spider API?

Rate limits are not specified in the OpenAPI spec. Check the vendor documentation for current limits. Through Jentic, rate limiting is handled automatically with retry logic built into the execution layer.

How do I crawl website through Jentic?

Install the Jentic SDK with pip install jentic, authenticate through Jentic One, the self-hosted execution layer, then search for 'crawl website'. Jentic returns the matching Spider API operation with its input schema. Load the schema and execute the call - credentials are injected automatically.

How many endpoints does the Spider API have?

The Spider API exposes 11 endpoints covering crawl, scrape, data operations.

Can I limit what my agent is allowed to do with the Spider API?

Yes. Jentic One is self-hosted by you, so your own rules decide which Spider operations and which credentials the agent may use. Because Spider passes its targets in the request body, you can allow only the operations the agent needs, such as scraping a page with POST /scrape or running a crawl with POST /crawl, while withholding actions like search or screenshot capture. The agent can call only the endpoints you permit and nothing else.

Spider API

Name: Spider API
Brand: Spider API
Availability: InStock

AI ReadinessAI-Aware (B+)68/100

See full scorecard

Official vendor OpenAPI document · agent-readyDeveloper ToolsSource Controlbearer11 EndpointsREST

For Agents

Programmatically crawl website, scrape page. Covers 11 operations with bearer authentication.

Use for: I need to crawl website, I want to scrape page, Search for bypass anti-bot protection, Find all web

Not supported: Does not handle payments, communications, or crm - use for developer tools only.

Web crawling, scraping, and data extraction API with advanced features like screenshot capture, anti-bot bypass, and content transformation. The API exposes 11 endpoints secured with bearer authentication.

Jentic One on Github View OpenAPI Document

Show dimension breakdown

100

Foundational Compliance

Base layer of spec validity and structural soundness.

Grade: A+Signals: 4

99%

Lint Results

Aggregated quality score from linter diagnostics, weighted by severity.

100%

Resolution Completeness

Percentage of `$ref` references that resolve successfully.

100%

Specification Validity

Checks whether the API description parses successfully and conforms to its declared specification (e.g., OpenAPI).

100%

Structural Integrity

Structural correctness score based on schema issues using logarithmic dampening.

Developer Experience & Jentic Compatibility

Clarity, completeness, and ingestion readiness for developers and tooling.

Grade: B-Signals: 4

Example Density

How richly the API is illustrated with examples.

100%

Example Validity

Percentage of examples that conform to their schemas.

50%

Response Coverage

Percentage of operations with complete response definitions (success, client error, server error).

100%

Tooling Readiness

Health of API ingestion, bundling, and resolution within Jentic pipelines.

AI-Readiness & Agent Experience

Semantic breadth, depth, and agent comprehension for AI systems.

Grade: DSignals: 4

45%

Description Coverage

Coverage of descriptions across API elements.

Error Standardization

Coverage of RFC 9457 Problem Details for error responses.

91%

OperationId Quality

Coverage, uniqueness, and casing consistency of operationIds for AI inference.

50%

Summary Coverage

Coverage of summaries across operations/tags/info.

Agent Usability

Functional utility, complexity comfort, and AI orchestration readiness.

Grade: A+Signals: 1

94%

Complexity Comfort

Agent comfort level based on API operational and structural complexity.

Security

Trust, risk posture, and security compliance.

Grade: B-Signals: 1

60%

Authentication Strength

Average quality of security schemes based on authentication method strength (weakest link for OAuth2).

100

AI Discoverability

Findability, semantic richness, and reasoning readiness.

Grade: A+Signals: 1

100%

Descriptive Richness

Clarity and depth of descriptions across API elements.

Connect the Spider API to your agent

Capabilities

Start building with Spider API

Use Cases

Key Endpoints

Why Jentic?

Related APIs

FAQs

Spider API

Jentic AI Readiness Score

Spider API

Foundational Compliance

Lint Results

Resolution Completeness

Specification Validity

Structural Integrity

Developer Experience & Jentic Compatibility

Example Density

Example Validity

Response Coverage

Tooling Readiness

AI-Readiness & Agent Experience

Description Coverage

Error Standardization

OperationId Quality

Summary Coverage

Agent Usability

Complexity Comfort

Security

Authentication Strength

AI Discoverability

Descriptive Richness