For Agents
Lets agents scrape one URL, batch up to 10 URLs, run web-scraper or crawl jobs, discover URLs on a domain, run AI-powered web search, or kick off agentic research jobs — all via X-API-Key authenticated POST/GET pairs.
Install Jentic One Beta
Jentic One is a self-hosted execution layer for AI agents. It lets your agent call the Anakin API, or any other public or private API you need. You set the rules, the agent never sees your credentials, and every call is logged.
Two steps, two machines. Install the instance in a safe environment, then register your agent from wherever it runs.
Step 1: Jentic One Host machine
# On the machine that will host your Jentic One instance:
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | shStep 2: Agent machine
# On the machine where your agent runs (keep this separate from the instance):
curl -fsSL https://raw.githubusercontent.com/jentic/jentic-one/main/tools/install.sh | sh
jentic register # connects your agent to your Jentic One instanceJentic One is in public beta. The setup above keeps your agent separate from the instance, which is what you want before using real credentials: an agent running as the same OS user as Jentic One can read its stored keys directly. Just evaluating? A single local install is fine to start. See the secure deployment guide for the tiers.
What an agent can do with Anakin API.
Scrape a single URL (or up to 10 in a batch) and retrieve the cleaned content as a job result
Run a web-scraper job that handles JavaScript-heavy pages and follows pagination rules
Crawl a website starting from a root URL and collect content across multiple pages
GET STARTED
Use for: I need to scrape a single URL and get the cleaned text, Batch scrape 10 URLs in one call, Crawl a website starting from a root URL and pull every page, Discover all URLs on a domain without scraping content
Not supported: Does not handle structured-data APIs, browser automation flows, or proxy management — use for content scraping, crawling, mapping, and AI-powered web search only.
The Anakin API is a web scraping and data extraction platform exposing 12 endpoints for URL scraping, batch scraping, web scraping jobs, web crawling, sitemap-style URL discovery (map), AI-powered web search, and longer-running agentic research jobs. Each long-running operation is a POST that returns a job id, with a paired GET /job/{id} or resource-specific status endpoint for polling. Authentication is via the X-API-Key header against api.anakin.io/v1. Typical consumers are LLM RAG pipelines, sales prospecting tools, and research agents that need clean structured content from arbitrary websites.
Discover the URL map of a website without fully crawling it
Run AI-powered web search with synthesised results suitable for grounding an LLM
Kick off an agentic-search job that performs multi-step research and returns a structured answer
Poll job status and pull final results once a long-running scrape, crawl, or research job completes
Patterns agents use Anakin API for, with concrete tasks.
★ RAG Ingestion of a Documentation Site
LLM platform teams can stand up a documentation RAG pipeline by calling POST /map to discover the docs URL set, POST /crawl to crawl the resulting site, then poll GET /crawl/{id} for the cleaned content. The two-step (kick off, then poll) pattern is consistent across long-running endpoints, so the same client code wraps every job type. Most teams stand up an end-to-end pipeline in a day.
Call POST /map with url='https://docs.example.com', then POST /crawl with the discovered root URL, then poll GET /crawl/{id} until status='completed' and return the cleaned page content
Sales Prospecting Page Extraction
Sales-prospecting tools can call POST /scrape on a company homepage or POST /scrape-batch on a list of 10 candidate URLs to pull pricing, headcount, and tech-stack signals. The job-based shape lets the tool process inbound leads asynchronously rather than blocking on every scrape. The X-API-Key model keeps the integration trivial.
Call POST /scrape-batch with 10 candidate company URLs, retrieve the resulting job id, then poll GET /job/{id} until completion and return each page's cleaned content
Agentic Research for Investment Notes
Investment and research teams can hand off a research question to POST /agentic-search, get back a job id, and poll GET /agentic-search/{id} for a multi-step researched answer. The agentic endpoint handles the search-and-synthesise loop server-side so the calling team does not need to wire up its own tool-calling scaffold. Pair with POST /search for lighter-weight grounded answers.
Call POST /agentic-search with a question like 'recent funding rounds for vector database startups', poll GET /agentic-search/{id}, and return the final structured answer
AI Agent Web Research Loop
Research and copywriting agents can use Anakin through Jentic to scrape, crawl, and search without managing an X-API-Key directly. The agent searches Jentic for a scrape, crawl, search, or agentic-search intent, loads the operation schema, and Jentic injects the key at execution time. Useful when an agent already chains LLM tool calls and just needs a clean web-data tool in the mix.
Search Jentic for 'scrape a single URL', execute the operation backed by POST /scrape with url='https://example.com/pricing', poll GET /job/{id} until completion, and return the cleaned page content
12 endpoints — the anakin api is a web scraping and data extraction platform exposing 12 endpoints for url scraping, batch scraping, web scraping jobs, web crawling, sitemap-style url discovery (map), ai-powered web search, and longer-running agentic research jobs.
METHOD
PATH
DESCRIPTION
/scrape
Scrape a single URL
/scrape-batch
Batch scrape up to 10 URLs
/job/{id}
Get scrape job status and results
/web-scraper
Start a JavaScript-aware web scraper job
/map
Discover URLs on a website
/crawl
Crawl multiple pages starting from a root URL
/search
AI-powered web search
/agentic-search
Start an agentic research job
/scrape
Scrape a single URL
/scrape-batch
Batch scrape up to 10 URLs
/job/{id}
Get scrape job status and results
/web-scraper
Start a JavaScript-aware web scraper job
/map
Discover URLs on a website
Three things that make agents converge on Jentic-routed access.
Credential isolation
The Anakin X-API-Key is stored encrypted in the Jentic vault. Agents receive scoped execution capability — the key never enters the agent's context, prompts, or logs.
Intent-based discovery
Agents search Jentic by intent (e.g. 'scrape a URL', 'crawl a website', 'run agentic research') and Jentic returns the matching Anakin operation with its input schema, so the agent calls the right endpoint without browsing anakin.io/docs.
Time to first call
Direct Anakin integration: 4-8 hours for auth, polling loops, and error handling on long-running jobs. Through Jentic: under 30 minutes — search, load schema, execute, poll.
Alternatives and complements available in the Jentic catalogue.
Specific to using Anakin API through Jentic.
What authentication does the Anakin API use?
Anakin uses API key authentication via the X-API-Key header on every request. There is no OAuth flow. Through Jentic, the X-API-Key is stored encrypted in the vault and injected at execution time, so the agent never sees the raw key.
Can I scrape multiple URLs in a single call with the Anakin API?
Yes. POST /scrape-batch accepts up to 10 URLs in one request and returns a job id. Poll GET /job/{id} until the status flips to completed and the response carries the cleaned content for each URL. For higher volume, queue multiple batches rather than expanding a single batch.
What are the rate limits for the Anakin API?
The spec does not document rate limits explicitly. Anakin applies them per plan tier — paid plans support more concurrent jobs and higher daily caps than the free trial. Treat HTTP 429 as a signal to back off and reduce concurrent jobs rather than retry tightly.
How do I run an agentic research job through Jentic?
Search Jentic for 'run agentic research'. Jentic returns the operation backed by POST /agentic-search, you load the schema (the question is the main input), execute, then poll GET /agentic-search/{id} for the result. Jentic injects the X-API-Key from the vault. Sign up at https://app.jentic.com/sign-up.
Are the long-running endpoints synchronous?
No. Scrape, web-scraper, map, crawl, and agentic-search endpoints are all kick-off-and-poll. Each POST returns a job id, and a paired GET endpoint (/job/{id}, /web-scraper/{id}, /map/{id}, /crawl/{id}, /agentic-search/{id}) returns status and final results. Plan for asynchronous handling in any client wrapping this API.
Can the Anakin API handle JavaScript-heavy pages?
Yes. POST /web-scraper is the JavaScript-aware path designed for SPAs and rendered pages, while POST /scrape is the lighter HTML-fetch path. Pick /web-scraper when the page needs rendering and /scrape when a clean HTML response is enough.
/crawl
Crawl multiple pages starting from a root URL
/search
AI-powered web search
/agentic-search
Start an agentic research job