Crawl multiple pages of a site, respecting link depth and domain limits
Crawl multiple pages of a site, respecting link depth and domain limits. Returns async job_id; results include markdown or HTML per page. Use it to fetch and process many pages of content at once.
by api.agishub.com · discovered, not yet claimed by its owner · updated 2026-09-11
Mirrored from Coinbase's public x402 Bazaar. The description is the provider's own text and has not been reviewed here; the payment requirements are copied as published.
What it does
Crawl multiple pages of a site, respecting link depth and domain limits. Returns async job_id; results include markdown or HTML per page. Use it to fetch and process many pages of content at once.
Tags: bazaar
Use with an agent
- Discover. Agents find this listing with
GET https://bazaar.saylorinnovations.com/discovery/search?query=…, the MCP toolsearch_resources, or/discovery/resources. - Inspect. Fetch
/resources/12916.jsonfor the price and payment options. - Pay. Call
GET https://api.agishub.com/paid/crawl. The provider answers402with its payment requirements. An x402 client signs one of the options below and retries withPAYMENT-SIGNATURE. - Execute. The provider returns
200with the data. Agent Bazaar is not in the request path and never sees your payment.
For humans
You don't need an account or an API key; you need a wallet holding a small amount of USDC on Base, and an x402-capable client (see the examples below). Call the endpoint unpaid first to see exactly what it asks for.
Pricing & payment
- Price
- $0.1 / request
- Protocol
- x402 v2
- Auth
- No account or API key. Pay per request.
- Networks
- Base
- Calls (30d)
- 2 · 1 unique payers (third-party data from the public x402 Bazaar)
x402 payment requirements
| Scheme | Network | Asset | Amount | Pay to | Timeout |
|---|---|---|---|---|---|
| exact | Base eip155:8453 | USDC | $0.1 | 0x23Fc725e8EAa1c58900C6507Ba89be99a6DFc75d | 300s |
accepts[] (raw JSON)
[
{
"scheme": "exact",
"network": "eip155:8453",
"payTo": "0x23Fc725e8EAa1c58900C6507Ba89be99a6DFc75d",
"asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
"amount": "100000",
"amountUsd": 0.1,
"maxTimeoutSeconds": 300
}
]Schemas
Output schema (JSON Schema)
{
"bazaar": {
"info": {
"input": {
"body": {
"limit": 5,
"url": "https://example.com"
},
"bodyType": "json",
"method": "POST",
"type": "http"
},
"output": {
"example": {
"result": "Crawl multiple pages of a site, respecting link depth and domain limits. Returns async job_id; results include markdown "
},
"type": "json"
}
},
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"properties": {
"input": {
"additionalProperties": false,
"properties": {
"body": {
"additionalProperties": false,
"properties": {
"formats": {
"description": "Output formats per page (default ['markdown']). 'html' adds raw HTML.",
"items": {
"enum": [
"markdown",
"html"
],
"type": "string"
},
"type": "array"
},
"limit": {
"description": "Maximum pages to crawl (default 100, max 100 per call). Returns 202 with job_id for async processing. For larger sites, issue several calls: each one is priced the same, so 1.000 pages cost ten calls.",
"exclusiveMinimum": true,
"maximum": 100,
"minimum": 0,
"type": "integer"
},
"max_depth": {
"description": "Maximum link depth from root (default 2, max 10). Depth 0 = root only, depth 1 = root + direct children.",
"exclusiveMinimum": true,
"maximum": 10,
"minimum": 0,
"type": "integer"
},
"same_domain": {
"description": "Only crawl URLs on the same domain (default true). Subdomain links are excluded when false.",
"type": "boolean"
},
"url": {
"description": "Root domain URL to crawl (e.g., https://example.com).",
"format": "uri",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"bodyType": {
"enum": [
"json"
],
"type": "string"
},
"method": {
"enum": [
"POST"
],
"type": "string"
},
"type": {
"enum": [
"http"
],
"type": "string"
}
},
"required": [
"type",
"method",
"bodyType",
"body"
],
"type": "object"
},
"output": {
"additionalProperties": false,
"properties": {
"example": {
"type": "object"
},
"type": {
"enum": [
"json"
],
"type": "string"
}
},
"required": [
"type",
"example"
],
"type": "object"
}
},
"required": [
"input",
"output"
],
"type": "object"
}
}
}Example shape — generated from the schema, not a captured live response
nullExamples
curl "https://api.agishub.com/paid/crawl"
# -> 402 Payment Required, accepts[] lists how to pay
# retry with a PAYMENT-SIGNATURE (or PAYMENT header) once paidconst res = await fetch("https://api.agishub.com/paid/crawl");
if (res.status === 402) {
const { accepts } = await res.json();
// pay one of accepts[] via an x402 client, then retry with the payment header
}import httpx
res = httpx.get("https://api.agishub.com/paid/crawl")
if res.status_code == 402:
accepts = res.json()["accepts"]
# pay one of accepts[] via an x402 client, then retry with the payment headerDetails
- Provider
- api.agishub.com
- Endpoint
- https://api.agishub.com/paid/crawl
- Machine-readable
/resources/12916.json