Discover all URLs reachable from a domain within a link depth limit
Discover all URLs reachable from a domain within a link depth limit. Returns a flat list of absolute URLs, respects robots.txt crawl delays. Use it to map a site's structure before crawling.
by api.agishub.com · discovered, not yet claimed by its owner · updated 2026-09-08
Mirrored from Coinbase's public x402 Bazaar. The description is the provider's own text and has not been reviewed here; the payment requirements are copied as published.
What it does
Discover all URLs reachable from a domain within a link depth limit. Returns a flat list of absolute URLs, respects robots.txt crawl delays. Use it to map a site's structure before crawling.
Tags: bazaar
Use with an agent
- Discover. Agents find this listing with
GET https://bazaar.saylorinnovations.com/discovery/search?query=…, the MCP toolsearch_resources, or/discovery/resources. - Inspect. Fetch
/resources/12935.jsonfor the price and payment options. - Pay. Call
GET https://api.agishub.com/v1/crawl-map. The provider answers402with its payment requirements. An x402 client signs one of the options below and retries withPAYMENT-SIGNATURE. - Execute. The provider returns
200with the data. Agent Bazaar is not in the request path and never sees your payment.
For humans
You don't need an account or an API key; you need a wallet holding a small amount of USDC on Base, and an x402-capable client (see the examples below). Call the endpoint unpaid first to see exactly what it asks for.
Pricing & payment
- Price
- $0.05 / request
- Protocol
- x402 v2
- Auth
- No account or API key. Pay per request.
- Networks
- Base
- Calls (30d)
- 1 · 1 unique payers (third-party data from the public x402 Bazaar)
x402 payment requirements
| Scheme | Network | Asset | Amount | Pay to | Timeout |
|---|---|---|---|---|---|
| exact | Base eip155:8453 | USDC | $0.05 | 0x23Fc725e8EAa1c58900C6507Ba89be99a6DFc75d | 300s |
accepts[] (raw JSON)
[
{
"scheme": "exact",
"network": "eip155:8453",
"payTo": "0x23Fc725e8EAa1c58900C6507Ba89be99a6DFc75d",
"asset": "0x833589fCD6eDb6E08f4c7C32D4f71b54bdA02913",
"amount": "50000",
"amountUsd": 0.05,
"maxTimeoutSeconds": 300
}
]Schemas
Output schema (JSON Schema)
{
"bazaar": {
"info": {
"input": {
"body": {
"limit": 5,
"url": "https://example.com"
},
"bodyType": "json",
"method": "POST",
"type": "http"
},
"output": {
"example": {
"result": "Discover all URLs reachable from a domain within a link depth limit. Returns a flat list of absolute URLs, respects robo"
},
"type": "json"
}
},
"schema": {
"$schema": "https://json-schema.org/draft/2020-12/schema",
"properties": {
"input": {
"additionalProperties": false,
"properties": {
"body": {
"additionalProperties": false,
"properties": {
"include_subdomains": {
"description": "Include URLs from subdomains (default false, same domain only).",
"type": "boolean"
},
"limit": {
"description": "Maximum URLs to return (default 100, max 200 per call). Respects robots.txt crawl-delay.",
"exclusiveMinimum": true,
"maximum": 200,
"minimum": 0,
"type": "integer"
},
"search": {
"description": "Optional regex or plain string to filter results (case-insensitive).",
"type": "string"
},
"url": {
"description": "Root domain URL to map (e.g., https://example.com).",
"format": "uri",
"type": "string"
}
},
"required": [
"url"
],
"type": "object"
},
"bodyType": {
"enum": [
"json"
],
"type": "string"
},
"method": {
"enum": [
"POST"
],
"type": "string"
},
"type": {
"enum": [
"http"
],
"type": "string"
}
},
"required": [
"type",
"method",
"bodyType",
"body"
],
"type": "object"
},
"output": {
"additionalProperties": false,
"properties": {
"example": {
"type": "object"
},
"type": {
"enum": [
"json"
],
"type": "string"
}
},
"required": [
"type",
"example"
],
"type": "object"
}
},
"required": [
"input",
"output"
],
"type": "object"
}
}
}Example shape — generated from the schema, not a captured live response
nullExamples
curl "https://api.agishub.com/v1/crawl-map"
# -> 402 Payment Required, accepts[] lists how to pay
# retry with a PAYMENT-SIGNATURE (or PAYMENT header) once paidconst res = await fetch("https://api.agishub.com/v1/crawl-map");
if (res.status === 402) {
const { accepts } = await res.json();
// pay one of accepts[] via an x402 client, then retry with the payment header
}import httpx
res = httpx.get("https://api.agishub.com/v1/crawl-map")
if res.status_code == 402:
accepts = res.json()["accepts"]
# pay one of accepts[] via an x402 client, then retry with the payment headerDetails
- Provider
- api.agishub.com
- Endpoint
- https://api.agishub.com/v1/crawl-map
- Machine-readable
/resources/12935.json