Home / Tools / Crawl & Site Mapping

Crawl & Site Mapping Tools for AI Agents

Full-site crawling and URL mapping APIs for agentic research and indexing.

Tools in Category
2
Providers
2
Free Tools
0
Metered Tools
2
← All Tools (313)LLMs & Chat (3)Web Search (11)Web Extraction (5)Deep Research & Answers (8)Crawl & Site Mapping (2)Audio & Text-to-Speech (3)Audio Transcription (1)Embeddings & Rerank (2)Finance, Markets & Crypto (67)Developer, Security & Registries (79)Science, Gov, Geo & Knowledge (114)Utilities & Media (18)
Firecrawl Website Sitemap / URL Map
firecrawl.map · firecrawl
Metered

Discover and filter all reachable URLs on a website using Firecrawl v2/map before extracting specific pages.

Replaces Firecrawl API account
~$0.0032/call (+15% Helve fee) Schema →
Tavily Website Map
tavily.map · tavily
Metered

Map a website's structure and discover relevant URLs with optional natural-language instructions via Tavily.

Replaces Tavily API account
~$0.008/call (+15% Helve fee) Schema →