Home / Tools / apify.website_content_crawler

Apify Website Content Crawler

Free

Tool ID: apify.website_content_crawler

Crawl websites and extract clean Markdown/text for RAG and LLMs via Apify (apify~website-content-crawler).

Provider
apify
Category
Crawl & Site Mapping
Pricing
Free tier ($0.00; $5/mo Apify credits)
Replaces Signup
Replaces Apify account
Get API Key ($1.00 Free Credit) Browse All 334 Tools

Input Parameters

ParameterTypeRequiredDescription
start_urlsarrayRequiredSeed URLs to crawl
max_crawl_pagesintegerRequiredMaximum pages to crawl (1-20)
crawler_typestring (cheerio | playwright:adaptive | playwright:firefox)RequiredCrawler engine (cheerio is fastest; playwright renders JS)

Call via REST API

curl -X POST https://helve.dev/v1/tools/call \
  -H "Authorization: Bearer sk_live_YOUR_KEY" \
  -H "Content-Type: application/json" \
  -d '{
  "tool_id": "apify.website_content_crawler",
  "arguments": {
    "start_urls": [
      "https://docs.helve.dev"
    ],
    "max_crawl_pages": 3,
    "crawler_type": "cheerio"
  }
}'

Call via MCP (call_tool)

{
  "name": "call_tool",
  "arguments": {
    "tool_id": "apify.website_content_crawler",
    "arguments": {
      "start_urls": [
        "https://docs.helve.dev"
      ],
      "max_crawl_pages": 3,
      "crawler_type": "cheerio"
    }
  }
}

Related Crawl & Site Mapping Tools