Configure the crawler to scrape a website
URLs to start crawling from (one per line)
Maximum number of pages to scrape (1-500)
How deep to follow links (1-10)
Only crawl URLs matching these regex patterns
Skip URLs matching these regex patterns