Site Discovery

Discover URLs from robots.txt and XML sitemaps. Pages are not crawled or analyzed.