Web Reconnaissance & Crawling Tools
Crawlers, scrapers, and recon tools for mapping a website's pages, technologies, and exposed files. Useful for security assessments and for gathering public content at scale.
Filter these in the interactive catalog
6 tools
- theHarvesterCollects public emails, names, subdomains, IPs, and URLs for a domain or organization, with passive and opt-in active modes.
- Fast Google Dorks ScanThe OSINT project, the main idea of which is to collect all the possible Google dorks search combinations and to find the information about the specific web-site: common admin panels, the widespread file types and path traversal. The 100% automated.
- HostHunterHostHunter a recon tool for discovering hostnames using OSINT techniques.
- InfooozeA OSINT tool which helps you to quickly find information effectively. All you need is to input and it will take take care of rest.
- Karma V2⡷⠂𝚔𝚊𝚛𝚖𝚊 𝚟𝟸⠐⢾ is a Passive Open Source Intelligence (OSINT) Automated Reconnaissance (framework)
- OSINT Team LinksLinks for the OSINT Team
Using web recon & crawling tools responsibly
- Respect robots.txt and rate limits; aggressive crawling can disrupt a site.
- Crawling or scanning systems you do not own may require authorization.
- Keep logs of what you collected and when.