theHarvester: Email, Subdomain, and Hostname Recon from Public Sources

Gathering Public Info

Every external recon engagement starts with gathering public info on the target.

TheHarvester collects email addresses, subdomains, hostnames, IP ranges. No packets are sent to the target.

Give it a domain, point to your data sources. Minutes later, you have a basic map.

It's a foundation. Not a dashboard. Not fancy. But unmatched in the free tools. The advantage is speed. You move fast. Get more done. That's it.

What theHarvester Collects

Primary outputs are email addresses, subdomains, hostnames, IPs, and open ports. These come from public sources, such as certificate transparency logs, search engine indices, DNS aggregators, and threat intelligence feeds.

Secondary outputs vary by source and may include employee names, LinkedIn data, URLs from search engines, and ASN info. This information maps IPs to their owning organization, helps build an organization chart, and confirms infrastructure ownership.

TheHarvester's default mode is passive; it queries third-party sources without direct contact with the target. The -c flag enables DNS brute-force enumeration, generating DNS queries against target nameservers, which constitutes active reconnaissance. Use this option with a defined scope.

Data Sources and the API Key Reality

theHarvester comes with over 30 source integrations. Several sources work right away, no authentication needed, such as crt.sh, DNSdumpster, HackerTarget, and Bing. You will encounter limitations with unauthenticated queries. Sources that provide valuable information require API keys, including Shodan, Censys, and ZoomEye.

Key registrations to prioritize:

• Hunter.io — best source for email address harvesting; free tier gives 25 searches/month
• Shodan — surfaces open ports and service banners on discovered IPs; free API access with a registered account
• FullHunt — strong subdomain enumeration; free tier covers light use
• SecurityTrails — historical DNS data and subdomain enumeration; free tier is limited but useful

API keys are stored in the api-keys.yaml file, located in the theHarvester root directory. Adding them there allows them to be picked up on the next run.

TheHarvester supports various APIs, including Hunter.io, FullHunt for email, Shodan, crt.sh, SecurityTrails for infrastructure.

Running the command with -b all without API keys is inefficient, as sources return nothing.

Installation and Setup

To get started, make sure you have Python 3.8 or higher installed. It's also recommended to use a virtual environment for a clean setup. The cleanest install path is:

pip install theHarvester

Installation

To get started, simply handle dependencies in one step. You can clone the repository from GitHub, then install from the requirements file. The GitHub version tracks the latest updates, providing source list updates more quickly than the pip package.

git clone https://github.com/laramies/theHarvester
cd theHarvester
pip install -r requirements/base.txt

Kali Linux comes with theHarvester pre-installed. The version tends to be outdated. Check your Kali install's version. If it's lagging, grab the latest from GitHub.

Running theHarvester in a Docker container is a good option when querying multiple paid APIs. Your credentials stay isolated, and you avoid dependency conflicts on a shared machine.

Running Effective Queries

The core invocation:

theHarvester -d target.com -b all -l 500 -f output

When using the tool, there are several flags to keep in mind. The target domain is specified with -d, while the data source is specified with -b. To query everything configured, use all, but use this option with caution. The -l flag allows you to limit the number of results per source. Additionally, the -f flag enables you to write output to HTML and XML files.

It's best to avoid using -b all, as sources may time out or return empty results. Instead, once you know which sources produce results for your target, specify them individually. It's also a good practice to save the output with the -f flag, which helps with operations.

theHarvester -d target.com -b hunter,shodan,crt -l 500 -f output

This runs faster and produces cleaner output.

Two additional flags are useful. The -n flag performs DNS resolution on hostnames, converting them to IPs in the output. It is a passive operation, so it can be safely enabled by default. The -c flag brute-forces DNS using a built-in list; it is an active operation and slower. Use it deliberately.

Interpreting and Acting on Output

Email addresses are easily accessible. The naming pattern is what you want, such as j.smith@target.com and a.jones@target.com. Now you know how they name their staff. Cross-referencing with LinkedIn provides a full employee list, making it unnecessary to scrape every address. Email addresses including first and last names, and job titles, help to identify staff.

Subdomains are often overlooked assets. Look for subdomains such as dev, staging, vpn, legacy, and admin. These may not match the main domain's security baseline.

IPs can be directly input into Shodan or Censys. Taking the IP list and querying Shodan yields a service and port map. All the information is external and comes from public data. No interaction is needed, just data. Operators often miss these.

Limitations

Source freshness is an issue. Some of theHarvester's sources update slowly, others cache results heavily, resulting in stale data that can be weeks or months old.

crt.sh and Shodan usually have current data, but search engines often do not; their results can be months behind.

Automated email scraping from search engines has decreased due to Google and Bing cracking down on bots. Although emails still appear in search results, they are sparse. Hunter.io and FullHunt have filled the gap.

TheHarvester's output requires manual review. The results contain duplicates and noise, with no duplicates removed, no confidence scores, and no false positive flags. You will need to manually review the results, remove duplicates, and eliminate noise.

Verdict

theHarvester gets you oriented fast. It is passive, no scanning needed. One run surfaces email patterns, subdomains, and IP ranges. That would take hours to dig up manually.

Pentesters mapping external attack surface and OSINT investigators building a profile love it. The value of theHarvester is its zero-to-oriented time: in ten minutes, with a few API keys set, and you've got a target map. Everything else builds from there.

The only cost is setup, which takes twenty minutes to set up API keys. Without them, you're barely scraping by. Add Hunter.io, Shodan, crt.sh, and you've got something. The tool is free, and the keys are free; it's just the setup.

TheHarvester is best for attack surface mapping, organizational profiling, email patterns. It is available on GitHub at laramies/theHarvester.