reNgine: Automated Web Reconnaissance Framework with Web UI and Scan Pipelines

reNgine: Recon Tool Orchestration

Running recon tools individually makes sense, you control each step. The pipeline is as complex as your shell skills.

The downside is equally clear. Setting up pipelines from scratch, managing output files, and reviewing results from multiple tools adds up. Across multiple clients, it eats into investigative time.

reNgine streamlines this. Define your pipeline in YAML once, run it on a schedule against multiple targets, and view results in a database.

The tools are familiar: subfinder, amass, httpx, naabu, nuclei, ffuf. reNgine handles orchestration and result storage.

What reNgine Is

reNgine is a self-hosted automated web reconnaissance framework. It executes configured scan pipelines through a web UI. A pipeline run covers subdomain enumeration, live host probing, port scanning, endpoint discovery, screenshot capture, and vulnerability checks, the full web recon workflow, in a single invocation against a defined target scope.

The scan engine model is the central concept. An engine is a YAML configuration defining which tools run, the parameters they run with, and the order of execution. You define a lightweight passive engine for quick weekly sweeps. A comprehensive active engine is used for monthly deep scans. An in-scope-only engine is used for sensitive engagements where scope control matters. Any engine can be applied to any target on demand or on a schedule.

The web interface handles everything. Target management, scan engine selection, scheduled scan configuration, and result review all occur in the browser. For teams with multiple analysts needing access to shared results, and for engagements where findings are reviewed without SSHing into a server, the interface provides operational value.

The full stack is deployed using Docker Compose. This includes the Django application, PostgreSQL database, Redis for queuing, Celery workers for pipeline execution, and the bundled recon tool binaries. A dedicated VPS with 4-8 gigabytes of RAM handles moderate scan loads comfortably.

Scan Engine Configuration

The scan engines are defined in YAML files. The modules that run include subdomain_discovery, port_scanning, screenshot, and vulnerability_scan. The tools specified are subfinder and amass for subdomains, naabu for ports, and nuclei for vulnerabilities.

The pipeline runs automatically, step by step. Subdomain discovery runs first, populating the database with hostnames. Live host probing uses those hostnames, and httpx probes them, marking live hosts.

Port scanning then hits live hosts. Screenshots are taken of live web services. Nuclei scans live hosts. The templates used in these scans are configurable. Each stage reads from the previous one, eliminating the need for manual work or temporary files.

The YAML files are easy to read and edit, requiring no coding knowledge. Changes can be made to subfinder sources, nuclei templates can be adjusted, and tools like ffuf can be added, all within a couple of minutes, and take effect at the next scan.

New scan engines can be created simply by adding new YAML files, with no need for shell scripts. Multiple engines can be run, providing a history of scans. For example, scanning a target eight times results in eight records.

The database allows for finding new subdomains, filtering by creation date, and comparing over time. This functionality is not available with manual runs, and is a key benefit of the persistence layer.

Web UI and Results Management

The database-backed interface adds significant value to reNgine. Every scan run stores results: subdomains, live hosts, open ports, screenshots, endpoints, vulnerabilities. You can query them all.

You can filter subdomains by HTTP status code, finding all hosts returning 403. You can filter by technology, finding targets running a specific CMS version. You can filter by vulnerability type, seeing all findings across clients where a nuclei template fired.

The screenshot gallery accelerates large-scale web reconnaissance. When triaging hundreds of live web hosts, the gallery helps you quickly identify which ones are worth investigating. Rather than opening URLs one by one, the gallery loads thumbnails in a paginated view, allowing you to conduct a visual triage pass. This helps you identify login panels, admin interfaces, default pages, and custom applications in a fraction of the time.

reNgine's notification integrations push findings to your team. Configure Slack or Discord webhooks to receive alerts on scan completions, new subdomains, and nuclei vulnerability findings. Scheduled scans run overnight or on weekends, with findings appearing in team channels immediately, eliminating the need for manual review.

reNgine vs bbot: Choosing the Right Tool

bbot is a CLI tool. It is built for digging deep on one target. It scans and rescans automatically, pulling in more info as it goes. Its module library is bigger than reNgine's. For solo ops doing deep recon on one target, bbot gives you more detail in one go.

reNgine is different. It is for checking on many targets over time. It has a web UI, a database, and scheduled scans. You can set it and forget it, then come back to see what's changed. Multiple analysts can use it.

bbot is for deep recon on one target. reNgine is for keeping tabs on many targets. They are not interchangeable, but they are both useful. Choose the right tool for the job.

Deployment Considerations

Recon Deployment

The Docker Compose stack is the way to go. One command sets up Django, PostgreSQL, Redis, and Celery workers.

The recon tools — subfinder, amass, nuclei, naabu, httpx, ffuf — are part of the app image. No extra tool installs are needed.

Sizing Your Infrastructure

Don't skimp on the infrastructure. PostgreSQL uses up disk space. A year's worth of scans across clients needs room to breathe. You don't want to run out of space mid-scan.

Celery workers need RAM. For light use, four gigs might be enough. For heavy use, eight to sixteen gigs are safer for multiple scan pipelines running at once.

Scope Control: Don't Overrun

Scheduled scans can get out of hand. For big orgs and big targets, configure CIDR restrictions. Limit recursion depth in YAML. Don't let nuclei templates fire willy-nilly.

Unlimited recursion against a big org's ASN will result in waiting and then reviewing for hours. Keep scans actionable, not exhaustive.

Who is this for?

This solution is for pentesters and security teams with clients who need scheduled scans, history, and shared findings: pentesters, security teams, clients.

Get it on GitHub: yogeshojha/rengine