Building a Recon Automation Pipeline

Manual recon finds bugs once. Automated recon finds the new subdomain the morning it appears, before anyone else has looked at it. The best bug bounty hunters treat recon as a system, not a task.

Automation multiplies mistakes. A pipeline pointed at the wrong scope sends thousands of unauthorized requests. Build the scope check in first, respect program rate limits, and identify your traffic when programs ask you to.

The shape of the pipeline

scope.txt ─▶ discover ─▶ resolve ─▶ filter to scope ─▶ probe ─▶ diff ─▶ notify

Each stage reads lines and writes lines. That’s deliberate: plain text pipes are easy to debug, resume, and extend.

Stage 1: scope as code

Keep the program’s scope in files, and filter every stage’s output through them:

# scope.txt
example.com
example.io

# out-of-scope.txt
status.example.com
legacy.example.com

in_scope() {
  grep -Ff scope.txt | grep -vFxf out-of-scope.txt
}

grep -F matches fixed strings, so a scope entry like example.com also matches api.example.com. Review that behavior against each program’s wording, since some programs exclude subdomains.

Stage 2: discovery

Passive sources first. They send no traffic to the target:

subfinder -dL scope.txt -all -silent > raw.txt
curl -s "https://crt.sh/?q=%25.example.com&output=json" | jq -r '.[].name_value' >> raw.txt
sort -u raw.txt | in_scope > subdomains.txt

Stage 3: resolve and probe

Resolve names, then probe for live web services. Set a rate limit that matches the program’s rules:

dnsx -l subdomains.txt -silent -a -resp > resolved.txt
httpx -l subdomains.txt -silent -rate-limit 10 \
  -status-code -title -tech-detect -follow-redirects \
  -H "X-Bug-Bounty: your-handle" > live.txt

The custom header lets the program’s security team identify your traffic. Many programs require one.

Stage 4: diff and notify

The pipeline’s real value is change detection. anew appends only lines it hasn’t seen before and prints them:

cut -d' ' -f1 live.txt | anew known-hosts.txt > new-hosts.txt
if [ -s new-hosts.txt ]; then
  notify -silent -bulk < new-hosts.txt
fi

A new host appearing overnight is often a staging environment, a fresh deployment, or an acquisition being integrated. Those are where the bugs are.

Running it on a schedule

Run the pipeline from a small VPS on a cron schedule, not from your laptop. Keep each target in its own directory with its own scope files:

targets/
  example/
    scope.txt
    out-of-scope.txt
    known-hosts.txt
    run.sh

Log every run with a timestamp. If a program ever asks what you sent and when, you can answer precisely.

What not to automate

• Vulnerability scanning at scale. Many programs ban automated scanners or restrict them heavily. Read the rules. Template scanners like nuclei are best used narrowly, on hosts you’ve already looked at, with templates you’ve read.
• Anything state-changing. Automation should observe. Form submissions, account actions, and fuzzing belong in focused manual testing.
• Out-of-scope assets you happen to find. Note them and move on.

Takeaway

Recon pipelines win on consistency, not cleverness. Make scope a hard gate, diff everything, and spend your manual time on what changed.