Shell Scripting to Automate Reconnaissance

Reconnaissance is where hacking is really won. Long before any exploit, the person who maps the target most thoroughly usually finds the way in. The trouble is that good recon is repetitive. The same commands, in the same order, against target after target. And repetition is exactly what computers are for.

So in this guide we are going to build your own reconnaissance pipeline with nothing more than Bash and a handful of free tools. Not a bloated framework you cannot see inside, but a script you own, understand, and can bend to any target in seconds. By the end you will have moved from typing the same commands by hand every time to running one script and going to make coffee while it works.

One line before we start, and I mean it every time. Everything here is for targets you own or are explicitly authorised to test, whether that is your own lab, a client engagement, or an asset inside a bug bounty program’s scope. Automation makes it trivially easy to scan things you should not, so scope carefully.

Why script it yourself

You might reasonably ask why bother writing your own scripts when polished all-in-one recon frameworks already exist. It is a fair question, and the honest answer is that both have their place. But building your own pipeline gives you three things the black boxes cannot.

The first is understanding. When you wire the tools together yourself, you learn exactly what each stage does and why it comes where it does. That knowledge pays off constantly, because when something behaves oddly on a real target you can reason about it instead of shrugging at a framework that failed silently.

The second is control. Your script, your rules. You decide which tools run, in what order, with what settings, and you can change any of it in seconds. When a target does something unusual, you adapt on the spot rather than fighting someone else’s assumptions.

The third is flexibility. Over time your script becomes yours, shaped by your habits and the kinds of targets you go after. It grows with you. That personal toolkit is one of the things that quietly separates hunters who plateau from those who keep getting sharper.

The building blocks of Bash you actually need

You do not need to be a shell wizard for this. Recon automation is mostly about gluing existing command-line tools together, and Bash is superb at exactly that. Here are the few pieces that do almost all the work.

Pipes are the single most important idea. The pipe symbol takes the output of one command and feeds it straight into the next. subfinder -d example.com | httpx sends every subdomain straight into a prober. Recon pipelines are, at heart, long chains of pipes.

Redirection means saving output to files with > and appending with >>. You will constantly want to capture results, subfinder -d example.com > subs.txt, so later stages can read them.

Variables store values so you do not repeat yourself. domain="example.com" lets you write the target once and reuse it everywhere, which is the first step toward a reusable script.

Loops do something to every line of a file. A simple while read loop lets you run a command against every host you discovered.

Arguments read input passed to your script with $1, so you can run ./recon.sh example.com and have it just work against whatever you point it at.

That really is the core. If those five ideas make sense to you, you can build everything below.

The shape of a recon pipeline

Before writing anything, picture the flow. Good recon moves in stages, each one feeding the next, gradually turning a single domain into a rich map of attack surface.

The typical pipeline looks like this. You start with a domain. You enumerate its subdomains to discover the wider footprint. You resolve and probe those subdomains to find which ones are actually alive and serving something. You gather historical and crawled URLs to uncover forgotten endpoints. Then you run content discovery and vulnerability templates against the live hosts to surface interesting paths and known issues. Each stage narrows and enriches the last.

Our job in scripting is simply to connect these stages so the output of one becomes the input of the next, automatically. Let us build it up piece by piece.

Stage one: subdomain enumeration

Everything begins with expanding one domain into all the hosts that belong to it. A tool like subfinder pulls subdomains from dozens of public sources quickly and passively. In a script it looks like this.

#!/bin/bash
domain="$1"
mkdir -p "$domain"
echo "[*] Enumerating subdomains for $domain"
subfinder -d "$domain" -silent > "$domain/subs.txt"
echo "[*] Found $(wc -l < "$domain/subs.txt") subdomains"

Look at what this small block already does. It takes the target as an argument, creates a folder to keep the results tidy, runs the enumeration silently, saves everything to a file, and tells you how many it found. That is a real, useful tool already, and we have barely started. If you want to go deeper on this stage specifically, the subdomain enumeration recon guide on this site covers the theory behind why these sources work.

Stage two: find what is actually alive

A list of subdomains is not much use on its own, because many will be dead, parked, or unreachable. The next stage probes each one to see which are actually responding, using a tool like httpx. We feed the previous stage straight in.

echo "[*] Probing for live hosts"
cat "$domain/subs.txt" | httpx -silent -o "$domain/live.txt"
echo "[*] $(wc -l < "$domain/live.txt") hosts are live"

Notice the cat ... | httpx pattern. We are piping the output of stage one directly into stage two. This is the whole philosophy of the pipeline in one line, the results flow forward on their own. Now live.txt holds only the hosts worth spending time on, which keeps every later stage faster and quieter.

Stage three: gather historical URLs

Live hosts often hide a wealth of old endpoints, forgotten parameters, and API paths that are not linked anywhere obvious. Tools like waybackurls and gau pull these from public archives and crawl data. We loop over our live hosts and collect everything.

echo "[*] Gathering historical URLs"
cat "$domain/live.txt" | waybackurls | sort -u > "$domain/urls.txt"
echo "[*] Collected $(wc -l < "$domain/urls.txt") URLs"

That sort -u at the end is a small but important habit. It sorts the results and strips duplicates, so you are not wading through the same URL a hundred times. Little touches like this are what make a homegrown script pleasant to actually use. There is a full write-up on this site about how attackers use waybackurls to find vulnerabilities if you want to understand what makes this stage so valuable.

Stage four: content discovery and templated checks

Now we hunt for hidden paths and known weaknesses. Content discovery with a tool like ffuf brute forces directories and files, while nuclei runs thousands of community templates that check for known misconfigurations and vulnerabilities against the live hosts.

echo "[*] Running nuclei against live hosts"
nuclei -l "$domain/live.txt" -silent -o "$domain/nuclei.txt"
echo "[*] Recon complete. Results saved in $domain/"

And that is a complete, end-to-end pipeline. One command in, a folder full of structured recon out. If content discovery is new to you, the guide to ffuf directory bruteforcing on this site walks through the tool that powers this stage in detail.

Bringing it together into one script

Here is the whole thing assembled into a single file. Save it as recon.sh, make it executable with chmod +x recon.sh, and run it with ./recon.sh example.com.

#!/bin/bash
# Simple recon pipeline. Authorised targets only.
domain="$1"

if [ -z "$domain" ]; then
  echo "Usage: ./recon.sh <domain>"
  exit 1
fi

mkdir -p "$domain"

echo "[*] Enumerating subdomains"
subfinder -d "$domain" -silent > "$domain/subs.txt"

echo "[*] Probing for live hosts"
cat "$domain/subs.txt" | httpx -silent -o "$domain/live.txt"

echo "[*] Gathering historical URLs"
cat "$domain/live.txt" | waybackurls | sort -u > "$domain/urls.txt"

echo "[*] Running templated checks"
nuclei -l "$domain/live.txt" -silent -o "$domain/nuclei.txt"

echo "[*] Done. Results in ./$domain/"

Notice the small guard near the top. If you forget to pass a domain, it tells you how to use it and stops rather than doing something confusing. That single check is the difference between a script that annoys you and one you actually trust. As you use this, you will keep adding these small quality-of-life touches, and that is exactly how a personal toolkit is born.

Making the script trustworthy: logging and error handling

The version above works, but a script you rely on needs to be a little more grown up. Two habits turn a fragile one-off into something you actually trust on real engagements.

The first is logging. When a run finishes and you find something interesting, you will want to know when it ran and how long each stage took. Adding timestamps costs almost nothing and pays off constantly. A simple helper function keeps it clean.

log() {
  echo "[$(date '+%H:%M:%S')] $1"
}

log "Enumerating subdomains"

Now every message in your pipeline is timestamped, so a log file becomes a record you can actually reason about later rather than an undated blur.

The second habit is not blindly charging ahead when a stage produces nothing. If subdomain enumeration returns an empty file, running every later stage against nothing just wastes time and clutters your output. A quick check prevents that.

if [ ! -s "$domain/subs.txt" ]; then
  log "No subdomains found, stopping."
  exit 1
fi

The -s test asks whether the file exists and is non-empty. Small guards like this are what separate a script that quietly does the wrong thing from one that tells you clearly what happened. As your pipeline grows, these checks between stages save you from chasing confusing results caused by an earlier step that silently failed.

Running it on a schedule

One of the real payoffs of scripting your recon is that a script can run without you. On engagements or bug bounty programs that last weeks, attack surface changes constantly. New subdomains appear, old services get exposed, and the person who notices first has the advantage. Rather than re-running your recon by hand every few days, you can schedule it.

On a Linux machine, cron is the classic tool for this. A single line in your crontab can run your pipeline automatically, say once a day, and because your script already saves organised, timestamped output per target, you build up a history over time. Compare today’s results against last week’s and the new entries jump out. That diffing of results over time, spotting what changed rather than re-reading everything, is one of the highest-value tricks in continuous recon, and it only becomes practical once your recon is a script rather than a manual ritual. You can even wire in a notification, a message to yourself when a run finds something new, so the pipeline effectively watches the target for you.

This is the moment the effort pays off. You stop performing recon and start operating a system that performs it for you, freeing your attention for the creative work that automation cannot do.

Making it yours, and using it responsibly

What you have now is a foundation, not a finished product, and that is the point. From here you will start customising. You might add screenshotting of live hosts so you can eyeball them quickly. You might pipe URLs through a parameter discovery tool before the vulnerability stage. You might add notifications so it pings you when a run finishes. Every one of those changes is a few lines, because the structure is already sound.

Two references will make the scripting half of this much less painful: ShellCheck catches the quoting and word-splitting bugs that silently corrupt pipelines like this one, and the Advanced Bash-Scripting Guide is a solid free reference when you need a construct you have not used before. The tools the pipeline drives (subfinder, httpx, nuclei, ffuf and waybackurls) are all worth reading the docs for rather than copying flags blindly.

A few habits will keep you out of trouble and make your results better. Be gentle with concurrency and speed, both to respect program rules and to avoid tripping defences that will feed you garbage data once they start blocking you. Keep your tools updated, since their data sources and templates improve constantly. And organise your output per target, as our script does with its folders, so a month from now you can still find what you gathered.

Above all, remember what makes this legitimate. The same automation that maps a target you are paid to test will happily map one you have no right to touch, and the tools do not know the difference. Scope carefully, stay inside your authorisation, and let the script do the boring work so your brain is free for the interesting part, which is turning all this reconnaissance into a way in.