Your attack surface changes every time someone spins up a staging host, forgets a DNS record, or opens a port for a vendor demo. A yearly asset spreadsheet misses all of that. This tutorial builds a self-hosted attack surface management monitor in 13 steps, using four open-source ProjectDiscovery tools (subfinder, httpx, naabu and Nuclei) plus one Bash script. Plan on about 75 minutes from empty server to first alert.
Asked to define the practice, a practitioner on the SL Cyber podcast Attack Surface Management 101 said: “It has a few key characteristics in that it’s holistic and it’s continuous and it’s focused on signal.” The monitor in this guide follows that shape. It covers every root domain you list, it reruns on a schedule, and it only pings you when something changes.
You finish with a working project: a scope file, a single script, a daily timer, and webhook alerts for new subdomains, new open ports, changed web stacks, and Nuclei findings that match CISA’s Known Exploited Vulnerabilities (KEV) catalog. If you only want the scanner half, our Nuclei scanner setup tutorial goes deeper on templates and CI.
Don't miss new tech stories on Google
Add Tech Insider once in the Google app and our stories appear in your news suggestions.
What Attack Surface Management Covers (and What It Skips)
Attack surface management is the routine of finding every internet-facing asset your organization owns, watching those assets for change, and checking them for exposure. The order matters. A vulnerability scanner starts from a list of targets you hand it and asks what is wrong with each one. The newer discipline starts earlier and asks which targets exist at all. In practice the asset question is where most breaches begin, because nobody patches a server nobody remembers.
CyCognito’s Rob Gurzeev made a related point in a Security Weekly interview, arguing that port scanners and CVE scanners on their own cannot solve the problem. They only see what you already told them about. That is the gap the discovery half of this build fills. Commercial platforms close it with large data pipelines, and our Defender EASM vs CyCognito vs Tenable ASM comparison shows what they charge for it. A small team can close a useful slice of it with free tools and a cron job.
| Aspect | Vulnerability scanning | Attack surface management |
|---|---|---|
| Starting point | A target list you maintain | Root domains, from which the target list is derived |
| Core question | What is wrong with this host? | Which hosts exist, and what changed since yesterday? |
| Typical cadence | Weekly or monthly scan windows | Daily discovery, with exposure checks on new or changed assets |
| Main output | A findings report sorted by severity | An inventory plus a change feed |
| Classic failure | Perfect scans of the wrong hosts | Noisy alerts that nobody triages |
This build is deliberately scoped. It runs from the outside with no credentials, so it sees what an attacker sees on day one. It does not inventory cloud accounts through provider APIs, scan internal networks, or review code. ProjectDiscovery announced an Internal Network Scanning product on July 21, 2026 that runs the same Nuclei detection engine inside a customer network through a lightweight agent, which is the commercial route for that missing half.
Why External Exposure Needs a Watcher in October 2026
Detection content now moves faster than most patch cycles. ProjectDiscovery’s own write-up on template speed reports that a Nuclei template for WordPress core CVE-2026-63030 merged about six hours after the NVD entry appeared. A template for Citrix NetScaler CVE-2026-3055 took six days. For Gitea CVE-2026-60004, a template existed three weeks before NVD published the record. The lesson is practical: a scanner you rerun every night picks up new checks within a day, while a scanner you run quarterly does not.
Edge products keep supplying the urgent cases. ProjectDiscovery’s vulnerability library lists an Ivanti Connect Secure detection for an authentication bypass tracked as CVE-2026-104935 (the entry names version 22.8R2.4 and earlier) with a published date of October 14, 2026. It recommends upgrading to the vendor patch released October 13. A VPN gateway, a mail gateway and a remote support portal share one trait: they sit on the public internet by design, so you cannot hide them behind a firewall rule.
The templates also carry exploitation context. The nuclei-templates repository ran a KEV and vKEV tag update on October 1, 2026, which means you can ask Nuclei for templates tied to vulnerabilities known to be exploited in the wild and skip the long tail of theoretical issues. That single filter keeps a small team’s alert volume manageable. If you also run a paid scanner, our Tenable vs Qualys vs Rapid7 pricing comparison explains how those products price per asset, which is the number this inventory will give you for free.
Prerequisites and Toolchain Versions
You need a Linux host with outbound internet access, shell access with sudo, and written permission to test every domain in scope. A small VPS works. Two vCPUs and 4 GB of RAM handle a few hundred hostnames comfortably. Run it from a fixed IP address so your own team can recognize the scan traffic in logs and allowlist it on a WAF.
| Component | Role in the monitor | Requirement |
|---|---|---|
| Linux host | Runs the scheduled job | Ubuntu 24.04 LTS or any recent distro with sudo |
| Go toolchain | Builds the four tools from source | Go 1.25 or newer, per ProjectDiscovery install notes |
| subfinder | Passive subdomain discovery | Installed in step 2, API keys optional but useful |
| httpx | Probes which hosts answer on HTTP and HTTPS | Installed in step 2 |
| naabu | Port discovery | libpcap headers for SYN scans, root or file capabilities |
| Nuclei | Template-based exposure checks | Installed in step 2, templates updated by the script |
| jq, curl, coreutils | Parsing, KEV download, diffing | Available from the package manager |
| Webhook URL | Delivers alerts | Slack, Discord or Teams incoming webhook |
On version pinning: GitHub activity on October 23, 2026 pointed to subfinder v2.17.0 and Nuclei v3.11.1 as the newest versions. We pin those in step 2 so a surprise upstream change cannot alter your results overnight. For httpx and naabu we install the latest tag, because we found no confirmed release page to pin against. Check each release page before you copy the pins, since all four projects ship often.
Two Nuclei details deserve attention before you start. First, since v3.11.0, custom templates that use the javascript: protocol must be digitally signed, and unsigned ones are skipped. The official nuclei-templates repository ships pre-signed, so only your own templates are affected. Second, an open issue on the Nuclei repository (opened October 20, 2026) reports that release builds use a Go version hit by a certificate-handling fix tracked as CVE-2026-111943, and it names Go 1.26.10 and Go 1.27.3 as the patched versions. Because go install compiles with your local toolchain, installing from source with a patched Go sidesteps the problem entirely.
Phase 1: Scope and Setup (Steps 1-2)
Step 1: Define Scope and Get Written Authorization
Start with a plain text list of root domains. Include brand domains, regional domains, and domains from acquisitions, because forgotten acquisitions are where forgotten servers pile up. Add a second file for hostnames you must never touch, such as a fragile legacy appliance or a partner-managed system. Passive discovery sends no traffic to your targets, but port scanning and Nuclei do, so the authorization requirement applies to every step after step 3.
Third-party infrastructure needs a separate decision. Your marketing SaaS, your support desk and your status page often live on subdomains you control while the servers belong to a vendor. Scanning those hosts without the vendor’s consent can breach their terms. Our piece on how supply chain attacks change cyber risk management explains why those vendor-hosted subdomains still deserve tracking. Put them on the exclude list for active steps and keep them in the inventory.
mkdir -p ~/asm/{scope,bin,state,runs}
cd ~/asm
cat > scope/roots.txt <<'EOF'
example.com
example.org
EOF
# hostnames that must never be probed or scanned (exact match, one per line)
cat > scope/exclude.txt <<'EOF'
status.example.com
legacy-erp.example.com
EOF
cat > .env <<'EOF'
WEBHOOK_URL=https://hooks.example.invalid/replace-me
EOF
chmod 600 .envReplace example.com with domains you own. The .env file holds your webhook secret, so lock its permissions now. Commit nothing from this directory to a public repository.
Step 2: Install the Toolchain With Pinned Versions
Install the build dependencies, then compile each tool at a pinned tag. The naabu SYN scanner needs libpcap headers, which is why libpcap-dev appears in the package list.
sudo apt-get update
sudo apt-get install -y jq curl libpcap-dev git
# Go 1.25+ from go.dev (distro packages usually lag). Pick the newest 1.26 or 1.27 release.
GO_VERSION=1.26.10
curl -fsSLO "https://go.dev/dl/go${GO_VERSION}.linux-amd64.tar.gz"
sudo rm -rf /usr/local/go
sudo tar -C /usr/local -xzf "go${GO_VERSION}.linux-amd64.tar.gz"
export PATH="$PATH:/usr/local/go/bin"
export PATH="$PATH:$(go env GOPATH)/bin"
echo 'export PATH="$PATH:/usr/local/go/bin:$HOME/go/bin"' >> ~/.bashrc
go version
go install github.com/projectdiscovery/subfinder/v2/cmd/[email protected]
go install github.com/projectdiscovery/httpx/cmd/[email protected]
go install github.com/projectdiscovery/naabu/v2/cmd/naabu@latest
go install github.com/projectdiscovery/nuclei/v3/cmd/[email protected]
subfinder -version
httpx -version
naabu -version
nuclei -versionThe tarball route avoids Ubuntu’s packaged Go, which usually lags behind the 1.25 minimum. The open Nuclei issue names Go 1.26.10 and 1.27.3 as patched, so set GO_VERSION to one of those or anything newer. Each -version command should print a banner and a version string. If a command reports “not found”, your PATH change has not loaded yet, so run source ~/.bashrc.
Last, give naabu permission to open raw sockets without running the whole job as root. This capability lets the binary craft SYN packets while the rest of the script stays unprivileged.
sudo setcap cap_net_raw,cap_net_admin,cap_net_bind_service+eip "$(command -v naabu)"
getcap "$(command -v naabu)"Repeat the setcap command after every naabu upgrade, because replacing the binary strips the capability.
Phase 2: Discovery (Steps 3-5)
Step 3: Discover Subdomains Passively With subfinder
Subfinder queries public data sources such as certificate transparency logs, DNS datasets and search engines. It never contacts your servers, so this step is safe to run without a change window. The more source API keys you add to its provider config, the more it finds. ProjectDiscovery says ChaosDB added 1.5 billion newly observed DNS records in a January 15, 2026 release, which shows how much of the discovery value sits in the data feeding the tool and not in the binary.
cd ~/asm
export LC_ALL=C
subfinder -dL scope/roots.txt -all -silent -o state/subs.raw
cat scope/roots.txt state/subs.raw | tr 'A-Z' 'a-z' | sort -u > state/subs.txt
wc -l state/subs.txt
head -5 state/subs.txtThe -all flag asks every configured source, which is slower but more thorough. The second line adds the root domains themselves and lowercases everything so later comparisons do not trip on case. Sample output with documentation domains looks like this, and yours will differ:
41 state/subs.txt
api.example.com
dev.example.com
example.com
mail.example.com
staging.example.comPer the subfinder release page, version 2.17.0 added bounded concurrent enumeration across domains, a new passive source called Subdomain Center, and regex match and filter options. The regex options help once your list grows, because you can drop noisy patterns such as per-customer wildcards. Check subfinder -h for the exact flag names in your build.
Step 4: Probe Live Web Services With httpx
Discovery gives you names. Probing tells you which names answer. Httpx connects to each hostname on common web ports, follows redirects, and records the status code, page title, detected technologies, resolved IP and whether the host sits behind a CDN. Rate limits keep you polite and keep your own firewall from flagging the scan.
httpx -l state/subs.txt -silent -json -sc -title -td -ip -cdn -fr \
-rl 50 -t 40 -timeout 10 -o state/httpx.jsonl
wc -l state/httpx.jsonl
head -1 state/httpx.jsonl | jq '{input, url, status_code, title, tech}'{
"input": "staging.example.com",
"url": "https://staging.example.com",
"status_code": 200,
"title": "Acme Staging - Login",
"tech": ["Nginx", "React"]
}That staging login page is exactly what this exercise exists to catch. A forgotten staging environment usually runs older code, weaker authentication and default credentials. Treat any unexpected 200 response, any login page on a hostname you did not expect, and any directory listing as a lead for the triage step.
One caution on screenshots. Httpx can capture headless browser screenshots, but an open issue reported on October 20, 2026 says it launches a full Chromium instance per capture, which makes large lists slow. Skip screenshots for the daily run and take them only for new hosts.
Step 5: Find Open Ports With naabu
Web servers are only part of the surface. Databases, remote desktop, SSH, message brokers and admin panels often listen on ports that httpx never touches. Naabu scans the resolved hosts for open TCP ports. Feed it the live hostnames from httpx so you spend scan time only on names that resolve.
jq -r '.input' state/httpx.jsonl | sort -u > state/live-hosts.txt
naabu -l state/live-hosts.txt -top-ports 1000 -rate 1000 \
-exclude-cdn -silent -o state/ports.raw
sort -u state/ports.raw > state/ports.txt
wc -l state/ports.txt203.0.113.10:22
203.0.113.10:443
203.0.113.14:3389
staging.example.com:8443The -exclude-cdn flag skips ports on CDN edge addresses and checks only 80 and 443 there, since scanning a shared CDN edge tells you nothing about your own servers and can annoy the provider. Port 3389 on a host with no business exposing remote desktop is the sort of line that should wake someone up. A change merged on September 29, 2026 keeps TCP and UDP ports distinct during deduplication, so upgrade naabu before you add UDP checks.
For a passive cross-check that touches nothing, compare your results with what a search engine for internet-connected devices already indexed about your address ranges. Our guide on using Shodan for OSINT and attack surface recon walks through that workflow, and a mismatch in either direction is worth investigating.
Phase 3: Inventory and Drift (Steps 6-7)
Step 6: Build a Normalized Inventory
Raw tool output changes shape between versions, so convert it into three flat, sorted text files that never change format: hosts, ports and web endpoints. Everything downstream compares these files and nothing else. If a future httpx release renames a JSON field, you fix one jq line and the rest of the pipeline keeps working.
jq -r '[.input, .url, (.status_code|tostring), (.title // ""),
((.tech // []) | sort | join(","))] | @tsv' state/httpx.jsonl \
| sort -u > state/web.tsv
jq -r '.url' state/httpx.jsonl | sort -u > state/urls.txt
wc -l state/subs.txt state/ports.txt state/web.tsv state/urls.txtThe web.tsv file holds one line per endpoint: hostname, URL, status code, title and a sorted, comma-joined technology list. Sorting the technology list matters. Without it, a scanner that reports “Nginx,React” on Monday and “React,Nginx” on Tuesday produces a fake change.
The inventory is also your first useful deliverable. Security teams without a current asset list often discover that the inventory alone answers audit questions that used to take a week of emails.
Step 7: Detect Drift Between Runs
Drift detection turns the inventory into a change feed. Keep yesterday’s files, compare them with today’s using comm, and report only the differences. The function below does that for any named file and writes both the additions and the removals. On the first run it only saves a baseline, so you start with an empty alert list instead of a flood.
export LC_ALL=C # comm needs identical sort order on both sides
drift() {
local name="$1" file="$2" prev="state/$1.prev"
if [ ! -f "$prev" ]; then cp "$file" "$prev"; return 0; fi
comm -13 "$prev" "$file" > "state/new-$name.txt"
comm -23 "$prev" "$file" > "state/gone-$name.txt"
cp "$file" "$prev"
}
drift subs state/subs.txt
drift ports state/ports.txt
drift web state/web.tsv
wc -l state/new-*.txt state/gone-*.txtA changed page title or technology list appears as one line in the gone file and one in the new file, since the whole line differs. That is fine. You read the pair together. Different kinds of drift deserve different urgency, and the table below is the triage order we use.
| Drift type | Source file | What it often means | Suggested response |
|---|---|---|---|
| New open port on a known host | new-ports.txt | A firewall change, a new service, or a forgotten debug listener | Same day, confirm with the host owner |
| New admin or database port (3389, 6379 and similar) | new-ports.txt | Direct exposure of a service that should sit behind a VPN | Immediate, close or restrict first, ask later |
| New subdomain | new-subs.txt | A new project, a staging copy, or a takeover candidate | Within one working day, find an owner |
| New web endpoint with status 200 | new-web.txt | A live application nobody registered | Within one working day, check authentication |
| Changed title or tech stack | new-web.txt and gone-web.txt | A deployment, a framework upgrade, or a defaced page | Weekly review unless the title looks wrong |
| Host disappeared | gone-subs.txt | Clean decommissioning, or a dangling DNS record left behind | Weekly review, check DNS for leftovers |
Phase 4: Exposure Checks and the KEV Cross-Check (Steps 8-9)
Step 8: Check Exposed Services With Nuclei
Nuclei runs community-written templates against your live URLs. Update the templates first, then run only the checks that matter for a daily job. Filtering by severity and by the kev tag keeps the run short and the results relevant. The first command below refreshes the template set. Run it at the start of each job so new checks apply the same night they ship.
nuclei -update-templates -silent
nuclei -l state/urls.txt -tags kev -severity critical,high \
-rl 50 -c 25 -timeout 10 -silent -jsonl -o state/nuclei-kev.jsonl
wc -l state/nuclei-kev.jsonlKeep the rate limit modest. A rate of 50 requests per second is polite for most targets and still finishes a few hundred URLs in minutes. If you scan production systems during business hours, lower it further. Add -etags dos to exclude denial-of-service templates, and never point the intrusive or fuzzing categories at systems you do not control.
Run a broader weekly pass without the KEV filter to catch misconfigurations, exposed panels and default credentials. When httpx reports WordPress on a host, hand that URL to a specialist tool. Our WPScan walkthrough for WordPress covers plugin and theme enumeration that generic templates only touch lightly.
Step 9: Cross-Check Findings Against CISA KEV
The kev tag in Nuclei comes from a snapshot. CISA’s catalog changes during the week, so download the live feed and match the CVE IDs from your findings against it. This second check catches templates that exist but are not yet tagged, and it gives you a clean list you can hand to an operations team: these are confirmed exploited in the wild, and they are on your perimeter.
curl -fsSL https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json \
-o state/kev.json
jq 'reduce .vulnerabilities[].cveID as $c ({}; .[$c]=true)' state/kev.json > state/kev-set.json
nuclei -l state/urls.txt -severity critical,high -rl 50 -c 25 -timeout 10 \
-silent -jsonl -o state/nuclei-all.jsonl
jq -r --slurpfile kev state/kev-set.json '
((.info.classification["cve-id"] // [])
| (if type=="array" then . else [.] end) | map(ascii_upcase)) as $ids
| select(any($ids[]; $kev[0][.] == true))
| [.host, ."template-id", .info.severity, ($ids|join(","))] | @tsv' \
state/nuclei-all.jsonl | sort -u > state/kev-hits.tsv
cat state/kev-hits.tsvhttps://vpn.example.com ivanti-connect-secure-auth-bypass critical CVE-2026-104935Two details in that jq filter prevent silent misses. Template authors sometimes write the CVE field as a single string and sometimes as an array, so the filter normalizes both. Case also varies, which is why it uppercases before the lookup. A hit in kev-hits.tsv is the one result in this whole pipeline that should page a human immediately. CISA’s remediation deadlines formally bind US federal civilian agencies, but the underlying signal applies to everyone: someone is already exploiting that bug.
Phase 5: Alerts, Scheduling and Triage (Steps 10-13)
Step 10: Send Alerts to a Webhook
Alerts must be short and rare, or people mute them within a week. Send one message per run with counts and the first handful of lines, and send nothing at all when nothing changed. The function posts JSON to any Slack-compatible webhook. For Discord, change the JSON key from text to content.
notify() {
[ -n "${WEBHOOK_URL:-}" ] || return 0
curl -fsS -X POST -H 'Content-Type: application/json' \
--data "$(jq -n --arg t "$1" '{text:$t}')" "$WEBHOOK_URL" > /dev/null
}
if [ -s state/new-subs.txt ] || [ -s state/new-ports.txt ] || [ -s state/kev-hits.tsv ]; then
notify "ASM $(date +%F): $(wc -l < state/new-subs.txt) new hosts, $(wc -l < state/new-ports.txt) new ports, $(wc -l < state/kev-hits.tsv) KEV hits"
fiStep 11: Schedule the Job
Run discovery daily and the broader Nuclei pass weekly. Use flock so a slow run can never overlap the next one, and send output to a log file you can read when something breaks. A cron entry is enough for a single host.
crontab -e
# add these two lines:
15 3 * * * flock -n /tmp/asm.lock "$HOME/asm/bin/asm-run.sh" daily >> "$HOME/asm/asm.log" 2>&1
30 4 * * 0 flock -n /tmp/asm.lock "$HOME/asm/bin/asm-run.sh" weekly >> "$HOME/asm/asm.log" 2>&1| Job | Cadence | Runs | Typical duration for a few hundred hosts |
|---|---|---|---|
| Daily | 03:15 every day | subfinder, httpx, naabu, drift, Nuclei with KEV tag | Tens of minutes |
| Weekly | 04:30 on Sundays | Everything in daily plus a broad Nuclei pass at medium severity and above | One to two hours |
| Manual | On demand | A single new domain after an acquisition | Minutes |
Step 12: Triage Findings and Assign Owners
Detection without ownership just produces a longer to-do list. For every new asset, answer three questions: who owns it, does it need to be public, and what is the fix. Keep a small CSV beside the inventory with columns for hostname, owner, status and decision date. A host marked “approved” stops generating attention until it changes again. Our vulnerability management program guide covers the ticketing and closed-loop tracking that make this stick.
- Known and intended: record the owner and move on.
- Known but should not be public: restrict access, then confirm with a rescan.
- Unknown: treat as hostile until an owner claims it, and check DNS, certificates and hosting records to trace where it came from.
- Matches KEV: patch or isolate first, investigate for compromise second.
The takeaway for a small team: the first week produces the most work, because the baseline reveals years of untracked assets. Expect that and block out the time. After the backlog clears, daily alerts usually shrink to a few lines.
Step 13: Prove the Monitor Works With a Canary
Never trust an alerting pipeline you have not tested. Add a harmless canary: create a DNS record such as asm-canary.example.com pointing at a test web server, wait for the next run (or run the script by hand), and confirm the alert arrives with the new hostname and the new port. Then delete the record and confirm the next run reports the host as gone. A canary also catches the quiet failures where a tool crashes, the script keeps going, and the alert never fires.
Add one more check: alert when a run produces suspiciously little data. If today’s subs.txt has fewer than half the lines of yesterday’s file, something broke (an expired API key, a blocked IP) and a flood of “gone” entries would mislead you. The final script below includes that guard.
The Complete Working Project
Here is the full job in one file. Save it as ~/asm/bin/asm-run.sh, make it executable with chmod +x, and run it once by hand before you trust cron. It takes a mode argument: daily runs the KEV-focused Nuclei pass, and weekly adds the broad pass.
#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C
MODE="${1:-daily}"
BASE="${ASM_HOME:-$HOME/asm}"
cd "$BASE"
# shellcheck disable=SC1091
source "$BASE/.env"
export PATH="$PATH:/usr/local/go/bin:$HOME/go/bin"
STATE="$BASE/state"; RUNS="$BASE/runs/$(date +%F)"
mkdir -p "$STATE" "$RUNS"
notify() {
[ -n "${WEBHOOK_URL:-}" ] || return 0
curl -fsS -X POST -H 'Content-Type: application/json' \
--data "$(jq -n --arg t "$1" '{text:$t}')" "$WEBHOOK_URL" > /dev/null
}
drift() {
local name="$1" file="$2" prev="$STATE/$1.prev"
: > "$STATE/new-$name.txt"; : > "$STATE/gone-$name.txt"
if [ ! -f "$prev" ]; then cp "$file" "$prev"; return 0; fi
comm -13 "$prev" "$file" > "$STATE/new-$name.txt"
comm -23 "$prev" "$file" > "$STATE/gone-$name.txt"
cp "$file" "$prev"
}
# 1. discovery
: > "$STATE/subs.raw"; : > "$STATE/httpx.jsonl"; : > "$STATE/ports.raw"
subfinder -dL scope/roots.txt -all -silent -o "$STATE/subs.raw" || true
cat scope/roots.txt "$STATE/subs.raw" | tr 'A-Z' 'a-z' | sort -u > "$STATE/subs.all"
if [ -s scope/exclude.txt ]; then
grep -v -F -x -f scope/exclude.txt "$STATE/subs.all" > "$STATE/subs.txt" || true
else
cp "$STATE/subs.all" "$STATE/subs.txt"
fi
# guard: a sudden collapse in results means a broken source, not a clean network
if [ -f "$STATE/subs.prev" ]; then
old=$(wc -l < "$STATE/subs.prev"); new=$(wc -l < "$STATE/subs.txt")
if [ "$new" -lt $(( old / 2 )) ]; then
notify "ASM WARNING: subdomain count fell from $old to $new. Check API keys and egress. Run aborted."
exit 1
fi
fi
# 2. probe and ports
httpx -l "$STATE/subs.txt" -silent -json -sc -title -td -ip -cdn -fr \
-rl 50 -t 40 -timeout 10 -o "$STATE/httpx.jsonl" || true
jq -r '.input' "$STATE/httpx.jsonl" | sort -u > "$STATE/live-hosts.txt"
jq -r '.url' "$STATE/httpx.jsonl" | sort -u > "$STATE/urls.txt"
jq -r '[.input,.url,(.status_code|tostring),(.title // ""),((.tech // [])|sort|join(","))]|@tsv' \
"$STATE/httpx.jsonl" | sort -u > "$STATE/web.tsv"
naabu -l "$STATE/live-hosts.txt" -top-ports 1000 -rate 1000 -exclude-cdn -silent \
-o "$STATE/ports.raw" || true
sort -u "$STATE/ports.raw" > "$STATE/ports.txt"
# 3. drift
drift subs "$STATE/subs.txt"
drift ports "$STATE/ports.txt"
drift web "$STATE/web.tsv"
# 4. exposure checks
nuclei -update-templates -silent || true
nuclei -l "$STATE/urls.txt" -tags kev -severity critical,high -etags dos \
-rl 50 -c 25 -timeout 10 -silent -jsonl -o "$RUNS/nuclei-kev.jsonl" || true
if [ "$MODE" = "weekly" ]; then
nuclei -l "$STATE/urls.txt" -severity critical,high,medium -etags dos \
-rl 50 -c 25 -timeout 10 -silent -jsonl -o "$RUNS/nuclei-all.jsonl" || true
fi
# 5. KEV cross-check across every finding file from this run
curl -fsSL https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json \
-o "$STATE/kev.json"
jq 'reduce .vulnerabilities[].cveID as $c ({}; .[$c]=true)' "$STATE/kev.json" > "$STATE/kev-set.json"
cat "$RUNS"/nuclei-*.jsonl 2>/dev/null | jq -r --slurpfile kev "$STATE/kev-set.json" '
((.info.classification["cve-id"] // [])
| (if type=="array" then . else [.] end) | map(ascii_upcase)) as $ids
| select(any($ids[]; $kev[0][.] == true))
| [.host, ."template-id", .info.severity, ($ids|join(","))] | @tsv' \
| sort -u > "$STATE/kev-hits.tsv" || true
# 6. alert only when something changed
n_s=$(wc -l < "$STATE/new-subs.txt"); n_p=$(wc -l < "$STATE/new-ports.txt")
n_k=$(wc -l < "$STATE/kev-hits.tsv")
if [ "$n_s" -gt 0 ] || [ "$n_p" -gt 0 ] || [ "$n_k" -gt 0 ]; then
msg="ASM $(date +%F) ($MODE): $n_s new hosts, $n_p new ports, $n_k KEV hits"
[ "$n_k" -gt 0 ] && msg="$msg"$'\n'"KEV: $(head -5 "$STATE/kev-hits.tsv")"
[ "$n_p" -gt 0 ] && msg="$msg"$'\n'"Ports: $(head -10 "$STATE/new-ports.txt" | tr '\n' ' ')"
[ "$n_s" -gt 0 ] && msg="$msg"$'\n'"Hosts: $(head -10 "$STATE/new-subs.txt" | tr '\n' ' ')"
notify "$msg"
fi
echo "done $(date -Is) mode=$MODE hosts=$(wc -l < "$STATE/subs.txt") ports=$(wc -l < "$STATE/ports.txt")"A typical alert from this script, using documentation values, looks like the block below. It is short on purpose. The first line tells you whether to keep reading.
ASM 2026-10-24 (daily): 2 new hosts, 1 new ports, 1 KEV hits
KEV: https://vpn.example.com ivanti-connect-secure-auth-bypass critical CVE-2026-104935
Ports: 203.0.113.14:3389
Hosts: asm-canary.example.com preview-42.example.comThe directory layout ends up looking like this. Raw outputs live in dated folders under runs/ so you can answer “what did we see last Tuesday” without rerunning anything, and the state/ folder holds only the current snapshot and the previous one.
~/asm
├── .env
├── asm.log
├── bin/asm-run.sh
├── scope/{roots.txt,exclude.txt}
├── state/{subs.txt,ports.txt,web.tsv,urls.txt,*.prev,new-*.txt,gone-*.txt,kev-hits.tsv}
└── runs/2026-10-24/{nuclei-kev.jsonl,nuclei-all.jsonl}Common Pitfalls to Avoid
- Scanning without authorization. Port scans and Nuclei traffic can trip intrusion detection, break fragile devices and violate a vendor’s terms. Get written permission, keep the exclude list current, and record who approved what.
- Trusting wildcard DNS. If a domain answers for every possible subdomain, brute force and some passive sources produce thousands of fake hosts. Test with a random name such as
zz9-not-real.example.com. If it resolves, filter by the wildcard IP or response size. - Forgetting
LC_ALL=C.commneeds both files sorted in the same locale. Mixed locales produce phantom differences that look like dozens of new hosts. Set the variable at the top of the script and sort with it on every file. - Alerting on everything. If every changed page title pages a human, nobody reads the channel by week two. Alert on hosts, ports and KEV hits. Batch the rest into a weekly digest.
- Tracking
latestblindly. A new upstream release can rename a JSON field or change defaults overnight. Pin versions, and upgrade on purpose after reading the release notes. - Ignoring what disappeared. A host that vanishes from the inventory but still has a DNS record is a takeover candidate. Review the gone files weekly.
- Scanning from a laptop on a hotel network. Captive portals and ISP filtering corrupt results and produce false closed ports. Run the job from a stable server with a fixed IP.
Troubleshooting: 10 Problems and Fixes
| Symptom | Likely cause | Fix |
|---|---|---|
command not found: subfinder after install | Go’s bin directory is not on PATH | Add $(go env GOPATH)/bin to PATH and reload the shell. Cron needs the export inside the script, as shown above. |
go install fails with a message that a newer Go is required | Packaged Go is older than the module’s minimum | Install Go 1.25 or newer from go.dev and run go version to confirm. |
| naabu prints a pcap or permission error | Missing libpcap headers or no raw socket permission | Install libpcap-dev, rerun the setcap command, or fall back to a connect scan. |
| subfinder returns only a handful of subdomains | No API keys configured for the strongest sources | Add keys to ~/.config/subfinder/provider-config.yaml and rerun. |
| httpx shows almost every host as timed out | Rate too high, egress filtering, or your IP was blocked | Lower -rl and -t, test one host with curl -I, and try a different source IP. |
| Thousands of fake subdomains appear | Wildcard DNS on a root domain | Detect the wildcard IP and filter it, or exclude that root from brute-force sources. |
| Nuclei says no templates were loaded | Template directory missing or update failed | Run nuclei -update-templates and check network access to GitHub. |
| Custom JavaScript templates are skipped | Unsigned templates are rejected since v3.11.0 | Sign the template with your key or rewrite it in YAML without the JavaScript protocol. |
| Alerts list hundreds of new hosts that already existed | Mixed locale or unsorted file fed to comm | Export LC_ALL=C and sort every file with sort -u before comparing. |
| Webhook returns an error and no message arrives | Wrong URL, wrong JSON key, or a payload that is too large | Test with curl by hand. Use text for Slack and content for Discord, and trim the message to a few lines. |
If a problem is not in the table, run the failing stage by itself with the -silent flag removed. Each tool prints useful errors when it is not in quiet mode, and the stage-by-stage layout of the script makes that easy. Also read the log file after the first few cron runs. Cron swallows errors by default, and a missing environment variable is the most common reason a job works by hand and fails on schedule.
Advanced Tips for a Larger Surface
- Add DNS and TLS enrichment. Two more ProjectDiscovery tools, dnsx and tlsx, record DNS answers and certificate details. Certificates reveal hostnames you did not know about through subject alternative names, and expiring certificates are a free signal.
- Watch for subdomain takeover. Flag hostnames whose CNAME points to a hosting service that no longer has the matching account. Nuclei has takeover templates, and the weekly pass should include them.
- Store history in a database. Once the text files outgrow you, load each run into SQLite with a first-seen and last-seen column. That turns “when did this port open” into one query.
- Map cloud ranges. Add your cloud accounts’ public IP ranges to the naabu input. Cloud providers recycle addresses, so an old IP in your DNS may now belong to a stranger.
- Filter noise with regex. When your list grows, use subfinder’s regex match and filter options (added in v2.17.0, according to its release page) to drop per-tenant or per-pull-request hostnames that churn every day.
- Add authenticated scanning for internal hosts. This external view cannot see missing patches behind a login. A commercial scanner can, and our Tenable, Qualys and Rapid7 scanner comparison lays out plugin coverage and the price gaps.
- Run it in CI. Package the script in a container and run it from your CI system on a schedule. That gives you audit logs, secret management and a second pair of eyes on changes to the script.
- Report through ProjectDiscovery’s free notification service, with care. ProjectDiscovery reportedly announced SecureTheWeb.org on October 22, 2026, a free service that checks its Chaos data for services running affected versions and emails the relevant security contact. Treat that as a single-source report and read the service terms before you rely on it.
Hunters in the bug bounty community chain these same tools in what one October 2026 write-up calls a recon-to-report workflow on Linux. Reading it from the attacker’s side is a useful exercise, because it shows how quickly an exposed asset gets noticed by someone who is not on your payroll.
DIY Toolchain vs Commercial Platforms
The self-hosted pipeline wins on cost and transparency. Every command is visible, every filter is yours, and nothing leaves your network except the traffic you chose to send. It loses on breadth. It will not trace ownership of acquired companies, correlate assets across thousands of subsidiaries, or give an executive a dashboard. Commercial platforms sell exactly those things.
| Approach | Strength | Weakness | Cost model |
|---|---|---|---|
| This DIY pipeline | Transparent, scriptable, free software | You maintain it and it covers only domains you list | Server time plus your hours |
| ProjectDiscovery Neo | Autonomous testing and attack surface mapping, generally available after beta | A separate commercial product, not a drop-in replacement for these CLIs | Per-seat monthly pay-as-you-go with included credits and paid top-ups, no numeric price on the pages we checked |
| Commercial EASM platforms | Large data pipelines and subsidiary mapping | Contract pricing and less transparency | Quote-based |
| Passive search engines | Fast view of what is already indexed | Misses new and unindexed assets | Free tiers and paid plans |
ProjectDiscovery announced Neo v1 as generally available after beta. The company describes it as an autonomous security platform for continuous exploitability testing, attack surface mapping, code and pull request review, internal network red teaming and vulnerability backlog triage. It runs in ProjectDiscovery’s managed cloud by default, with self-hosting inside a customer network available for enterprise use. Nothing in the pages we reviewed says Neo replaces subfinder, httpx, naabu or Nuclei, and the open-source tools remain active projects.
A sensible path for most teams runs in three stages. Start with this pipeline to get a baseline and prove the value. Add a commercial scanner for authenticated and internal coverage if your risk or audit scope demands it, and compare vendors on price and coverage before you sign. Revisit the build-versus-buy question once your asset count or compliance burden outgrows a single server. The wider cybersecurity coverage on this site tracks how those vendors and threats shift.
Frequently Asked Questions
Is it legal to run attack surface scans?
Scanning assets you own, or have written permission to test, is routine. Scanning someone else’s systems can break computer misuse laws and terms of service in many countries. Passive discovery with subfinder queries third-party datasets and is generally low risk, but port scans and Nuclei send traffic to targets. Get written authorization, keep the exclude list current, and do not scan vendor-hosted systems without their consent. This article is not legal advice, so involve counsel for anything unclear.
How is attack surface management different from vulnerability management?
Vulnerability management works on a list of known assets and ranks flaws on them. Attack surface management finds the assets first and watches them for change. The two feed each other: discovery supplies the target list, and the vulnerability process fixes what discovery surfaces. Most mature programs run both.
How often should I run the monitor?
Daily discovery suits most teams, because new subdomains and ports appear at any time and a day is short enough to catch a mistake before it is widely found. Run broader Nuclei passes weekly. Large organizations with rapid deployment cycles may run discovery every few hours, as long as the rate limits stay polite.
Will these scans disrupt production systems?
Rarely, if you keep rate limits low and exclude denial-of-service templates. Naabu and httpx generate light traffic at the settings used here. The risk comes from fragile devices such as old embedded systems and legacy appliances, which can misbehave under a port scan. Put those on the exclude list and test changes against a staging environment first.
Do I need paid API keys for subfinder?
No, the tool works without them. Free keys from several sources noticeably increase the number of subdomains found, and paid tiers from some providers add more. Start without keys, measure how many hosts you find, then add free keys one at a time and watch the count.
Can this replace a commercial external attack surface management product?
For a single organization with a known set of root domains, it can cover the core job of discovery, change tracking and exposure checks. It cannot replace the data scale, subsidiary mapping and reporting of a commercial platform. Many teams use the DIY pipeline to validate what a commercial tool reports.
What should I do first when the KEV check fires?
Confirm the finding by hand, then patch or isolate the asset according to the vendor guidance. After that, review logs for signs of earlier compromise, since the flaw has been exploited elsewhere and your exposure window may already have been used. Document the timeline for your incident record.
