Penetration Testing: Methodology and Tools
Penetration testing is authorized simulated attack against computer systems to evaluate security. Unlike vulnerability scanning—which runs automated tools to identify potential weaknesses—penetration...
Key Insights
- Penetration testing follows a structured five-phase methodology—reconnaissance, scanning, exploitation, post-exploitation, and reporting—that transforms chaotic “hacking” into repeatable, professional security assessment.
- The difference between a vulnerability scan and a penetration test is exploitation: scanners find potential weaknesses, pentesters prove they’re exploitable and demonstrate real business impact.
- Your pentest is only as valuable as your report; technical findings without clear severity ratings, business context, and actionable remediation guidance end up ignored in someone’s inbox.
Introduction to Penetration Testing
Penetration testing is authorized simulated attack against computer systems to evaluate security. Unlike vulnerability scanning—which runs automated tools to identify potential weaknesses—penetration testing involves actively exploiting vulnerabilities to demonstrate real-world impact.
Organizations need regular pentests because attackers don’t run Nessus and call it a day. They chain vulnerabilities, pivot through networks, and find creative paths to your crown jewels. A vulnerability scanner might flag 500 “medium” findings. A penetration tester shows you which three actually let someone steal your customer database.
Before touching a keyboard, get written authorization. A signed Rules of Engagement (RoE) document should specify target scope, testing windows, emergency contacts, and explicitly permitted techniques. Without this, you’re committing computer crimes regardless of intent. Most organizations require proof of liability insurance and background checks for external pentesters.
The Five Phases of Penetration Testing
Professional penetration testing follows a structured methodology that ensures thorough coverage and repeatable results.
Phase 1: Reconnaissance involves gathering information about the target without directly interacting with their systems. You’re building a map of the attack surface.
Phase 2: Scanning means actively probing target systems to identify open ports, running services, and potential vulnerabilities.
Phase 3: Exploitation is where you attempt to leverage discovered vulnerabilities to gain unauthorized access.
Phase 4: Post-Exploitation explores what an attacker could do after initial compromise—privilege escalation, lateral movement, and data access.
Phase 5: Reporting documents everything with severity ratings, proof-of-concept evidence, and remediation guidance.
Here’s a bash script that automates passive reconnaissance:
#!/bin/bash
# passive_recon.sh - Passive reconnaissance automation
# Usage: ./passive_recon.sh example.com
TARGET=$1
OUTPUT_DIR="./recon_$TARGET"
mkdir -p "$OUTPUT_DIR"
echo "[*] Starting passive recon for $TARGET"
# WHOIS lookup
echo "[+] Running WHOIS lookup..."
whois "$TARGET" > "$OUTPUT_DIR/whois.txt"
# DNS records
echo "[+] Gathering DNS records..."
dig ANY "$TARGET" +noall +answer > "$OUTPUT_DIR/dns_records.txt"
dig "$TARGET" MX +short >> "$OUTPUT_DIR/dns_records.txt"
dig "$TARGET" TXT +short >> "$OUTPUT_DIR/dns_records.txt"
dig "$TARGET" NS +short >> "$OUTPUT_DIR/dns_records.txt"
# Reverse DNS on discovered IPs
echo "[+] Performing reverse DNS lookups..."
for ip in $(dig "$TARGET" +short); do
dig -x "$ip" +short >> "$OUTPUT_DIR/reverse_dns.txt"
done
# theHarvester for email and subdomain discovery
echo "[+] Running theHarvester..."
theHarvester -d "$TARGET" -b google,bing,linkedin -f "$OUTPUT_DIR/harvester"
# Certificate transparency logs
echo "[+] Checking certificate transparency..."
curl -s "https://crt.sh/?q=%25.$TARGET&output=json" | \
jq -r '.[].name_value' | sort -u > "$OUTPUT_DIR/ct_subdomains.txt"
echo "[*] Recon complete. Results in $OUTPUT_DIR/"
Reconnaissance and Information Gathering
Reconnaissance divides into passive and active techniques. Passive recon never touches target systems—you’re gathering publicly available information. Active recon involves direct interaction that might appear in target logs.
Passive techniques include WHOIS lookups, DNS record queries, certificate transparency logs, social media research, job postings (which reveal technology stacks), and cached/archived web pages. These leave no trace on target systems.
Active techniques include port scanning, banner grabbing, directory brute-forcing, and DNS zone transfers. These generate traffic that security teams can detect.
This Python script performs subdomain enumeration using both DNS queries and wordlist brute-forcing:
#!/usr/bin/env python3
"""
subdomain_enum.py - Subdomain enumeration tool
Combines DNS resolution with wordlist-based discovery
"""
import socket
import dns.resolver
import concurrent.futures
from typing import Set, List
def load_wordlist(filepath: str) -> List[str]:
"""Load subdomain wordlist from file."""
with open(filepath, 'r') as f:
return [line.strip() for line in f if line.strip()]
def resolve_subdomain(subdomain: str, domain: str) -> dict | None:
"""Attempt to resolve a subdomain and return results."""
fqdn = f"{subdomain}.{domain}"
result = {"subdomain": fqdn, "a_records": [], "cname": None}
try:
# A record lookup
answers = dns.resolver.resolve(fqdn, 'A')
result["a_records"] = [rdata.address for rdata in answers]
# Try CNAME lookup
try:
cname_answers = dns.resolver.resolve(fqdn, 'CNAME')
result["cname"] = str(cname_answers[0].target)
except (dns.resolver.NoAnswer, dns.resolver.NXDOMAIN):
pass
return result
except (dns.resolver.NXDOMAIN, dns.resolver.NoAnswer,
dns.resolver.NoNameservers, dns.exception.Timeout):
return None
def enumerate_subdomains(domain: str, wordlist_path: str,
threads: int = 50) -> List[dict]:
"""Enumerate subdomains using threaded DNS resolution."""
wordlist = load_wordlist(wordlist_path)
discovered = []
print(f"[*] Enumerating subdomains for {domain}")
print(f"[*] Loaded {len(wordlist)} words, using {threads} threads")
with concurrent.futures.ThreadPoolExecutor(max_workers=threads) as executor:
futures = {
executor.submit(resolve_subdomain, word, domain): word
for word in wordlist
}
for future in concurrent.futures.as_completed(futures):
result = future.result()
if result:
discovered.append(result)
print(f"[+] Found: {result['subdomain']} -> {result['a_records']}")
return discovered
if __name__ == "__main__":
import sys
if len(sys.argv) != 3:
print(f"Usage: {sys.argv[0]} <domain> <wordlist>")
sys.exit(1)
results = enumerate_subdomains(sys.argv[1], sys.argv[2])
print(f"\n[*] Discovered {len(results)} subdomains")
Scanning and Vulnerability Assessment
Network scanning identifies live hosts, open ports, and running services. Nmap remains the industry standard, but understanding what each scan type does matters more than memorizing flags.
# Host discovery - find live hosts on a network
nmap -sn 192.168.1.0/24 -oG live_hosts.txt
# TCP SYN scan - fast, stealthy, requires root
sudo nmap -sS -p- --min-rate 1000 -oA full_syn target.com
# Service version detection
nmap -sV -sC -p 22,80,443,3306 -oA services target.com
# UDP scan - slow but finds DNS, SNMP, TFTP
sudo nmap -sU --top-ports 100 -oA udp_scan target.com
# Vulnerability scripts
nmap --script vuln -p 80,443 target.com
For programmatic scanning, the python-nmap library provides a clean interface:
#!/usr/bin/env python3
"""
network_scanner.py - Automated network scanning with python-nmap
"""
import nmap
import json
from datetime import datetime
def scan_network(target: str, ports: str = "1-1000") -> dict:
"""Perform comprehensive network scan and return structured results."""
scanner = nmap.PortScanner()
results = {
"target": target,
"scan_time": datetime.now().isoformat(),
"hosts": []
}
print(f"[*] Scanning {target} ports {ports}")
# SYN scan with version detection
scanner.scan(hosts=target, ports=ports,
arguments='-sS -sV -O --script=banner')
for host in scanner.all_hosts():
host_info = {
"ip": host,
"hostname": scanner[host].hostname(),
"state": scanner[host].state(),
"os_match": [],
"ports": []
}
# OS detection results
if 'osmatch' in scanner[host]:
host_info["os_match"] = [
{"name": os['name'], "accuracy": os['accuracy']}
for os in scanner[host]['osmatch'][:3]
]
# Port information
for proto in scanner[host].all_protocols():
for port in scanner[host][proto].keys():
port_info = scanner[host][proto][port]
host_info["ports"].append({
"port": port,
"protocol": proto,
"state": port_info['state'],
"service": port_info['name'],
"version": port_info.get('version', ''),
"product": port_info.get('product', '')
})
results["hosts"].append(host_info)
return results
if __name__ == "__main__":
results = scan_network("192.168.1.0/24", "22,80,443,3389,8080")
print(json.dumps(results, indent=2))
Exploitation Techniques and Tools
Exploitation proves vulnerabilities are actually exploitable. Metasploit Framework remains the dominant tool, but understanding the underlying techniques matters more than point-and-click exploitation.
This Metasploit resource script automates common testing workflows:
# auto_exploit.rc - Metasploit resource script for automated testing
# Usage: msfconsole -r auto_exploit.rc
# Set global options
setg RHOSTS file:/tmp/targets.txt
setg THREADS 10
# Database setup
db_status
workspace -a pentest_engagement
# Run discovery
use auxiliary/scanner/portscan/tcp
set PORTS 21,22,23,25,80,443,445,3306,3389,8080
run
back
# SMB enumeration
use auxiliary/scanner/smb/smb_version
run
back
# Check for EternalBlue
use auxiliary/scanner/smb/smb_ms17_010
run
back
# Web server enumeration
use auxiliary/scanner/http/http_version
set RHOSTS file:/tmp/web_targets.txt
run
back
# Export results
hosts -o /tmp/discovered_hosts.csv
services -o /tmp/discovered_services.csv
vulns -o /tmp/discovered_vulns.csv
For web application testing, this script detects basic SQL injection vulnerabilities:
#!/usr/bin/env python3
"""
sqli_detector.py - Basic SQL injection detection
For authorized testing only
"""
import requests
from urllib.parse import urljoin, parse_qs, urlparse, urlencode
from typing import List, Tuple
SQL_PAYLOADS = [
"'",
"''",
"' OR '1'='1",
"' OR '1'='1'--",
"1' ORDER BY 1--",
"1' UNION SELECT NULL--",
]
SQL_ERRORS = [
"sql syntax",
"mysql_fetch",
"sqlite3",
"postgresql",
"ora-01756",
"unclosed quotation",
"quoted string not properly terminated",
]
def test_parameter(url: str, param: str, value: str) -> List[Tuple[str, str]]:
"""Test a single parameter for SQL injection."""
findings = []
parsed = urlparse(url)
params = parse_qs(parsed.query)
for payload in SQL_PAYLOADS:
test_params = params.copy()
test_params[param] = [value + payload]
test_url = f"{parsed.scheme}://{parsed.netloc}{parsed.path}"
try:
response = requests.get(
test_url,
params={k: v[0] for k, v in test_params.items()},
timeout=10
)
response_lower = response.text.lower()
for error in SQL_ERRORS:
if error in response_lower:
findings.append((payload, error))
print(f"[!] Potential SQLi: {param}={payload}")
print(f" Error indicator: {error}")
except requests.RequestException as e:
print(f"[-] Request failed: {e}")
return findings
def scan_url(url: str) -> dict:
"""Scan all parameters in a URL for SQL injection."""
parsed = urlparse(url)
params = parse_qs(parsed.query)
results = {"url": url, "vulnerable_params": {}}
for param, values in params.items():
print(f"[*] Testing parameter: {param}")
findings = test_parameter(url, param, values[0])
if findings:
results["vulnerable_params"][param] = findings
return results
Post-Exploitation and Reporting
Post-exploitation demonstrates business impact. After initial access, you need to show what an attacker could actually achieve.
This PowerShell script enumerates Windows privilege escalation vectors:
# privesc_enum.ps1 - Windows privilege escalation enumeration
# Run from compromised Windows host
Write-Host "[*] Windows Privilege Escalation Enumeration" -ForegroundColor Cyan
# Current user context
Write-Host "`n[+] Current User Info:" -ForegroundColor Green
whoami /all
# Unquoted service paths
Write-Host "`n[+] Unquoted Service Paths:" -ForegroundColor Green
Get-WmiObject win32_service | Where-Object {
$_.PathName -notlike '"*' -and
$_.PathName -like '* *' -and
$_.StartMode -eq 'Auto'
} | Select-Object Name, PathName, StartMode
# Writable service executables
Write-Host "`n[+] Checking Service Executable Permissions:" -ForegroundColor Green
Get-WmiObject win32_service | ForEach-Object {
$path = ($_.PathName -split '"')[1]
if ($path -and (Test-Path $path)) {
$acl = Get-Acl $path
$acl.Access | Where-Object {
$_.FileSystemRights -match 'Write|FullControl' -and
$_.IdentityReference -match 'Users|Everyone|Authenticated'
} | ForEach-Object {
Write-Host " [!] Writable: $path" -ForegroundColor Yellow
}
}
}
# Scheduled tasks
Write-Host "`n[+] Scheduled Tasks:" -ForegroundColor Green
schtasks /query /fo LIST /v | Select-String "TaskName|Run As User|Task To Run"
# AlwaysInstallElevated check
Write-Host "`n[+] AlwaysInstallElevated Check:" -ForegroundColor Green
$hklm = Get-ItemProperty -Path "HKLM:\SOFTWARE\Policies\Microsoft\Windows\Installer" -Name "AlwaysInstallElevated" -ErrorAction SilentlyContinue
$hkcu = Get-ItemProperty -Path "HKCU:\SOFTWARE\Policies\Microsoft\Windows\Installer" -Name "AlwaysInstallElevated" -ErrorAction SilentlyContinue
if ($hklm.AlwaysInstallElevated -eq 1 -and $hkcu.AlwaysInstallElevated -eq 1) {
Write-Host " [!] AlwaysInstallElevated is ENABLED - privesc possible!" -ForegroundColor Red
}
# Stored credentials
Write-Host "`n[+] Stored Credentials:" -ForegroundColor Green
cmdkey /list
Your report structure should follow this template:
# Penetration Test Report - [Client Name]
## Executive Summary
- **Testing Period:** [Dates]
- **Scope:** [Systems/Networks tested]
- **Critical Findings:** [Count]
- **Overall Risk Rating:** [Critical/High/Medium/Low]
## Methodology
Brief description of testing approach and tools used.
## Findings Summary
| ID | Title | Severity | CVSS | Status |
|----|-------|----------|------|--------|
| F01 | SQL Injection in Login | Critical | 9.8 | Open |
## Detailed Findings
### F01: SQL Injection in Login Form
**Severity:** Critical | **CVSS:** 9.8
**Description:** The login form at /auth/login is vulnerable to SQL injection...
**Evidence:**
- Screenshot of successful exploitation
- Request/response pairs
- Extracted data samples (sanitized)
**Impact:** An attacker could bypass authentication, extract all user credentials...
**Remediation:**
1. Implement parameterized queries
2. Apply input validation
3. Deploy WAF rules as interim mitigation
**References:** OWASP SQL Injection, CWE-89
Building a Pentest Lab and Continuous Practice
You can’t practice exploitation on production systems. Build a lab using virtualization—VirtualBox or VMware Workstation for local setups, or cloud providers for more resources.
Start with these vulnerable-by-design applications:
- DVWA (Damn Vulnerable Web Application): Classic web vulnerabilities
- Metasploitable 2/3: Network-level exploitation practice
- HackTheBox/TryHackMe: Guided challenges with increasing difficulty
- VulnHub: Downloadable vulnerable VMs
For certifications, consider OSCP (Offensive Security Certified Professional) for hands-on exploitation skills, or CEH (Certified Ethical Hacker) for broader theoretical coverage. OSCP carries more weight in technical roles because it requires demonstrating actual exploitation skills in a 24-hour practical exam.
The key to improvement is consistent practice. Set aside time weekly to work through new machines, document your methodology, and refine your scripts. Penetration testing is a craft that improves with deliberate practice.