Log Analysis — where they live, what to look for, and suspicious events

Logs are the primary source of evidence in a security incident. Knowing where they are, what they mean, and how to filter out the noise is a core skill for any security analyst.

Linux — main log files

/var/log/auth.log       — SSH, sudo, su authentication (Debian/Ubuntu)
/var/log/secure         — same, on Red Hat/CentOS
/var/log/syslog         — general system messages
/var/log/kern.log       — kernel messages
/var/log/cron           — cron job executions
/var/log/audit/audit.log — auditd events (detailed)
/var/log/apache2/       — Apache access and error logs
/var/log/nginx/         — Nginx logs

What to look for in auth.log

# SSH authentication failures (brute-force)
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn | head -20

# Successful login after multiple failures — sign of successful brute-force
grep "Accepted password\|Accepted publickey" /var/log/auth.log

# Sudo usage
grep "sudo:" /var/log/auth.log | grep -v "pam_unix"

# User creation or password change
grep "useradd\|passwd\|usermod" /var/log/auth.log

Windows — Event Viewer and critical IDs

ID 4624 — Successful logon
ID 4625 — Failed logon (brute-force indicator)
ID 4648 — Logon using explicit credentials (pass-the-hash, runas)
ID 4672 — Special privileges assigned at logon
ID 4688 — Process created (with Sysmon, includes command line)
ID 4698 — Scheduled task created
ID 4720 — User account created
ID 4728 — Member added to privileged group
ID 7045 — Service installed
ID 1102 — Audit log cleared (sign of covering tracks)

# Filter logon failures in the last 24 hours
Get-WinEvent -FilterHashtable @{
  LogName='Security'; Id=4625;
  StartTime=(Get-Date).AddHours(-24)
} | Select-Object TimeCreated, Message | Format-List

Suspicious patterns

SSH brute-force:
  Dozens of "Failed password" from the same IP within minutes
  Followed by "Accepted" — confirmed compromise

Lateral movement:
  Logon type 3 (network) from an unusual internal IP
  Logon outside business hours

Backdoor creation:
  useradd + usermod -aG sudo in sequence
  New service installed (7045) + binary outside System32

Log tampering:
  Event ID 1102 (log cleared) or log suddenly truncated
  Time gap in logs — indicates manipulation

Correlation and centralization

Logs isolated on each machine are hard to correlate. Use a SIEM or centralized log stack:

Typical flow:
  Host → rsyslog/winlogbeat → Logstash/Fluentd → Elasticsearch → Kibana/Grafana

Simple correlation rule:
  If the same IP generates > 10 logon failures in 5 minutes → alert
  If a successful logon follows consecutive failures → critical alert

Useful tools

# grep + awk for quick analysis
grep "Failed password" /var/log/auth.log | awk '{print $11}' | sort | uniq -c | sort -rn

# journalctl (systemd)
journalctl -u sshd --since "1 hour ago" --no-pager

# last — login history
last -a | head -30
lastb | head -30   # failed attempts

Preserve logs: configure rotation to keep at least 90 days and ship copies to immutable storage (e.g., S3 with Object Lock).