LINUX FIELD GUIDE // 03
Turn Linux logs into a reproducible incident report
Filter, normalize, count, and preview derived data without modifying the original log.
Large logs hide a small but important signal. Find it and reshape it into useful output.
BEFORE YOU START
Prerequisites and boundaries
- Copy sample logs into a writable practice directory.
- Confirm the timestamp and field layout before relying on positional extraction.
- Estimate input size and available disk space before creating intermediate files.
OPERATIONAL REASONING
Why this sequence matters
The goal is not merely to memorize commands. Each inspection, change, and verification step reduces a different failure mode.
Preserve the source
Derived files make each transformation inspectable and keep the original evidence unchanged.
Sort before uniq
uniq only combines adjacent duplicate lines, so ordering is a required prerequisite.
Preview bounded output
Counts and a small sample catch parsing mistakes before a report is shared.
STANDARD PROCEDURE
Extract error lines from a log, normalize their order, and produce a reusable report.
Filter first, sort before deduplication, count the resulting report, then preview a bounded sample.
- 01
grep -n "ERROR" app.log > errors.logGNU grep manual ↗ - 02
sort errors.log -o errors.logGNU sort documentation ↗ - 03
uniq errors.log errors-unique.logGNU uniq documentation ↗ - 04
wc -l errors-unique.logGNU wc documentation ↗ - 05
head -n 20 errors-unique.logGNU head documentation ↗
INCIDENT RESPONSE
Find repeated failed-login sources in an authentication log without modifying the original.
Create derived files, extract the source field, sort before counting duplicates, and rank the final counts numerically.
- 01
grep "Failed password" auth.log > failed.logGNU grep manual ↗ - 02
awk '{print $NF}' failed.log > sources.logGNU awk manual ↗ - 03
sort sources.log -o sources.logGNU sort documentation ↗ - 04
uniq -c sources.log > source-counts.logGNU uniq documentation ↗ - 05
sort -nr source-counts.log -o source-counts.logGNU sort documentation ↗
COMMON FAILURE MODES
What usually goes wrong
- A broad grep pattern matched informational lines that merely mentioned an error.
- awk extracted the wrong field because the log format changed.
- Redirection overwrote the input file before the command could read it.
VERIFICATION
Evidence to collect
- wc compares the input, filtered, and unique record counts.
- head and tail confirm the first and last records retain the expected structure.
- A manual sample of source lines matches the derived report.