Linux Log Files & Directory Structure: The Complete Guide for Ubuntu & Debian

Introduction: Why Every Server Admin Needs This Knowledge

You just deployed a new server. Your application is running. Then — a user reports the site is down. Or SSH stops responding. Or disk space hits 100% at 2 AM. Where do you look? What do you check first?

The answer is always the same: logs and directory structure. These two things — knowing where every file lives and how to read what the system tells you — are the foundation of competent server administration. Without them, you are flying blind.

This guide covers everything: the complete Linux directory structure with purpose-driven explanations, every essential log file location, practical troubleshooting scenarios, and the commands you need to work with logs efficiently. It is written for Ubuntu and Debian servers, with notes on differences between the two distributions.

Part 1: The Linux Directory Structure — What Lives Where

The Filesystem Hierarchy Standard (FHS) defines where files belong on a Linux system. Understanding this structure means you always know where to look — whether for configuration files, application data, or logs.

The Complete Directory Tree

/
├── bin/          → Essential user command binaries (ls, cp, bash)
├── boot/         → Bootloader, kernel, initramfs
├── dev/          → Device files (disks, terminals, null)
├── etc/          → ALL system configuration files
│   ├── nginx/        → Nginx config
│   ├── apache2/      → Apache config
│   ├── php/          → PHP config (by version)
│   ├── mysql/        → MySQL config
│   ├── postgresql/   → PostgreSQL config
│   ├── ssh/          → SSH daemon config
│   ├── cron.d/       → Cron job definitions
│   ├── logrotate.d/  → Log rotation rules
│   ├── fail2ban/     → Fail2ban rules
│   └── ufw/          → UFW firewall rules
├── home/         → User home directories (/home/username/)
├── lib/          → Shared libraries for /bin and /sbin
├── media/        → Removable media mount points (USB, CD)
├── mnt/          → Temporary manual mount points
├── opt/          → Third-party / self-contained software
├── proc/         → Virtual filesystem — running processes, kernel state
├── root/         → Root user's home directory
├── run/          → Runtime data (PID files, sockets) — tmpfs, cleared on reboot
├── sbin/         → System administration binaries (fsck, iptables)
├── srv/          → Service data (web, FTP)
├── sys/          → Virtual filesystem — kernel and device information
├── tmp/          → Temporary files — cleared on reboot
├── usr/          → User programs and data (read-only after install)
│   ├── bin/          → Most user commands (grep, vim, git)
│   ├── sbin/         → System commands for admin (useradd, nginx)
│   ├── lib/          → Libraries for /usr/bin and /usr/sbin
│   ├── local/        → Manually compiled/installed software
│   │   ├── bin/          → Custom user binaries
│   │   ├── lib/          → Custom libraries
│   │   └── sbin/         → Custom system binaries
│   ├── share/        → Architecture-independent data (man pages, docs)
│   └── include/      → Header files for development
└── var/          → Variable data that changes during operation
    ├── log/          → LOG FILES — all system and service logs
    ├── www/          → Web server document roots
    ├── lib/          → Application state data (databases, package lists)
    ├── cache/        → Application cache files
    ├── spool/        → Queued data (mail, print, cron)
    ├── run/          → Symlink to /run (legacy compatibility)
    ├── backups/      → System backup files
    ├── mail/         → Local mail storage
    └── tmp/          → Temporary files — preserved across reboots

The Most Important Directories Explained

/etc/ — Configuration Central

/etc/ stands for etcetera — historically, but today it means "everything configuration." Every daemon, service, and application stores its configuration here. Editing files in /etc/ changes how services behave. Key rule: always back up before editing.

/etc/
├── hosts             → Local DNS overrides (hostname → IP)
├── hostname          → This server's hostname
├── fstab             → Filesystem mount table
├── resolv.conf       → DNS resolver configuration
├── ssh/sshd_config   → SSH server configuration
├── crontab           → System-wide cron jobs
├── sudoers           → Sudo permission rules
├── passwd            → User account database
├── shadow            → Hashed user passwords (root-only)
├── group             → Group definitions
├── environment       → System-wide environment variables
├── profile           → Shell initialization for all users
├── apt/              → APT package manager configuration
└── systemd/          → Systemd unit files and overrides

/var/ — Where the Action Happens

/var/ is the most dynamic part of the filesystem. Logs grow here. Databases store data here. Mail queues here. If your disk fills up, the culprit is almost always in /var/.

/var/
├── log/              → ALL logs (primary focus of this guide)
├── www/              → Web document roots
│   └── html/             → Default Nginx/Apache document root
├── lib/
│   ├── mysql/            → MySQL database files
│   ├── postgresql/       → PostgreSQL database files
│   ├── apt/lists/        → APT package index cache
│   └── dpkg/             → Installed package database
├── cache/
│   ├── apt/              → Downloaded .deb package cache
│   └── nginx/            → Nginx proxy cache
├── spool/
│   ├── cron/             → Per-user cron tables
│   ├── mail/             → Local mail inbox
│   └── cups/             → Print queue
├── backups/          → dpkg, apt snapshot backups
└── mail/             → Alternative mail storage location

/tmp/ vs /var/tmp/ — The Difference Matters

/proc/ and /sys/ — The Kernel's Window

These directories do not contain real files. They are virtual filesystems that expose kernel internals in real time.

# CPU information
cat /proc/cpuinfo

# Memory usage
cat /proc/meminfo

# Running processes
ls /proc/          # Each numbered dir = a PID

# Network connections
cat /proc/net/tcp

# Kernel parameters
cat /sys/kernel/ostype
ls /sys/class/net/  # Network interfaces

/run/ — Runtime Data

/run/ (and its legacy symlink /var/run/) stores runtime state: PID files, Unix sockets, lock files. It is a tmpfs mount, meaning it lives in RAM and is completely cleared on reboot. Services write their PID here so other processes (and systemd) can track them.

# Example contents
/run/nginx.pid          → Nginx process ID
/run/mysql/mysql.sock   → MySQL Unix socket
/run/sshd.pid           → SSH daemon PID
/run/php/               → PHP-FPM sockets and PIDs

/opt/ — Third-Party Software

/opt/ is designed for self-contained software that does not follow the FHS convention of spreading files across /usr/bin/, /etc/, etc. Tools like Panelica, Google Chrome, and many commercial applications install here because they can be fully isolated and removed cleanly.

/usr/bin/ vs /usr/sbin/ vs /usr/local/

/usr/bin/         → Commands for all users (ls, grep, curl, php)
/usr/sbin/        → Commands for system administration (useradd, nginx, sshd)
/usr/local/bin/   → Manually installed user commands (not from apt)
/usr/local/sbin/  → Manually installed admin commands
/usr/local/lib/   → Libraries for manually installed software

The local/ subtree is intentionally separate so that manually compiled software does not conflict with distribution-packaged software. If you compile PHP from source, it goes in /usr/local/.

Part 2: Essential Log Files — The Complete Reference

Every service on Linux writes logs. Here is a complete reference of where to find them and what each file contains.

System Logs

Package Management Logs

Web Server Logs

# Nginx
/var/log/nginx/access.log      → Every HTTP request (IP, method, status, URL)
/var/log/nginx/error.log       → Errors, upstream failures, config problems

# Per-site logs (common pattern)
/var/log/nginx/site.com-access.log
/var/log/nginx/site.com-error.log

# Apache
/var/log/apache2/access.log    → HTTP request log
/var/log/apache2/error.log     → Errors and warnings
/var/log/apache2/other_vhosts_access.log  → Virtual host requests

Reading an Nginx Access Log

# Default combined log format:
# remote_addr - remote_user [time] "request" status body_bytes "referrer" "user_agent"

203.0.113.42 - - [28/Mar/2026:14:23:01 +0000] "GET /wp-login.php HTTP/1.1" 200 4521 "-" "Mozilla/5.0..."

# Field breakdown:
# 203.0.113.42  → Client IP
# -             → Identd (always -)
# -             → Auth user (if no auth)
# [timestamp]   → Request time
# "GET /path"   → HTTP method + URI + protocol
# 200           → HTTP status code
# 4521          → Response body size in bytes
# "-"           → Referrer URL
# "Mozilla..."  → User agent string

Database Logs

# MySQL / MariaDB
/var/log/mysql/error.log         → Startup errors, crashes, InnoDB issues
/var/log/mysql/mysql-slow.log    → Slow queries (if slow_query_log=ON)
/var/log/mysql/general.log       → All queries (WARNING: very large, disabled by default)

# PostgreSQL
/var/log/postgresql/postgresql-*.log   → All Postgres messages, errors, connections

Mail Server Logs

# Ubuntu/Debian
/var/log/mail.log        → Postfix delivery, SMTP connections, Dovecot auth
/var/log/mail.err        → Mail errors only
/var/log/mail.warn       → Warnings

# OpenDKIM (if installed)
/var/log/syslog          → DKIM signing/verification in syslog

Reading a Mail Log Entry

# Postfix delivery attempt:
Mar 28 14:23:01 server postfix/smtp[12345]: A1B2C: to=,
  relay=mail.example.com[93.184.216.34]:25, delay=2.3,
  delays=0.1/0.0/1.8/0.4, dsn=2.0.0, status=sent (250 OK)

# Fields:
# A1B2C         → Message queue ID
# to=           → Recipient address
# relay=        → Outbound mail server used
# delay=        → Total time in queue
# status=sent   → Delivery result

Security and Firewall Logs

/var/log/auth.log           → SSH logins, sudo commands, PAM failures
/var/log/fail2ban.log       → Fail2ban bans, unbans, filter matches
/var/log/ufw.log            → UFW firewall allow/block events
/var/log/syslog             → Also captures firewall kernel messages

Reading an auth.log SSH Entry

# Successful SSH login:
Mar 28 14:23:01 server sshd[1234]: Accepted publickey for admin from 203.0.113.42 port 52341 ssh2

# Failed password:
Mar 28 14:23:05 server sshd[1235]: Failed password for root from 198.51.100.7 port 44123 ssh2

# Sudo usage:
Mar 28 14:23:10 server sudo: admin : TTY=pts/0 ; PWD=/var/www ; USER=root ; COMMAND=/usr/bin/systemctl restart nginx

PHP Logs

# Location depends on php.ini configuration:
/var/log/php8.3-fpm.log              → PHP-FPM pool errors
/var/log/php/php-error.log           → Custom PHP error log (if configured)

# Or check php.ini for the actual path:
php -i | grep error_log

# Typical php.ini setting:
error_log = /var/log/php/error.log

Cron Logs

# Ubuntu (rsyslog routes cron to syslog):
grep CRON /var/log/syslog

# Dedicated cron log (if configured in /etc/rsyslog.d/):
/var/log/cron.log

# Specific user crontab output — redirect in the crontab itself:
* * * * * /path/to/script.sh >> /var/log/mycron.log 2>&1

Systemd Journal — The Modern Log System

On systems using systemd (Ubuntu 16.04+ and Debian 8+), the journal captures all service output in a structured binary format. The journalctl command queries it.

# View all journal entries (recent first)
journalctl -r

# Follow live (like tail -f)
journalctl -f

# Specific service
journalctl -u nginx
journalctl -u mysql
journalctl -u ssh

# Time range
journalctl --since "2026-03-28 00:00:00"
journalctl --since "1 hour ago"
journalctl --since yesterday

# Error level and above only
journalctl -p err
journalctl -p warning

# Combine filters
journalctl -u nginx -p err --since "1 hour ago"

# Current boot
journalctl -b

# Previous boot (useful after a crash)
journalctl -b -1

Part 3: Ubuntu vs Debian Differences

Part 4: Essential Commands for Log Analysis

Reading and Following Logs

# Real-time log follow (Ctrl+C to exit)
tail -f /var/log/syslog
tail -f /var/log/nginx/error.log

# Last 100 lines
tail -n 100 /var/log/auth.log

# Better follow with less (supports scroll and search)
less +F /var/log/syslog
# Press Ctrl+C to stop following, then /search to search, then F to resume following

# View beginning of log
head -n 50 /var/log/dpkg.log

# Entire file with line numbers
cat -n /var/log/fail2ban.log

Searching Logs

# Basic search (case sensitive)
grep "error" /var/log/syslog

# Case insensitive
grep -i "failed" /var/log/auth.log

# Show 3 lines before and after each match (context)
grep -B3 -A3 "Out of memory" /var/log/kern.log

# Count matches
grep -c "Failed password" /var/log/auth.log

# Show filename with matches (useful for multiple files)
grep -l "error" /var/log/nginx/*.log

# Invert match (show lines NOT matching)
grep -v "GET /assets" /var/log/nginx/access.log

# Multiple patterns (OR)
grep -E "error|warning|critical" /var/log/syslog

# Regex: find IP addresses
grep -oE '([0-9]{1,3}\.){3}[0-9]{1,3}' /var/log/auth.log | sort | uniq -c | sort -rn

Working with Compressed Rotated Logs

# Rotated logs are compressed with gzip (.gz extension)
ls /var/log/nginx/
# access.log  access.log.1  access.log.2.gz  access.log.3.gz

# Read compressed log
zcat /var/log/nginx/access.log.2.gz

# Search compressed log
zgrep "500" /var/log/nginx/access.log.2.gz

# Read and follow across rotated files
cat /var/log/auth.log.1 /var/log/auth.log | grep "Failed"

# Search ALL rotated versions
zgrep -h "Failed password" /var/log/auth.log* | wc -l

Login History Commands

# All login/logout history
last

# Failed login attempts (reads /var/log/btmp)
lastb

# Last login time for each user
lastlog

# Who is currently logged in
who

# Who is logged in with extended info
w

Kernel Ring Buffer

# Current kernel messages (human-readable timestamps)
dmesg -T

# Follow kernel messages live
dmesg -Tw

# Filter for errors
dmesg -T --level=err,crit,alert,emerg

# Hardware errors (disk, memory)
dmesg -T | grep -i "error\|fail\|warn"

# OOM (Out of Memory) killer events
dmesg -T | grep -i "oom\|killed process"

Finding Recently Modified Log Files

# Files modified in the last 24 hours
find /var/log -mtime -1 -type f

# Files modified in the last 1 hour
find /var/log -mmin -60 -type f

# Largest log files
du -sh /var/log/* 2>/dev/null | sort -rh | head -20

# Total log directory size
du -sh /var/log/

Part 5: Practical Troubleshooting Scenarios

Here is how to apply this knowledge to real problems.

Scenario 1: "My Server Is Slow"

# 1. Check current load
top
htop

# 2. Look for OOM killer events (killed processes due to RAM)
dmesg -T | grep -i "oom\|killed process"

# 3. Check kernel hardware errors
grep -i "error\|fail" /var/log/kern.log | tail -50

# 4. Check syslog for unusual activity
tail -n 200 /var/log/syslog | grep -i "error\|warn\|fail"

# 5. Check disk I/O
iostat -x 1 5
iotop

Scenario 2: "I Can't SSH Into My Server"

# On the server (if you have console access):

# 1. Is SSH running?
systemctl status ssh
journalctl -u ssh -n 50

# 2. Check auth log for recent events
tail -n 100 /var/log/auth.log

# 3. Are you blocked by fail2ban?
grep "Ban" /var/log/fail2ban.log | tail -20
fail2ban-client status sshd

# 4. Is the firewall blocking you?
grep "BLOCK" /var/log/ufw.log | tail -20
ufw status numbered

# 5. Is SSH listening?
ss -tlnp | grep :22
netstat -tlnp | grep :22

Scenario 3: "Disk Is Full"

# 1. Where is the space used?
df -h
du -sh /var/* 2>/dev/null | sort -rh | head -20
du -sh /var/log/* 2>/dev/null | sort -rh | head -20

# 2. Check for massive log files
find /var/log -type f -size +100M

# 3. Check for large files system-wide
find / -type f -size +500M 2>/dev/null

# 4. Check MySQL binary logs (often huge)
ls -lh /var/lib/mysql/mysql-bin.*

# 5. Clear APT cache
apt-get clean
du -sh /var/cache/apt/

# 6. Force log rotation
logrotate -f /etc/logrotate.conf

Scenario 4: "Website Returns 502 / 503"

# 1. Nginx error log
tail -n 100 /var/log/nginx/error.log

# 2. PHP-FPM log
tail -n 100 /var/log/php8.3-fpm.log
journalctl -u php8.3-fpm -n 100

# 3. Is PHP-FPM running?
systemctl status php8.3-fpm

# 4. Is the upstream (Node.js, Gunicorn, etc.) running?
journalctl -u your-app-service -n 50

# 5. Check PHP max connections / pool exhaustion
grep "max_children\|request_terminate_timeout" /var/log/php8.3-fpm.log

Scenario 5: "Mail Is Not Sending"

# 1. Postfix queue status
mailq
postqueue -p

# 2. Mail log - last delivery attempts
tail -n 100 /var/log/mail.log

# 3. Is Postfix running?
systemctl status postfix
journalctl -u postfix -n 50

# 4. Test SMTP locally
echo "test" | mail -s "test subject" [email protected]
# Then check:
tail -f /var/log/mail.log

# 5. Check for DSN bounce messages
grep "status=bounced" /var/log/mail.log | tail -20

Scenario 6: "Suspect the Server Was Compromised"

# 1. Who has logged in recently?
last -n 50
lastlog | grep -v "Never"

# 2. Failed login attempts by IP
grep "Failed password" /var/log/auth.log | grep -oE '([0-9]{1,3}\.){3}[0-9]{1,3}' | sort | uniq -c | sort -rn

# 3. Successful logins from unexpected IPs
grep "Accepted" /var/log/auth.log | tail -50

# 4. New users created
grep "useradd\|adduser" /var/log/auth.log
cat /var/log/auth.log | grep "new user"

# 5. Sudo usage history
grep "sudo:" /var/log/auth.log | tail -50

# 6. Recently modified system files
find /etc /bin /sbin /usr/bin /usr/sbin -mtime -7 -type f 2>/dev/null

# 7. Unusual cron jobs
cat /etc/crontab
ls -la /etc/cron.d/ /etc/cron.daily/ /etc/cron.hourly/
crontab -l  # root's crontab

Part 6: Log Rotation — Preventing Disk Emergencies

Without log rotation, log files grow indefinitely until they fill your disk. logrotate is the standard solution on Debian and Ubuntu.

How logrotate Works

A cron job (usually in /etc/cron.daily/) runs logrotate daily. It checks /etc/logrotate.conf and all files in /etc/logrotate.d/ for rules, then compresses, renames, and deletes old logs according to those rules.

The Main Config File

# /etc/logrotate.conf (global defaults)
weekly          # Rotate logs weekly
rotate 4        # Keep 4 rotated versions
create          # Create new empty log file after rotation
compress        # Compress rotated logs with gzip
delaycompress   # Compress on the next rotation cycle (not immediately)
include /etc/logrotate.d   # Load per-service configs

Per-Service Config Example

# /etc/logrotate.d/nginx
/var/log/nginx/*.log {
    daily               # Rotate daily
    missingok           # Don't error if log file is missing
    rotate 14           # Keep 14 days of logs
    compress            # Compress old logs
    delaycompress       # Don't compress the most recent rotated log
    notifempty          # Don't rotate empty logs
    sharedscripts       # Run postrotate once for all matched files
    postrotate
        /bin/kill -USR1 $(cat /run/nginx.pid 2>/dev/null) 2>/dev/null || true
        # Sends USR1 signal to Nginx to reopen log files
    endscript
}

Custom Log Rotation for Your Apps

# /etc/logrotate.d/myapp
/var/log/myapp/*.log {
    daily
    rotate 30
    compress
    delaycompress
    missingok
    notifempty
    create 0640 www-data adm
    postrotate
        systemctl reload myapp 2>/dev/null || true
    endscript
}

Testing and Forcing Rotation

# Test without actually rotating (dry run)
logrotate --debug /etc/logrotate.conf

# Force rotation right now (ignores time-based rules)
logrotate -f /etc/logrotate.conf

# Force rotation for specific service only
logrotate -f /etc/logrotate.d/nginx

Part 7: Quick Reference Cheat Sheet

Log File Locations at a Glance

Command Quick Reference

Conclusion: Build the Habit

The difference between a confident server administrator and one who panics under pressure is usually not skill — it is familiarity. Knowing that auth.log is the first place to look when SSH behaves oddly, knowing that a 502 error points to PHP-FPM before Nginx, knowing that dmesg -T shows hardware problems in real time — these are reflexes built through repetition.

Bookmark this guide. The next time something breaks at 3 AM, you will know exactly where to look.