Linux journalctl: Viewing systemd Logs

Key Insights

The systemd journal replaces traditional syslog with a binary format that enables structured queries, persistent storage, and efficient indexing—making log analysis significantly faster than grepping through text files.
Filtering by time, service, and priority level simultaneously is journalctl’s killer feature: journalctl -u nginx.service -p err --since "1 hour ago" gives you exactly what you need without parsing gigabytes of irrelevant logs.
Journal maintenance is critical for production systems—without proper vacuum policies, journals can consume hundreds of gigabytes, and a simple --vacuum-time=7d command prevents disk space emergencies.

Introduction to journalctl and systemd Logging

The systemd journal fundamentally changed how Linux systems handle logging. Unlike traditional syslog, which writes plain text files to /var/log, systemd’s journal stores logs in a structured binary format with automatic indexing. This design enables fast queries across millions of log entries without the performance hit of grepping through text files.

The journalctl command is your interface to this journal. It provides a unified view of all system logs—kernel messages, service output, authentication events, and application logs—in one place. This consolidation eliminates the need to know which specific log file contains the information you’re looking for.

Let’s start with the basics. Running journalctl without arguments displays all available logs:

journalctl

This outputs everything from the oldest entry to the newest, using your system’s default pager (typically less). You can navigate with standard pager controls: space for next page, b for previous page, / to search, and q to quit.

Before diving deeper, check how much disk space your journal consumes:

journalctl --disk-usage

On a typical server running for months, you might see output like Archived and active journals take up 2.1G in the file system. This awareness is crucial for capacity planning.

Most troubleshooting doesn’t require reading the entire journal history. You typically want recent entries or real-time monitoring. Here’s how to control what you see.

To view just the last 50 log entries:

journalctl -n 50

The -n flag works like tail, showing the most recent entries. Change the number to whatever makes sense for your investigation.

For real-time monitoring—watching logs as they’re written—use follow mode:

journalctl -f

This behaves like tail -f, continuously displaying new log entries as they appear. It’s invaluable when you’re actively testing a service or debugging an issue in real-time.

Sometimes you want to investigate recent events by working backwards. Reverse chronological order puts the newest entries first:

journalctl -r

This is particularly useful when you know a problem just occurred and want to see the most recent relevant entries immediately.

If you’re piping output to another command or redirecting to a file, disable the pager:

journalctl --no-pager

Without this flag, journalctl waits for interactive pager input, which breaks scripts and pipelines.

Filtering Logs by Time and Date

Time-based filtering is where journalctl starts showing its superiority over traditional log files. Instead of calculating timestamps and using complex awk commands, you use natural language.

To view logs from a specific date range:

journalctl --since "2024-01-01" --until "2024-01-31"

The date format is flexible. ISO 8601 works, but so do relative times:

journalctl --since "1 hour ago"

This shows everything logged in the past hour. It’s perfect for investigating recent issues without sifting through days of logs.

For daily troubleshooting, the yesterday keyword is incredibly convenient:

journalctl --since yesterday

You can also specify exact timestamps when you need precision:

journalctl --since "2024-01-15 14:30:00"

Combine --since and --until to create precise time windows. This is essential when correlating events across multiple services or investigating issues that occurred during specific maintenance windows.

Filtering by Service, Unit, and Priority

Service-specific filtering is journalctl’s most frequently used feature. Every systemd service logs to the journal, and you can isolate those logs instantly.

To view all logs from nginx:

journalctl -u nginx.service

The .service suffix is optional but recommended for clarity. Combining unit filtering with time filtering creates powerful queries:

journalctl -u ssh.service --since today

This shows all SSH-related activity since midnight, perfect for security audits or investigating authentication issues.

Priority filtering lets you focus on problems. The -p flag accepts syslog priority levels: emerg, alert, crit, err, warning, notice, info, and debug.

To see only errors:

journalctl -p err

You can specify priority ranges to capture multiple severity levels:

journalctl -p warning..err

This shows warnings and errors but excludes informational messages. It’s the sweet spot for troubleshooting—serious enough to matter but not so verbose that you’re drowning in debug output.

Combining service and priority filters creates targeted queries:

journalctl -u postgresql.service -p err --since "1 hour ago"

This command answers the question: “What went wrong with PostgreSQL recently?” in one line.

Advanced Filtering and Output Formats

Process ID filtering helps when you’re tracking a specific application instance:

journalctl _PID=1234

The underscore prefix indicates a journal field. The systemd journal stores structured metadata with each log entry, and you can filter on any field.

Kernel messages deserve special attention. The -k flag shows only kernel logs:

journalctl -k

This replaces dmesg for most purposes and provides better timestamp accuracy and filtering capabilities.

Boot logs are critical for diagnosing startup issues. To see logs from the current boot:

journalctl -b

To view logs from the previous boot (useful after a crash):

journalctl -b -1

The number indicates how many boots ago: -b -2 for two boots ago, and so on. Use journalctl --list-boots to see all available boot sessions.

The default output format is human-readable, but sometimes you need structured data. JSON output enables programmatic log analysis:

journalctl -o json-pretty

This outputs each log entry as formatted JSON, perfect for parsing with jq or importing into log analysis tools.

For maximum detail, verbose output shows all available metadata:

journalctl -o verbose

This displays every field the journal captured, including process IDs, user IDs, systemd unit information, and more. It’s overwhelming for casual viewing but invaluable when you need complete context.

Journal Management and Maintenance

Journals grow continuously. Without maintenance, they’ll consume all available disk space. The vacuum commands provide control over journal size.

To delete logs older than seven days:

journalctl --vacuum-time=7d

This is the most common retention policy for non-critical systems. Adjust the time period based on your compliance requirements and disk capacity.

To limit journal size regardless of age:

journalctl --vacuum-size=500M

This keeps the journal under 500 megabytes, deleting the oldest entries when necessary. It’s useful for systems with limited disk space.

Verify journal integrity after system crashes or disk errors:

journalctl --verify

This checks for corruption and reports any issues. Run it periodically on critical systems.

For permanent retention policies, edit /etc/systemd/journald.conf. Key settings include:

SystemMaxUse=500M
SystemMaxFileSize=100M
MaxRetentionSec=7day

After modifying the configuration, restart the journal service:

systemctl restart systemd-journald

Practical Troubleshooting Examples

Real-world troubleshooting combines multiple filters. Here’s how to apply journalctl to common scenarios.

When a service fails, immediately check its error logs from today:

journalctl -u myapp.service -p err --since today

This shows exactly what went wrong without the noise of informational messages or old errors you’ve already fixed.

After a system crash, investigate boot issues from the previous boot:

journalctl -b -1 -p err

This reveals what failed during the boot sequence, helping you identify hardware issues, configuration problems, or service dependencies.

For monitoring multiple related services simultaneously:

journalctl -u nginx -u mysql -f

This follows logs from both nginx and MySQL in real-time, showing how they interact. It’s perfect for debugging application issues that span multiple services.

When tracking down authentication failures, combine service filtering with grep:

journalctl -u sshd --since "1 hour ago" | grep -i failed

While journalctl has powerful filtering, sometimes grep is still the right tool for text pattern matching.

For capacity planning and performance analysis, export a time range to JSON and analyze with jq:

journalctl --since "2024-01-01" --until "2024-01-31" -o json | jq 'select(.PRIORITY <= 3)'

This extracts all critical errors from January for trend analysis or reporting.

The systemd journal is a robust, performant logging system that rewards learning its query syntax. Master these journalctl commands, and you’ll troubleshoot faster and more effectively than you ever could with traditional log files. The investment in learning these tools pays dividends every time you need to diagnose a production issue quickly.