Bash Here Documents: Multi-Line Input

Key Insights

• Here documents provide a clean way to handle multi-line input in Bash scripts, eliminating the need for messy echo chains or external files for configuration generation and templated content.
• The <<- operator strips leading tabs (not spaces) from heredoc content, enabling proper indentation of multi-line text within your script structure without affecting the output.
• Quoting the delimiter (<<'EOF') disables variable expansion entirely, giving you precise control over whether your heredoc content is treated as a literal string or processed for substitution.

Introduction to Here Documents

Here documents (heredocs) are a redirection mechanism in Bash that allows you to pass multi-line input to commands without creating temporary files or chaining multiple echo statements. They’re particularly valuable when you need to generate configuration files, construct complex queries, or provide structured input to commands.

The syntax is straightforward: you use << followed by a delimiter, then your content, and close with the same delimiter on its own line. Here’s the simplest example:

cat <<EOF
This is line one.
This is line two.
This is line three.
EOF

This sends three lines to cat, which outputs them to stdout. The delimiter EOF (End Of File) is conventional but arbitrary—you can use any string that won’t appear in your content.

When should you use heredocs versus alternatives? Use heredocs when you need multiple lines of structured text within your script. For single lines, echo or printf is clearer. For truly large content, external files are better for maintainability. Heredocs shine in the middle ground: generating configs, creating SQL statements, or building templates where the content logically belongs in the script itself.

Basic Here Document Syntax and Usage

The standard heredoc structure uses the << operator followed by a delimiter. The content begins on the next line and continues until the delimiter appears alone on a line:

<<DELIMITER
content goes here
more content
DELIMITER

Writing to files is accomplished through standard redirection:

cat <<EOF > /etc/myapp/config.conf
server_name=production
port=8080
max_connections=100
EOF

You can also pipe heredocs to commands:

grep "error" <<LOGS
2024-01-15 10:23:45 INFO Application started
2024-01-15 10:24:12 ERROR Connection timeout
2024-01-15 10:24:15 ERROR Database unreachable
2024-01-15 10:25:01 INFO Retry successful
LOGS

Common delimiter conventions include EOF, END, HEREDOC, and context-specific names like SQL or CONFIG. Choose delimiters that make your intent clear. For nested heredocs or content that might contain common delimiters, use more specific markers like END_SQL or CONFIG_BLOCK.

Variable Expansion and Quoting

By default, heredocs process variable expansion and command substitution just like double-quoted strings:

#!/bin/bash
USER="admin"
TIMESTAMP=$(date +%Y-%m-%d)

cat <<EOF
Configuration generated for: $USER
Date: $TIMESTAMP
Home directory: $HOME
EOF

Output:

Configuration generated for: admin
Date: 2024-01-15
Home directory: /home/youruser

To disable expansion and treat content literally, quote the delimiter:

#!/bin/bash
PRICE=100

cat <<'EOF'
The variable $PRICE will not expand.
Neither will $(date) execute.
Everything is literal: $HOME, `whoami`, etc.
EOF

Output:

The variable $PRICE will not expand.
Neither will $(date) execute.
Everything is literal: $HOME, `whoami`, etc.

This is critical when generating scripts or configuration files that contain their own variable syntax. For instance, when creating a shell script that will use its own variables, you need the literal $ characters preserved.

You can also escape individual characters within an expanding heredoc:

cat <<EOF
This will expand: $HOME
This will not: \$HOME
EOF

Indentation and Tab Stripping

When writing heredocs inside indented code blocks (functions, conditionals, loops), maintaining script readability while controlling output formatting becomes challenging. The <<- operator solves this by stripping leading tabs from each line:

#!/bin/bash

generate_config() {
    if [ "$1" == "production" ]; then
        cat <<-EOF
		[database]
		host=prod-db.example.com
		port=5432
		
		[cache]
		enabled=true
		EOF
    fi
}

generate_config production

The heredoc content is indented with tabs in the script for readability, but the <<- operator removes those leading tabs from the output. This produces clean configuration files while keeping your script properly structured.

Critical limitation: The <<- operator only strips tabs, not spaces. If you indent with spaces, they’ll appear in the output. This is a common source of frustration:

# This WON'T work as expected with spaces
cat <<-EOF
    These leading spaces will appear in output
    Because <<- only strips tabs
EOF

Many developers work around this by using tabs only for heredoc indentation, even if the rest of their script uses spaces.

Practical Use Cases

Generating Configuration Files

Heredocs excel at creating configuration files during deployment or setup scripts:

#!/bin/bash

create_nginx_config() {
    local domain=$1
    local port=$2
    
    cat <<EOF > "/etc/nginx/sites-available/$domain"
server {
    listen 80;
    server_name $domain;
    
    location / {
        proxy_pass http://localhost:$port;
        proxy_set_header Host \$host;
        proxy_set_header X-Real-IP \$remote_addr;
    }
}
EOF
}

create_nginx_config "api.example.com" 3000

Note the escaped \$host and \$remote_addr—these are Nginx variables that should appear literally in the config file, not Bash variables to be expanded.

Multi-Line SQL Queries

Database operations often require complex queries that are more readable as heredocs:

#!/bin/bash

mysql -u admin -p"$DB_PASSWORD" <<SQL
USE analytics;

CREATE TABLE IF NOT EXISTS user_events (
    id INT AUTO_INCREMENT PRIMARY KEY,
    user_id INT NOT NULL,
    event_type VARCHAR(50),
    timestamp DATETIME DEFAULT CURRENT_TIMESTAMP,
    INDEX idx_user_id (user_id),
    INDEX idx_event_type (event_type)
);

INSERT INTO user_events (user_id, event_type) VALUES
    (1001, 'login'),
    (1002, 'purchase'),
    (1001, 'logout');
SQL

Remote Command Execution

Heredocs work beautifully for sending multiple commands to remote systems:

#!/bin/bash

ssh user@remote-server <<'COMMANDS'
cd /var/www/app
git pull origin main
npm install --production
pm2 restart app
echo "Deployment completed at $(date)"
COMMANDS

The quoted delimiter prevents local variable expansion, ensuring commands execute in the remote environment’s context.

Here Strings (Brief Comparison)

Here strings (<<<) provide single-line input redirection, useful when you need to pass a string to a command that expects stdin:

# Using here string
grep "error" <<< "$log_line"

# Equivalent to
echo "$log_line" | grep "error"

# Here string with literal content
bc <<< "scale=2; 22/7"

Here strings are more concise than heredocs for single-line input and avoid the overhead of piping from echo. Use heredocs when you need multiple lines; use here strings for single-line stdin input.

Best Practices and Tips

Choose meaningful delimiters. Use SQL for database queries, HTML for markup, CONFIG for configuration files. This self-documents your code’s intent.

Be cautious with variable expansion in security-sensitive contexts. If your heredoc content comes from user input or external sources, use quoted delimiters to prevent unintended expansion or command injection:

# Dangerous if $user_input contains malicious content
cat <<EOF
User data: $user_input
EOF

# Safer
cat <<'EOF'
User data: $user_input
EOF

Keep heredocs readable. Don’t nest them too deeply or make them excessively long. If your heredoc exceeds 50 lines, consider whether it belongs in an external template file instead.

Watch for trailing whitespace on the delimiter line. The closing delimiter must appear on a line by itself with no trailing spaces or tabs (unless using <<- with tabs). This is a common source of “unexpected end of file” errors.

Use <<- consistently with tabs if you need indentation. Configure your editor to insert tabs for heredoc content even if you use spaces elsewhere.

Test your output. When generating configuration files, always verify the output matches your expectations, especially regarding whitespace and variable expansion.

Heredocs are a powerful tool for making Bash scripts more maintainable and readable. They eliminate the need for awkward echo chains and temporary files while keeping related content together in your script. Master the quoting and indentation rules, and you’ll find countless uses for this elegant feature.