Eric Campbell 6 min read Linux

Linux SSH Keys: Authentication Without Passwords

• SSH key authentication uses asymmetric cryptography to eliminate password transmission over networks, making brute-force attacks ineffective and enabling secure automation

Key Insights

• SSH key authentication uses asymmetric cryptography to eliminate password transmission over networks, making brute-force attacks ineffective and enabling secure automation • Ed25519 keys offer the best balance of security and performance for modern systems, while RSA-4096 remains necessary for legacy compatibility • Disabling password authentication after deploying SSH keys reduces your server’s attack surface by 90%+ according to security audits

Introduction to SSH Key Authentication

Password authentication is fundamentally broken for SSH access. Every login attempt transmits credentials that attackers can intercept, guess, or brute-force. SSH keys solve this by using public-key cryptography—a mathematical system where you prove identity without ever sending secret information across the network.

The practical benefits are immediate. Automated deployments become possible without embedding passwords in scripts. Brute-force attacks become computationally infeasible. Password fatigue disappears when you’re managing dozens of servers. Most importantly, compromising one server doesn’t expose credentials for your entire infrastructure.

SSH keys work through asymmetric cryptography: you generate a key pair consisting of a private key (kept secret) and a public key (distributed freely). The private key never leaves your machine. The server challenges you to prove you possess the private key without actually transmitting it. This mathematical handshake is what makes SSH keys dramatically more secure than passwords.

How SSH Key Pairs Work

Understanding the authentication flow helps you troubleshoot issues and appreciate why SSH keys are secure. When you connect to a server:

  1. Your SSH client announces which public key it wants to use for authentication
  2. The server checks if that public key exists in ~/.ssh/authorized_keys
  3. The server generates a random challenge and encrypts it with your public key
  4. Your SSH client decrypts the challenge using your private key
  5. Your client sends back a hash of the decrypted challenge
  6. The server verifies the hash matches—proving you have the private key without ever transmitting it

This cryptographic handshake is mathematically guaranteed secure. An attacker intercepting the network traffic sees only encrypted challenges and hashed responses—useless without the private key. Even if someone steals your public key, they cannot derive the private key from it. That’s the power of asymmetric cryptography.

The relationship between public and private keys is one-way. You can freely distribute public keys because they only encrypt and verify—they cannot decrypt or sign. The private key must remain secret because it’s the only thing that can decrypt challenges or create valid signatures.

Generating SSH Key Pairs

Creating SSH keys takes seconds but requires choosing the right algorithm. Modern systems should use Ed25519, which offers excellent security with smaller key sizes and faster operations. Use RSA-4096 only when connecting to legacy systems that don’t support Ed25519.

Generate an Ed25519 key pair:

ssh-keygen -t ed25519 -C "your-email@example.com"

The -C flag adds a comment (typically your email) to help identify keys later. You’ll be prompted for a file location (accept the default ~/.ssh/id_ed25519) and a passphrase.

For RSA keys on legacy systems:

ssh-keygen -t rsa -b 4096 -C "your-email@example.com"

The -b 4096 flag specifies a 4096-bit key size. Never use less than 4096 bits for RSA—smaller keys are vulnerable to modern attacks.

When prompted for a passphrase, use one. A passphrase encrypts your private key on disk, so even if someone steals your laptop, they can’t use your keys without the passphrase. This adds a crucial security layer without sacrificing the automation benefits of SSH keys.

Verify your keys were created:

ls -la ~/.ssh/

You should see:

  • id_ed25519 (private key) with permissions -rw------- (600)
  • id_ed25519.pub (public key) with permissions -rw-r--r-- (644)

The private key must be readable only by you. SSH will refuse to use keys with overly permissive settings.

Copying Public Keys to Remote Servers

The easiest method uses ssh-copy-id, which handles everything automatically:

ssh-copy-id user@remote-host

You’ll be prompted for your password one final time. The command copies your public key to the server’s ~/.ssh/authorized_keys file and sets correct permissions.

If ssh-copy-id isn’t available, copy manually:

cat ~/.ssh/id_ed25519.pub | ssh user@remote-host "mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys"

This creates the .ssh directory if needed and appends your public key to authorized_keys. After copying, verify permissions on the remote server:

ssh user@remote-host
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Wrong permissions are the most common reason SSH keys fail. The .ssh directory must be readable/writable/executable only by the owner (700). The authorized_keys file must be readable/writable only by the owner (600). SSH deliberately refuses to use files with broader permissions because they could be modified by other users.

For cloud providers, you typically paste your public key into a web interface during server creation. View your public key:

cat ~/.ssh/id_ed25519.pub

Copy the entire output—it should be a single line starting with ssh-ed25519.

Configuring SSH Client for Easy Access

The SSH config file transforms complex connection commands into simple aliases. Create or edit ~/.ssh/config:

# Production web server
Host prod-web
    HostName 203.0.113.10
    User deploy
    IdentityFile ~/.ssh/id_ed25519
    Port 22

# Development database server
Host dev-db
    HostName 198.51.100.50
    User admin
    IdentityFile ~/.ssh/id_rsa
    Port 2222

# Wildcard for all production servers
Host prod-*
    User deploy
    ForwardAgent no
    StrictHostKeyChecking yes

Now connect using the alias:

ssh prod-web

This is equivalent to ssh -i ~/.ssh/id_ed25519 -p 22 deploy@203.0.113.10 but infinitely more convenient.

Useful SSH config options:

  • IdentityFile: Specify which private key to use
  • Port: Non-standard SSH ports
  • User: Default username for connections
  • ForwardAgent: Enable/disable SSH agent forwarding
  • StrictHostKeyChecking: Prevent MITM attacks by refusing unknown host keys

The config file supports wildcards and pattern matching, letting you apply settings to multiple servers at once.

Security Best Practices

After deploying SSH keys, disable password authentication on your servers. Edit /etc/ssh/sshd_config:

# Disable password authentication
PasswordAuthentication no

# Disable challenge-response authentication
ChallengeResponseAuthentication no

# Disable root login (use sudo instead)
PermitRootLogin no

# Only allow key-based authentication
PubkeyAuthentication yes

Restart the SSH service to apply changes:

sudo systemctl restart sshd

Warning: Test SSH key login before disabling passwords, or you’ll lock yourself out. Keep your current SSH session open while testing a new connection.

Use SSH agent to avoid entering passphrases repeatedly:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

The agent holds decrypted keys in memory for the session duration. You enter your passphrase once, then the agent handles authentication automatically. This combines security (passphrase-protected keys) with convenience (no repeated passphrase entry).

For workstations, configure SSH agent to start automatically. On most Linux distributions, the desktop environment handles this. Verify it’s running:

echo $SSH_AUTH_SOCK

If you see a path like /tmp/ssh-XXX/agent.123, the agent is active.

Rotate SSH keys annually. Generate new keys, deploy them to all servers, then remove old keys from authorized_keys files. This limits the damage if a key is compromised.

Troubleshooting Common Issues

Connection refused or permission denied errors usually stem from file permissions or configuration mistakes.

Debug connections with verbose output:

ssh -v user@remote-host

Use -vv or -vvv for increasingly detailed output. Look for lines indicating which keys are being tried and why they’re rejected.

Verify your key fingerprint matches what’s on the server:

ssh-keygen -lf ~/.ssh/id_ed25519.pub

On the server, check the fingerprint of authorized keys:

ssh-keygen -lf ~/.ssh/authorized_keys

If fingerprints don’t match, you’re using the wrong key or the public key wasn’t copied correctly.

Common permission errors:

  • .ssh directory isn’t 700: chmod 700 ~/.ssh
  • authorized_keys isn’t 600: chmod 600 ~/.ssh/authorized_keys
  • Private key isn’t 600: chmod 600 ~/.ssh/id_ed25519
  • Home directory is group-writable: chmod g-w ~

Check server logs for authentication failures:

sudo tail -f /var/log/auth.log  # Debian/Ubuntu
sudo tail -f /var/log/secure     # RHEL/CentOS

These logs show exactly why authentication failed, including permission issues and configuration problems.

If the server doesn’t recognize your key, verify PubkeyAuthentication yes is set in /etc/ssh/sshd_config and the SSH service was restarted after changes.

SSH key authentication eliminates entire categories of security vulnerabilities while simplifying server access. The initial setup takes minutes, but the security and convenience benefits last for years. Start with Ed25519 keys, protect them with passphrases, and disable password authentication once keys are deployed. Your infrastructure will be more secure and easier to manage.

Liked this? There's more.

Every week: one practical technique, explained simply, with code you can use immediately.