SSH Configuration Explained: Learn to Control Servers Like a Pro

Introduction

In today’s distributed and cloud-centric environments, SSH configuration is a non-negotiable skill for any serious system administrator or developer. SSH (Secure Shell) is not just a tool for remote login – it’s a gateway to securely control, automate, and audit your infrastructure.

Yet many only scratch the surface of what SSH can do. Misconfigured SSH servers often expose systems to unnecessary risk, use outdated authentication methods, or lack essential hardening and access controls.

In this comprehensive guide, you’ll learn how to properly configure SSH both on the client and server side, enforce least privilege, use SSH keys securely, and tap into advanced features that help you control your servers like a pro.

Whether you’re managing cloud VMs, bare-metal servers, or large multi-tenant networks, understanding SSH deeply gives you a serious operational edge.

What Is SSH and Why Configuration Matters

SSH (Secure Shell) is a cryptographic protocol widely used to securely connect to remote systems over untrusted networks. It replaces legacy protocols like Telnet with encrypted communication, authentication mechanisms, and channel multiplexing.

By default, SSH:

Encrypts data in transit
Verifies hosts and users using keys or passwords
Supports command execution, tunneling, and file transfer

However, its true power lies in its configurability and extensibility. Poor SSH configuration is one of the most common vectors for brute-force attacks, unauthorized access, and operational inefficiencies.

Let’s explore how to configure SSH for maximum security, usability, and performance.

SSH Server Configuration (`/etc/ssh/sshd_config`)

The heart of SSH server behavior lies in /etc/ssh/sshd_config. This file defines how the server listens for connections, authenticates users, authorizes commands, and manages security.

Essential Hardening Directives

Below are the most important settings for a secure and functional SSH setup:

Port 22                      # Default TCP port – change for obscurity, not primary security
Protocol 2                   # Only use protocol 2 (protocol 1 is insecure and deprecated)
PermitRootLogin no           # Do not allow root login remotely
PasswordAuthentication no    # Enforces key-based authentication
PermitEmptyPasswords no      # Never allow blank passwords
MaxAuthTries 3               # Limit brute-force attempts
LoginGraceTime 30s           # Reduced login window
UsePAM yes                   # Required for 2FA and other auth integrations
AllowUsers admin devops      # Restrict access to trusted usernames
AllowGroups ssh-users        # Use group-based access control
X11Forwarding no             # Disable unless you need GUI forwarding
AllowTcpForwarding no        # Disable port forwarding unless strictly needed

Tip: After modifying sshd_config, restart the SSH daemon:

sudo systemctl restart sshd

Securing with Non-Standard Ports

Using a port other than 22 (e.g., 2200) can deter automated scanners:

Port 2200

Be sure to open this port in your firewall and update client configs.

SSH Client Configuration (`~/.ssh/config`)

The SSH client config lives at ~/.ssh/config and helps automate logins, simplify repetitive commands, and enhance organization when accessing multiple servers.

Example Multi-Host SSH Config

Host prod-server
    HostName prod.example.com
    User ubuntu
    IdentityFile ~/.ssh/prod_ed25519
    Port 2200

Host test-server
    HostName test.local
    User tester
    IdentityFile ~/.ssh/test_key

You can now connect using:

ssh prod-server

Time-Saving Options

ForwardAgent – Enables key authentication with remote servers (use cautiously)
ControlMaster and ControlPersist – Enable connection multiplexing for faster repeated connections

Example:

Host *
  ControlMaster auto
  ControlPath ~/.ssh/sockets/%r@%h-%p
  ControlPersist 10m

Key-Based Authentication: Best Practices

SSH keys significantly improve security over password-based logins. They’re nearly uncrackable when properly managed.

Generate a Secure Key

# Recommended modern key
ssh-keygen -t ed25519 -C "[email protected]"

# Or for legacy compatibility
ssh-keygen -t rsa -b 4096 -C "[email protected]"

Your keys will be saved to:

Private key: ~/.ssh/id_ed25519 (keep secure!)
Public key: ~/.ssh/id_ed25519.pub (copy to remote)

Install Keys on Remote Server

ssh-copy-id -i ~/.ssh/id_ed25519.pub user@host

Alternatively, manually append the public key to:

~/.ssh/authorized_keys

Recommended Permissions

chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys
chmod 600 ~/.ssh/id_ed25519

Never share your private key or leave it on public systems.

Port Forwarding and Tunnels

SSH tunnels allow you to securely access systems behind firewalls or proxies.

Local Port Forwarding

Exposes a remote service on your local machine:

ssh -L 8080:localhost:80 user@remote

Now localhost:8080 forwards to the remote machine’s localhost:80.

Remote Port Forwarding

Exposes a local service to the remote machine:

ssh -R 2022:localhost:22 user@remote

Dynamic Port Forwarding (SOCKS Proxy)

Turns SSH into a proxy tunnel:

ssh -D 1080 user@remote

Configure browser to use localhost:1080 as a SOCKS5 proxy.

Using SSH Agent Securely

Avoid typing your passphrase every session using ssh-agent and ssh-add.

Start agent and add key:

eval "$(ssh-agent -s)"
ssh-add ~/.ssh/id_ed25519

Add timeout:

ssh-add -t 3600 ~/.ssh/id_ed25519

Use keychains or tools like gnome-keyring or macOS Keychain to persist across reboots securely.

SSH in DevOps, CI/CD, and Cloud

Infrastructure as Code Integration

Ansible, Terraform, and Jenkins rely heavily on SSH for remote provisioning.

Use dedicated deploy users
Rotate keys periodically
Use SSH config for multiple environments (dev/stage/prod)

Ephemeral Keys in Cloud

Modern cloud providers (AWS, GCP, Azure) allow:

Dynamic key injection on VM boot
SSM or Identity-based access (bypassing SSH keys entirely)
Role-based access via SSH Certificate Authorities (SSH CA)

Advanced Method: Issue signed certificates using custom CA:

ssh-keygen -s ca_key -I user_identity -n username user_key.pub

Troubleshooting SSH Issues

Cannot Connect to Custom Port

Ensure the port is open in firewall (ufw, iptables)
Check with:
```
sudo ss -tuln | grep [port]
```

Host Key Verification Failed

SSH stores server fingerprints in ~/.ssh/known_hosts
If server’s identity changed:
```
ssh-keygen -R [hostname]
```

Permission Denied

Check insecure permissions on .ssh files
Confirm user entry is in authorized_keys
Run with ssh -vvv to see detailed debug logs

Advanced Strategies for the Real-World

Best Practices Checklist

Disable root login
Enforce public key authentication
Use strong key types (ED25519)
Set MaxAuthTries and LoginGraceTime sensibly
Disable unused modules like X11, TCP forwarding
Configure multi-user control with AllowUsers or AllowGroups
Regularly rotate keys
Monitor with tools like fail2ban, auditd, or Wazuh

Pro Strategies

Use isolated users for each system/component
Enable 2FA via google-authenticator PAM module
Maintain SSH access logs with auditd
Replace keys with SSH CA as your org scales
Implement Bastion Hosts to gate SSH traffic and record sessions

Resources & Further Reading

Next Steps

Audit your current SSH server config
Replace password-based logins with keys
Set up your own SSH CA for your team

Conclusion

Let’s review the most important takeaways:

SSH configuration is not optional – it determines how safe and scalable your access is
Use key-based authentication, disable unnecessary options, and enforce tight permissions
Master both sshd_config and ~/.ssh/config for full control
Leverage tunneling, agent forwarding, and multiplexing for powerful workflows
Integrate with DevOps, CI/CD, and cloud-native tools
Move toward modern practices like SSH CA and ephemeral access

By implementing secure and efficient SSH configurations, you equip yourself with the tools to manage infrastructure confidently, efficiently, and securely.