How to Set Up SSH Access for Secure Server Management

How to Set Up SSH Access for Secure Server Management

Introduction

When it comes to managing your server remotely, SSH (Secure Shell) is the gold standard. It provides an encrypted, secure way to access and control Linux servers—allowing you to perform everything from software installation to file transfers and firewall configuration.

But SSH isn’t just about convenience—it’s about security. With cyber threats on the rise, properly configuring SSH access is critical for protecting your server and data.

In this post, you’ll learn:

• What SSH is and how it works
• How to connect to your server via SSH
• How to set up SSH key authentication
• How to harden your SSH security
• Best practices to keep your server safe

Whether you’re using a Vicservers VPS or a dedicated machine, this step-by-step guide will get you running securely in no time.

---

What Is SSH?

SSH (Secure Shell) is a cryptographic network protocol that allows secure remote login and command execution on a server over an unsecured network. It replaces older, insecure protocols like Telnet and FTP.

SSH uses port 22 by default and encrypts the communication between your local machine and your remote server.

With SSH, you can:

• Access your server’s command line
• Transfer files securely using SCP or SFTP
• Automate server management tasks
• Configure firewalls, install packages, restart services

---

Prerequisites

To follow this guide, you’ll need:

✅ A Linux server (Ubuntu/Debian/CentOS) — e.g. from VicServers
✅ A local computer with SSH installed (macOS/Linux: built-in, Windows: use PowerShell or PuTTY)
✅ Server login credentials (IP address, username, and password or SSH key)

 Step 1: Connecting to Your Server Using SSH

The most basic way to connect to your server:

ssh username@your-server-ip

Example:

ssh root@192.168.1.100

The server will ask for the user’s password. Once entered, you’re inside!

Step 2: Setting Up SSH Key Authentication (Recommended)

SSH key authentication is far more secure than using passwords. Here’s how to set it up:

1. Generate SSH Key Pair (on your local machine)

ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

Press Enter to accept the default file location (~/.ssh/id_rsa) and optionally set a passphrase.

This generates:

• id_rsa — your private key (keep safe!)
• id_rsa.pub — your public key

2. Copy the Public Key to the Server

ssh-copy-id username@your-server-ip

Or manually:

cat ~/.ssh/id_rsa.pub | ssh username@your-server-ip 'mkdir -p ~/.ssh && cat >> ~/.ssh/authorized_keys'

3. Connect Using the SSH Key

Now you can log in without a password:

ssh username@your-server-ip

---

 Step 3: Hardening SSH Security

Once SSH key access is working, tighten security to reduce risk.

1. Disable Password Authentication

Edit your SSH configuration file:

sudo nano /etc/ssh/sshd_config

Find these lines and update:

PasswordAuthentication no
PermitRootLogin no

This:

• Disables password-based login (use keys only)
• Prevents root login (use sudo instead)

Then restart SSH:

sudo systemctl restart ssh

2. Change Default SSH Port

Using a non-default port adds a layer of protection against bots.

In /etc/ssh/sshd_config, change:

Port 2222

Restart SSH:

sudo systemctl restart ssh

And allow the new port through the firewall:

sudo ufw allow 2222/tcp

---

Step 4: Using SSH Config for Easier Management

If you manage multiple servers, create a config file at ~/.ssh/config:

Host vicserver
    HostName 192.168.1.100
    User youruser
    Port 2222
    IdentityFile ~/.ssh/id_rsa

Now you can connect with:

ssh vicserver

---

 Step 5: File Transfers Using SCP or SFTP

SSH also lets you transfer files safely.

Using SCP:

Upload a file:

scp file.txt user@your-server-ip:/home/user/

Download a file:

scp user@your-server-ip:/home/user/file.txt .

Using SFTP:

sftp user@your-server-ip

This opens a secure FTP-like session over SSH.

---

Step 6: Managing SSH Access for Multiple Users

To add a new user:

sudo adduser newuser

Give them SSH access:

sudo mkdir /home/newuser/.ssh
sudo cp ~/.ssh/authorized_keys /home/newuser/.ssh/
sudo chown -R newuser:newuser /home/newuser/.ssh

Restrict sudo access if necessary with:

sudo usermod -aG sudo newuser

---

 Step 7: Enable Two-Factor Authentication (Optional)

For added security, enable 2FA on SSH.

1. Install Google Authenticator:

sudo apt install libpam-google-authenticator

2. Run setup:

google-authenticator

3. Edit PAM:

sudo nano /etc/pam.d/sshd

Add this line at the top:

auth required pam_google_authenticator.so

4. Update SSH config:

sudo nano /etc/ssh/sshd_config

Set:

ChallengeResponseAuthentication yes

Restart SSH.

Now users need their SSH key and 2FA code.

---

Step 8: Enforcing Security Best Practices

Use Strong Keys

• Use RSA 4096-bit or better
• Or switch to ed25519 for faster, modern encryption:

ssh-keygen -t ed25519

 Rotate Keys Regularly

Change or revoke keys for users who no longer need access.

 Limit Login Attempts

Use tools like Fail2Ban to block repeated login failures:

sudo apt install fail2ban

Configure /etc/fail2ban/jail.local to protect SSH.

 Monitor SSH Logs

Check login attempts:

sudo journalctl -u ssh
sudo cat /var/log/auth.log | grep sshd

---

Troubleshooting SSH Issues

• Connection refused?
  → Make sure the SSH service is running:
  sudo systemctl status ssh
• Permission denied (publickey)?
  → Check file permissions:
  ~/.ssh/authorized_keys must be 600
~/.ssh/ directory must be 700
• Lost your private key?
  → You’ll need console access or another user account to restore access.

---

 Using Vicservers? You’re Already Ahead

Vicservers makes SSH setup fast and secure by default. With full root access and instant provisioning, every VPS or dedicated server includes:

✅ Preinstalled OpenSSH
✅ Full SSH key support
✅ Firewall controls for SSH ports
✅ 24/7 assistance for key setup or security hardening

Need help setting up your first SSH session? Our support team can walk you through it.

---

 Final Thoughts

SSH is a fundamental tool for server management—but only when used securely. By setting up SSH keys, disabling password login, and limiting access, you protect your server from the most common attacks.

When paired with best practices like rotating keys and monitoring logs, SSH becomes your secure gateway to full server control.

---

Ready to Deploy Secure Servers?

Start your secure VPS with Vicservers
SSH, firewalls, backups, and more — all included
Need help? Our engineers are on call 24/7

 

Vicservers — Secure. Scalable. Ready for anything.