Server Hardening: SSH, Firewalls, and Fail2ban
Keeping my servers secure is always a priority, and a good starting point is tightening up SSH, configuring a basic firewall, and using Fail2ban to block brute-force attempts. It's not about perfect security, but about making things harder for attackers.
I spend a lot of time making sure my servers, and those of my clients, are reasonably secure. It’s a constant battle, and I don’t aim for absolute invulnerability. My goal is to make it significantly harder for attackers to get in, and to be alerted if someone does try. This post walks through a few basic steps I take for server hardening, focusing on SSH, firewalls, and Fail2ban. It’s a good starting point for anyone running Linux servers, especially those of us in Bangladesh who might be dealing with a higher volume of automated attacks.
SSH Hardening
SSH is a common target, so securing it is a must. I prefer disabling password authentication entirely and relying on SSH keys. It’s more secure and faster, too.
First, generate an SSH key pair on your local machine if you don’t already have one:
ssh-keygen -t rsa -b 4096
Then, copy your public key to the server. There are many ways to do this, ssh-copy-id is the easiest, but if you don’t have it:
cat ~/.ssh/id_rsa.pub | ssh user@server 'mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys'
Next, edit the SSH configuration file. The location varies by distribution, but it’s usually /etc/ssh/sshd_config.
nano /etc/ssh/sshd_config
Make these changes (or similar, depending on your needs):
PasswordAuthentication no
PubkeyAuthentication yes
ChallengeResponseAuthentication no
UsePAM no
After making these changes, reload the SSH service:
systemctl reload sshd # Ubuntu/Debian
# systemctl reload sshd.service # AlmaLinux/CentOS/RHEL
If you’re using a non-standard SSH port (which I recommend), make sure to change the Port directive in sshd_config.
Firewall Setup with UFW
UFW (Uncomplicated Firewall) is a user-friendly front-end for iptables. It’s easy to set up and manage. I find it much simpler than directly configuring iptables.
First, enable UFW:
uFW enable
By default, UFW denies all incoming connections and allows all outgoing connections. Let’s allow SSH (assuming you’re still using the default port 22, or your custom port):
uFW allow 22/tcp
If you're running a web server (like Nginx or Apache), you’ll also want to allow HTTP (port 80) and HTTPS (port 443):
uFW allow 80/tcp
uFW allow 443/tcp
Check the status of UFW:
uFW status
Fail2ban
Fail2ban monitors log files for suspicious activity and automatically blocks offending IP addresses. It's great for mitigating brute-force attacks against SSH, web servers, and other services.
Install Fail2ban:
sudo apt update # Ubuntu/Debian
sudo apt install fail2ban
# sudo dnf install fail2ban # AlmaLinux/CentOS/RHEL
Fail2ban uses “jails” to define which log files to monitor and what actions to take when suspicious activity is detected. The default configuration is usually sufficient for basic protection. You can customize jails in /etc/fail2ban/jail.local.
Start and enable Fail2ban:
systemctl start fail2ban
systemctl enable fail2ban
Check Fail2ban status:
systemctl status fail2ban
My setup
On my servers, I’m running Ubuntu 24.04 and AlmaLinux 9. I always disable password authentication for SSH and use key-based authentication. I also change the default SSH port to something random, between 1024 and 65535, to reduce the noise from automated bots. I don't bother with complex firewall rules on small VPSes; UFW's default settings are usually enough. I also find the default Fail2ban configuration works well for most services. Remember to regularly review your Fail2ban logs (/var/log/fail2ban/fail2ban.log) to ensure it's working correctly and not blocking legitimate traffic.
One gotcha I’ve run into before is forgetting to reload services after making changes to configuration files. Always double-check that your changes have taken effect. Another common issue is SELinux contexts, particularly when dealing with custom applications. Make sure your files have the correct SELinux contexts to avoid permission problems. Finally, always verify your DNS TTL (Time To Live) is reasonably low (e.g., 300 seconds) so that changes propagate quickly if you need to block an IP address.
I also regularly check my server logs for any unusual activity. Early detection is key to preventing a full-blown security breach. I frequently link back to Linux Server Administration Checklist for Ubuntu, Debian, and AlmaLinux for more thorough setup steps.
That's a basic overview of how I approach server hardening. It's an ongoing process, and you should always adapt your security measures to your specific needs and environment. Keeping your systems updated and staying informed about new security threats is important. Next up: automating some of these tasks with scripting.
Related reading
Frequently asked questions
How often should I review my server hardening configuration?
At least quarterly, or whenever you update your server software. New vulnerabilities emerge regularly, so staying on top of things is important.
Is Fail2ban a silver bullet for server security?
No, it’s a good layer of defense against brute-force attacks, but it’s not a replacement for other security best practices like keeping software updated and using strong passwords.
What's the difference between a firewall and Fail2ban?
A firewall controls network traffic in and out of your server, while Fail2ban monitors log files for suspicious activity and blocks offending IP addresses.
