How Should I Set Up My Linux Web Server?

That’s really a much broader question than I can answer here, however, my recent experiences both being hacked, and building out a new server to replace the hacked one, are worth recording. Not only might you find it a helpful laundry list of ideas for securing your server, but you might also have some ideas and experiences to share.

So, these are the steps I’ve taken to “harden” or more tightly secure a newly built out Linux server. These steps are intended to reduce the probability the server being hacked into again.

The New Server Itself

In this particular case our brand new server is built out with Redhat Enterprise Linux, with the cPanel management interface. A 2.0Ghz Xeon, with 512Meg RAM, and a 54gig hard drive.

The goal at this step is actually fairly simple: start out with a popular, current, and supported, version of the OS.

WHM

Step one was to login to Web Host Manager (the server management front-end for cPanel), and run their configuration wizard. The only things that required non-default action were:

• setting the hostname
• added a resolver (DNS) server ip

After the wizard was complete, these additional settings were performed manually:

• Server Contact E-Mail
• Turned on awstats web stats reporting tool
• Changed hostname. (Even though the wizard set it, this was an additional, different place.)
• Changed system mail preferences to send root email to an admin alias.
• Changed root password.
• Set Nameservers

Also ran Update Server Software and Update System Software within WHM to update as many components as reasonable.

Linux Kernel

The original configuration as delivered by the server farm:

WHM 10.1.0 cPanel 10.2.0-R82
RedHat Enterprise 3 i686 – WHM X v3.1.0
Linux kernel version: 2.4.21-27.ELsmp

Since that’s not the latest kernel at the time of delivery, and cPanel will not update the kernel automatically, I was advised to update. I upgraded the kernel to 2.4.21-32.0.1.ELsmp, which went smoothly.

Remote Access

One of the areas for almost immediate security tweaking is remote access. If you’ve ever examined a server’s logs, you know that they’re regularly probed with various types of attacks. There are several modifications to render those attacks pointless, as well as secure remote access in general.

SSH

SSH, or secure shell, is the remote command-line access mechanism of choice. Its single biggest claim to fame is simply that everything is encrypted, though it includes many more features as well.

I first created a couple of non-root login accounts, and installed RSA public keys for eventual RSA/public key-only logins. (Also created end-user instructions for creating, and using, SSH key pairs.)

I then made the following changes to /etc/ssh/sshd_config:

• Protocol 2 – this disables protocol version 1, which is known to have security issues.
• PermitRootLogin without-password – this disables root login, unless a public key is used.
• PasswordAuthentication set to no – this disables the ability to login with just a username and password. A username and public key are required.

Telnet

I disabled telnet, as it allows passwords to be transmitted in the clear, and is a known security risk.

FTP

Normally at this point I would also disable the FTP service for the same reasons
as Telnet – it’s a known security risk. For reasons I didn’t bother to investigate further, cPanel either complained repeatedly via email if the service was stopped, or it would restart it, depending on the technique I was attempting to use. I gave up on this and left the FTP service running, but blocked at the firewall, discussed below.

TMP

The /tmp directory is an interesting and apparently common place for hackers to exploit to breech a system. One approach to securing it without breaking its functionality is to create it as a new partition that disallows program execution. Where in the past a hacker could use an exploit to place a file in /tmp and then execute it, the ability to execute is now removed by creating a secure /tmp partition.

Firewall

Installed APF which is, essentially, an intelligent front end to the built in Linux firewall, IP tables. I found a good how-to, though I did not yet configure the anti-dos (denial of service) component. The one change that I did make to the default configuration is that I disabled or blocked all ports that we know are not explicitly used for something specific. For example, this is where I blocked all FTP access. I also whitelisted my own static IP address, and that of the client.

Logging

Logwatch is a server log summary tool that emails a report to the server admins nightly. An older version was installed. I updated to latest, and increased the reporting level to “medium”.

Rootkit Checks

Installed nightly backup script, which includes nightly reporting of both rkhunter and chkrootkit runs.

Apache

Installed mod_security, which guards against exploits via the Apache web server, using a basic ruleset. This how to was helpful in setting that up.

General

A few additional tweaks and settings …

• Installed LES – “LES is intended as a facility to quickly & easily secure Redhat/RPM based environments (i.e: turbo Linux, open Linux). It does such by enforcing root-only permissions on system binaries (binaries that have no place being executed by normal users), enforcing root-only path traversal on system paths, enforcing immutable bit on essential rpm package contents (i.e: coreutils), and enforcing immutable bit on shell profile scripts.”
• Added a mail line to /etc/rc.d/rc.local, to automatically notify on reboots.
• Removed the cpbackup cron job, since we don’t use cPanel’s backup approach.
• Modified reverse-DNS to be correct.

---

Ongoing Maintenance

Even with all the tools and updates in place, there’s still on-going work and vigilance required.

• Monitor the nightly logs and reports generated for anomalies.
• Manually install updates as reported available by up2date.
• Monitor Redhat for kernel updates, and install as appropriate.
• Monitor chkrootkit for updates, and install as appropriate.

---

Still More Options

I haven’t implemented these, but they’ve been recommended, and they’re on the list to evaluate at some point.

• mod_doevasive.
  “mod_dosevasive is an evasive maneuvers module for Apache to provide evasive action in the event of an HTTP DoS or DDoS attack or brute force attack. It is also designed to be a detection and network management tool, and can be easily configured to talk to ipchains, firewalls, routers, etcetera. mod dosevasive presently reports abuses via email and syslog facilities.”
• How To Evaluate BFD.
  “BFD is a modular shell script for parsing applicable logs and checking for authentication failures.”