Linux Server Setup – Part 1

If you are new to Linux, setting up a Linux server is a great way to learn more about Linux and server management. Whether you want to host services, run apps or build a homelab, a Linux server offers unmatched flexibility, performance, and control.

This beginner’s guide will walk you through the basics of setting up an efficient, stable and secure Linux server, so you can move on to more advanced stuff. This guide also includes many links to resources that I’ve found useful or written about before.

Key Takeaways

• Distro Matters: Select a distro that suits your experience level and project goals.
• Secure from the Start: SSH key authentication, firewalls, and tools like Fail2Ban protect against unauthorized access.
• System Stability: Monitoring services (htop, glances) and log analysis (journalctl, /var/log files) help you spot issues early.
• Automate Wisely: Scheduled updates and backups save time—but test them in a safe environment first.
• Practice & Explore: Hands-on tinkering with containers, web servers, or media services is the best way to learn.
• Also see Part 2.

Why Build a Linux Server?

Screenshot displaying ufw, fail2ban, and /var/log/syslog, as discussed in the article.

Unlike preconfigured hosting options, building and managing your own Linux server allows you to see under the hood of the technologies that power modern applications and services. So it’s a great choice if you want to learn more or establish a foundation for future Linux-powered projects.

By building your own Linux server, you can:

• Learn networking, system administration and security practices.
• Run custom applications or services for your needs.
• Control your data and keep it private.
• Avoid vendor lock-in and proprietary limitations of preconfigured solutions.

Also read: Mastering Linux Administration: 20 Powerful Commands to Know.

Whether you want to run a homelab, host your own website or explore advanced IT stuff, a Linux server is a tool that can grow with you as you learn and grow.

Choosing the Right Linux Distribution

Choosing the right Linux distro is one of the most important decisions when setting up your server. The Linux distribution you choose will influence the tools you have available, the level of support you can expect, and the overall management experience you’ll have throughout the life of your server.

Some of the most popular options include:

• Ubuntu Server: Popular, easy-to-use and large community. If you’re new to Linux, Ubuntu Server is a great place to start!
• Debian: Has a huge software repository, making it a great choice if you want a rock-solid server to build on.
• OpenSUSE: With strong tools like YaST, OpenSUSE is a versatile option for managing server configurations.
• Fedora: Based on Red Hat Enterprise Linux (RHEL), Fedora is popular for those who like to stay on the bleeding-edge of Linux.
• AlmaLinux and Rocky Linux: These distributions also offer compatibility with RHEL, making them great for enterprise.

Each Linux distribution has its unique strengths, so consider your needs and goals before making a selection.

Also, check out my guide on The Best Linux Server Distros, consider these 5 ‘other’ Independent Linux Distros You Should Try and also What CentOS alternative distro should you choose?

Choosing Your Hardware or Virtual Machine (VM)

Homelab Linux server rack with monitoring display. Learn how to build your own.

Choosing the right hardware or virtual machine (VM) for your Linux server is a critical step in setting up a Linux server that fits your needs. The first question to ask yourself is: Do I want my server to be self-hosted or hosted by a cloud hosting provider?

Hardware

For a self-hosting (or homelab), even old hardware can be sufficient to get you started and is very cost-effective. However, for production environments and hosting resource intensive applications, cloud infrastructure is usually the way to go.

Also read: Learn Linux: 4 Devices to boost your Linux skills.

From a processor perspective, consumer-grade hardware like Intel Core i5/i7 or AMD Ryzen 5/7 is more than sufficient for self-hosted setups. However, for production workloads, server-grade processors like Intel Xeon or AMD EPYC are the way to go.

Also, read: PHP Performance: Additional CPU cores vs Faster CPU cores.

RAM is another important aspect. As little as 1 GB is typically more than enough for very basic setups, but for databases or container orchestration you will need more. More is better when it comes to RAM.

SSDs (Solid State Drives) are a must for faster read/write performance. Especially if you plan to host databases or serve tons of files. You may also benefit from RAID arrays. RAID stands for Redundant Array of Independent Disks, a technique used to combine multiple drives for improved redundancy or performance.

Also read: What is iowait and how does it affect Linux performance?

Your network connection is also important. Ethernet is much preferred over Wi-Fi for consistency and performance reasons. Gigabit Ethernet is now the norm for most setups. If you plan on hosting your server in the cloud, providers like DigitalOcean or StackLinux give you a high-performance network connection to your box since they’re backed by robust infrastructure.

Also read: Guide to Network Troubleshooting in Linux.

Virtual Machine (VM)

If you’d rather run a Virtual Machine (VM) on a host machine, your host machine will need to have enough resources to run the VM. A hypervisor like Proxmox, VirtualBox, VMware, or KVM allows you to easily create and manage virtual servers. Similar to the above “Hardware” advice, be sure to allocate enough CPU cores, RAM, and disk space, depending on the workload.

Also, read: Linux server needs a RAM upgrade? Check with top, free, vmstat, sar.

Popular cloud providers offer Linux VMs on solid AMD or Intel hardware. If you’d rather not mess around with hardware, this is a simple and cost-effective approach.

VMs are also great because you can take seamless snapshots of them. This is extremely helpful if you’re testing configurations or updating software. I usually take snapshots of my VMs nightly, others I have set up with hourly CDP. If I mess up a configuration or screw up some software, I can simply revert to a good snapshot or roll back and not waste a lot of time recovering.

By taking the time to choose your hardware or VM setup carefully, you’ll be well on your way to a solid Linux server that fits your needs. If you’d rather not mess with configuring Linux VMs, services like StackLinux.com offer preconfigured and custom-tuned VMs that can save you tons of time and hassle.

Installing Linux and Initial Setup

Ubuntu Server installation menu.

The first step is to install Linux. Whether you’re using hardware or a self-hosted virtual machine, the process very similar. We will focus on installing Linux on self-hosted hardware.

Note: For a Linux VM from a cloud hosting provider, these 3 steps are unnecessary because the installation is typically completed with a few clicks on the provider’s website.

1. Download the ISO: Go to the website of your chosen Linux distribution and download the latest ISO. For example, if you’re installing Ubuntu Server go to ubuntu.com/server.
2. Create Bootable Media: Use tools like Rufus (Windows), BalenaEtcher (Linux/Windows/Mac) or dd (Linux command-line) to create a bootable USB or virtual CD-ROM.
3. Boot and Install: Put the bootable media in your machine and boot the system. Follow the installer’s prompts to set up partitions, select software and configure the base system. Most distributions have a guided installation for beginners.

Once Linux is installed, you’ll need to configure the system for your specific server use.

Setting a Static IP

The static IP is usually set during the installation wizard/process. A static IP ensures your server always has the same IP, making it easier to access and manage. This is especially important for servers hosting services, as dynamic IPs can cause problems when they change.

Here’s another way to manually set a static IP on Ubuntu server using Netplan (a Ubuntu utility that simplifies network configuration using YAML files, making it easier to manage IP addresses and routes):

1. Find your network interface with ip a or ifconfig.
2. dit the network configuration file. The location of the file is typically in the /etc/netplan/ directory. The specific file name often ends with .yaml (e.g., 01-netcfg.yaml or 50-cloud-init.yaml).
   
   Example File Path:

   /etc/netplan/01-netcfg.yaml
   

   You can list all Netplan configuration files in the directory to confirm:

   ls /etc/netplan/
   

   Edit the file with a text editor like nano or vim:

   sudo vi /etc/netplan/01-netcfg.yaml
   

   Example file changes:

   network:
     version: 2
     renderer: networkd
     ethernets:
       eth0:
         addresses:
           - 192.168.1.100/24
         gateway4: 192.168.1.1
         nameservers:
           addresses:
             - 8.8.8.8
             - 8.8.4.4
   

   Save and apply the changes with sudo netplan apply.

3. Restart your network service to ensure the new configuration takes effect.

Static IPs can also be configured via your router’s settings.

Creating a New User with Administrator Privileges

Fun illustration highlighting sudo privileges. (Wallpaper)

Working as root is discouraged as it gives full access to the system and can lead to accidental or malicious damage. Instead, create a new user with admin (sudo) privileges:

1. Add a new user:

   adduser yourusername

2. Grant administrative privileges:

   usermod -aG sudo yourusername

3. Test the new user:
   – Log out of the root account and log in with the new username.
   – Use the sudo command to ensure administrative access works:

   sudo apt update && sudo apt upgrade

Now your server will have a stable identity on the network and a secure user setup to manage it.

Fail2Ban and Similar Tools

Fail2Ban is a security tool that helps protect your server from brute-force and other attacks by monitoring log files and banning IP addresses that exhibit malicious behavior.

Installing and Configuring Fail2Ban

1. Install Fail2Ban:

   sudo apt install fail2ban  # For Debian/Ubuntu
   sudo dnf install fail2ban  # For Fedora/AlmaLinux

2. Create a local configuration file:

   sudo cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local

3. Enable SSH protection: Edit the jail.local file and ensure the [sshd] section is enabled:

   [sshd]
   enabled = true
   port = ssh
   logpath = /var/log/auth.log  # Adjust for your distribution
   maxretry = 5

4. Restart Fail2Ban:

   sudo systemctl restart fail2ban

5. Check active bans:

   sudo fail2ban-client status sshd

Other tools like DenyHosts and CSF (ConfigServer Security & Firewall) are also useful for server security and can work alongside or instead of Fail2Ban depending on your needs.
With this in place,  your server will be significantly more secure against unauthorized access and malicious activity.

Automating Updates

Automation can save time and ensure your server remains up to date and secure; however, be cautious, as unattended updates can occasionally introduce compatibility issues or break services. So is usually not recommended in production environments. Always test updates in a staging environment if possible.

• Unattended Upgrades: Automatically install security updates on Debian/Ubuntu systems.

  sudo apt install unattended-upgrades
  sudo dpkg-reconfigure unattended-upgrades

• Cron Jobs: Schedule tasks such as cleaning up temporary files or running system updates.Edit your crontab:

  sudo crontab -e

  Add a job to update and upgrade the system every day:

  0 3 * * * apt update && apt upgrade -y

By leveraging these tools and practices, you can maintain optimal server performance with minimal manual intervention.

Also, read: How to Enable Unattended Upgrades on Ubuntu/Debian, Enable Automatic Updates – Fedora/Red Hat, and Linux Updates: Command Line Guide.

Key Concepts for Linux Server Setup

When setting up a Linux server, you need to understand a few basic concepts that will help you manage and secure your system. The building blocks of server administration, for secure connections, stable operation, and successful troubleshooting.

From setting up secure access protocols, to firewalls and system performance monitoring, understanding these will let you unlock your server’s full potential. Let’s take a look.

SSH (Secure Shell)

SSH is an important protocol used for securely accessing and managing your server. By default, it uses encryption so the data between your device and the server is secure. This makes SSH an essential tool for remote Linux server administration.

Also, read: SSH command in Linux, with examples.

You can use SSH to log in to your server remotely and execute commands in a terminal emulator, restart services, transfer files with SFTP (Secure File Transfer Protocol), and monitor your server’s health. It is an indispensable tool for remote Linux server management.

Also, read: SSH Security: Protecting Your Linux Server from Threats.

Most Linux distributions include OpenSSH, making it easy to get started. To increase security, we recommend setting up SSH key authentication instead of password-based login. Not only is SSH key authentication more secure against brute-force attacks, but it’s also more convenient, and less prone to authentication errors.

Configuring SSH Key Authentication

Here’s how to set it up:

1. Generate an SSH key pair on your local machine:

   ssh-keygen -t rsa -b 4096 -C "your_email@example.com"

   Save the key in the default location (e.g., ~/.ssh/id_rsa) and set a passphrase for added security.

2. Copy the public key to your server: Use the ssh-copy-id command to add your public key to the server:

   ssh-copy-id username@your_server_ip

   Alternatively, you can manually copy the contents of your ~/.ssh/id_rsa.pub file to the server’s ~/.ssh/authorized_keys file.

3. Test the setup: log in again using your private key:

   ssh username@your_server_ip

4. Disable password-based authentication: Edit the SSH configuration file on your server:

   sudo nano /etc/ssh/sshd_config

   Set the following options:

   PasswordAuthentication no
   PermitRootLogin no

   Restart the SSH service:

   sudo systemctl restart sshd

Firewalls

A firewall is a security feature that controls incoming and outgoing network traffic to your server. It acts as a barrier, preventing unauthorized access while allowing legitimate traffic to pass through.

Firewalls work by managing “ports,” which are like doors that allow data to flow to specific services on your server. By closing unnecessary ports, you reduce the risk of unauthorized access.

Also, read: How to Secure Your Linux Server from Emerging Cyber Attacks.

Configuring a firewall, such as UFW (Uncomplicated Firewall), firewalld, or Nftables is one of the first steps in securing your server. UFW often includes profiles for services like SSH, Nginx and others, making it easy to allow necessary traffic while blocking others. Reviewing and updating your firewall rules periodically ensures your server remains secure as you add or remove services.

Setting Up a Basic Firewall

The most common tools for managing firewalls on Linux are UFW (Uncomplicated Firewall) and firewalld. Here are two quick examples:

Using UFW (Ubuntu):

1. Allow SSH traffic:

   sudo ufw allow SSH

2. Enable the firewall:

   sudo ufw enable

3. Check the status:

   sudo ufw status

For more commands and details, read: UFW – Uncomplicated Firewall.

Using firewalld (Fedora/AlmaLinux):

1. Start and enable firewalld:

   sudo systemctl start firewalld
   sudo systemctl enable firewalld

2. Allow SSH traffic:

   sudo firewall-cmd --permanent --add-service=ssh
   sudo firewall-cmd --reload

3. Verify the rules:

   sudo firewall-cmd --list-all

Also, read: iptables: The two variants and their relationship with nftables and Firewalld: A Beginner’s Guide.

System Services

A Linux server is a blank canvas for innovation and learning. Services are background programs running on your Linux server, such as web servers (e.g., Apache, Nginx), databases (e.g., MySQL, MongoDB), file-sharing systems (e.g., Samba, Nextcloud), scripting platforms (e.g., PHP, Python), containerization platforms (e.g., Docker, Podman), mail servers (e.g., Postfix, Exim), and many others.

They are essential to delivering functionality. Properly managing these services ensures that your server performs efficiently and remains secure.

Running a Media Server

Turn your server into a multimedia hub by setting up a media server. Tools like Plex, Emby or Jellyfin can stream movies, music and photos to devices across your network.

• Plex: Offers a user-friendly interface and remote streaming capabilities.
• Jellyfin: A free and open-source alternative with no subscription required.
• Emby: Intuitive media organization for seamless access across devices.
• Universal Media Server: A free, versatile media server supporting many devices.
• Serviio: Stream media to TVs, Blu-ray players, and consoles with ease.

Host a Website or Blog

With a Linux server, you can host your own website or blog using web servers like Apache or Nginx. Combine them with tools like WordPress or Hugo for easy content management.

• LAMP/LEMP Stack: Set up a Linux, Apache/Nginx, MySQL, and PHP stack for dynamic websites.
• CMS: Content Management Systems like WordPress, Magento, Joomla, or Discourse for websites and e-commerce.
• Hugo: A static site generator for creating fast, secure, and simple websites.
• Ghost: A modern platform for professional blogging, optimized for speed and flexibility.
• Security: Use SSL certificates with tools like Let’s Encrypt to secure your site.

Build Your Own Cloud or NAS

Turn your server into a network-attached storage (NAS) device or personal cloud with tools like Nextcloud, ownCloud, or alternatives like Seafile and Syncthing.

• Nextcloud: A popular option for hosting personal or shared storage with additional features like calendars and document editing.
• ownCloud: Similar to Nextcloud, offering reliable file synchronization and sharing with enterprise-focused extensions.
• Seafile: Known for its high-performance and efficient data synchronization.
• Syncthing: A fully open-source and decentralized file synchronization solution.
• RAID: For local NAS setups, consider using RAID for data redundancy.

Also read: Linux Server DIY Projects for Beginners and Discover LLMs and Chatbots for Linux.

Conclusion

Building and managing your first Linux server provides unlimited opportunities to learn and experiment. With a solid distro choice, proper hardware or VM resources, secure configuration, and reliable monitoring tools, you’ve laid a strong foundation.

Now that you’ve set up a solid foundation for your Linux server, you’re ready to dive into Part 2 of this series to explore monitoring, logging, and additional tools that will help you get even more out of your server.