Linux Performance: Almost Always Add Swap Space

We know that using Linux swap space instead of RAM (memory) can severely slow down performance. So, one might ask, since I have more than enough memory available, wouldn’t it be better to delete swap space? The answer is: No. There are performance benefits when swap space is enabled, even when you have more than enough RAM. Update: also see Part 2: Linux Performance: Almost Always Add Swap (ZRAM).

Even with more than adequate server memory installed, you will often find that swap space will be used after long periods of uptime. See the below example from a live chat server with around one month of uptime:

              total        used        free      shared  buff/cache   available
Mem:           3.7G        1.0G        445M         84M        2.2G        2.2G
Swap:          1.8G        308M        1.5G

The output of free -h here shows 308M of swap space used. When I ran checks for swapping, there were no signs of ongoing or untimely swap I/O activity. Also, the kswap service didn’t consume much CPU time. In fact, the kswap process was nowhere to be found in top (top processes sorted by CPU time). To confirm, I used the following command:

ps -A | grep kswap
 40 ? 00:00:29 kswapd0

…so in this case, as in many, swap usage is not hurting Linux server performance. Now, let’s look at how swap space can actually help Linux server performance.

Update: Recently, I installed Manjoro i3 on my Pinebook Pro. It now comes with ZRAM enabled by default.

Advantages of swap space on systems with adequate RAM

It is normal and can be a good thing for Linux systems to use some swap, even if there is still available RAM. The Linux Kernel will move memory pages that are hardly ever used into swap space to ensure that even more cachable space is made available in-memory for more frequently used memory pages (a page is a piece of memory). Swap usage becomes a performance problem when the Kernel is pressured to continuously move memory pages in and out of memory and swap space.

Another advantage is that swap gives admins time to react to low memory issues. We will often notice the server acting slowly and, upon login, will notice heavy swapping. Without swap (as described in the next section), running out of memory can create much more sudden and severe chain reactions. So, usually, I would advise setting swap space to about the size of your largest process. For example, MySQL’s configured memory in my.cnf. It can even be smaller, especially if you have monitoring and/or alerting in place.

Some recommend no swap or swap size slightly larger than the total RAM. If you can come up with valid reasons for this, then that may be your choice. However, this is hardly the case on servers, and you should instead balance your decision with the effects swap will have on your specific applications. Swap does not change the amount of RAM required for a healthy server, or desktop for that matter. It’s designed to be complementary to the performance of healthy systems.

To summarize:
— Even if there is still available RAM, the Linux Kernel will move memory pages that are hardly ever used into swap space.
— It’s better to swap out memory pages that have been inactive for a while, keeping often-used data in cache, and this should happen when the server is most idle, which is the aim of the Kernel.
— Avoid setting your swap space too large if it will result in prolonging performance issues, outages, or your response time (without proper monitoring/alerts).

Swap Space vs. No Swap when available memory is low

Unlike the case above, if you don’t have enough memory, swap will be used quite often and noticeably more during any memory requirement spikes. If you don’t have enough memory and no swap space, this will often cause failure to allocate memory for requests needing more memory pages. As a last resort, the Kernel will deploy OOM killer to nuke high-memory processes (usually MySQL, java, etc.).

For a more detailed look at Linux swap space, read the Swap Management and Page Frame Reclamation chapters from Kernel.org docs. Also, look at the last section, “Kernel cache pressure and swappiness,” of my other blog post for tips on tuning Linux swap space usage by the Kernel. If your swap space “used” is always ‘0’ then you do indeed have a ton of freely available RAM, in which case it may be safe to remove the swap space… or you can adjust your Kernel’s cache pressure to make use of even more RAM.

To summarize:
— Swap I/O scales very poorly. If memory pages cannot be swapped only when the server is idle, you should tune or disable swap. This is usually not the case, thus the “almost always” title of this blog post.
— With swap disabled, performance issues become noticeable very fast, and the OOM killer may get you! :)

For comparison, here’s the output of free using an older version of free from procps-ng-3.3.1 on the same server:

             total       used       free     shared    buffers     cached
Mem:          3.7G       3.3G       445M         0B       4.2M       1.7G
-/+ buffers/cache:       1.6G       2.1G
Swap:         1.8G       308M       1.5G

Kernel cache pressure and swappiness

Now that you have swap enabled. Consider adjusting your server’s cache pressure and tendency to swap (vm.swappiness) by following the guide below, which is from the previous article: Linux server needs a RAM upgrade? Check with top, free, vmstat, and sar:

vfs_cache_pressure – Controls the kernel’s tendency to reclaim the memory, which is used for caching of directory and inode objects. (default = 100, recommend value 50 to 200)

swappiness – This control is used to define how aggressively the kernel will swap memory pages. Higher values will increase aggressiveness; lower values decrease the amount of swap. (default = 60, recommended values between 1 and 60) Remove your swap for 0 value, but it is usually not recommended in most cases.

To edit, you can add or replace these lines in /etc/sysctl.conf file. For example, if you have low memory until you upgrade, you can try something such as:

vm.swappiness=10
vm.vfs_cache_pressure=200

This will increase the cache pressure, which may seem somewhat counterproductive since caching is good for performance. However, too frequent swapping reduces your server’s overall performance significantly more. So not keeping as much cache in memory will help reduce swap activity. Also, with vm.swappiness set to 10 or as low as 1, it will reduce disk swapping.

On a healthy server with lots of available memory, use the following:

vm.swappiness=10
vm.vfs_cache_pressure=50

This will decrease the cache pressure. Since caching is good for performance, we want to keep cached data in memory longer. Since the cache will grow larger, we still want to reduce swapping to not cause increased swap I/O.

To check current values using these commands use:

sudo cat /proc/sys/vm/swappiness
sudo cat /proc/sys/vm/vfs_cache_pressure

To enable these settings temporarily without rebooting, use the following commands:

sudo sysctl -w vm.swappiness=10
sudo sysctl -w vm.vfs_cache_pressure=50

The man pages for both versions of free should be considered independently. See the screenshot of the most recent version below.

Conclusion

The debate over whether to enable or disable swap space on a Linux system with adequate RAM often boils down to nuanced performance considerations and specific use cases. As we’ve explored, even when you have more than enough memory, having some swap space can offer advantages.

One notable advantage is that the Linux Kernel intelligently uses swap space to optimize memory usage. It moves rarely-used memory pages into swap, freeing up more cacheable space for frequently used pages. This helps maintain a balance between performance and memory management.

Additionally, having swap space provides a safety net. It allows administrators to react to low memory issues gradually, preventing sudden and severe performance degradation. Properly monitoring and alerting for swap usage can further enhance this safety net.

However, the decision regarding swap space should be tailored to your specific server and applications. Setting swap space too large can prolong performance issues or outages, so it’s essential to strike a balance.

Conversely, when you have limited memory and no swap space, you risk running out of memory quickly, leading to the deployment of the OOM (Out Of Memory) killer, which can terminate high-memory processes.

Ultimately, the use of swap space should be considered in the context of your system’s overall performance and resource utilization. Properly tuning parameters like vm.swappiness and vm.vfs_cache_pressure can further optimize how swap is used.

In summary, swap space is not just a backup for low-memory situations; it can be a valuable tool for maintaining optimal performance on a Linux server, even when you have ample RAM. The key is to strike the right balance between swap space and RAM based on your specific server’s requirements and performance characteristics.