Swapon: Orchestrating Virtual Memory Paging, Prioritising Multi-Device Swap Pools, and Mitigating OOM Thrashing in Production

When a server suddenly grinds to a halt under pressure, the culprit is rarely a mysterious hardware fault. More often, it is a quiet, brutal battle taking place deep inside the machine's memory. When physical RAM fills up, the Linux operating system faces a stark choice: it can abruptly kill running applications to free up space, or it can sacrifice the background file cache that keeps disks running smoothly. Without an escape valve, disk access slows to an absolute crawl, application threads freeze, and the server locks up completely.

That escape valve is swap space, and the command that orchestrates it is swapon. Often dismissed as an obsolete relic from the era of slow mechanical hard drives, swapon is in fact a sophisticated control dial for modern virtual memory. It tells the Linux kernel where, when, and with what priority to park dormant data onto secondary storage—protecting vital system caches, absorbing sudden traffic surges, and preventing catastrophic crashes.

Before you make changes during an incident or configure a new server, your first practical step is to audit your machine's active memory topology. The single most useful diagnostic command you can run is:

swapon --show=NAME,TYPE,SIZE,USED,PRIO

This single command provides an instant snapshot of your server's memory safety net, detailing the storage medium, total allocation, current usage, and priority hierarchy:

NAME      TYPE      SIZE  USED PRIO
/dev/zram0 partition   4G  1.2G  100
/swapfile file        16G  2.4G   10
/dev/nvme0n1p3 partition  32G    0B    5

In one quick view, you can see whether your machine has an active safety net, how much buffer remains, and how the kernel distributes the load across different drives.

What It Does in Plain English

At its heart, swapon tells the Linux kernel to activate secondary storage targets—such as raw disk partitions, lightning-fast NVMe drives, or dedicated files—as auxiliary virtual memory.

When your server's physical RAM runs short, the kernel identifies inactive, resting memory pages and writes them out to these designated swap targets. By offloading dormant memory, the operating system keeps its essential file caches intact, ensuring that active programs continue reading and writing to disk without delay. Through swapon, administrators can dynamically orchestrate how and when these overflow buffers engage, keeping infrastructure stable even when workloads spike unpredictably.

Core Options & Everyday Flags

The swapon utility offers a concise set of flags to inspect, activate, and fine-tune swap devices across your system:

How Linux Manages Virtual Memory Under the Hood

To make effective operational decisions, it helps to understand how the Linux kernel balances physical memory, background caches, and secondary storage.

Anonymous Memory versus the Page Cache

The Linux virtual memory manager divides physical RAM into two primary categories:

1. File-Backed Pages (The Page Cache): These contain data read directly from files on disk (such as running binaries, libraries, and cached documents). When memory is tight, the kernel can discard clean file-backed pages instantly without saving them anywhere, because it can reload them from disk whenever needed.
2. Anonymous Pages: These belong directly to running applications—their execution heaps, process stacks, and private working buffers. Because they do not exist as files on disk, they cannot simply be discarded.

Without swap configured via swapon, the kernel has no way to evict anonymous pages. Under heavy memory load, it is forced to cannibalize the page cache down to zero. Once the page cache vanishes, every shared library execution and basic file read requires slow, synchronous access to disk. The system enters thrashing—a crippling state where the CPU spends all its time waiting on storage I/O until the Out-Of-Memory (OOM) killer abruptly terminates critical processes.

The Lifecycle of a Swapped Page

The background eviction and reclamation of memory pages are handled by a dedicated kernel thread named kswapd, working alongside direct reclamation routines. Pages are tracked across Least Recently Used (LRU) lists:

1. Scanning: The kernel scans active memory pages, identifying those that haven't been accessed recently, and moves them to the inactive list.
2. Swap Cache Allocation: The inactive page is assigned an identifier (swp_entry_t) matching a slot on the swap device and is registered in the kernel's swap cache to prevent race conditions.
3. Paging Out: The page contents are written to the storage device or swap file via the kernel's block layer (submit_bio).
4. Page Table Update: Once the write completes, the process's Page Table Entry (PTE) is updated: the "present" bit is cleared, and the physical memory address is replaced with the swap identifier. The physical RAM frame is then freed for active workloads.
5. Paging In (Swap Fault): When an application tries to access that address again, the CPU triggers a page fault. The kernel intercepts the fault, reads the data back from storage into a fresh RAM frame, updates the page table, and seamlessly resumes the application.

The Swap Header Layout (SWAPSPACE2)

Before swapon can register any partition or file, the target must be prepared with the mkswap utility. This initialises the storage with the SWAPSPACE2 metadata structure:

• Magic Signature: The final 10 bytes of the first 4-kilobyte page contain the ASCII string SWAPSPACE2. The kernel verifies this signature; if missing, activation fails immediately with EINVAL.
• Version Number: Modern swap formats use version 1.
• Last Usable Page: An unsigned 32-bit integer defining the highest valid swap slot on the device.
• UUID & Volume Label: A unique 128-bit identifier and optional label, allowing reliable references in /etc/fstab via /dev/disk/by-uuid/.

The System Call Boundary

When you run swapon, the utility checks your parameters and invokes the underlying kernel system call: sys_swapon(const char *specialfile, int swap_flags).

Inside the kernel: 1. Path Resolution: The path is resolved to a block device or filesystem inode. 2. Exclusive Locking: The kernel requests exclusive access to the target (BLK_OPEN_RESTRICTED). If another process or swap instance holds an open lock, the call halts with EBUSY. 3. Header Validation: The first page is inspected for the SWAPSPACE2 signature and alignment. 4. Map Construction: An internal control structure (struct swap_info_struct) and tracking map (swap_map) are created to maintain reference counters for every slot. 5. Priority Linking: The device is inserted into the kernel's swap list (swap_avail_heads), sorted by its assigned priority.

Swappiness and Memory Balancing

The balance between reclaiming anonymous memory and shedding file cache is governed by sysctl settings described in the Kernel Sysctl vm.txt Documentation:

• /proc/sys/vm/swappiness: A scale from 0 to 200 (in modern 5.x/6.x kernels) that sets the ratio between anonymous and file-cache reclamation:

$$\text{Reclaim Ratio} = \frac{\text{Scan Anonymous}}{\text{Scan File-Cache}} \approx \frac{\text{swappiness}}{200 - \text{swappiness}}$$

Setting swappiness = 0 tells the kernel to avoid swapping anonymous memory unless physical RAM reaches near-total exhaustion. A setting of 100 balances anonymous and file-cache eviction equally, while values approaching 200 aggressively page out idle anonymous buffers to prioritize disk caching.

5 Real-World Production Scenarios

1. Dynamic NVMe Swap File Provisioning with SSD Discard

The Scenario

A high-traffic PostgreSQL primary node deployed on NVMe storage experiences sudden memory pressure during end-of-month reporting runs. Physical RAM is nearly full, risking database termination by the OOM killer. You must quickly provision an emergency 32-gigabyte swap file on the fast NVMe volume, enforce strict root-only security permissions, and enable continuous SSD TRIM discard to preserve drive longevity and write performance.

The Command Sequence

# Allocate a contiguous 32GB file on the NVMe volume
dd if=/dev/zero of=/var/lib/swap/swapfile_nvme bs=1G count=32 status=progress

# Restrict permissions strictly to root
chmod 0600 /var/lib/swap/swapfile_nvme

# Initialize the SWAPSPACE2 signature with a descriptive label
mkswap -L SWAP_NVME_01 /var/lib/swap/swapfile_nvme

# Activate swap with continuous page-level TRIM discard enabled
swapon --verbose --discard=pages --priority 50 /var/lib/swap/swapfile_nvme

# Verify active registration
swapon --show=NAME,TYPE,SIZE,USED,PRIO

Realistic Terminal Output

34359738368 bytes (34 GB, 32 GiB) copied, 14.821 s, 2.3 GB/s
Setting up swapspace version 1, size = 32 GiB (34359734272 bytes)
LABEL=SWAP_NVME_01, UUID=b4a7d65c-28f0-464a-9ceb-782da67b2d5a
swapon: /var/lib/swap/swapfile_nvme: found signature [pagesize=4096, signature=SWAPSPACE2]
swapon: /var/lib/swap/swapfile_nvme: pagesize=4096, swapsize=34359738368, devsize=34359738368
swapon /var/lib/swap/swapfile_nvme
NAME                         TYPE SIZE USED PRIO
/var/lib/swap/swapfile_nvme  file  32G   0B   50

Line-by-Line Explanation

1. dd if=/dev/zero of=...: Creates a fully allocated 32GB file filled with zeroes, guaranteeing contiguous disk blocks and avoiding filesystem fragmentation.
2. chmod 0600 ...: Restricts read and write permissions exclusively to the superuser to safeguard memory contents.
3. mkswap -L SWAP_NVME_01 ...: Formats the file with the SWAPSPACE2 header, page-size metadata, and a distinct UUID.
4. swapon --verbose --discard=pages --priority 50 ...: Registers the swap target with priority 50 and commands the kernel to issue non-blocking TRIM discard requests (BLKDISCARD) to the SSD controller whenever swapped pages are freed.

What the Sysadmin Does Next

Persist the configuration in /etc/fstab so the swap file mounts automatically on reboot:

echo "/var/lib/swap/swapfile_nvme none swap sw,pri=50,discard=pages 0 0" >> /etc/fstab

2. Multi-Tiered Hierarchical Paging Architecture

The Scenario

A virtualisation host contains three storage tiers: compressed in-memory RAM (zram0), a high-speed NVMe partition (/dev/nvme0n1p3), and a large mechanical hard drive (/dev/sda2). To maximize throughput and avoid latency bottlenecks, you must configure a tiered hierarchy where memory overflow spills sequentially across devices based on speed.

The Command Sequence

# Activate Tier 1: In-memory compressed zram with maximum priority
swapon --priority 100 /dev/zram0

# Activate Tier 2: NVMe flash partition with secondary priority
swapon --priority 20 --discard=pages /dev/nvme0n1p3

# Activate Tier 3: Mechanical drive partition as emergency backup
swapon --priority 1 /dev/sda2

# Inspect the active tiered hierarchy
swapon --show=NAME,TYPE,SIZE,USED,PRIO

Realistic Terminal Output

NAME           TYPE      SIZE USED PRIO
/dev/zram0     partition   8G   0B  100
/dev/nvme0n1p3 partition  64G   0B   20
/dev/sda2      partition 128G   0B    1

Line-by-Line Explanation

1. swapon --priority 100 /dev/zram0: Positions the compressed RAM device at the head of the kernel's swap queue. The kernel will utilize all 8GB of compressed RAM before writing anything to disk.
2. swapon --priority 20 --discard=pages /dev/nvme0n1p3: Provides the fast secondary tier. If zram0 fills, the kernel moves directly to the NVMe drive with minimal latency.
3. swapon --priority 1 /dev/sda2: Serves as the final safety net, preventing memory exhaustion crashes under extreme load.

What the Sysadmin Does Next

Test the tiered spillover behavior under controlled load using stress-ng while monitoring device usage across /proc/swaps.

3. Provisioning zram Compressed In-Memory Swap for High-Density Hosts

The Scenario

You are tuning a fleet of Kubernetes worker nodes running high-density microservices. Physical memory is limited, and disk input/output must be reserved exclusively for container storage. You need to provision a zram device using modern zstd compression, effectively doubling available memory capacity without touching physical drives.

The Command Sequence

# Load the kernel zram module
modprobe zram num_devices=1

# Select the Zstandard compression algorithm
echo zstd > /sys/block/zram0/comp_algorithm

# Set virtual uncompressed capacity to 16GB
echo 16G > /sys/block/zram0/disksize

# Format the virtual memory block with SWAPSPACE2
mkswap -L ZRAM_SWAP /dev/zram0

# Activate the zram device with top priority
swapon --priority 32767 /dev/zram0

# Inspect compression efficiency and device statistics
zramctl /dev/zram0

Realistic Terminal Output

Setting up swapspace version 1, size = 16 GiB (17179865088 bytes)
LABEL=ZRAM_SWAP, UUID=f10c3b84-9844-4822-a7e6-81a1795026df
NAME       ALGORITHM DISKSIZE DATA COMPR TOTAL STREAMS MOUNTPOINT
/dev/zram0 zstd           16G 1.8G  480M  540M      32 [SWAP]

Line-by-Line Explanation

1. modprobe zram ...: Loads the kernel's RAM-based compressed block storage module.
2. echo zstd > ...: Configures the fast zstd compression engine, which typically compresses application heap data by 2.5x to 3.5x.
3. echo 16G > ...: Defines the virtual boundary. Physical RAM is allocated dynamically on demand as pages arrive.
4. mkswap -L ZRAM_SWAP ...: Writes the swap superblock to the device.
5. swapon --priority 32767 ...: Engages the device at the maximum possible priority (32767), guaranteeing that initial page-outs stay in fast compressed RAM.

What the Sysadmin Does Next

Adjust kernel memory parameters via sysctl to encourage proactive offloading of dormant heap memory into zram:

sysctl -w vm.swappiness=150
sysctl -w vm.watermark_scale_factor=200

4. Structured Telemetry Ingestion for Alerting and Triage

The Scenario

During incident triage, automation scripts and monitoring systems need clean, unformatted metrics on swap capacity. Standard human-readable suffixes (such as M or G) can break parsing scripts. You need to gather raw swap metrics with swapon and export them to Prometheus Node Exporter text files.

The Command Sequence

# Output raw, machine-readable metrics without table headers
swapon --show=NAME,TYPE,SIZE,USED,PRIO --raw --noheadings --bytes

# Create the Prometheus Node Exporter custom metrics file
cat << 'EOF' > /var/lib/node_exporter/textfile_collector/swap_metrics.prom
# HELP node_swap_partition_bytes_total Total size of configured swap device in bytes.
# TYPE node_swap_partition_bytes_total gauge
# HELP node_swap_partition_bytes_used Used space of configured swap device in bytes.
# TYPE node_swap_partition_bytes_used gauge
# HELP node_swap_partition_priority Assigned priority of the swap device.
# TYPE node_swap_partition_priority gauge
EOF

swapon --show=NAME,TYPE,SIZE,USED,PRIO --raw --noheadings --bytes | while read -r name type size used prio; do
  device_clean=$(echo "${name}" | sed 's/[^a-zA-Z0-9_]/_/g')
  cat << EOF >> /var/lib/node_exporter/textfile_collector/swap_metrics.prom
node_swap_partition_bytes_total{device="${name}",clean_id="${device_clean}",type="${type}"} ${size}
node_swap_partition_bytes_used{device="${name}",clean_id="${device_clean}",type="${type}"} ${used}
node_swap_partition_priority{device="${name}",clean_id="${device_clean}",type="${type}"} ${prio}
EOF
done

# Validate the generated metrics output
cat /var/lib/node_exporter/textfile_collector/swap_metrics.prom

Realistic Terminal Output

# HELP node_swap_partition_bytes_total Total size of configured swap device in bytes.
# TYPE node_swap_partition_bytes_total gauge
# HELP node_swap_partition_bytes_used Used space of configured swap device in bytes.
# TYPE node_swap_partition_bytes_used gauge
# HELP node_swap_partition_priority Assigned priority of the swap device.
# TYPE node_swap_partition_priority gauge
node_swap_partition_bytes_total{device="/dev/zram0",clean_id="_dev_zram0",type="partition"} 17179869184
node_swap_partition_bytes_used{device="/dev/zram0",clean_id="_dev_zram0",type="partition"} 2147483648
node_swap_partition_priority{device="/dev/zram0",clean_id="_dev_zram0",type="partition"} 32767
node_swap_partition_bytes_total{device="/swapfile",clean_id="_swapfile",type="file"} 34359738368
node_swap_partition_bytes_used{device="/swapfile",clean_id="_swapfile",type="file"} 10737418240
node_swap_partition_priority{device="/swapfile",clean_id="_swapfile",type="file"} 10

Line-by-Line Explanation

1. swapon ... --raw --noheadings --bytes: Emits exact byte counts without headers, ensuring deterministic parsing.
2. device_clean=$(echo ... | sed ...): Sanitizes device paths (e.g. turning /dev/zram0 into _dev_zram0) to comply with Prometheus metric formatting standards.
3. Appends structured gauge entries directly into the Prometheus textfile collector path.

What the Sysadmin Does Next

Configure a Prometheus alerting rule to notify the team whenever a primary swap tier reaches 85% utilization:

- alert: HighSwapTierSaturation
  expr: (node_swap_partition_bytes_used{clean_id="_dev_zram0"} / node_swap_partition_bytes_total{clean_id="_dev_zram0"}) > 0.85
  for: 2m
  labels:
    severity: warning
  annotations:
    summary: "Primary zram compressed swap tier nearly exhausted on {{ $labels.instance }}"

5. Zero-Downtime Swap Evacuation and Migration

The Scenario

Monitoring alerts report failing physical sectors on an NVMe SSD hosting an active 64GB swap partition (/dev/nvme1n1p1). The server is handling live traffic with 28GB of anonymous memory currently residing on the failing drive. Running swapoff outright would attempt to force all 28GB back into physical RAM, immediately crashing the host. You need to perform a zero-downtime swap evacuation by attaching a healthy replacement drive with swapon and cleanly migrating the active memory pages.

The Command Sequence

# 1. Audit memory headroom and current swap usage
free -h
swapon --show=NAME,TYPE,SIZE,USED,PRIO

# 2. Format the new replacement NVMe partition
mkswap -L SWAP_NVME_REPLACE /dev/nvme2n1p1

# 3. Activate the new target with higher priority to receive writes
swapon --priority 100 --discard=pages /dev/nvme2n1p1

# 4. Drain and deactivate the failing swap partition
swapoff -v /dev/nvme1n1p1

# 5. Confirm complete migration
swapon --show=NAME,TYPE,SIZE,USED,PRIO

Realistic Terminal Output

               total        used        free      shared  buff/cache   available
Mem:           125Gi        98Gi       5.2Gi       1.1Gi        22Gi        24Gi
Swap:           64Gi        28Gi        36Gi

NAME           TYPE      SIZE  USED PRIO
/dev/nvme1n1p1 partition  64G   28G   10

Setting up swapspace version 1, size = 64 GiB (68719476736 bytes)
LABEL=SWAP_NVME_REPLACE, UUID=9d2c4183-4318-466d-88b9-e16089ecbcf3
swapon: /dev/nvme2n1p1: found signature [pagesize=4096, signature=SWAPSPACE2]
swapon /dev/nvme2n1p1

swapoff /dev/nvme1n1p1
swapoff: /dev/nvme1n1p1: reading swap space...
swapoff: /dev/nvme1n1p1: relocating pages to other swap spaces...
swapoff: /dev/nvme1n1p1: done

NAME           TYPE      SIZE  USED PRIO
/dev/nvme2n1p1 partition  64G   28G  100

Line-by-Line Explanation

1. free -h: Audits physical headroom. Available RAM (24Gi) is less than the active swap data (28Gi), meaning a direct unmount would trigger an immediate OOM crash.
2. swapon --priority 100 ... /dev/nvme2n1p1: Activates the healthy replacement drive at top priority.
3. swapoff -v /dev/nvme1n1p1: Commands the kernel to read each page from the failing device and funnel it through the swap cache into the new target without overwhelming RAM.
4. swapon --show: Verifies that the failing drive has been safely detached and all 28GB of memory pages now reside on the replacement drive.

What the Sysadmin Does Next

Update /etc/fstab with the new UUID (9d2c4183-4318-466d-88b9-e16089ecbcf3) before unmounting and physically replacing the failing drive.

What Can Go Wrong: Pitfalls & Operational Gotchas

1. The Sparse File Trap on Modern Filesystems

A common mistake occurs when attempting to create a swap file using fallocate -l on filesystems like ext4 or XFS:

fallocate -l 16G /swapfile_broken
chmod 0600 /swapfile_broken
mkswap /swapfile_broken
swapon /swapfile_broken

Diagnostic Error:

swapon: /swapfile_broken: swapon failed: Invalid argument

The Root Cause

fallocate creates unallocated blocks ("holes") in filesystem metadata. The kernel's sys_swapon call requires strictly contiguous physical allocations with zero holes to avoid filesystem deadlocks during page faults. If unallocated holes are detected, activation fails with EINVAL (Invalid argument).

The Fix

Always allocate swap files with contiguous block writes using dd:

dd if=/dev/zero of=/swapfile bs=1M count=16384 status=progress
chmod 0600 /swapfile
mkswap /swapfile
swapon /swapfile

2. File Permissions and Secret Exposure

Swap targets store raw, unencrypted snapshots of application memory—including TLS certificates, private keys, and database buffers.

# HAZARDOUS: Overly permissive file rights expose memory data
chmod 0644 /swapfile
swapon /swapfile

The Security Risk

Any non-root user on the host can read /swapfile directly, exposing sensitive credentials from co-located services.

The Fix

1. Enforce strict ownership: chmod 0600 /swapfile.
2. For production environments, configure ephemeral encrypted swap using LUKS and cryptsetup:

cryptsetup open --type plain --key-file /dev/urandom /dev/nvme0n1p2 swap_crypt
mkswap /dev/mapper/swap_crypt
swapon /dev/mapper/swap_crypt

3. Continuous versus One-Time Discard Overhead

While SSDs benefit from TRIM commands to clear unused blocks, using --discard=pages on certain enterprise drives can trigger latency spikes:

# Can cause I/O latency stalls on SATA SSDs
swapon --discard=pages /dev/sdb1

The Root Cause

When the kernel frees an individual 4KB page, --discard=pages sends a synchronous TRIM request to the storage controller. On SATA drives or busy controllers lacking queued TRIM support, these operations briefly stall ongoing application reads and writes.

The Fix

Use one-time discard at activation on SATA drives, or schedule periodic TRIMs via fstrim:

swapon --discard=once /dev/sdb1

Today's Takeaway

Open a terminal on your machine right now and run swapon --show to inspect your current virtual memory setup. If your system reports no configured swap devices, your machine is running without a safety net—leaving it vulnerable to sudden page-cache starvation and abrupt application crashes under heavy load. In less than five minutes, you can configure a lightweight, compressed zram device or a secure, root-protected swap file. Doing so gives your operating system the headroom it needs to handle workload surges gracefully and stay online when it matters most.

Authoritative Technical References

• Linux Kernel Virtual Memory & Concepts Documentation
• Linux Kernel Sysctl vm.txt Documentation
• Linux man7.org: swapon(8) Administration Manual
• Linux man7.org: swapon(2) System Calls Reference
• Linux man7.org: mkswap(8) System Administration Manual
• ArchWiki: Comprehensive Linux Swap Configuration
• ArchWiki: Advanced Zram and In-Memory Compression