Hexdump: Inspecting Binary Offsets, Validating Filesystem Superblock Magic Bytes, and Decoding Raw Serialized Streams in Production

When high-level operating system tools throw their hands in the air, guessing is dangerous. Standard filesystem drivers, volume managers, and disk utilities depend on structured metadata. If that metadata is corrupted, truncated, or misaligned by even a single byte, those tools blindfold themselves and emit generic, unhelpful error messages. To discover what has actually occurred on that physical disk or solid-state drive, you must bypass the operating system's assumptions and look directly at the underlying reality: the raw binary stream.

This is where hexdump, an essential utility residing in the core Linux util-linux package, becomes an indispensable tool. Instead of relying on operating system layers that are already failing, hexdump functions as an unfiltered digital microscope. It reads sequential 8-bit bytes directly from any storage device, memory segment, or network pipe, rendering them into a side-by-side display of hexadecimal notation and plain text.

The single most practical and versatile command you can run to inspect any raw file or binary structure safely is the canonical byte dump:

hexdump -C -n 64 /bin/ls

00000000  7f 45 4c 46 02 01 01 00  00 00 00 00 00 00 00 00  |.ELF............|
00000010  03 00 3e 00 01 00 00 00  b0 6d 00 00 00 00 00 00  |..>......m......|
00000020  40 00 00 00 00 00 00 00  38 48 02 00 00 00 00 00  |@.......8H......|
00000030  00 00 00 00 40 00 38 00  09 00 40 00 1f 00 1e 00  |....@.8...@.....|
00000040

By combining the canonical view flag (-C) with a length constraint (-n 64 to inspect just the first 64 bytes), you obtain an immediate, non-destructive snapshot of the file's inner anatomy. The leftmost column displays the exact byte offset in hexadecimal notation. The central two columns show sixteen individual bytes per row in hexadecimal. The right-hand column, bounded by vertical bars, prints the corresponding ASCII characters, using full stops (.) as placeholders for non-printable control characters. Right at offset zero, the sequence 7f 45 4c 46 spells out \x7fELF—the standardized magic signature of a Linux ELF (Executable and Linkable Format) binary.

What Hexdump Does in Plain English

Every piece of digital information—whether it is a photo, an encrypted database record, a compiled program, or a disk partition header—is stored as a continuous series of 8-bit numbers (bytes). High-level software reads these bytes through specific lenses, translating them into text files, spreadsheets, or running processes. When a file format becomes corrupted or an unexpected character sneaks into a configuration file, standard text viewers like cat, less, or vim may hide the problem, render garbage characters, or crash altogether.

hexdump strips away all formatting assumptions. It ingests arbitrary streams of binary data and displays the exact numerical value of each byte alongside its text equivalent. By peering underneath application layers, you can spot invisible control characters, detect missing structural headers, confirm encryption signatures, and examine partition tables without making any modifications to the underlying disk.

Core Flags and Practical Quick-Start Reference

The hexdump utility offers several output modes, from standard canonical tables to custom formatting strings. The table below details the most important flags used in troubleshooting and systems analysis:

Architectural Foundations: Stream Buffering and the Endianness Trap

To interpret binary output accurately, it helps to understand how the Linux kernel passes byte streams from disk storage or input pipes into hexdump.

The Endianness Trap: Why -C is Essential

A frequent trap for engineers troubleshooting low-level issues is the difference between single-byte canonical mode (-C) and multi-byte word modes (-x, -d):

• Canonical Mode (-C): Treats input strictly as an array of individual 8-bit bytes. It displays bytes in the exact physical order in which they reside in storage or across a network cable, regardless of the CPU architecture you are running on.
• Multi-Byte Modes (-x): Ingests bytes as 16-bit or 32-bit machine words. On modern x86_64 and ARM64 processors—which are Little-Endian—the CPU stores the least significant byte first. As a result, hexdump -x swaps the positions of adjacent bytes in the terminal output.

Consider the four-byte ASCII sequence "ABCD" (Hex: 0x41, 0x42, 0x43, 0x44):

# Canonical display preserves the real physical byte order:
printf "ABCD" | hexdump -C
# 00000000  41 42 43 44                                       |ABCD|

# Two-byte hex mode swaps bytes within each 16-bit word on little-endian hardware:
printf "ABCD" | hexdump -x
# 00000000    4241    4443

5 Real-World Production Use Cases

1. Validating Filesystem Superblock Magic Bytes and Partition Integrity

The Operational Scenario

A storage area network (SAN) volume mapped to /dev/sdb1 refuses to mount following an unexpected cluster failover. The kernel reports EXT4-fs (sdb1): VFS: Can't find ext4 filesystem. Before running repair utilities that might make destructive changes, you need to verify whether the partition table or the ext4 superblock located at byte offset 1024 has been overwritten.

Execution Command

# 1. Inspect the Master Boot Record (MBR) boot signature (offset 510, 2 bytes)
hexdump -v -s 510 -n 2 -C /dev/sdb

# 2. Inspect the ext4 superblock signature (offset 1024 + 56 = 1080, 2 bytes)
hexdump -v -s 1080 -n 2 -C /dev/sdb1

Realistic Terminal Output

# Command 1: MBR Boot Signature Output
000001fe  55 aa                                             |U.|
00000200

# Command 2: Ext4 Superblock Magic Number Output
00000438  53 ef                                             |S.|
0000043a

Line-by-Line Technical Deconstruction

• hexdump -v -s 510 -n 2 -C /dev/sdb:
• -s 510: Seeks directly to byte 510 (hex offset 0x01FE), the standard location of the MBR boot signature.
• -n 2: Restricts the read to exactly two bytes.
• 000001fe 55 aa: Shows that the valid MBR signature 0xAA55 is present in little-endian order (55 aa), confirming the base partition table is intact.
• hexdump -v -s 1080 -n 2 -C /dev/sdb1:
• According to the Linux Kernel Ext4 Documentation, the ext4 superblock starts at byte offset 1024 (0x0400). Within that superblock structure, the 16-bit magic number s_magic is located at relative offset 56 (0x38). Adding these together gives offset 1080 (0x0438).
• 00000438 53 ef: Confirms the standard ext4 magic number 0xEF53 stored in little-endian byte order (53 ef).

What the Admin Does Next

The presence of 55 aa and 53 ef proves that neither the partition boundaries nor the primary superblock magic bytes have been destroyed. The mount failure is likely due to damaged secondary metadata, such as block group descriptors or journal pointers. Confident that the partition alignment is sound, you can safely run fsck.ext4 -b 32768 /dev/sdb1 to recover the filesystem using an intact backup superblock.

2. Dissecting Raw Network Telemetry and Compressed Stream Headers

The Operational Scenario

An internal microservice cluster communicating over a high-performance binary protocol starts dropping connections. Upstream proxy logs report HTTP 502 Bad Gateway: Corrupted Payload Framing. You capture incoming payload packets using tcpdump into a file named telemetry_dump.bin. You must verify whether the stream contains valid compression framing headers or if an intermediate proxy is truncating the data.

Execution Command

hexdump -C -n 32 telemetry_dump.bin

Realistic Terminal Output

00000000  00 00 00 1a 28 b5 2f fd  04 00 40 00 00 12 a0 01  |....(./...@.....|
00000010  3f 99 c2 4a 65 6e 76 6f  79 2d 70 72 6f 78 79 00  |?..Jenvoy-proxy.|
00000020

Line-by-Line Technical Deconstruction

• 00000000: The start of the binary stream at offset zero.
• 00 00 00 1a: A 4-byte big-endian integer representing decimal 26, the custom protocol's declared frame length header.
• 28 b5 2f fd: The 4-byte magic signature for a Zstandard (zstd) compressed frame (0xFD2FB528).
• 04 00 40 00 ...: The start of the Zstandard compression descriptor parameters.
• |....(./...@.....|: Visible printable ASCII characters showing the metadata string envoy-proxy.

What the Admin Does Next

The magic bytes 28 b5 2f fd confirm the payload is compressed with Zstandard. However, the leading length integer (00 00 00 1a = 26 bytes) shows that the upstream proxy miscalculated the payload size by declaring only 26 bytes for a stream that exceeds 128 bytes. You adjust the proxy's buffer allocation configuration so that its content-length calculations account for the post-compression binary size.

3. Custom Struct Extraction with Advanced Format Strings

The Operational Scenario

An industrial sensor telemetry daemon writes packed binary records directly to shared memory at /dev/shm/telemetry.raw. The C data structure is defined in the kernel as:

struct __attribute__((__packed__)) telemetry_record {
    uint32_t timestamp_sec; // 4-byte unsigned integer (Unix epoch)
    uint16_t sensor_id;     // 2-byte unsigned short
    uint16_t status_flags;   // 2-byte bitmask
    float    reading;        // 4-byte IEEE 754 single-precision float
}; // Total size = 12 bytes

Standard text-processing tools cannot parse this raw binary file. You need to decode these records into readable decimal and floating-point metrics on the command line without having to write and compile a separate script.

Execution Command

hexdump -v -e '"Epoch: " 1/4 "%u" " | "' \
           -e '"Sensor: " 1/2 "0x%04X" " | "' \
           -e '"Flags: " 1/2 "0x%04X" " | "' \
           -e '"Reading: " 1/4 "%9.3f" "\n"' \
           -n 48 /dev/shm/telemetry.raw

Realistic Terminal Output

Epoch: 1729605840 | Sensor: 0x0A12 | Flags: 0x0001 | Reading:   104.250
Epoch: 1729605841 | Sensor: 0x0A12 | Flags: 0x0001 | Reading:   104.375
Epoch: 1729605842 | Sensor: 0x0A12 | Flags: 0x0003 | Reading:   108.125
Epoch: 1729605843 | Sensor: 0x0A12 | Flags: 0x0000 | Reading:     0.000

Line-by-Line Technical Deconstruction

The -e flag allows you to define custom format strings structured as [iteration/byte_length "format"]: * -e '"Epoch: " 1/4 "%u" " | "': Reads 1 unit of 4 bytes (uint32_t), formatting it as an unsigned decimal integer (%u) preceded by the label Epoch:. * -e '"Sensor: " 1/2 "0x%04X" " | "': Ingests the next 2 bytes (uint16_t), printing them as a 4-digit hexadecimal identifier. * -e '"Flags: " 1/2 "0x%04X" " | "': Ingests the subsequent 2 bytes representing operational status flags. * -e '"Reading: " 1/4 "%9.3f" "\n"': Reads the final 4 bytes as an IEEE 754 single-precision float (%f), formatted to 3 decimal places and terminated with a newline (\n). * -n 48: Limits output to 48 bytes (exactly 4 continuous 12-byte struct records).

What the Admin Does Next

The decoded telemetry immediately reveals an anomaly at timestamp 1729605842: status flags changed from 0x0001 (Normal Operation) to 0x0003 (Over-temperature Warning) right before the reading dropped to 0.000 at 1729605843. You identify a hardware sensor failure on unit 0x0A12 and initiate an automated switchover to the secondary monitoring bus.

4. Isolating Invisible Bytes, UTF-8 BOMs, and Corrupted Config Files

The Operational Scenario

Following a security certificate rotation, an NGINX ingress controller fails to reload with the error: SSL_CTX_use_PrivateKey_file() failed (SSL: error:0B080074:x509 certificate routines:X509_check_private_key:key values mismatch). The private key and certificate files look perfectly valid when viewed with cat or vim. You suspect that non-printable characters, Windows line endings (\r\n), or a UTF-8 Byte Order Mark (BOM) were introduced during a copy-paste operation between different operating systems.

Execution Command

# Compare the first 32 bytes of the rotated certificate file
hexdump -C -n 32 /etc/ssl/certs/production.crt

Realistic Terminal Output

00000000  ef bb bf 2d 2d 2d 2d 2d  42 45 47 49 4e 20 43 45  |...-----BEGIN CE|
00000010  52 54 49 46 49 43 41 54  45 2d 2d 2d 2d 2d 0d 0a  |RTIFICATE----...|
00000020

Line-by-Line Technical Deconstruction

• 00000000: The file offset starting at byte zero.
• ef bb bf: The UTF-8 Byte Order Mark (BOM). These three hex bytes (0xEF 0xBB 0xBF) are added by some text editors to indicate UTF-8 encoding. Standard text editors often hide them, but cryptographic parsers in OpenSSL treat them as unexpected raw binary input, breaking certificate validation.
• 2d 2d 2d 2d 2d 42 45 47 49 4e ...: The ASCII text representation of -----BEGIN CE.
• 0d 0a: DOS/Windows Carriage Return and Line Feed (CRLF). The byte 0x0D (\r) followed by 0x0A (\n) confirms the file was saved with Windows line endings rather than POSIX standard newlines (0x0A).

What the Admin Does Next

You strip the three-byte BOM and convert the line endings back to Unix format using standard command-line tools:

# Strip UTF-8 BOM and convert CRLF to standard LF line endings
sed -i '1s/^\xEF\xBB\xBF//' /etc/ssl/certs/production.crt
dos2unix /etc/ssl/certs/production.crt

# Verify the cleaned file using hexdump
hexdump -C -n 16 /etc/ssl/certs/production.crt
# 00000000  2d 2d 2d 2d 2d 42 45 47  49 4e 20 43 45 52 54 49  |-----BEGIN CERTI|

With the hidden bytes removed, NGINX reloads smoothly without errors.

5. Auditing Pseudo-Random Entropy Streams and Kernel Device Nodes

The Operational Scenario

During the provisioning of a virtual machine, cryptographic key generation operations hang indefinitely. You suspect that the system's /dev/urandom entropy pool is starved or delivering repeating byte sequences due to a hypervisor timing issue. You need to inspect raw kernel device streams without allowing hexdump to suppress identical lines.

Execution Command

# Inspect 64 bytes of raw kernel entropy directly from the character device
# Passing -v is essential to prevent deduplication of repeating blocks
hexdump -v -n 64 -C /dev/urandom

Realistic Terminal Output

00000000  a4 72 e1 9c 5b 89 23 f0  8b 31 77 19 cc de 02 a1  |.r..[.#..1w.....|
00000010  1e 84 f5 2a 90 d3 b4 67  51 0a e9 3f 88 14 55 bc  |...*...gQ..?..U.|
00000020  7c 11 02 e8 d6 f2 c9 80  3a 4f b0 91 e4 60 7d 34  ||.......:O...`}4|
00000030  ff c8 92 1b 4d 7e 33 5a  21 b9 8a 04 e7 d1 f8 62  |....M~3Z!......b|
00000040

Line-by-Line Technical Deconstruction

• hexdump -v -n 64 -C /dev/urandom:
• -v (Verbose): Disables automatic line deduplication. By default, when hexdump encounters repeating 16-byte blocks, it suppresses them with an asterisk (*). In an entropy audit, an asterisk indicates identical, repeating data—a critical failure for cryptographic security. Passing -v ensures that every individual 16-byte block is explicitly printed.
• -n 64: Bounds the read to 64 bytes, preventing an endless stream from the character device.
• a4 72 e1 ... f8 62: Shows a healthy, non-patterned distribution of random hexadecimal values across all columns.

What the Admin Does Next

The audit confirms that /dev/urandom is producing varied, non-repeating entropy streams. The cryptographic hang is therefore isolated to an application-level lock contention rather than kernel entropy starvation, saving your team hours of chasing false leads in the operating system.

Tool Comparison: Hexdump vs. XXD vs. OD

Linux systems provide three core byte-level inspection tools: hexdump, xxd, and od (octal dump). Choosing the right tool depends on your specific debugging requirements and environment constraints:

• hexdump (Part of util-linux): Pre-installed across virtually all Linux distributions. Its format string engine (-e) allows you to unpack binary structs, network frames, and custom data types directly on the command line without writing custom code.
• xxd (Distributed with vim-common): Excels at bidirectional conversion. With the -r (reverse) flag, you can dump a binary file to text, edit the hexadecimal values in a text editor, and reassemble it into a modified binary: bash # Dump binary to editable hex text, edit, and reassemble: xxd file.bin > dump.hex # (modify dump.hex in text editor) xxd -r dump.hex patched_file.bin
• od (POSIX Coreutils): The official POSIX standard byte-inspection utility. It is guaranteed to exist on any UNIX-like system, but its default output uses octal words rather than hexadecimal, requiring specific flags (-t x1) for readable output.

Common Pitfalls and Safe Operating Practices

Working directly with raw binary data and storage devices carries specific operational risks. Here are the three most common hazards and how to avoid them:

1. Hexadecimal vs. Decimal Offset Confusion

When specifying seek offsets with -s, remember that hexdump defaults to interpreting bare integers as decimal, not hexadecimal. If documentation indicates that a header starts at offset 0x400, running hexdump -s 400 will seek to decimal byte 400 (hex offset 0x0190), missing the target data structure entirely.

# INCORRECT: Seeks to decimal 400 (hex 0x0190)
hexdump -s 400 -C /bin/ls

# CORRECT: Explicitly declare the hexadecimal prefix '0x'
hexdump -s 0x400 -C /bin/ls

Rule of thumb: Always prefix hexadecimal offsets with 0x, or use explicit multipliers such as 1024, 1k, or 1m.

2. Accidental Omission via Line Suppression Asterisks (*)

When hexdump encounters multiple identical 16-byte blocks, it suppresses them by default, printing a single asterisk (*). While convenient for saving terminal space, this can mask unexpected single-byte errors located in the middle of zero-padded blocks:

00000000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00001000  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

Rule of thumb: During security audits, cryptographic checks, or disk recovery, always add the -v (verbose) flag to guarantee that every single byte is printed.

3. Accidental Block Device Overwrite via Redirection

While hexdump is strictly a read-only inspection tool, engineers often combine it with shell pipelines. A typing error using shell output redirection (>) rather than input redirection (<) when inspecting raw block devices can instantly destroy partition tables and filesystem metadata.

# CATASTROPHIC ERROR: Instantly truncates /dev/sdb
hexdump -C /dev/sdb > /dev/sdb

The Safe Inspection Sandbox

When diagnosing live production storage, temporarily set the target block device to read-only in the Linux kernel before running your diagnostic commands:

# Set device to read-only mode in the Linux kernel
blockdev --setro /dev/sdb

# Execute diagnostics safely
hexdump -C -n 512 /dev/sdb

# Restore write access only when ready to perform remediation
blockdev --setrw /dev/sdb

Today's Takeaway

hexdump is the vital link between high-level software abstractions and physical, low-level data structures. In the next five minutes, open a terminal on your computer and run a safe, non-destructive canonical audit of your local user database metadata:

hexdump -v -n 64 -C /etc/passwd

Look at how the readable text aligns with its hexadecimal byte representations on the left. Make hexdump -C your instinctive first step whenever an application fails with a vague parsing error, when a configuration file refuses to load, or when a disk volume refuses to mount. Learning to inspect raw bytes directly gives you a clear, objective view of how data is truly stored and processed across your systems.

Authoritative Technical References & Documentation

• Linux Kernel Organization: Ext4 Filesystem Disk Layout Specification
• Linux Standard Man Pages: hexdump(1) Reference Manual
• Executable and Linkable Format (ELF) System V Application Binary Interface
• ArchWiki: Core Utilities and Binary Inspection Guide
• The Open Group Base Specifications: POSIX od (Octal Dump) Utility
• The util-linux Source Code Repository