How to Benchmark and Optimize Disk Performance on Linux with fio

Linux disk performance benchmark with fio

Learn how to benchmark and optimize disk performance on Linux using fio. This comprehensive guide covers installation, test scenarios, interpreting results, and tuning tips to maximize throughput and minimize latency.

Table of Contents

🔈Introduction

When you’re managing Linux systems—whether they’re servers, virtual machines, or embedded devices—understanding your disk performance is critical. The versatile utility fio (Flexible I/O tester) offers a comprehensive way to benchmark and optimize storage throughput, latency, and IOPS. In this guide we’ll walk through a structured approach to benchmarking disk performance with fio, interpreting results, and applying optimization techniques to get the most out of your disk subsystem.

✅ Why Benchmark Disk Performance?

Before diving into commands, it’s worth asking: why use fio, and why benchmark at all?

  • Typical tools like dd may only test sequential throughput with minimal queue depths, giving an optimistic or unrealistic view of performance. 
  • Real-world workloads often involve random I/O, concurrent threads, deep queues, and latency sensitivity—scenarios fio can simulate.
  • Benchmarking helps you:
    • Validate claims from hardware vendors
    • Identify bottlenecks or misconfigured disks/file systems
    • Track performance changes over time (after upgrades, filesystem moves, RAID changes)
    • Align storage configuration (RAID, NVMe, SSD vs HDD) to workload requirements.

📤 Getting Started with fio

🟢 Install fio

On Debian/Ubuntu:

				
					sudo apt update
				
			
				
					sudo apt install –y fio

On RHEL/CentOS:

				
					sudo yum install -y fio

On RHEL/CentOS version 8 or higher:

				
					sudo dnf install -y fio

fio is available in standard repositories.

🟢 Choose the Target Device or Directory

Decide whether to test a raw block device (e.g., /dev/nvme0n1) or a mounted filesystem (e.g., /mnt/data).

  • ✅ Testing a raw device eliminates filesystem and cache overhead, giving hardware-level performance.
  • ✅ Testing a filesystem gives realistic performance of your actual workload.
💡NOTE: Use fio to benchmark either the block device or a mounted filesystem.

🟢 Define Test Parameters

Key parameters you’ll want to specify in your fio command or job file:

ParameterPurposeExample
--rwType of I/O: read, write, randread, randwrite, randrw--rw=randread
--bsBlock size for each I/O operation--bs=4k, --bs=1M
--iodepthQueue depth (number of I/Os in flight)--iodepth=32
--numjobsNumber of threads/processes--numjobs=4
--direct=1Use direct I/O (bypass OS cache)--direct=1
--time_basedRun for a specific runtime rather than size--time_based --runtime=300
--sizeAmount of data to operate on--size=10G
--ioengineMethod of issuing I/O (libaio, posixaio…)--ioengine=libaio

For example, to measure random reads:

				
					sudo fio --filename=/mnt/testfile --direct=1 --ioengine=libaio --bs=4k --iodepth=64 --rw=randread --time_based --runtime=300 --group_reporting --name=rand_read_test

This approach is consistent with cloud-vendor guidance.

✅ Recommended Benchmark Scenarios

Here are a few key test cases to run regularly. Use them as templates, and adjust sizes or depths based on your hardware.

Test nameDescriptionSample command
Sequential ReadLarge block reads (e.g., backup)fio --filename=... --direct=1 --ioengine=libaio --bs=1M --iodepth=64 --rw=read --time_based --runtime=300 --group_reporting --name=seq_read
Sequential WriteLarge block writes... --bs=1M --iodepth=64 --rw=write --name=seq_write
Random Read IOPS4 K random reads (database)... --bs=4k --iodepth=256 --rw=randread --name=rand_read_iops 
Random Write IOPS4 K random writes... --bs=4k --iodepth=256 --rw=randwrite --name=rand_write_iops
💡NOTE: When interpreting results, the key numbers you’ll see in the output include: BW (bandwidth), IOPS, latency (clat/slat), and utilization. One practical example: “random-read …bw=864KiB/s, iops=211”

📤 Interpreting the Results

After running the test, fio will display an output summary. Here’s what to focus on:

  • Bandwidth (BW): Shows throughput in MB/s.
  • IOPS: Important for random I/O workloads (small block sizes).
  • Latency (clat/slat): Indicates responsiveness. High latency may impact application performance more than raw throughput.
  • Utilization and queue depth: If your device is near 100% utilization for extended periods, the workload is saturating the disk.

🟢 Example interpretation

Suppose you ran a 4 K random read test and got 70 k IOPS but latencies over 4 ms. If the application expects sub-millisecond responses, this may not be acceptable—even though the IOPS number looks high.

🔄 Optimizing Disk Performance Based on Results

Once you’ve benchmarked and interpreted your results, you can apply optimizations. Here are some areas to review:

🔹Ensure correct alignment and file system settings

Misalignment of partitions (especially on older hardware) can degrade performance. For example, start sectors not aligned to 4 KiB boundaries may cause penalty. Use tools like fdisk -lu or lsblk -t to inspect alignments.

🔹Adjust I/O scheduler and queue depth

On modern NVMe or high-performance SSDs, using scheduler none or mq-deadline often yields better results than legacy CFQ. Experiment with echo none > /sys/block/<device>/queue/scheduler or use nvme specific tuning.

🔹Use appropriate filesystem options

Choose a filesystem and mount options aligned with your workload (e.g., noatime, appropriate inode size). Also ensure that if you’re testing raw devices you use direct=1 to bypass caches.

🔹Match your benchmark to expected workload

It’s pointless to benchmark with a huge queue depth if your production workload uses depth = 1. Tailor iodepth, numjobs, bs, and rw to match what your applications actually do. Many users on the web highlight mismatches causing misleading “great” results.

🔹RAID / virtualization / caching layers

If you have underlying RAID arrays, virtualization, caching layers (like LVM cache, ZFS cache, etc), your results will reflect that stack—not just raw disk performance. Example: “while using fio I’ve run into … got write 20,000 MiB/s” when it was hitting cache rather than real disk. If you suspect caches, you may want to invalidate caches, mount fresh, or use disks with clean state.

🔹Re-run benchmarks when things change

Any time you change hardware (SSD model, RAID level), firmware update, kernel, or filesystem, re-benchmark to validate the new configuration. Trend your results over time to detect degradation or anomalies (e.g., wear on SSDs, firmware bugs).

📊 Benchmark Report Table

Here’s an example of how you might record results from different tests:

TestBlock sizeDepthBW (MB/s)IOPSLatency (avg)Comment
Seq Read1 M641200—0.5 msGood throughput
Seq Write1 M64950—1.0 msAcceptable write speed
Rand Read (4 K)4 K256—75,0000.8 msMeets database requirement
Rand Write (4 K)4 K256—55,0001.2 msSlightly high latency

Using a table like this simplifies comparison across hardware, configurations, or time.

🏆 Caveats and Best Practices

  • Avoid testing on a live production volume unless you know the workload and can absorb the impact. Better to use an unused mount point or device. Cloud providers also caution that benchmarking active disks may not reflect true limits.
  • Be careful with caches: If your test writes are hitting cache rather than disk, you’ll get inflated numbers that don’t reflect sustained performance.
  • Use realistic test sizes: If you only test with small data (say 256 MB) on a huge SSD, you might only see cache-performance, not steady-state.
  • Interpret latency as well as throughput: Some workloads are more sensitive to tail latency than raw MB/s.
  • Document your environment: CPU, RAM, filesystem, queue depth, RAID level, SSD model—all matter for reproducibility and comparison.

🔚 Wrapping It Up

Using fio to benchmark and optimize disk performance on Linux gives you actionable insight. The process in summary:

  • ✅ Install fio.
  • ✅ Select target device or filesystem, be clear about what you’re testing.
  • ✅ Run well-defined tests (sequential, random, read, write) with parameters aligned to your workload.
  • ✅ Record bandwidth, IOPS, latency, utilization.
  • ✅ Analyze results, compare to expectations, identify bottlenecks.
  • ✅ Tune system (alignment, scheduler, filesystem options, queue depth) and rerun to validate improvement.
  • ✅ Periodically repeat benchmark after changes to hardware or configuration to maintain performance visibility.

By adopting this systematic approach, you can make data -driven decisions about storage upgrades, workload matching, and disk configuration on Linux. Whether you’re optimizing a database server, a VM cluster, or an embedded device, fio offers the flexibility to test and verify performance with confidence.

Did you find this article helpful? Your feedback is invaluable to us! Feel free to share this post with those who may benefit, and let us know your thoughts in the comments section below.

📕 Related Posts
Tags