Four Drives. Production Grade. Nothing Extra to Pay.
Storage Series — Part 2
Last week we established that EBS is a network device and that the wire has a cost — in latency, in user experience, and in your monthly bill. This week we look at what replaces it and why four NVMe drives in the right configuration do not just match EBS. They make EBS look like a budget option you are overpaying for.
Why ECC RAM is Not Optional
Before the drives, the memory.
ZFS is not like other filesystems. It maintains the integrity of your data from the moment it enters RAM to the moment it lands on disk. It checksums everything. It detects and corrects silent corruption. It is one of the most trustworthy storage stacks available on Linux.
But that trust has one hard dependency. If the data in RAM is silently corrupted by a memory error before ZFS can checksum it, ZFS will faithfully write corrupt data to disk and have no way of knowing. ECC RAM — Error Correcting Code memory — detects and corrects single-bit memory errors in hardware before they reach the filesystem. Without it, ZFS’s integrity guarantees are incomplete.
This is not a theoretical risk. It is a documented failure mode. ECC RAM is the foundation everything else sits on.
A Dell PowerEdge R750 with dual Xeon Gold processors and 128GB ECC RDIMM runs $3,500–5,000 refurbished. That is your platform. Everything after this is built on something you can trust.
The RAID-10 Architecture
Four Samsung 9100 PRO 8TB NVMe drives. Two mirrored pairs, striped across both.
In RAID-10 reads are served by whichever drive in the mirror responds first. Writes commit to both drives in each mirror simultaneously across two stripes. You get the redundancy of mirroring and the throughput of striping at the same time.
The numbers: sequential reads approaching 28,000 MB/s, random IOPS exceeding 4 million. Two drives can fail — one per mirror — and your data survives intact. Rebuild times on NVMe are measured in minutes, not the hours you wait watching a spinning rust array sweat through a resilver.
EBS gp3 caps at 1,000 MB/s and 16,000 IOPS. You cannot provision past those limits regardless of what you spend.
What ZFS Adds on Top
This is where the comparison stops being fair to EBS entirely.
Transparent compression. LZ4 compression is on by default, effectively free in CPU terms, and on typical database and application workloads it delivers 1.5–2x compression ratios. Your 16TB usable array starts behaving like 24–32TB in practice.
ARC — the Adaptive Replacement Cache. By default ZFS uses available RAM as an intelligent read cache, keeping hot data in memory and serving it without touching the drives at all. A server with 128GB ECC RAM has a very large ARC. A database with hot working sets fits largely in memory. Query latency drops to microseconds.
Snapshots. Instantaneous, space-efficient, no IO penalty at creation time. Point-in-time recovery without a backup window. Something you pay separately for in AWS and schedule around.
End-to-end checksumming. Silent corruption — the failure mode you never see coming — is detected and corrected automatically when you have a mirror to heal from.
None of this costs extra. It is the filesystem.
What You Have Built
A storage platform that outperforms EBS gp3 by an order of magnitude on throughput and IOPS, with built-in redundancy, compression, caching, snapshots, and data integrity guarantees — on hardware that paid for itself before the end of Q1.
Your users get the snappy experience that used to require premium spend. Your database gets microsecond local IO instead of network round trips. Your team gets a storage layer they can actually reason about.
EBS is a managed service. This is infrastructure you own. The difference shows up in your bill and in every interaction your users have with your application.
Next: the numbers that close the argument — a direct performance comparison between PostgreSQL on local NVMe RAID-10 and PostgreSQL on EBS gp3. Same queries. Same schema. Different latency by an order of magnitude.