You are not logged in.
I just got a new Kingston KC3000 2TB NVMe SSD for my computer. Reviews have listed it as among the fastest SSDs available, with sustained 6500 MB/s write speeds [1]
My machine only supports PCIe 3.0 [2] so I should be reaching ~3500 MB/s read/write. But with KDiskMark and O_DIRECT disabled (very few apps use O_DIRECT), write speeds are capped at 1100 MB/s (https://imgur.com/tsNQbmM).
I'm at a loss here. Is Linux I/O stack just hopelessly bad, that it cannot saturate an SSD at PCIe 3 rates without O_DIRECT?
And read speed does reach those levels, but write speed is nowhere near in real world.
According to UDisks benchmark, write speed only goes up to ~520 MB/s: https://imgur.com/CpmRPB1. At one point I even saw ~60 MB/s write at the beginning of the test, but it recovered mid-way in the test: https://imgur.com/eBBcN1M
Testing with "dd" via an ext4 file system gets better results, but still falls very short of expectations:
# dd if=/dev/zero of=./tempfile bs=1M count=8192 conv=fdatasync
8192+0 records in
8192+0 records out
8589934592 bytes (8,6 GB, 8,0 GiB) copied, 6,67135 s, 1,3 GB/sFinally I found KDiskMark (which uses "fio" backend). With its default settings (O_DIRECT ON), I can see the promised write up to 3500 MB/s: https://imgur.com/hnWPDaF \o/
But if I disable O_DIRECT, it's devastating, speed is capped at 1100 MB/s: https://imgur.com/tsNQbmM
O_DIRECT makes this a purely synthetic benchmark; besides a few databases, almost no applications use O_DIRECT.
These are things I tried... I ran the benchmarks after these:
I disabled PCIe/NVMe power management with "nvme_core.default_ps_max_latency_us=0 pcie_aspm=off" on the kernel command line.
I reformatted the new SSD with 4096 byte sectors.
Could not find any firmware updates for this drive.
This seems to have helped somewhat. Before these changes "dd" test gave me only ~700-800 MB/s. But UDisks results did not change.
But this is still deeply unsatisfcatory. Even testing with my years old worn down Intel 660p SSD, I can reach write speeds of 1,2 GB/s with the same "dd" test (!!).
Googling around, I found a bunch of similar reports from users, where read speeds are close to PCIe limits, but write speeds are around 600-700 MB/s:
[1] This graph shows ~6500 MB/s sequential write, sustained for nearly 110 seconds: https://cdn.mos.cms.futurecdn.net/YzH49 … 0.png.webp full review https://www.tomshardware.com/reviews/ki … renegade/2
[2] My motherboard is ASUS ROG STRIX B450-E GAMING. There is plenty of cooling, I'm using the motherboard-provided SSD heatsink in addition to the plate included with SSD.
Last edited by intgr (2024-02-12 17:50:39)
Offline
And how about hdparm benchmark: 'hdparm -Tt /dev/your_drive/partition_name' or 'hdparm -Tt --direct /dev/nvme0n1' ?
Offline
hdparm is only a read benchmark, the main issue here is write speed. But sure, why not...
# hdparm -Tt --direct /dev/nvme1n1p2
/dev/nvme1n1p2:
Timing O_DIRECT cached reads: 5846 MB in 2.00 seconds = 2924.68 MB/sec
Timing O_DIRECT disk reads: 9710 MB in 3.00 seconds = 3236.27 MB/sec
# hdparm -Tt /dev/nvme1n1p2
/dev/nvme1n1p2:
Timing cached reads: 25468 MB in 1.99 seconds = 12770.07 MB/sec
Timing buffered disk reads: 7106 MB in 3.00 seconds = 2368.08 MB/secOffline