I still struggle to get my 200k files (coll/index) read but as I understand now the “problem” I am trying to get it speed up as much as possible.
I’ve switched from a pd-standard to a pd-ssd (GKE). I would have expected a read performance of a few factors to be faster but at the end it got like 50% faster. As I discovered the disks in GCP have higher IOPS as the size increases. So I went from a 128B to a 512GB, where the measured IOPS (via fio) is like 10x more. And somehow surprised the startup time is the same. Also the CPU is in sleeping mode:
%Cpu7 : 20.6 us, 7.2 sy, 0.0 ni, 57.7 id, >>>10.3 wa<<<, 0.0 hi, 4.1 si, 0.0 st
Could it be that your distribution has not set the read ahead setting correctly ?
I am not able to execute (cant find sdb) → blockdev --getra /dev/sdb
Measurments with FIO in /data/db
Jobs: 1 (f=1): [m(1)][19.9%][r=1210KiB/s,w=407KiB/s][r=302,w=101 IOPS][eta 09m:39s]
Jobs: 1 (f=1): [m(1)][100.0%][r=11.3MiB/s,w=3844KiB/s][r=2884,w=961 IOPS][eta 00m:00s]
Jobs: 1 (f=1): [m(1)][100.0%][r=43.9MiB/s,w=14.8MiB/s][r=11.2k,w=3784 IOPS][eta 00m:00s]
=> I would expect that the read of the files would go much faster - which is not the case