vSAN Lab

Hello,

I'm configuring a new home lab based on 3x Supermicro Superserver E300-9D-4CN8TP with the following config:

Intel Xeon D-2123IT

2x SFP+ 10GbE LAN

128 GB RAM

1x SATADOM 64GB (boot)

1x M.2 Samsung 970 Pro NVMe 500GB (cache)

1x M.2 WD 2 TB SATA SSD WDS200T2B0B (capacity)

vCenter Server Appliance 6.7.0.21000

ESXi -> VMware ESXi, 6.7.0, 11675023

I've configured vSAN (no manual adjustments or modifications, just turned on vSAN and claimed disks) and made some perf tests with HCI bench 1.6.8.7 (easy run) and I get results like these:

1-vdb-8vmdk-100ws-4k-70rdpct-100randompct-4threads-1551300003-res.txt

Datastore: Compute-vsanDatastore

=============================

Version: vdbench50407

Run Def: RD=run1; I/O rate: Uncontrolled MAX; elapsed=3600 warmup=1800; For loops: None

VMs = 6

IOPS = 59242.30 IO/s

THROUGHPUT = 231.41 MB/s

LATENCY = 3.2033 ms

R_LATENCY = 3.6867 ms

W_LATENCY = 2.0755 ms

95%tile_LAT = 10.1144 ms

=============================

Resource Usage:

CPU USAGE = 93.05%

RAM USAGE = 20.19%

VSAN PCPU USAGE = 45.3539%

=============================

2-vdb-8vmdk-100ws-4k-100rdpct-100randompct-4threads-1551309960-res.txt

Datastore: Compute-vsanDatastore

=============================

Version: vdbench50407

Run Def: RD=run1; I/O rate: Uncontrolled MAX; elapsed=3600 warmup=1800; For loops: None

VMs = 6

IOPS = 73090.60 IO/s

THROUGHPUT = 285.51 MB/s

LATENCY = 2.6077 ms

R_LATENCY = 2.6077 ms

W_LATENCY = 0.0000 ms

95%tile_LAT = 6.0918 ms

=============================

Resource Usage:

CPU USAGE = 97.47%

RAM USAGE = 20.87%

VSAN PCPU USAGE = 44.7564%

=============================

3-vdb-8vmdk-100ws-256k-0rdpct-0randompct-1threads-1551319607-res.txt

Datastore: Compute-vsanDatastore

=============================

Version: vdbench50407

Run Def: RD=run1; I/O rate: Uncontrolled MAX; elapsed=3600 warmup=1800; For loops: None

VMs = 6

IOPS = 7800.10 IO/s

THROUGHPUT = 1950.00 MB/s

LATENCY = 6.6960 ms

R_LATENCY = 0.0000 ms

W_LATENCY = 6.6960 ms

95%tile_LAT = 23.3727 ms

=============================

Resource Usage:

CPU USAGE = 82.94%

RAM USAGE = 22.2%

VSAN PCPU USAGE = 40.9864%

=============================

I was thinking if those results are "good" or not.. I see that Samsung 970 pro should be capable of:

• 500.000 IOPS RANDOM WRITE (4KB, QD32),

• RANDOM READ (4KB, QD32)

  512 GB: Up to 370,000 IOPS
  1,024 GB: Up to 500,000 IOPS

• RANDOM READ (4KB, QD1)

  Up to 15,000 IOPS

• RANDOM WRITE (4KB, QD1)

          Up to 55,000 IOPS

( source: Samsung SSD 970 PRO | Samsung V-NAND Consumer SSD | Samsung Semiconductor Global Website )

Also, looking on some blogs (for example https://www.virtualizationhowto.com/2017/01/samsung-960-evo-m-2-1tb-nvme-in-vmware-home-lab/  )   seems that even with the 960 much higher results can be achieved.. ( >200000 IOPS)

I've started to check basic conf (like bios, firmware etc ) and all seems correctly configured as per vendor best practices. Also I've tested the network connections and looks performing quite well ( 5]   0.00-10.11  sec  11.5 GBytes  9.75 Gbits/sec receiver).

Any ideas for those results?

Thanks you all for any input!

Message was edited by: iNik26