Adjusting Round Robin IOPS limit from default 1000 to 1

These settings ensure optimal distribution and availability of load between I/O paths to the storage, which significantly improves performance on ESXI host due to high IO utilisation. It would recommend to use 4 HBA port along with this setting, it will share the load across 4 paths.

Symptoms

ESXi hosts using iSCSI/FC/FC/FCoE storage experiences latency issues with no signs of latency on the SAN side.

Refer below article for more details:
https://kb.vmware.com/kb/2069356

Steps to adjust the IOPS parameter

Its a  host based change, so you have to run this command on each ESXi host and against all the LUNs mapped to the host. You do not need to restart host for the changes to take effect. This recommendation should come from storage vendor.

1.      First thing to do is, change the datastore path selection policy to RR (from vSphere client – select host – configure – storage adapters – vmhba adapter – at bottom, right click the device and select manage paths – for path selection select Round Robin (VMware) and click change, closed).

2.      Login to ESXi using SSH.

To adjust the IOPS parameter from the default 1000 to 1, run this command:

for i in `esxcfg-scsidevs -c |awk '{print $1}' | grep naa.xxxx`; do esxcli storage nmp psp roundrobin deviceconfig set --type=iops --iops=1 --device=$i; done

where .xxxx matches the first few characters of your naa IDs.

To get LUN information, please run below ESXi CLI command in ESXi host.

#esxcfg-scsidevs -c

3. To verify if the changes are applied, run this command: 

esxcli storage nmp device list

You see output similar to:

Path Selection Policy: VMW_PSP_RR
Path Selection Policy Device Config: {policy=iops,iops=1,bytes=10485760,useANO=0;lastPathIndex=1: NumIOsPending=0,numBytesPending=0}
Path Selection Policy Device Custom Config:
Working Paths: vmhba33:C1:T4:L0, vmhba33:C0:T4:L0
Is Local SAS Device: false
Is Boot USB Device: false

                         --------------------End---------------