Raw Device Mapping (RDM) in detail

RDM is mapping file used to map a LUN directly to VM bypassing VMFS layer. This LUN can be formatted using any File System (NTFS or FAT32) without the need to format it using VMFS and placing VMDK file on top of it.

It is method to provide direct access to a LUN on an iscsi or fibre channel storage system for a virtual machine. RDM is basically a Mapping file acts as a proxy for a raw physical storage device placed in a VMFS volume.

RDM is located with the virtual machine directory. Here is an example of RDM mapping file for a VM called VM-01 which has pRDM LUN and vRDM LUN (will cover them later in details).

~ #  cd /vmfs/volumes/SAN-DATASTORE/VM-01

/vmfs/volumes/507fd84d-e9b583ac-b370-001b244d6cb2/VM-01 # ls -la

drwxr-xr-x    1 root     root          2800 Nov  3 12:18 .

drwxr-xr-t    1 root     root          2100 Nov  2 21:33 ..

-rw-r--r--    1 root     root            73 Oct 20 10:21 VM-01-e61f5754.hlog

-rw-------    1 root     root     2147483648 Nov  3 11:39 VM-01-e61f5754.vswp

-rw-------    1 root     root     42949672960 Nov  3 12:49 VM-01-flat.vmdk

-rw-------    1 root     root          8684 Nov  3 11:40 VM-01.nvram

-rw-------    1 root     root           541 Nov  3 11:38 VM-01.vmdk

-rw-r--r--    1 root     root             0 Oct 19 15:10 VM-01.vmsd

-rwxr-xr-x    1 root     root          3679 Nov  3 12:18 VM-01.vmx

-rw-------    1 root     root             0 Oct 20 19:04 VM-01.vmx.lck

-rw-r--r--    1 root     root           260 Nov  3 12:18 VM-01.vmxf

-rwxr-xr-x    1 root     root          3680 Nov  3 12:18 VM-01.vmx~

-rw-------    1 root     root     32212254720 Nov  3 12:18 VM-01_1-rdm.vmdk ---------------------> vRDM mapping file

-rw-------    1 root     root           482 Nov  3 12:18 VM-01_1.vmdk ----------------------> Descriptor File

-rw-------    1 root     root     32212254720 Nov  3 12:18 VM-01_2-rdmp.vmdk ---------------------> pRDM mapping file

-rw-------    1 root     root           494 Nov  3 12:18 VM-01_2.vmdk ----------------------> Descriptor File

-rw-r--r--    1 root     root        579252 Oct 20 10:21 vmware-1.log

-rw-r--r--    1 root     root        139522 Oct 20 19:03 vmware-2.log

-rw-r--r--    1 root     root        307364 Nov  3 12:19 vmware.log

Types of Raw Device Mapping

1. Virtual compatibility mode:

·        All SCSI commands from the VM to LUN will pass through VMFS layer except READ & WRITE. Those will be passed directly to the LUN.

·        If you check the device manager of the VM you will see the RDM LUN listed as VMware Virtual Disk SCSI Disk Device. This is exactly the same as VMDK Drive added to the VM. The physical characteristics of the LUN are hidden from the VM.

·        It preserves the ability to perform virtual machine snapshots.

2. Physical compatibility mode:

All SCSI commands are passed to RDM LUN directly from the VM except REPORT command virtualized by VMkernal.

If you check the device manager of the VM you will see the RDM LUN listed based on the physical name/characteristics of the storage array.

The advantage of the full exposure of the LUN characteristics to the VM can be seen when using SAN Management Tools and MSCS services. Also, RDM LUNs in physical mode can have a size greater than 2TB.

Another important feature inherited with RDM is Dynamic Name Resolution. Each LUN is having unique ID in RDM mapping file used by VMFS. In case the path to the LUN modified due to failover, for example, the ID of the LUN will be change. Dynamic Name Resolution feature will automatically discover the new name and associate it in the RDM mapping file to resume operations.

RDM Restrictions

1. RDM can't be used with NFS datastore.
2. SVM can't be used with NPIV

Common task for the RDM

In case RDM is greyed out:

1. Make sure that you have a free LUN which support RDM 

2. Make sure that RDMFiler Advanced parameter is unchecked

Note: In case a LUN is used by ESXi host (even if partially), you can't use it as RDM to VM and vice versa. The reason is that OS or Hypervisor will apply lock into the LUN which will prevent it to be used by other system.

If you have a suitable controller, you can configure the local device as an RDM. Configuration of a local storage device as an RDM must be done using the Command Line Interface (CLI), it cannot be done through the vSphere Client.

To configure a local device as an RDM disk:

1.      Open an SSH session to the ESXi/ESX host.

2.      Run this command to list the disks that are attached to the ESXi host:

# ls -l /vmfs/devices/disks

3.      From the list, identify the local device you want to configure as an RDM and copy the device name.

Note: The device name is likely be prefixed with t10. and look similar to: 

t10.F405E46494C4540046F455B64787D285941707D203F45765

4.      To configure the device as an RDM and output the RDM pointer file to your chosen destination, run this command:

# vmkfstools -z /vmfs/devices/disks/diskname /vmfs/volumes/datastorename/vmfolder/vmname.vmdk

For example:

# vmkfstools -z /vmfs/devices/disks/t10.F405E46494C4540046F455B64787D285941707D203F45765 /vmfs/volumes/Datastore2/localrdm1/localrdm1.vmdk

Note: The size of the newly created RDM pointer file appears to be the same size and the Raw Device it it mapped to, this is a dummy file and is not consuming any storage space.

5.      When you have created the RDM pointer file, attach the RDM to a virtual machine using the vSphere Client:

a.      Right click the virtual machine you want to add an RDM disk to.

b.      Click Edit Settings.

c.      Click Add.

d.      Select Hard Disk.

e.      Select Use an existing virtual disk.

f.       Browse to the directory you saved the RDM pointer to in step 5 and select the RDM pointer file and click Next.

g.      Select the virtual SCSI controller you want to attach the disk to and click Next.

h.      Click Finish.

6.      You should now see your new hard disk in the virtual machine inventory as Mapped Raw LUN.

• Notes: As this virtual machine now has an attached local disk migration, using vMotion is not possible.

To list all virtual disks pointing to an RDM device using PowerCLI:

This operation is generally time-consuming, as PowerCLI must iteratively inquire about the disk type of every VMDK file on the remote hosts. 

1.      Open the vSphere PowerCLI command-line. For more information, see the vSphere PowerCLI Documentation.

2.      Run the command:

Get-Datastore | Get-HardDisk -DiskType "RawPhysical","RawVirtual" | Select "Filename","CapacityKB" | fl

Example output:

Filename    : [DatastoreName] DirectoryName/virtualrdm.vmdk
CapacityKB  : 5760

To list all virtual disks pointing to an RDM device using the local console:

This operation is generally quick, as there is no overhead of network communication.

1.      Open a console to the ESX or ESXi host.

2.      Run the command:

#find /vmfs/volumes/ -type f -name '*.vmdk' -size -1024k -exec grep -l '^createType=.*RawDeviceMap' {} \; > /tmp/rdmsluns.txt

#for i in `cat /tmp/rdmsluns.txt`; do vmkfstools -q $i; done

Example output:

o   Virtual Mode RDM:

Disk /vmfs/volumes/.../virtualrdm.vmdk is a Non-passthrough Raw Device Mapping
Maps to: vml.02000000006006048000019030091953303030313253594d4d4554

o   Physical Mode RDM:

Disk /vmfs/volumes/.../physicalrdm.vmdk is a Passthrough Raw Device Mapping
Maps to: vml.02000000006006048000019030091953303030313253594d4d4554

Thanks for Reading !!!!