Dell PowerStore and VMware – NVMe/TCP Made Simple
A guest post by Jodey Hogeland
The primary purpose of this article is to answer a very simple question – “If you are a VMware vSphere customer, a Dell PowerStore customer, and you use iSCSI as a SAN connection protocol – do you know that you can switch to NVMe/TCP today?”
Approximately one year ago (October, 2021) Dell Technologies made a public splash about our commitment to driving NVMe/TCP within the industry. This idea of a standardized, high-performance connection methodology has garnered attention due to its simplicity – one of PowerStore’s key tenants. Simplification of storage management is a primary focus of Dell’s PowerStore product staff and a portion of that strategy revolves around connectivity like NVMe/TCP.
NVMe/TCP falls into the category of NVMe Over Fabrics, or NVMe-oF. There are now two primary methods for deploying NVMe-oF:
-
Fibre Channel – NVMe/FC – Using dedicated NVMe capable 16G and 32G fibre channel fabrics for SAN based host to storage connectivity.
-
Ethernet – a bit more broad with technologies like RoCE (RDMA over Converged Ethernet), iWARP (internet wide area RDMA protocol), and now, NVMe/TCP.
Where RoCE and iWARP can require dedicated hardware and network deployment methodologies, NVMe/TCP can use a standard ethernet network as is. This is a result of the only requirement being TCP/IP. Since TCP/IP is very flexible, so is NVMe/TCP. Leveraging TCP/IP – a standard ethernet protocol – there is no need for proprietary connectivity options like special NICs, HBAs, or network topologies. Also, NVMe/TCP is routable and enables edge and cloud based connection methodologies that have never been possible.
Simplicity
Why is it that NVMe/TCP has gained such a strong industry focus? Once again, it’s simple. Because NVMe/TCP can leverage existing ethernet infrastructure it is very simple to adopt. This is one of the reasons Dell Technologies is a industry leading pioneer in this space.
Considering that Dell Technologies has tens of thousands of customers and tens of thousands of storage arrays running mission critical customer environments, you can imagine that customers are leveraging multiple protocol connectivity options (FCP, iSCSI, SMB/NFS, NVMe-oF). With this market dominance, Dell is committed to bringing customers simplified, high performant solutions that enhance a customers connection between applications and infrastructure.
At the exact same time Dell was making announcements about NVMe/TCP availability with PowerStore (PowerStore OS 2.1 and higher), the worlds leading virtualization company was making announcements as well. VMware announced NVMe/TCP starting with vSphere 7.0 Update 3 which, in their words, “Allows vSphere customers a fast, simple and cost-effective way to get the most out of their existing storage investments.”
VMware highlighted their partnership with Dell, and specifically PowerStore, with a focus on several key aspects:
-
High performance storage connectivity – comparable to Fibre Channel at a fraction of the cost
-
Less complexity than existing solutions (simplicity)
-
Leverage existing investments in Ethernet
-
Automation of the SAN configuration process for accelerated operations and lower OPEX with SmartFabric Storage Software (again… simplicity)
What about performance?
With NVMe being known for performance and CPU efficiency, it hardly needs a mention that switching from traditional iSCSI deployment to NVMe/TCP shows tremendous benefits.
In his blog post on Software Defined Networking for Storage Connectivity, Ihab Tarazi (SVP/CTO) points to impressive results that VMware / PowerStore iSCSI customers could potentially realize immediately:
As mentioned previously, we decided to implement NVMe/TCP first because we felt it was the best “general purpose” IP based storage protocol available. We also noticed that it could provide a tremendous performance boost when compared to iSCSI on ESX. For example, initial internal testing has provided the following results:
-
IOPs – NVMe/TCP provided 2.5-3.5x the IOPs as iSCSI
-
Latency – NVMe/TCP reduced latency by 70-75%
-
CPU per IO – NVMe/TCP reduced CPU utilization by 40-50%
Here are a few other advantages NVMe/TCP has when compared to other connectivity options:
Again we ask the question in plain speak – If you are running vSphere 7.0u3 or higher, have a PowerStore array running PowerStore OS 2.1 or higher, and your are using iSCSI – why would you not unlock this simplistic, high performance capability at no additional cost?
Just how simple?
Just how simple is it to start leveraging NVMe/TCP? If you are using iSCSI today and you are running vSphere 7u3 or higher, then you can approach this in one of two ways. Which method you choose is up to you and what works best for your environment.
First, ensure that your PowerStore arrays is enabled to support NVMe/TCP by opening PowerStore Manager and navigating to SETTINGS > NETWORKING > Network IPs. From the Network IPs screen:
-
Select the STORAGE tab
-
Ensure that NVMe/TCP is enabled on the storage network. If not, select the MORE ACTIONS drop down and RECONFIGURE.
Above is the only setup required to enable NVMe/TCP on PowerStore. Now let’s move to the VMware side.
Option 1 – Unmap from iSCSI and Map to NVMe/TCP
Special shout out to my friend Ethan Stokes – Product Manager for PowerStore with a focus on NVMe/TCP. Ethan validated the steps in a lab and provided the screenshots below.
This method will require unmapping the volume/datastore and remapping.
-
Make sure the VMkernel is enabled for “NVMe over TCP”
2. Add Software NVMe over TCP adapter and select your Physical Network Adapter
3. Select the VMware NVMe over TCP Storage Adapter, select the Controllers tab, and click ADD CONTROLLER.
4. Enter the Global Storage Discover IP, set the Port Number to 8009, and DISCOVER CONTROLLERS. Select the targets you wish to connect to and click OK.
5. Create a new host on PowerStore. Specify your name, select the OS (ESXi), and click next. Select NVMe for the initiator type again and click NEXT.
6. On the Host Initiators page, you’ll see any auto-discovered initiators here. The ESXi nqn you just logged in will appear. Select it. You can also click ADD INITIATOR to do it manually. Click NEXT
7. Click ADD Host to complete
8. In vSphere, Power off any VMs on the Datastore
9. On PowerStore, unmap the volume from iSCSI host
10. Rescan storage in vSphere, the datastore will be inaccessible. This is expected.
11. On PowerStore, map the volume to the newly created NVMe/TCP version of the ESXi host
12. In vSphere, with your host selected, confirm the volume appears under Storage Adapters > VMware NVMe over TCP Storage Adapter > Namespaces tab:
13. Right click your host in vSphere and create a new Datastore. Select VMFS. Select your NVMe TCP Disk. Leave the name as is (it will change to the original name anyway).
-
On the mount option step, vSphere should detect a signature.
-
Leave the default option selected to keep existing signature, this will bring the datastore back into vSphere with the existing data.
You are now accessing the same VMFS formatted block volume on PowerStore over NVMe/TCP!
Option 2 – Create a new datastore and Storage vMotion
As an alternative to the unmap / remap option, you can also create a new NVMe/TCP host in Powerstore and create a new volume that will be used as a datastore. Map the NVMe host to the new volume and then create a new datastore on the newly created volume. You can then perform an online Storage vMotion of the VM’s from the iSCSI datastore to the new NVMe/TCP datastore.
Conclusion
Within a matter of minutes a preexisting iSCSI vSphere environment can begin to leverage the performance and efficiencies of NVMe/TCP. If you haven’t already, reach out to your Dell Technologies account team to discuss PowerStore and the simplified next generation capabilities that are available today!
The process listed above is common for customer using direct discovery methods for IQN’s and iSCSI. However, in a follow-on post I will detail NVMe/TCP simplicity at scale with Dell Technologies SmartFabric Storage Software (SFSS). This is another industry first capability that makes managing NVMe/TCP hosts at scale a very simple process. No more manual discovery and mapping! With SFSS you can maximize the benefits of NVMe/TCP and the Centralized Discovery Controller (CDC) capabilities of the protocol – something that was never possible with iSCSI.
More Information
For more details regarding NVMe/TCP and PowerStore:
-
NVMe/TCP on PowerStore with VMware YouTube video
