Dell EMC PowerProtect 19.9, Part 2, Protecting VMware Virtual Machines, Using Transparent Snapshots

A few months ago, I wrote a post about the performance impact of VMware Snapshots with vVOLs and VMFS datastores and explored these performance aspects of VM snapshot creation/deletion on the performance of guest applications running inside the VMs. In this post, I’ll introduce Transparent Snapshots, a feature within Dell EMC PowerProtect Data Manager 19.9, which changes the way we protect VMware virtual machines to ensure availability of all your VMs at scale without business disruption.
Before we start, if you are new to PowerProtect 19.9, I suggest you start reading about another feature, I blogged about here: Dell EMC PowerProtect 19.9, Part1 – Dynamic NAS Protection

Organizations are continuously expanding their virtualized environments, which are becoming the norm for hosting applications, and their associated VM datasets are constantly growing.
A natural outcome of increased adoption of virtualization is that virtualized environments host more mission-critical applications that cannot experience downtime. These applications demand high, transaction rates, stable IOPS, and low read/write latency during backup.
The technology traditionally used for backing up VMware assets is VMWare’s vStorage APIs for Data Protection – VADP. Though widely used, VADP-based backup solutions, and other similar solutions such as storage array snaps and journaling, introduce challenges that lead to application disruption, ultimately compromising day-to-day operation.
These challenges include:

Slow determination of deltas in the VM dataset, making the halt time for an application longer than desired, prior to each backup
Undesirable need for resource-intensive external agents to move VM snapshot data to secondary storage
Longer-than-acceptable application read and write latencies during backup
Overly impactful application IOPS reduction during backup

The result is “stunned” mission-critical applications, impacting business operations.
Even as VADP evolves to address some of its performance issues, such as by adding Change Block Tracking, VADP-based solutions continue to struggle with latency and IOPS issues due to their need for external data movers, outside the ESX host.
The answer is to modernize how you protect your VMs. Transparent Snapshots technology is a new VM image method that eliminates application disruption during VM backups, all without compromise.
Transparent Snapshots is a new feature available with Dell EMC PowerProtect Data Manager 19.9 that delivers unique VMware VM protection. Transparent Snapshots simplifies and automates VM image-level backups and enables backing up VMs WITHOUT the need to pause the VM during the backup process. The result is significantly reduced impact on VMs, especially large, high-change-rate VMs.
Additionally, the simplified backup process via Transparent Snapshots reduces infrastructure by removing the reliance on proxies for data movement.
The bottom line is that with Transparent Snapshots you can effectively and efficiently backup up your VMs via a process that requires fewer steps, leading to less impact to your entire VM environment, thus ensuring availability of all your VMs without business disruption.
So how does it work?
PowerProtect Data Manager uses Transparent Snapshots by using a variant of the vSphere API for I/O (VAI/O) Filtering framework with version 7.0u3. It uses an internal component (TSDM) that is created through the PowerProtect Data Manager VIB to write the copies to the protection storage (Dell EMC PowerProtect appliance).

During a restore, these same components use the backup copies. PowerProtect Data Manager assumes the role of an orchestrator where it identifies the VM assets in the VMware environment and provides scheduling capabilities.

PowerProtect Data Manager uses VM Direct Engine to communicate with the VMware vCenter level APIs provided by VMware. The VM Direct Engine connects to a component provided by VMware called Data Protection Service (DPS) which is a vCenter level service and has two key responsibilities in various scenarios:
- − Creates and tracks the progress of the vCenter level tasks that are visible to the end users such as sync, restore, and snapshot operations
- − Responsible for locating the relevant ESXi host on which the operation (backup or restore) is to be performed, based on the placement of the VM asset to be protected
On each ESXi host, there is a Data Protection Daemon (DPD) provided by VMware that facilitates the protection-related APIs and workflows from VMware.
The DPD also communicates to the Dell EMC Transparent Snapshot Data Mover (TSDM) component, which is responsible for the VM-backup data movement.
The backup and restore processes transfer the Transparent Snapshots respectively to and from the PowerProtect Appliance.
TSDM also consists of the PowerProtect Appliance SDK (DDBoost Library) which helps the framework access the storage units on the PowerProtect Appliance. It also helps write and read data from those storage units.

Look Mom! No vProxies!
Most prominent in the Transparent Snapshots method is the use of a new plug-in that embeds DD Boost on the ESX host, allowing direct data transfer to PowerProtect appliances without the need for external proxies to move either full-image or incremental data at backup time. The new plug-in is installed and deployed once on the ESX host itself as part of VM Image protection workflow.
There are three simple steps of the VM image backup lifecycle with Transparent Snapshots:

Change monitoring, which tracks the VM deltas since the last snapshot was taken
Snapshot processing, during which the VM deltas are accessed in memory. Data is then transferred directly to PowerProtect appliances
Snapshot release, at which time delta tables and temporary data blocks are removed.

Because it does not require external proxies or temporary copies to be used as part of the VM image backup lifecycle, Transparent Snapshots is superior to traditional backups at each step of the lifecycle.
Transparent Snapshots is a simple and efficient method to backup VMs and offers significant advantages.

Auto-scaling: An ESX-based plugin – which is a VMware certified VMware Installation Bundle (VIB) – gets deployed real-time, as needed, eliminating the need to use proxies (that is needed with traditional methods) and allowing for scaling by leveraging additional ESXi hosts where the data mover resides, versus needing manual intervention.
Continuous in-memory deltas: Monitors the deltas of full VMs, resulting in no impact to VM or ESX host – versus full real-time continuous data protection with traditional methods that requires extensive resources and does not scale for hundreds of VMs.
Storage agnostic: Transparent Snapshots deploys seamlessly into your existing storage environment. It does not require hardware-specific snapshots to be used in order to perform a single VM backup. Traditional methods utilize vendor-specific storage hardware that supports, and is specialized for, that storage type. Therefore, Transparent Snapshots eliminates the need for expensive, specialized storage. No new storage arrays or upgrades are required.
Direct data movement to PowerProtect appliances: Direct access to data eliminates the need for working copies. Traditional options require intermediary copies (for example, clones) that use additional compute and storage resources and impact backup processes.

VADP is the standard. It’s a low-cost snapshot method that is not optimized for large or performant VMs and requires proxies to move the data off the ESX.
Custom VM Tools is a way to help address the VM latency by impacting the performance of the ESX and adds the additional overhead of custom agents.
Storage Array Snaps leverages the built-in hardware snapshot capability to address the VM latency but is only available on Storage Array enabled appliances.
Journaling is another method for backing up the image but requires additional resources to monitor and track IO changes between backups. This solution is expensive to scale because of the increased resources.

With Transparent Snapshots, the benefits are huge:

Increase backup performance
- Near zero impact to VMs or environment
- Up to 5X faster backups
- Up to 5X reduction in VM latency
Lower costs & simplify management
- Zero proxies for data movement
- Storage agnostic
- Environment auto-scales via automation

In conclusion, Transparent Snapshots offers significant benefits to backup performance, costs and simplified management as well as reducing risk of data loss.
Backup performance is increased since there is no need to pause VMs during backup processes and the compute and network resources used to perform the backup are separate from the VMs. The result is near-zero impact to VMs or the environment. Restore performance is also faster and VM latency is reduced.
Transparent Snapshots does not require proxies for data movement, which lowers infrastructure, costs, and simplifies management. Transparent Snapshots is also storage agnostic, supporting NFS and vSAN, and automatically scales to support the number of ESX hosts you want to backup without additional impact to the host.
Here you can see a video comparing a VMware Snapshot with Transparent Snapshot and showing how to configure and run a Transparent Snapshot backup using PowerProtect Data Manager 19.9