How to do Rapid VM Backup and Clone by Using Native Storage APIs?

Introduction

VMware supports native snapshot and clone technology at VM level where users can take snapshots or clones for a VM backup and fast VM provisioning by using the clone. In the vSphere UI, user can right click on VM and initiate a snapshot or clone operations.
There are also command line options available to initiate the snapshot of a VM. VMware provides option to revert to given snapshot through “Snapshot Manager” in case of any data corruption at VM level or if the user intentionally wants to revert VM to particular snapshot.
However, considering the technologies stack, this is the best offering by VMware, though performance degrades as we increase the size of VMs which eventually happens in enterprise DC, caveats, it can longer be instantaneous. How do we make VM backup or clone even faster in enterprise Datacenter deployment?
It is a well-known fact that snapshot and clone features are also offered by storage vendors though the granularity is at block or at file level depending on type of storage solution offerings.  By leveraging the storage snapshot and clone technologies for a VM backup, it is possible to increase the VM backup and clone performance.
Please refer to the “VMware Infrastructure” diagram below:

Some limitations of existing snapshot or clone offerings are:

• Hypervisor Layer
• Enterprise Server Layer
• Enterprise Network Layer
• Enterprise Storage Layer

Any snapshot or clone offering by hypervisors is at top of the technological stack, i.e. at Hypervisor Layer. This reduces the performance since each i/o needs to traverse the technology stack before committing to disk. What if we could bypass some layers of the stack or minimize the above technological stack? This could be achieved by taking advantage of the storage vendor’s snapshot and clone technologies.

Details

Storage vendors offer their snapshot and clone technologies solution and they are made available to end user via REST/SOAP SDKs. On leveraging the storage APIs, it is possible to take the snapshot/clone of a volume. Since it is volume level backup, the respective VMware APIs and Storage APIs could be used to amalgamate the VM, datastore and its associated volume, in order to achieve VM level backup or clone.

If we consider VMware technologies, VMs are made up of files *.vmdk,*.vmx , *.vswap etc and gets stored in data stores. Datastore is directly mapped to a Volume.

Using VMware APIs, you can get the file structure of VMs and its storage details like volume properties, host properties, storage details etc. Once this information is available, invoke native Storage APIs and initiate a snapshot of a volume. You then need to maintain the relationship between VM and volume snapshot and present this association to user.

Similarly, for clone, take the clone of a volume and present its VM clone to vSphere.
The above solution could be offered as:

1. Command line interface (CLI)
2. VMware UI Plugin

Design

VMware exposes a vSphere API as a web service, running on VMware vSphere server systems. The API provides access to vSphere management components that can be used to manage and control life-cycle operations of virtual machines. The APIs are made available via VMware vSphere Web Services SDK.

Storage vendors also expose their APIs for snapshot and clone operations and could be used for building any integration solutions. Thus by leveraging the VMware APIs and storage vendor’s APIs, the following solution is developed:

Plugin UI

This is a user visible component and sits inside the vSphere GUI from where user can list all the VMs and also request any snapshot or clone. Any user driven request comes to Plugin Server via REST APIs. It offers the following features to end user:

1. List view of VMs
2. Drop down menu option for snapshot and clone of a VM.
3. List view of snapshots of a VM

Plugin Server

This is a REST based server application which acts as client as well as server. It takes request from Plugin UI and acts as client for VMware vCenter Server and Storage Arrays. Primary responsibilities of this application is to process the Plugin UI request and invoke VMware vCenter Server APIs to get the necessary VM details, if the plugin server requests to take snapshot or clone, it further invokes Storage APIs and take snapshot or clone at volume level. The relationship between storage volumes, snapshot and VM is locally stored.

Conclusion

The performance of a snapshot or a clone of a VM is significantly increased due to the fact that it has not only minimized the technological stack but also by using the best native snapshot or clone technology offered by storage vendors.  The intention here is to give fair perspective of various redundancy features and technology solutions available which could be used in a manner to achieve desired end user performance results.