What Is Unique About Oracle Cloud VMware Solution?

What Is Unique About Oracle Cloud VMware Solution?

Everyone talks about multi-cloud and in most cases they mean the so-called big 3 that consist of Amazon Web Services (AWS), Microsoft Azure and Google Cloud. If we are looking at the 2021 Gartner Magic Quadrant for Cloud Infrastructure & Platform Services, one can also spot Alibaba Cloud, Oracle, IBM and Tencent Cloud.

VMware has a strategic partnership with 6 of these hyperscalers and all of these 6 public clouds offer VMware’s software-defined data center (SDDC) stack on top of their global infrastructure:

While I mostly have to talk about AWS, AVS and GCVE, I am finally getting the chance to attend a OCVS customer workshop led by Oracle. That is why I wanted to prepare myself accordingly and share my learnings with you.

Amazon Web Services, Microsoft Azure and Google Cloud dominate the cloud market, but Oracle has unique capabilities and characteristics that no one else can deliver. Additionally, Oracle’s Cloud Infrastructure (OCI) has shown an impressive pace of innovation in the past two years, which led to a 16% increase on Gartner’s solution scorecard for OCI (November 2021, from 62% to 78%), which put them into the fourth place behind Alibaba Cloud!

What is Oracle Cloud VMware Solution?

Oracle Cloud VMware Solution or OCVS is a result of the strategic partnership announced by VMware and Oracle in September 2019. Like the other VMware Cloud solutions like VMC on AWS, AVS or GCVE, Oracle Cloud VMware Solution will enable customers to run VMware Cloud Foundation on Oracle’s Generation 2 Cloud Infrastructure.

Meaning, running an on-premises VMware-based infrastructure combined with OCVS should make cloud migrations easier and faster, because it is the same foundation with vSphere, vSAN and NSX.

Oracle Cloud VMware Solution Key Differentiator #1 – Different SDDC Bundles

Customers can choose between a multi-host SDDC (minimum of 3 production hosts) and a single-host SDDC, that is made for test and dev environments. Oracle guarantees a monthly uptime percentage of at least 99.9% for the OCVS service.

OCVS offers three different ESXi software versions and supports the following versions of other components:

  • ESXi 7.0, 6.7 or 6.5
  • vCenter 7.0, 6.7 or 6.5
  • vSAN 7.0, 6.7 or 6.5
  • NSX-T 3.0
  • HCX Advanced 4.0, 3.5 (default option)
  • HCX Enterprise (billed upgrade)

Note: vSphere 6.5 and vSphere 6.7 reach the End of General Support from VMware on October 15, 2022.

Key Differentiator #2 – Customer-Managed & Baremetal Hosts

The VMware Cloud offerings from AWS, Azure or Google are all vendor-controlled and customers get limited access to the VMware hosts and infrastructure components. With Oracle Cloud VMware Solution, customers get baremetal servers and the same operational experience as on-premises. This means full control over VMware infrastructure and its components:

  • SSH access to ESXi
  • Edit vSAN cluster settings
  • Browse datastores; upload and delete files
  • Customer controls the upgrade policy (version, time, defer)
  • Oracle has NO ACCESS after the SDDC provisioning!

Note: According to Oracle it takes about 2 hours to deploy a new SDDC that consists of 3 production hosts.

Customers can choose between Intel- and AMD-based hosts:

  • Two-socket BM.DenseIO2.52 with two CPUs each running 26 cores (Intel)
  • Two-socket BM.DenselO.E4.128 with two CPUs each running 16 cores (AMD)
  • Two-socket BM.DenselO.E4.128 with two CPUs each running 32 cores (AMD)
  • Two-socket BM.DenselO.E4.128 with two CPUs each running 64 cores (AMD)

Details about the compute shapes can be found here.

Key Differentiator #3 – Availability Domains

To provide high throughput and low latency, an OCVS SDDC is deployed by default across a minimum of three fault domains within a single availability domain in a region. But, upon request it is also possible to deploy your SDDC across multiple availability domains (AD), which comes with a few limitations:

  • While OCVS can scale from 3 up to 64 hosts in a single SDDC, Oracle recommends a maximum of 16 ESXi hosts in a multi-AD architecture
  • This architecture can have impacts on vSAN storage synchronization, and rebuild and resync times

Most hyperscaler only let you use two availability zones and fault domains in the same region. With Oracle it is possible to distribute the minimum of 3 hosts to 3 different availability domains.  An availability domain consists of one or more data centers within the same region.

Note: Traffic between ADs within a region is free of charge.

Key Differentiator #4 – Networking

Because OCVS is customer-managed and can be operated like your on-premises environment, you also get “full” control over the network. OCVS is installed within a customers’ tencancy, which gives customer the advantage to run their VMware SDDC workloads in the same subnet as OCI native services. This provides lower latency to the OCI native services, especially for customers that are using Exadata for example.

Another important advantage of this architecture is capability to create VLAN-backed port groups on your vSphere Distributed Switch (VDS).

Key Differentiator #5 – External Storage

Since March 2022 the OCI File Storage service (NFS) is certified as secondary storage for an OCVS cluster. This allows customers to scale the storage layer of the SDDC without adding new compute resources at the same time.

And just announced on 22 August 2022, with Oracle’s summer ’22 release, OCVS customers can now connect to a certified OCI Block Storage through iSCSI as a second external storage option.

Block Storage provides high IOPS to OCI, and data is stored redundantly across storage servers with built-in repair mechanisms with a 99.99% uptime SLA.

Key Differentiator #6 – Billing Options

OCVS is currently only sold and supported by Oracle. Like with other cloud providers and VMware Cloud offerings, customers have different pricing options depending upon their commitment levels:

  • On-demand (hourly)
  • 1 month
  • 1 year
  • 3 years

The rule of thumb for any hyperscaler says, that a 1-year commitment get around 30% discount and the 3-year commitments are around 50% discount.

The unique characteristic here is the monthly commitment option, which is caluclated with a discount of 16-17% depending on the compute shape.

Note: OCVS is not part (yet) of the VMware Cloud Universal subscription (VMCU).

Key Differentiator #7 – Global Reach

Currently, OCI is available in 39 different cloud regions (21 countries) and Oracle announced five more by the end of 2022. On day one of each region, OCVS is available with a consistent and predictable pricing that doesn’t vary from region to region.

To compare: AWS has launched 27 different regions with 19 being able to host the VMware Cloud on AWS service. In Switzerland, AWS just opened their new data center without having the VMware Cloud on AWS service available, while OCVS is already available in Zurich.

Use Cases

While OCVS is a great solution for joint VMware and Oracle customers, it is not necessary for customers to using Oracle Cloud Infrastructure native solutions.

Data Center Expansion

As you just learned before, OCVS is a great fit if you want to maintain the same VMware software versions on-premises and in OCI. The classic use case here is the pure data center expansion scenario, which allows you to stretch your on-premises infrastructure to OCI, without the need to use their native services.

VMware Horizon on OCVS

As I mentioned at the beginning, Oracle Cloud VMware Solution is based on VMware Cloud Foundation and so it is no surprise that Horizon on OCVS is fully supported.

The Horizon deployment on OCVS works a little bit different compared to the on-premises installation and there is no feature parity yet:

  • Horizon on OCVS does not support vGPUs yet.
  • Horizon on OCVS does not support IPv6 yet.
  • Horizon on OCVS does not support vTPM yet. In this situation it is recommended to use shielded OCVS instances.

Note: The support of NSX Advanced Load Balancer (Avi) is still a roadmap item

VMware Tanzu for OCVS

Since April 2022 it is possible for joint VMware and Oracle customers to use Tanzu Standard and its components with Oracle Cloud VMware Solution. Tanzu Standard comes with VMware’s Kubernetes distribution Tanzu Kubernetes Grid (TKG) and Tanzu Mission Control, which is the right solution for multi-cloud, multi-cluster K8s management.

With TMC you can deploy and manage TKG clusters on vSphere on-premises or on Oracle Cloud VMware Solution. You can even attach existing Kubernetes clusters from other vendors like RedHat OpenShift, Amazon EKS or Azure Kubernetes Service (AKS).

OCVS Tanzu Standard 

Oracle Cloud VMware Solution FAQ

VMware’s OCVS FAQ can be found here.

Oracle’s OCVS FAQ can be found here.

Additional Resources

Here is a list of additional resources:

VMware Cloud on AWS – The Power of VMware and AWS

VMware Cloud on AWS – The Power of VMware and AWS

VMware Cloud on AWS (VMC on AWS) brings VMware’s software-defined data center (SDDC) stack to the AWS cloud. By using the same vSphere-based virtualization/cloud technology on-premises and in the public cloud, you can create a true hybrid cloud architecture, that enables you to get consistent operations by using consistent infrastructure.

VMC on AWS Overview

This solution comes with optimized access to the AWS services and is delivered, sold and supported by VMware, AWS and their partner networks.

As you can see above, VMC on AWS comes with the same VMware tools and integrates the VMware Cloud Foundation stack (vSphere for compute, vSAN for storage, NSX for networking) along with vCenter for management.

VMware Cloud on AWS runs on dedicated Amazon EC2 bare-metal infrastructure.

Instance Types

VMware Cloud on AWS comes with two different host configurations, which both require a minimum of two hosts per cluster.

VMC on AWS Instances

For identifying the right host types for specific use cases, check out the VMware Cloud on AWS sizer.

Note: 99.9% SLA for non-stretched clusters, 99.99% for stretched clusters

Single Host Starter Configuration

VMC on AWS allows you to deploy a starter configuration with a single host only (not available with i3en.metal hosts).

This small SDDC configuration allows customers to get their first experiences with this hybrid cloud offering during a 60-day time period. Such a setup is only appropriate for test and development or proof of concept use cases. You can run production workloads on this small VMC on AWS environment if you scale up to the minimum of two hosts before the 60-day period ends, otherwise your evaluation ends with you losing data.

Note: Not all features of the standard VMC service offering are available in this limited setting. The VMC on AWS service level offering also does not apply to this one-node offering.

Included VMware Software

The following software is included in single host and production configurations:

Single Hosts (non-production environments) Production (minimum 2 hosts)

Includes

  • VMware SDDC software: vSphere, vSAN, NSX-T, vCenter Server
  • VMware HCX
  • Dedicated Amazon EC2 Bare Metal Instances
  • VMware Global Support

Purchase separately

  • VMware Site Recovery
  • VMware Cloud Disaster Recovery
  • VMware vRealize Automation Cloud
  • VMware vRealize Operations Cloud
  • VMware vRealize Log Insight Cloud
  • VMware vRealize Network Insight Cloud
  • VMware Tanzu Standard

Not supported

  • Lifecycle management by VMware (updates, patches and upgrades)
  • High Availability (HA) and Stretched Clusters
  • Service Level Agreement (SLA)

Includes

  • VMware SDDC software: vSphere, vSAN, NSX-T, vCenter Server
  • VMware HCX
  • VMware Tanzu Services: TKG Service + TMC Essentials
  • Dedicated Amazon EC2 Bare Metal Instances
  • VMware Global Support
  • Lifecycle management by VMware (updates, patches and upgrades)
  • Support for High Availability (HA) and Stretched Clusters
  • Service Level Agreement (SLA)

Purchase separately

  • VMware Site Recovery
  • VMware Cloud Disaster Recovery
  • VMware NSX Advanced Firewall
  • VMware vRealize Automation Cloud
  • VMware vRealize Operations Cloud
  • VMware vRealize Log Insight Cloud
  • VMware vRealize Network Insight Cloud
  • VMware Tanzu Standard

VMware Cloud on AWS Outposts

If you want to get the agility and innovation of (VMware) Cloud in your own data center, delivered as a service, then VMC on AWS Outposts is for you.

VMC on AWS Outposts is a fully managed on-premises as-a-service offering, that stretches VMC on AWS to your data center or edge location. You’ll get dedicated Amazon Nitro-based EC2 bare-metal instances delivered on-premises with VMware Cloud Foundation running on top.

VMC on AWS Outposts

What’s included in the offering?

  • AWS Outposts 42u rack (we can also expect a half-rack offering in the future)
    • 3-8 hosts configurations based on i3en.metal
    • Dark host capacity included (for remediation, EDRS, scale-out and lifecycle management purposes)
    • Installed by AWS
  • AWS managed dedicated Nitro-based i3en.metal EC2 instance with local SSD storage
  • VMware managed SDDC software – vSphere, vSAN, NSX-T, vCenter Server
  • VMware HCX
  • VMware Cloud Console
  • Support by VMware SREs
  • Supply chain, shipment logistics and onsite installation by AWS
  • Ongoing hardware monitoring with break/fix support.

Use Cases

VMware Cloud on AWS Outposts is made for multiple use cases:

  • Data/App Locality
  • Low latency
  • Local data processing
  • Data sovereignty/compliance
  • Infrastructure modernization
  • Branche Office or large edge modernization

But this offering and VMC on AWS in general come with multiple other use cases which help orgnaizations to fulfill their cloud strategy.

App Modernization

VMware Cloud on AWS provides an infrastructure platform option for customers to modernize their existing enterprise applications on and enables them to run their enterprise workloads of today and tomorrow. With VMware Cloud on AWS, customers can run, monitor, and manage their Kubernetes clusters and virtual machines – all on the same infrastructure. VMware Tanzu Kubernetes Grid provides a consistent, upstream-compatible distribution of Kubernetes, that is tested, signed, and supported by VMware. Tanzu Kubernetes Grid is central to many of the offerings in the VMware Tanzu portfolio.

Solution Brief

Cloud Migration / Data Center Extension

VMC on AWS can help customers to expand to new locations. Maybe it’s an unplanned project or there are temporary or seasonal capacity needs. Some customers are also using such an offering to build a flexible test, lab or training environment in the public cloud.

Solution Brief

Cloud VDI

Adopt a robust, feature-rich cloud platform for virtual desktops and applications that can be used to deliver complete VDI infrastructure from the cloud. Or you can extend an existing on-premises VDI environment for desktop bursting, protection or proximity to applications running in AWS. Optimize infrastructure costs with flexible, consumption-based billing while paying only for what you use.

Solution Brief

Disaster Recovery

Another typical use case is disaster recovery. Customers are looking for an offsite approach with which they can prepare themselves for different kind of scenarios with “warm standby” or “active/active” configurations. There are different architectural options and also different solutions from VMware available, e.g.:

Hybrid Cloud Extension (HCX)

How can you bridge the gap between on-premises data centers and VMC on AWS to enable application migrations or workload mobility? HCX creates an encrypted, high-throughput, WAN-optimized, load-balanced, traffic-engineered hybrid interconnect automates the creation of network extensions.

In short: VMware HCX can interconnect different vSphere-based clouds and with that you achieve a fabric for workload mobility by using vMotion over different clouds. It even preserves existing network connections!

Imagine how much easier and faster application migrations can be done now.

Let’s see if there is a future, that customers need full workload mobility where regular migrations from and to different clouds can be done. Maybe there is a customer, who migrates workloads today from on-prem to VMC on AWS, tomorrow to Azure VMware Solution, the next week to Google Cloud VMware Engine, and in the end back to an on-premises data center where another fully managed service like VMC on Dell EMC is deployed. 😀

VMware Cloud on AWS with Tanzu Services

It was mentioned above already, VMware Cloud on AWS includes “Tanzu Kubernetes Service” and “Tanzu Mission Control Essentials”.

VMware Cloud with Tanzu Services has been introduced at VMworld 2021 as the “Easy path to enterprise-grade Kubernetes on a fully managed, multi-cloud ready IaaS and CaaS platform”:

VMware Cloud with Tanzu Services

 

This was also when Tanzu Services became available for VMC on AWS with the following capabilities:

  • Managed Tanzu Kubernetes Grid Service: Provision Tanzu Kubernetes clusters within a few minutes using a simple, fast, and self-service experience in the VMware Cloud console. The underlying SDDC infrastructure and capacity required for Kubernetes workloads is fully managed by VMware. Use vCenter Server for managing Kubernetes workloads by deploying Kubernetes clusters, provisioning role-based access and allocating capacity for Developer teams. Manage multiple TKG clusters as namespaces with observability, troubleshooting and resiliency in vCenter Server.
  • Built in support for Tanzu Mission Control Essentials: Attach upstream compliant Kubernetes clusters including Amazon EKS and Tanzu Kubernetes Grid clusters. Manage lifecycle for Tanzu Kubernetes Grid clusters and centralize platform operations for Kubernetes clusters using the Kubernetes management plane offered by Tanzu Mission Control. Tanzu Mission Control provides a global visibility across clusters and clouds and increases security and governance by automating operational tasks such as access and security management at scale.

VMware Cloud with Tanzu Services

Take a look at the VMware Tanzu Mission Control Feature Comparison Chart to better understand the feature set of TMC Essentials.

Did you know that the Tanzu Mission Control Standard Package is included with TMC Essentials?

As of November 2021, new clusters registered with TMC will have the Carvel package manager (the kapp-controller), deployed within the cluster. The “Catalog” page in the Tanzu Mission Control console allows you to view packages available from the Tanzu Standard repository (and your own custom Carvel package repositories) and install them in your Kubernetes clusters.

Tanzu Mission Control Packages

Application Transformer for VMware Tanzu for VMC on AWS

VMware announced the tech preview for Application Transformer for VMware Tanzu for VMware Cloud on AWS in September 2021.

Application Transformer for VMware Tanzu is a tool that aids organizations in discovering application types, visualizing application topology, choosing a modernization approach based on scores, and containerizing and migrating suitable legacy applications to enhance business outcomes. As an agentless tool, Application Transformer for Tanzu utilizes the VMware vCenter API to introspect VMs across an entire vSphere or VMware Cloud on AWS-based data center.

Application Transformer can help you to convert virtual machines and application components to OCI-compliant container images, that then can be deployed into the Tanzu Kubernetes stack.

There are several ways how customers get access to Application Transformer for VMware Tanzu:

Good news for everyone is that Application Transformer for VMware Tanzu became generally available in February 2022. With this, VMware Cloud on AWS customers also have limited access to this offering from now on. The access is through integration with VMware Cloud console. If customers desire full access to Application Transformer, they need to buy Tanzu Standard, Tanzu Advanced, Tanzu for Kubernetes Operations, or App Navigator.

Features & Roadmap

VMware provides a lot of information about the features and roadmap of VMware Cloud on AWS.

VMC on AWS FAQ

There is a large collection of FAQs available that can be found here.

Multi-Tenancy on VMware Cloud Foundation with vRealize Automation and Cloud Director

Multi-Tenancy on VMware Cloud Foundation with vRealize Automation and Cloud Director

In my article VMware Cloud Foundation And The Cloud Management Platform Simply Explained I wrote about why customers need a VMware Cloud Foundation technology stack and what a VMware cloud management platform is.

One of the reasons and one of the essential characteristics of a cloud computing model I mentioned is resource pooling.

By the National Institute of Standards and Technology (NIST) resource pooling is defined with the following words:

The provider’s computing resources are pooled to serve multiple
consumers using a multi-tenant model, with different physical and virtual
resources dynamically assigned and reassigned according to consumer demand.
There is a sense of location independence in that the customer generally has no
control or knowledge over the exact location of the provided resources but may be
able to specify location at a higher level of abstraction (e.g., country, state, or
data center).

This time I would like to focus on multi-tenancy and how you can achieve that on top of VMware Cloud Foundation (VCF) with Cloud Director (formerly known as vCloud Director) and vRealize Automation, which both could be part of a VMware cloud management platform (CMP).

Multi-Tenancy

There are many understandings around about multi-tenancy and different people have different definitions for it.

If we start from the top of an IT infrastructure, we will have application or software multi-tenancy with a single instance of an application serving multiple tenants. And in the past even running on the same virtual or physical server. In this case the multi-tenancy feature is built into the software, which is commonly accessed by a group of users with specific permissions. Each tenant gets a dedicated or isolated share of this application instance.

Coming from the bottom of the data center, multi-tenancy describes the isolation of resources (compute, storage) and networks to deliver applications. The best example here are (cloud) services providers.

Their goal is to create and provide virtual data centers (VDC) or a virtual private cloud (VPC) on top of the same physical data center infrastructure – for different tenants aka customers. Normally, the right VMware solution for this requirement and service providers would be Cloud Director, but this is maybe not completely true anymore with the release of vRealize Automation 8.x. 

To make it easier for all of us, I’ll call Cloud Director and vCloud Director “vCD” from now on.

VMware Cloud Director (formerly vCloud Director)

Cloud Director is a product exclusively for cloud service providers via the VMware Cloud Provider Program (VCPP). Originally released in 2010, it enables service providers (SPs) to provision SDDC (Software-Defined Data Center) services as complete virtual data centers. vCD also keeps resources from different tenants isolated from each other.

Within vCD a unit of tenancy is called Organization VDC (OrgVDC). It is defined as a set of dedicated compute (CPU, RAM), storage and network resources. A tenant can be bound to a single OrgVDC or can be composed of multiple Organization VDCs. This is typically known as Infrastructure as a Service (IaaS).

A provider virtual data center (PVDC) is a grouping of compute, storage, and network resources from a single vCenter Server instance. Multiple organizations/tenants can share provider virtual data center resources.

Cloud Director Resource Abstraction

A lot of customers and VCPP partners have now started to offer their cloud services (IaaS, PaaS, SaaS etc.) based on VMware Cloud Foundation. For private and hybrid cloud scenarios, but also in the public cloud as a managed cloud service (VMware Cloud on AWS, Azure VMware Solution, Google Cloud VMware Engine, Alibaba Cloud VMware Solution and more).

Important: I assume that you are familiar with VCF, its core components (ESXi, vSAN, NSX, SDDC Manager) and architecture models (standard as the preferred).

Cloud Director components are currently not part of the VCF lifecycle automation, but it is a roadmap item!

Cloud Director Resource Hosting Models

vCD offers multiple hosting models:

  • In the shared hosting model, multiple tenant workloads run all together on the same
    resource groups without any performance assurance
  • In the reserved hosting model, performance of workloads is assured by resource
    reservation.
  • In the physical hosting model, hardware is dedicated to a single tenant and performance
    is assured by the allocated hardware

Tenant Using Shared Hosting on VCF Workload Domain

In this use case a tenant is using shared hosting backed by a VMware Cloud Foundation workload domain. A workload domain, which is mapped to a provider VDC.

vCD VCF Shared

Tenant Using Shared Hosting and Reserved Hosting on Multiple VCF Workload Domains

This use case describes the example of customer using shared and reserved hosting backed by multiple VCD workload domains. Here each cluster has a single resource pool mapped to a single PVDC.

vCD VCF Shared Reserved

Tenant Using Physical Hosting and Central Point of Management (CPOM)

The last example shows a single customer using physical hosting. You will notice that there is also a vSphere with
Kubernetes workload domain. VMware Cloud Foundation automates the installation of vSphere with Kubernetes (Tanzu) which makes it incredibly easy to deploy and manage.

You can see that there is an “SDDC” box on top of the Kubernetes Cluster vCenter, which is attached to
the “SDDC Proxy” entity. vCD can act as an HTTP/S proxy server between tenants and the
underlying vSphere environment in VMware Cloud Foundation. An SDDC proxy is an
access point to a component from an SDDC, for example, a vCenter Server instance, an ESXi host, or
an NSX Manager instance.

The vCD becomes the central point of management (CPOM) in this case and the customer gets a complete dedicated SDDC with vCenter access.

vCD VCF Physical CPOM

Note: Since vCD 9.7 it is possible to present for example a vCenter Server instance securely to a tenant’s organization using the Cloud Director user interface. This is how you could build your own VMC-on-AWS-like cloud offering!

Cloud Director CPOM

All 3 Tenants Together

Finally, we put it all together. In the first use case we can see that different customers are sharing resources from a
single PVDC. We can also see that resources from a single vCenter can be split across different provider virtual datacenters and that we can mix and match multi-tenants workload domains and workload domains offering dedicated private cloud all together.

vCD VCF All Together

Cloud Director Service and VMware Cloud on AWS

If you don’t want to extend or operate your own data center or cloud infrastructure anymore and provide a managed service to multiple customer, there are still options for you available backed by VMware Cloud Foundation as well.

Since October 2020 you have Cloud Director Service globally available, which delivers multi-tenancy to VMware Cloud on AWS for managed service providers (MSP).

VMware sees not only new, but also existing VCPP partners moving towards a mixed-asset portfolio, where their cloud management platform consists of a VCPP and MSP (VMware SaaS offerings) contract. This allows them for example to run vCD on-premises for their current customers and the onboarding of new tenants would happen in the public cloud with CDS and VMC on AWS.

vCD CDS Mixed Mode

Enterprise Multi-Tenancy with vRealize Automation

With the release of vRealize Automation 8.1 (vRA) VMware offered support for dedicated infrastructure multi-tenancy, created and managed through vRealize Suite Lifecycle Manager. This means vRealize Automation enables customers or IT providers to set up multiple tenants or organizations within each deployment.

Providers can set up multiple tenant organizations and allocate infrastructure. Each tenant manages its own projects (team structures), resources and deployments.

Enabling tenancy creates a new Provider (default) organization. The Provider Admin will create new tenants, add tenant admins, setup directory synchronization, and add users. Tenant admins can also control directory synchronization for their tenant and will grant users access to services within their tenant. Additionally, tenant admins will configure Policies, Governance, Cloud Zones, Profiles, access to content and provisioned resources; within their tenant. A single shared SDDC or separate SDDCs can be used among tenants depending on available resources.

vRealize Automation 8.1 Multi-Tenancy

With vRealize Automation 8.2, provider administrators got the ability to share infrastructure by creating and assigning Virtual Private Zones (VPZ) to tenant organizations.

Think of VPZs as a kind of container of infrastructure capacity and services which can be defined and allocated to a Tenant. You can add unique or shared cloud accounts, with associated compute, flavors, images, storage, networking, and tags to each VPZ. Each component offers the same configuration options you would see for a standalone configuration.

vRealize Automation 8.2 Multi-Tenancy

vRealize Automation and VMware Cloud Foundation

With the pretty new multi-tenancy and VPZ capability a new consumption model on top of VCF can be built. You (provider) would map the Cloud Zones (compute resources on vSphere (or AWS for example)) to a VCF workload domain.

The provider sets these cloud zones up for their customers and provides dedicated or shared infrastructure backed by Cloud Foundation workload domains.

This combination would allow you to build an enterprise VPC construct (like AWS for example), a logically isolated section of your provider cloud.

vRealize Automation and VMware Cloud Foundation

SDDC Manager Integration and VMware Cloud Foundation (VCF) Cloud Account

Since the vRA 8.2 release customers are also able to configure a SDDC Manager integration and on-board workload domains as VMware Cloud Foundation cloud accounts into the VMware Cloud Assembly service.

VMware Cloud Director or vRealize Automation?

You wonder if vRealize Automation could replace existing vCD installations? Or if both cloud management platforms can do the same?

I can assure you, that you can provide a self-service provisioning experience with both solutions and that you can provide any technology or cloud service “as a service”. Both have in common to be backed by Cloud Foundation, have some form of integration (vRA) and can be built by a VMware Validated Design (VVD).

vCD is known to be a service provider solution, where vRA is more common in enterprise environments. VMware has VCPP partners, that use Cloud Director for their external customers and vRealize Automation for their internal IT and customers.

If you are looking for a “cloud broker” and Infrastructure as Code (IaC), because you also want to provision workloads on AWS, Azure or GCP as well, then vRealize Automation is the better solution since vCD doesn’t offer this deep integration and these deployment options yet.

Depending on your multi-tenant needs and if you for example only have chosen vCD in the past, because of the OrgVDC and resource pooling feature, vRealize Automation would be enough and could replace vCD in this case.

It is also very important to understand how your current customer onboarding process and operational model look like:

  • How do you want to create a new tenant? 
  • How do you want to onboard/migrate existing customer workloads to your provider infrastructure?
  • Do you need versioning of deployments or templates?
  • Do customers require access to the virtual infrastructure (e.g. vCenter or OrgVDC) or do you just provide SaaS or PaaS?
  • Do customers need a VPN or hybrid cloud extension into your provider cloud?
  • How would you onboard non-vSphere customers (Hyper-V, KVM) to your vSphere-based cloud?
  • Does your customer rely on other clouds like AWS or Azure?
  • How do you do billing for your vSphere-based cloud or multi-cloud environment?
  • What is your Kubernetes/container strategy?
  • And 100 other things 😉

There are so many factors and criteria to talk about, which would influence such a decision. There is no right or wrong answer to the question, if it should be VMware Cloud Director or vRealize Automation. Use what makes sense.

Which could also be a combination of both.

Introduction to Alibaba Cloud VMware Solution (ACVS)

Introduction to Alibaba Cloud VMware Solution (ACVS)

VMware’s hybrid and multi-cloud strategy is to run their Cloud Foundation technology stack with vSphere, vSAN and NSX in any private or public cloud including edge locations. I already introduced VMC on AWS, Azure VMware Solution (AVS), Google Cloud VMware Engine (GCVE) and now I would like to briefly summarize Alibaba Cloud VMware Solution (ACVS).

VMware Multi-Cloud Offerings

A lot of European companies, this includes one of my large Swiss enterprise account, defined Alibaba Cloud as strategic for their multi-cloud vision, because they do business in China. The Ali Cloud is the largest cloud computing provider in China and is known for their cloud security, reliable and trusted offerings and their hybrid cloud capabilities.

In September 2018, Alibaba Cloud (also known as Aliyun), a Chinese cloud computing company that belongs to the Alibaba Group, has announced a partnership with VMware to deliver hybrid cloud solutions to help organizations with their digital transformation.

Alibaba Cloud was the first VMware Cloud Verified Partner in China and brings a lot of capabilities and services to a large number of customers in China and Asia. Their current global infrastructure operates worldwide in 22 regions and 67 availability zones with more regions to follow. Outside Main China you find Alibaba Cloud data centers in Sydney, Singapore, US, Frankfurt and London.

As this is a first-party offering from Alibaba Cloud, this service is owned and delivered by them (not VMware). Alibaba is responsible for the updates, patches, billing and first-level support.

Alibaba Cloud is among the world’s top 3 IaaS providers according to Gartner and is China’s largest provider of public cloud services. Alibaba Cloud provides industry-leading flexible, cost-effective, and secure solutions. Services are available on a pay-as-you-go basis and include data storage, relational databases, big-data processing, and content delivery networks.

Currently,  Alibaba Cloud has been declared as a Niche player according to the actual Gartner Magic Quadrant for Cloud Infrastructure and Platform Services (CIPS) with Oracle, IBM and Tencent Cloud.

Alibaba Gartner CIPS MQ

Note: If you would like to know more about running the VMware Cloud Foundation stack on top of the Oracle Cloud as well, I can recommend Simon Long’s article, who just started to write about Oracle Cloud VMware Solution (OCVS).

This partnership with VMware and Alibaba Cloud has the same goals like other VMware hybrid cloud solutions like VMC on AWS, OCVS or GCVE – to provide enterprises the possibility to meet their cloud computing needs and the flexibility to move existing workloads easily from on-premises to the public cloud and have highspeed access to the public cloud provider’s native services.

ACVS vSphere Architecture

In April 2020, Alibaba Cloud and VMware finally announced the general availability of Alibaba Cloud VMware Solution for the Main China and Hongkong region (initially). This enables customers to seamlessly move existing vSphere-based workloads to the Alibaba Cloud, where VMware Cloud Foundation is running on top of Aliyun’s infrastructure.

As already common with such VMware-based hybrid cloud offerings, this let’s you move from a Capex to a Opex-based cost model based on subscription licensing.

Joint Development

X-Dragon – Shenlong in Chinese – is a proprietary bare metal server architecture developed by Alibaba Cloud for their cloud computing requirements. It offers direct access to CPU and RAM resources without virtualization overheads that bare metal servers offer (built around a custom X-Dragon MOC card). The virtualization technology, X-Dragon, behind Alibaba Cloud Elastic Compute Service (ECS) is now in its third generation. The first two generations were called Xen and KVM.

X-Dragon  NIC

VMware works closely together with the Alibaba Cloud engineers to develop a VMware SDDC (software-defined data center based on vSphere and NSX) which runs on this X-Dragon bare metal architecture.

The core of the MOC NIC is the X-Dragon chip. The X-Dragon software system runs on the X-Dragon chip to provide virtual private cloud (VPC) and EBS disk capabilities. It offers these capabilities to ECS instances and ECS bare metal instances through VirtIO-net and VirtIO-blk standard interfaces.

Note: The support for vSAN is still roadmap and comes later in the future (no date committed yet). Because the X-Dragon architecture is a proprietary architecture, running vSAN over it requires official certification. 

Project Monterey

Have you seen VMware’s announcement at VMworld 2020 about Project Monterey which allows you to run VMware Cloud Foundation on a SmartNIC? For me, this looks similar to the X-Dragon architecture 😉

Project Monterey VMware Cloud Foundation Use Cases

Data Center extension or retirement. You can scale the data center capacity in the cloud on-demand, if you for example don’t want to invest in your on-premises environment anymore. In case you just refreshed your current hardware, another use case would be the extension of your on-premises vSphere cloud to Alibaba Cloud.ACVS Disaster Recovery

Disaster Recovery and data protection. Here we’ll find different scenarios like recovery (replication) or backup/archive (data protection) use cases. You can use your ACVS private clouds as a disaster recovery (DR) site for your on-premises workloads. This DR solution would be based on VMware Site Recovery Manager (SRM) which can be also used together with HCX. At the moment Alibaba Cloud offers 9 regions for DR sites.

Cloud migrations or consolidation. If you want to start with a lift & shift approach to migrate specific applications to the cloud, then ACVS is the right choice for you. Maybe you want to refresh your current infrastructure and need to relocate or migrate your workloads in an easy and secure way? Another perfect scenario would be the consolidation of different vSphere-based clouds.

ACVS Migration to Alibaba Cloud

Multicast Support with NSX-T

Like with Microsoft Azure and Google Cloud, an Alibaba Cloud ECS instance or VPC in general doesn’t support multicast and broadcast. That is one specific reason why customers need to run NSX-T on top of their public cloud provder’s global cloud infrastructure.

Connectivity Options

For (multi-)national companies Alibaba Cloud has different enterprise-class networking offerings to connect different sites or regions in a secure and reliable way.

Cloud Enterprise Network (CEN) is a highly-available network built on the high-performance and low-latency global private network provided by Alibaba Cloud. By using CEN, you can establish private network connections between Virtual Private Cloud (VPC) networks in different regions, or between VPC networks and on-premises data centers.  The CEN is also available in Europe in Germany (Frankfurt) and UK (London).

Alibaba Cloud Cloud Enterprise Network

Alibaba Cloud Express Connect helps you build internal network communication channels that feature enhanced cross-network communication speed, quality, and security. If your on-premises data center needs to communicate with an Alibaba Cloud VPC through a private network, you can apply for a dedicated physical connection interface from Alibaba Cloud to establish a physical connection between the on-premises data center and the VPC. Through physical connections, you can implement high-quality, highly reliable, and highly secure internal communication between your on-premises data center and the VPC. 

Alibaba Cloud Express Connect

ACVS Architecture and Supported VMware Cloud Services

Let’s have a look at the ACVS architecture below. On the left side you see the Alibaba Cloud with the VMware SDDC stack loaded onto the Alibaba bare metal servers with NSX-T connected to the Alibaba VPC network.

This VPC network allows customers to connect their on-premises network and to have direct acccess to Alibaba Cloud’s native services.

Customers have the advantage to use vSphere 7 with Tanzu Kubernetes Grid and could leverage their existing tool set from the VMware Cloud Management Platform like vRealize Automation (native integration of vRA with Alibaba Cloud is still a roadmap item) and vRealize Operations.

Alibaba Cloud VMware Solution Architecture

The right side of the architecture shows the customer data centers, which run as a vSphere-based cloud on-premises managed by the customer themselves or as a managed service offering from any service provider. In between, with the red lines, the different connectivity options like Alibaba Direct Connect, SD-WAN or VPN connections are mentioned with different technologies like NSX-T layer 3 VPN, HCX and Site Recovery Manager (SRM).

To load balance the different application services across the different vSphere-based or native clouds, you can use NSX Advanced Load Balancer (aka Avi) to configure GSLB (Global Server Load Balancing) for high availability reasons.

Because the entire stack on top of Alibaba Cloud’s infrastructure is based on VMware Cloud Foundation, you can expect to run everything in VMware’s product portfolio like Horizon, Carbon Black, Workspace ONE etc. as well.

You can also deploy AliCloud Virtual Edges with VMware SD-WAN by VeloCloud.

Node Specifications

The Alibaba Cloud VMware Solution offering is a little bit special and I hope that I was able to translate the Chinese presentations correctly.

First, you have to choose the amount of hosts which gives you specific options.

1 Host (for testing purposes): vSphere Enterprise Plus, NSX Data Center Advanced, vCenter

2+ Hosts (basic type): vSphere Enterprise Plus, NSX Data Center Advanced, vCenter

3+ Hosts (flexibility and elasticity): vSphere Enterprise Plus, NSX Data Center Advanced, vCenter, (vSAN Enterprise)

Site Recovery Manager, vRealize Log Insight and vRealize Operations need to be licensed separately as they are not included in the ACVS bundle.

The current ACVS offering has the following node options and specifications (maximum 32 hosts per VPC):

ACVS Node Specifications

All sixth-generation ECS instance come equipped with Intel® Xeon® Platinum 8269CY processors. These processors were customized based on the Cascade Lake microarchitecture, which is designed for the second-generation Intel® Xeon® Scalable processors. These processors have a turbo boost with an increased burst frequency of 3.2 GHz, and can provide up to a 30% increase in floating performance over the fifth generation ECS instances.

Component Version License
vCenter 7.0 vCenter Standard
ESXi 7.0 Enterprise Plus
vSAN (support coming later) n/a Enterprise
NSX Data Center (NSX-T) 3.0 Advanced
HCX n/a Enterprise

Note: Customers have the possibility to install any VIBs by themselves with full console access. This allows the customer to assess the risk and performance impacts by themselves and install any needed 3rd party software (e.g. Veeam, Zerto etc.).

If you want to more about how to accelerate your multi-cloud digital transformation initiatives in Asia, you can watch the VMworld presentation from this year. I couldn’t find any other presentation (except the exact same recording on YouTube) and believe that this article is the first publicy available summary about Alibaba Cloud VMware Solution. 🙂

Google Cloud VMware Engine (GCVE)

Google Cloud VMware Engine (GCVE)

In June 2020 VMware and Google made the announcement that Google Cloud VMware Engine (GCVE) is generally available. Almost exactly one year ago, the market received the information that VMware’s Cloud Foundation (vSphere, vSAN and NSX) stack will come to Google Cloud.

With this milestone VMware is now present on top of all the so-called “big three” hyperscalers.

GCVE has the same goals like the other similar offerings like VMware Cloud on AWS or Azure VMware Solution and belongs to the VMware multi-cloud strategy – to seamlessly migrate and run applications in the public cloud. In this case in Google Cloud! Run your applications in the public cloud exactly the same way as you already do now withh your on-premises VMware environment. With the very important addition, that you have high speed access to Google Cloud services like Cloud SQL, Cloud Storage, big data or AI/ML services.

To be able to run VMware workloads on top of the Google Cloud global infrastructure, Google acquired CloudSimple (with which they partnered with already) last November 2019.

At the moment of writing, the VMware hybrid cloud experience on Google Cloud is sold, operated and supported by Google and their partners.

Many customers are already looking at this very interesting offer, which is going to be available in more regions until the end of 2020. But there are also already a few customers using the joint offering. Google just published a customer reference story about the “Deutsche Börse Group”, a large and international financial organization, which extended their on-premises environment to Google Cloud with Google Cloud VMware Engine. One of the reasons why Deutsche Börse went for this vSphere-based cloud approach, was, to keep migrations to the cloud easy. I expect we can hear more about this success story at VMworld 2020.

Cloud Migration and Workload Mobility

A lot of customers underestimate the amount of work, time and costs involved in refactoring or re-platforming applications and the overall challenges when it comes to migrations from on-prem to the cloud. To build this secure hybrid cloud extension with GCVE, you’ll need VMware HCX, which is included in the GCVE offering.

There are different options available to connect both worlds:

GCVE Connectivity Options

  • VPN Gateway for point-to-point connections, used for the secure admin access to vCenter. Useful for the initial setup of the GCVE environment.
  • Cloud VPN for site-to-site connections, a secure layer 3 connection over the internet. This is one of the lower cost options for use cases, that don’t require high bandwidth.
  • Dedicated Cloud Interconnect with a direct traffic flow to Google with 10Gbps or 100Gbps circuits with 50Mbps to 50 Gbps connection capacities. This direct connection is required for HCX and the preferable connectivity option for customers requiring high speed and low latency.
  • Partner (Cloud) Interconnect is another option of a Cloud Interconnect, where your traffic flows through one of the supported service providers (e.g. Colt, Equinix, BT, e-shelter, Verizon, InterCloud, Interxion, Megaport)

Note: One unique feature of GCVE is the ability to route between different GCVE environments in the same region, without the need for additional configuration. 

Use Cases

These use cases, if you made yourself already familiar with a hybrid cloud approach, shouldn’t be new to you.

Data Center extension or retirement. You can scale the data center capacity in the cloud on-demand, if you for example don’t want to invest anymore in your on-premises environment. In case you just refreshed your current hardware, another use case would be the extension of your on-premises vSphere cloud to Google Cloud.

Disaster Recovery and data protection. Here we’ll find different scenarios like recovery (replication) or backup/archive (data protection) use cases. You can also still use your existing 3rd party tools from Zerto or Veeam to replace or complement existing DR locations and leverage the Cloud Storage service. You can also use your GCVE private clouds as a disaster recovery (DR) site for your on-premises workloads. This DR solution would be based on VMware Site Recovery Manager (SRM) which can be also used together with HCX.

Cloud migrations or consolidation. If you want to start with a lift & shift approach to migrate specific applications to the cloud, then GCVE is definitely right for you. Maybe you want to refresh your current infrastructure and need to relocate or migrate your workloads in an easy and secure way? Another perfect scenario would be the consolidation of different vSphere-based clouds.

Application modernization. Re-architecting or refactoring applications is not that easy. Most customers start with a partial approach to modernize their applications and leverage cloud-native services (e.g. databases, AI/ML engines).

Interesting: Did you know that Google’s on-prem GKE (Google Anthos) is running on vSphere?

VMware Horizon on VMware Engine

The advantages of a public cloud like Google Cloud are the “endless” capacity, agility and high-bandwidth connections. These items are very important for a virtual desktop infrastructure (VDI) and specially during disaster scenarios, when onboardings have to happen fast or if you look for on-demand growth.

Another regular example could be a merger & acquisition use case, where we the main infrastructure doesn’t have the necessary physical resources to onboard to new company and their employees.

Because something like this has always happen as easy and fast as possible. Running virtual desktops in Google Cloud VMware Engine can help in such situations. Together with VMware Horizon, organizations could install a VDI environment in GCVE and connect it to their Horizon on-premises infrastructure using the Cloud Pod Architecture (CPA). 

Note: When migrating applications to the cloud (GCVE), it is a best practice to keep the virtual desktop close to the application, which is a general use case we see when talking about application locality.

Horizon Global Pod GCVE

With the release of Horizon 2006 (aka Horizon 8) it is also possible to choose “Google Cloud” as deployment option during the connection server installation.

C:\\Users\\mrebmann\\OneDrive - VMware, Inc\\cloud13\\2020 - Google Cloud VMware Engine\\Horizon on GCVE.png

In case you need a load balancer (for your Horizon components and in general) for your on-premises environment and the public cloud, have a look at NSX Advanced Load Balancer.

GCVE Node Specs

When planning your GCVE resource needs, be aware of the following specifications and limits:

CPU: Intel Xeon Gold 6240 (Cascade Lake) 2.6 GHz (x2), 36 Cores, 72 Hyper-Threads

Memory: 768 GB

Storage (vSAN): 2 × 1.6 TB (3.2 TB) NVMe (Cache), 6 × 3.2 TB (19.2 TB) NVMe (Data)

Number of nodes required to create a private cloud: 3 (up to 64 hosts per private cloud)

Number of nodes allowed in a cluster on a private cloud: 16

3rd party tools compatibility: Yes, you can use existing tools (elevated privileges allow you to install 3rd party software)

Interesting facts: It only takes about a half hour to spin up your private cloud with three nodes! The addition of a new node takes approximately 15 minutes.

GCVE Elevated Privileges

Software License and Versions

Please find the current software versions and licenses below used for the GCVE offering (purchased with a 1- or 3- year commitment). The listed software versions are fixed and all updates are managed by Google. Google is responsible for the lifecycle management of the VMware software, which includes ESXi, vCenter and NSX.

Component Version License
vCenter 6.7 U3 vCenter Standard
ESXi 6.7 U3 Enterprise Plus
vSAN 6.7 U3 Enterprise
NSX Data Center (NSX-T) 2.5.1 Advanced
HCX 3.5.3 Advanced

Shared Responsibilities

Google Cloud VMware Engine is coming with all components you need to securely run VMware natively in a dedicated private cloud. Google takes care of the infrastructure (service) and their native service integrations. As a customer you only need to take care of your virtual machines or containers with your applications and data. Besides that, you also need to make sure that your configurations, policies, network portgroups, authentication and capacity management are properly configured.

GCVE Shared Responsibilities

If you want to know and learn more about Google Cloud VMware Engine, have a look at the following resources: 

Cross-Cloud Mobility with VMware HCX

Cross-Cloud Mobility with VMware HCX

Update 10th September 2020: vSphere 7.0 (VDS 7.0) and NSX-T 3.0.1+ are supported since the HCX R143 release which has been made available on September 8, 2020

https://docs.vmware.com/en/VMware-HCX/services/rn/VMware-HCX-Release-Notes.html 

Most people think that VMware HCX is a only migration tool that helps you moving workloads to a vSphere based cloud like VMware Cloud on AWS, Azure VMware Solution or Google Cloud VMware Engine. But it can do so much more for you than only application or workload migrations. HCX is also designed for workload rebalancing and business continuity across data centers or VMware clouds. Why I say “across VMware clouds” and not only “clouds”?

A few years ago everyone thought or said that customers will move all their workloads to the public cloud and the majority of them don’t need local data centers anymore. But we all know that this perception has changed and that the future cloud operation is model hybrid.

A hybrid cloud environment leverages both the private and public clouds to operate. A multi-cloud environment includes two or more public cloud vendors which provide cloud-based services to a business that may or may not have a private cloud. A hybrid cloud environment might also be a multi-cloud environment.

We all know that the past perception was an illusion and we didn’t have a clue where the hyperscalers like AWS, Azure or GCP would be in the next 5 or 7 years. And I believe that even the AWS and Microsoft didn’t expect what is going to happen since we observed interesting shifts in the last few years.

Amazon Web Services (AWS) has been launched 14 years ago (2006) to provide web services from the cloud. At AWS re:Invent 2018 the CEO Andy Jassy announced AWS Outposts because their customers have been asking for an AWS option on-premises. In the end, Outpost is just an extension of an AWS region into the own data center, where you can launch EC2 instances or Amazon EBS volumes locally. AWS already had some hybrid services available (like Storage Gateway) but here we talk about infrastructure and making your own data center part of the AWS Global Infrastructure.

Microsoft Azure was released in 2010 and the first technical preview of Azure Stack has been announced in 2016. So, Microsoft also realized that the future cloud model is a hybrid approach “that provides consistency across private, hosted and public clouds”.

Google Cloud Platform (GCP) offers cloud computing services since 2008. Eleven years later (in 2019) Google introduced Anthos that you can “run an app anywhere –  simply, flexibly and securely”.

All the hyperscalers changed their cloud model to provide customers a consistent infrastructure with consistent operations and management as we understand now.

VMware is coming from the other end of this hybrid world and has the same overall goal or vision to make a hybrid or multi-cloud reality. But with one very important difference. VMware helps you to abstract the complexity of a hybrid environment and gives you the choice to run your workloads in any cloud infrastructure with a cloud-agnostic approach.

As organizations try to migrate their workloads to the public, they face multiple challenges and barriers:

  • How can I migrate my workload to the public cloud?
  • How long does it take to migrate?
  • What about application downtime?
  • Which migration options do I have?
  • Which cloud is the best destination for which workloads?
  • Do I need to refactor or develop some applications?
  • Can I do a lift and shift migration and modernize the application later?
  • How can I consistently deploy workloads and services for my multi-cloud?
  • How can I operate and monitor (visibility and observability) all the different clouds?
  • What if tomorrow one the public cloud provider is not strategic anymore? How can I move my workloads away?
  • How can I control costs over all clouds?
  • How can I maintain security?
  • What about the current tools and 3rd party software I am using now?
  • What if I want to migrate VMs back from the public cloud?
  • What if I want to move away/back somewhen from a specific cloud provider?

In summary, the challenges with a hybrid cloud are about costs, complexity, tooling and skills. Each public cloud added to your current on-premises infrastructure is in fact a new silo. If you have the extra money and time and don’t need consistent infrastructures and consistent operations and management, you’ll accept the fact that you have a heterogeneous environment with different technology formats, skill/tool sets, operational inconsistencies and security controls.

If you are interested in a more consistent platform then you should build a more unified hybrid cloud. Unified means that you provide consistent operations with the existing skills and tools (e.g. vCenter, vRealize Automation, vRealize Operations) and the same policies and security configuration over all clouds – your data center, public cloud or at the edge.

To provide such a cloud agnostic platform you need to abstract the technology format and infrastructure in the public cloud. This is why VMware built the VMware Cloud Foundation (VCF) platform that delivers a set of software-defined services for compute, storage, networking, security and cloud management for any cloud.

VMC on AWS, Azure VMware Solution, Google Cloud VMware Engine and all the other hybrid cloud offerings (IBM, Oracle, Alibaba Cloud, VCPP) are based on VMware Cloud Foundation. This is the underlying technology stack you would need if your goal is to be independent and to achieve workload mobility between clouds. With this important basic understanding we can take a closer look at VMware HCX.

VMware HCX Use Cases

HCX provides an any-to-any vSphere workload mobility service without requiring retrofit as we use the same technology stack in any cloud. 

VMware HCX Use Cases

HCX enables you to schedule application migrations of hundreds or thousands of vSphere VMs within your data centers and across public clouds without requiring a reboot or a downtime.

If you would like to change the current platform or have to modernize your current data center, HCX allows you to migrate workloads from vSphere 5.x and non-vSphere (Hyper-V and KVM) environments.

VMware HCX Migration

Workload rebalancing means providing a mobility platform across cloud regions and cloud providers to move applications and workloads at any time for any reason.

Workload mobility is cool and may be the future but is not possible today as the public cloud’s egress costs would be way too high at the moment. Let’s say you pay $0.05 per GB when you move data away from the public cloud to any external destination, this would generate costs of $2.50 for a 50GB virtual machine.

Not that much, right? If you move away 500 VMs, the bill would list $1’250 for egress costs. Evacuating VMs from one public cloud to another one is not so expensive if it happens three or four times a year. But if the rebalancing should happen at a higher cadence, the egress costs would get too high. But we can assume that this fact will change in the future as the public cloud computing prices will come down in the future. 

HCX Components and Services

HCX is available with an Advanced and Enterprise license. The Advanced license services are standard with HCX and are also included in the HCX Enterprise license. The Enterprise license is needed when you migrate non-vSphere workloads into a vSphere environment. This capability is called OS Assisted Migration (OSAM).

HCX Services

The HCX Advanced features are included in a NSX Data Center Enterprise Plus license. With a managed service like VMware Cloud on AWS or Azure VMware Solution HCX Advanced is already be included.

HCX Connector Advanced License

If you want to move workloads from a vSphere environment to a vSphere enabled public cloud, you don’t need the complete VMware Cloud Foundation stack at the source site:

  • On-premises vSphere version 5.5 and above
  • Minimum of 100 Mbps bandwidth between source and destination
  • Virtual switch based on vDS (vSphere Distributed Switch), Cisco Nexus 1000v or vSphere Standard Switch
  • Minimum of virtual machine hardware version 9
  • VMs with hard disks not larger than 2TB

Depending on the HCX license and services you need, you have to deploy some or all of the HCX components. HCX comprises components and appliances at the source and destination sites.

HCX Manager Destination

The HCX Connector services and appliances are deployed at the destination site first before you are going to deploy the virtual appliances at the source site (HCX Interconnect appliance).

HCX Interconnect Appliance Download Link

After you deployed the appliances at the source site, you can create the site pairing.

HCX Site Pairing

As soon as you have installed HCX in both sites, you can manage and configure the services within the vSphere Client.

HCX in vSphere Client

After a successful site pairing, you can start to create the HCX Service Mesh.

The Multi-Site Service mesh is used to create a secure optimized transport fabric between any two sites managed by HCX. When HCX Migration, Disaster recovery, Network Extension, and WAN Optimization services are enabled, HCX deploys Virtual Appliances in the source site and corresponding “peer” virtual appliances on the destination site. The Multi-Site Service Mesh enables the configuration, deployment, and serviceability of these Interconnect virtual appliance pairs.

HCX Service Mesh

In the HCX site-to-site architecture, there is notion of an HCX source and HCX destination environment. Depending on the environment, there is a specific HCX installer:

HCX Connector (previously HCX Enterprise) or HCX Cloud. HCX Connector is always deployed as the source. HCX Cloud is typically deployed as the destination, but it can be used as the source in cloud-to-cloud deployments. In HCX-enabled public clouds, the cloud provider deploys HCX Cloud. The public cloud tenant deploys HCX Connector on-premises.
The source and destination sites are paired together for HCX operations. 

In both the source and destination environments, HCX is deployed to the management zone, next to each site’s vCenter Server, which provides a single plane (HCX Manager) for administering VMware HCX. This HCX Manager provides a framework for deploying HCX service virtual machines across both the source and destination sites. VMware HCX administrators are authenticated, and each task authorized through the existing vSphere SSO identity sources. VMware HCX mobility, extension, protection actions can be initiated from the HCX User Interface or from within the vCenter Server Navigator screen’s context menus.

In the NSX Data Center Enterprise Plus (HCX for Private to Private deployments), the tenant deploys both source and destination HCX Managers.

The HCX-IX service appliance provides replication and vMotion-based migration capabilities over the Internet and private lines to the destination site whereas providing strong encryption, traffic engineering, and virtual machine mobility.

The VMware HCX WAN Optimization service appliance improves performance characteristics of the private lines or Internet paths by applying WAN optimization techniques like the data de-duplication and line conditioning. It makes performance closer to a LAN environment. It accelerates on-boarding to the destination site using Internet/VPN- without waiting for Direct Connect/MPLS circuits.

The VMware HCX Network Extension service appliance provides a late Performance (4–6 Gbps) Layer 2 extension capability. The extension service permits keeping the same IP and MAC addresses during a Virtual Machine migration. Network Extension with Proximity Routing provides the optimal ingress and egress connectivity for virtual machines at the destination site.

 

Using VMware HCX OS Assisted Migration (OSAM), you can migrate guest (non-vSphere) virtual machines from on-premise data centers to the cloud. The OSAM service has several components: the HCX Sentinel software that is installed on each virtual machine to be migrated, a Sentinel Gateway (SGW) appliance for connecting and forwarding guest workloads in the source environment, and a Sentinel Data Receiver (SDR) in the destination environment.

The HCX Sentinel Data Receiver (SDR) appliance works with the HCX Sentinel Gateway appliance to receive, manage, and monitor data replication operations at the destination environment.

HCX Migration Types

VMs can be moved from one HCX-enabled data center using different migration technologies or types.

HCX Migration Types

HCX cold migration uses the VMware NFC (Network File Copy) protocol and is automatically selected when the source VM is powered off.

HCX vMotion

  • This option is designed for moving a single virtual machine at a time
  • There is no service interruption during the HCX vMotion migration
  • Encrypted vMotion between legacy source and SDDC target
  • Bi-directional (Cross-CPU family compatibility without cluster EVC)
  • In-flight Optimization (deduplication/compression)
  • Compatible from vSphere 5.5+ environments (VM HW v9)

HCX Bulk Migration

  • Bulk migration uses the host-based replication (HBR) to move a virtual machine between HCX data centers
  • This option is designed for moving virtual machines in parallel (migration in waves)
  • This migration type can set to complete on a predefined schedule
  • The virtual machine runs at the source site until the failover begins. The service interruption with the bulk migration is equivalent to a reboot
  • Encrypted Replication migration between legacy source and SDDC target
  • Bi-directional (Cross-CPU family compatibility)
  • WAN Optimized (deduplication/compression)
  • VMware Tools and VM Hardware can be upgraded to the latest at the target.

HCX Replication Assisted vMotion

VMware HCX Replication Assisted vMotion (RAV) combines advantages from VMware HCX Bulk Migration (parallel operations, resiliency, and scheduling) with VMware HCX vMotion (zero downtime virtual machine state migration).

HCX OS Assisted Migration

This migration method provides for the bulk migration of guest (non-vSphere) virtual machines using OS Assisted Migration to VMware vSphere on-premise or cloud-based data centers. Enabling this service requires additional HCX licensing.

 

  • Utilizes OS assisted replication to migrate (conceptually similar to vSphere replication)
  • Source VM remains online during replication
  • Quiesce the source VM for final sync before migration
  • Source VM is powered off and the migrated VM is powered on in target site, for low downtime switchover
  • VMware tools is installed on the migrated VM

Cross-Cloud Mobility

Most customers will probably start with one public cloud first, e.g. VMC on AWS, to evaluate the hybridity and mobility HCX delivers. Cross-cloud monility is maybe not a requirement today or tomorrow but gets more important when your company has a multi-cloud strategy which becomes reality very soon.

If you want to be able to move workloads seamlessly between clouds, extend networks and protect workloads the same way across any cloud, then you should consider a VMware platform and use HCX.

HCX Cross-Cloud Mobility

Let’s take network and security as an example. How would you configure and manage all the different network, security, firewall policies etc. in your different clouds with the different infrastructure and security management consoles?

If you abstract the hyperscaler’s global infrastructure and put VMware on top, you could in this case use NSX (software-defined networking) everywhere. And because all the different policies are tied to a virtual machine, it doesn’t matter anymore if you migrate a VM from host to host or from cloud to cloud.

This is what you would call consistent operations and management which is enabled by a consistent infrastructure (across any cloud).

And how would you migrate workloads in a very cost and time efficient way without a layer 2 stretch? You would have to take care of re-IPing workloads and this involves a lot of changes and dependencies. If you have hundreds of applications then the cloud migration would be a never ending project with costs you could not justify.

In the case you need to move workload back to your own on-premises data center, HCX also gives you this advantage.

You have the choice in which cloud you want to run your applications, at any time.

 

HCX and vSphere 7

At the time of writing HCX has no official support for vSphere 7.0 yet. I tested it in my home lab and ran into an error while creating the Service Mesh. At least one other colleague had the same issue with vSphere 7 using NSX-T 3.0 and VDS 7.0.

HCX vSphere 7 Error

I would like to thank Danny Stettler for reviewing and contributing. 🙂 Big kudos to you, Danny! 🙂

I hope the article has helped you to get an overview what HCX and a hybrid cloud model really mean. Drop a comment and share your view and experience when it comes to cloud strategies and migrations.