Application Modernization and Multi-Cloud Portability with VMware Tanzu

Application Modernization and Multi-Cloud Portability with VMware Tanzu

It was 2019 when VMware announced Tanzu and Project Pacific. A lot has happened since then and almost everyone is talking about application modernization nowadays. With my strong IT infrastructure background, I had to learn a lot of new things to survive initial conversations with application owners, developers and software architects. And in the same time VMware’s Kubernetes offering grew and became very complex – not only for customers, but for everyone I believe. 🙂

I already wrote about VMware’s vision with Tanzu: To put a consistent “Kubernetes grid” over any cloud

This is the simple message and value hidden behind the much larger topics when discussing application modernization and application/data portability across clouds.

The goal of this article is to give you a better understanding about the real value of VMware Tanzu and to explain that it’s less about Kubernetes and the Kubernetes integration with vSphere.

Application Modernization

Before we can talk about the modernization of applications or the different migration approaches like:

  • Retain – Optimize and retain existing apps, as-is
  • Rehost/Migration (lift & shift) – Move an application to the public cloud without making any changes
  • Replatform (lift and reshape) – Put apps in containers and run in Kubernetes. Move apps to the public cloud
  • Rebuild and Refactor – Rewrite apps using cloud native technologies
  • Retire – Retire traditional apps and convert to new SaaS apps

…we need to have a look at the palette of our applications:

  • Web Apps – Apache Tomcat, Nginx, Java
  • SQL Databases – MySQL, Oracle DB, PostgreSQL
  • NoSQL Databases – MongoDB, Cassandra, Prometheus, Couchbase, Redis
  • Big Data – Splunk, Elasticsearch, ELK stack, Greenplum, Kafka, Hadoop

In an app modernization discussion, we very quickly start to classify applications as microservices or monoliths. From an infrastructure point of view you look at apps differently and call them “stateless” (web apps) or “stateful” (SQL, NoSQL, Big Data) apps.

And with Kubernetes we are trying to overcome the challenges, which come with the stateful applications related to app modernization:

  • What does modernization really mean?
  • How do I define “modernization”?
  • What is the benefit by modernizing applications?
  • What are the tools? What are my options?

What has changed? Why is everyone talking about modernization? Why are we talking so much about Kubernetes and cloud native? Why now?

To understand the benefits (and challenges) of app modernization, we can start looking at the definition from IBM for a “modern app”:

“Application modernization is the process of taking existing legacy applications and modernizing their platform infrastructure, internal architecture, and/or features. Much of the discussion around application modernization today is focused on monolithic, on-premises applications—typically updated and maintained using waterfall development processes—and how those applications can be brought into cloud architecture and release patterns, namely microservices

Modern applications are collections of microservices, which are light, fault tolerant and small. Microservices can run in containers deployed on a private or public cloud.

Which means, that a modern application is something that can adapt to any environment and perform equally well.

Note: App modernization can also mean, that you must move your application from .NET Framework to .NET Core.

I have a customer, that is just getting started with the app modernization topic and has hundreds of Windows applications based on the .NET Framework. Porting an existing .NET app to .NET Core requires some work, but is the general recommendation for the future. This would also give you the option to run your .NET Core apps on Windows, Linux and macOS (and not only on Windows).

A modern application is something than can run on bare-metal, VMs, public cloud and containers, and that easily integrates with any component of your infrastructure. It must be something, that is elastic. Something, that can grow and shrink depending on the load and usage. Since it is something that needs to be able to adapt, it must be agile and therefore portable.

Cloud Native Architectures and Modern Designs

If I ask my VMware colleagues from our so-called MAPBU (Modern Application Platform Business Unit) how customers can achieve application portability, the answer is always: “Cloud Native!”

Many organizations and people see cloud native as going to Kubernetes. But cloud native is so much more than the provisioning and orchestration of containers with Kubernetes. It’s a about collaboration, DevOps, internal processes and supply chains, observability/self-healing, continuous delivery/deployment and cloud infrastructure.

There are so many definitions around “cloud native”, that Kamal Arora from Amazon Web Services and others wrote the book “Cloud Native Architecture“, which describes a maturity model. This model helps you to understand, that cloud native is more a journey than only restrictive definition.

Cloud Native Maturity Model

The adoption of cloud services and applying an application-centric design are very important, but the book also mentions that security and scalability rely on automation. And this for example could bring the requirement for Infrastructure as Code (IaC).

In the past, virtualization – moving from bare-metal to vSphere – didn’t force organizations to modernize their applications. The application didn’t need to change and VMware abstracted and emulated the bare-metal server. So, the transition (P2V) of an application was very smooth and not complicated.

And this is what has changed today. We have new architectures, new technologies and new clouds running with different technology stacks. We have Kubernetes as framework, which requires applications to be redesigned for these platforms.

That is the reason why enterprises have to modernize their applications.

One of the “five R’s” mentioned above is the lift and shift approach. If you don’t want or need to modernize some of your applications, but move to the public cloud in an easy, fast and cost efficient way, have a look at VMware’ hybrid cloud extension (HCX).

In this article I focus more on the replatform and refactor approaches in a multi-cloud world.

Kubernetize and productize your applications

Assuming that you also define Kubernetes as the standard to orchestrate your containers where your microservices are running in, usually the next decision would be about the Kubernetes “product” (on-prem, OpenShift, public cloud).

Looking at the current CNCF Cloud Native Landscape, we can count over 50 storage vendors and over 20 networks vendors providing cloud native storage and networking solutions for containers and Kubernetes.

Talking to my customers, most of them mention the storage and network integration as one of their big challenges with Kubernetes. Their concern is about performance, resiliency, different storage and network patterns, automation, data protection/replication, scalability and cloud portability.

Why do organizations need portability?

There are many use cases and requirements that portability (infrastructure independence) becomes relevant. Maybe it’s about a hardware refresh or data center evacuation, to avoid vendor/cloud lock-in, not enough performance with the current infrastructure or it could be about dev/test environments, where resources are deployed and consumed on-demand.

Multi-Cloud Application Portability with VMware Tanzu

To explore the value of Tanzu, I would like to start by setting the scene with the following customer use case:

In this case the customer is following a cloud-appropriate approach to define which cloud is the right landing zone for their applications. They decided to develop new applications in the public cloud and use the native services from Azure and AWS. The customers still has hundreds of legacy applications (monoliths) on-premises and didn’t decide yet, if they want to follow a “lift and shift and then modernize” approach to migrate a number applications to the public cloud.

Multi-Cloud App Portability

But some of their application owners already gave the feedback, that their applications are not allowed to be hosted in the public cloud, have to stay on-premises and need to be modernized locally.

At the same time the IT architecture team receives the feedback from other application owners, that the journey to the public cloud is great on paper, but brings huge operational challenges with it. So, IT operations asks the architecture team if they can do something about that problem.

Both cloud operations for Azure and AWS teams deliver a different quality of their services, changes and deployments take longer with one of their public clouds, they have problems with overlapping networks, different storage performance characteristics and APIs.

Another challenge is the role-based access to the different clouds, Kubernetes clusters and APIs. There is no central log aggregation and no observability (intelligent monitoring & alerting). Traffic distribution and load balancing are also other items on this list.

Because of the feedback from operations to architecture, IT engineering received the task to define a multi-cloud strategy, that solves this operational complexity.

Notes: These are the regular multi-cloud challenges, where clouds are the new silos and enterprises have different teams with different expertise using different management and security tools.

This is the time when VMware’s multi-cloud approach Tanzu become very interesting for such customers.

Consistent Infrastructure and Management

The first discussion point here would be the infrastructure. It’s important, that the different private and public clouds are not handled and seen as silos. VMware’s approach is to connect all the clouds with the same underlying technology stack based on VMware Cloud Foundation.

Beside the fact, that lift and shift migrations would be very easy now, this approach brings two very important advantages for the containerized workloads and the cloud infrastructure in general. It solves the challenge with the huge storage and networking ecosystem available for Kubernetes workloads by using vSAN and NSX Data Center in any of the existing clouds. Storage and networking and security are now integrated and consistent.

For existing workloads running natively in public clouds, customers can use NSX Cloud, which uses the same management plane and control plane as NSX Data Center. That’s another major step forward.

Using consistent infrastructure enables customers for consistent operations and automation.

Consistent Application Platform and Developer Experience

Looking at organization’s application and container platforms, achieving consistent infrastructure is not required, but obviously very helpful in terms of operational and cost efficiency.

To provide a consistent developer experience and to abstract the underlying application or Kubernetes platform, you would follow the same VMware approach as always: to put a layer on top.

Here the solution is called Tanzu Kubernetes Grid (TKG), that provides a consistent, upstream-compatible implementation of Kubernetes, that is tested, signed and supported by VMware.

A Tanzu Kubernetes cluster is an opinionated installation of Kubernetes open-source software that is built and supported by VMware. In all the offerings, you provision and use Tanzu Kubernetes clusters in a declarative manner that is familiar to Kubernetes operators and developers. The different Tanzu Kubernetes Grid offerings provision and manage Tanzu Kubernetes clusters on different platforms, in ways that are designed to be as similar as possible, but that are subtly different.

VMware Tanzu Kubernetes Grid (TKG aka TKGm)

Tanzu Kubernetes Grid can be deployed across software-defined datacenters (SDDC) and public cloud environments, including vSphere, Microsoft Azure, and Amazon EC2. I would assume, that the Google Cloud is a roadmap item.

TKG allows you to run Kubernetes with consistency and makes it available to your developers as a utility, just like the electricity grid. TKG provides the services such as networking, authentication, ingress control, and logging that a production Kubernetes environment requires.

This TKG version is also known as TKGm for “TKG multi-cloud”.

VMware Tanzu Kubernetes Grid Service (TKGS aka vSphere with Tanzu)

TKGS is the option vSphere admins want to hear about first, because it allows you to turn a vSphere cluster to a platform running Kubernetes workloads in dedicated resources pools. TKGS is the thing that was known as “Project Pacific” in the past.

Once enabled on a vSphere cluster, vSphere with Tanzu creates a Kubernetes control plane directly in the hypervisor layer. You can then run Kubernetes containers by deploying vSphere Pods, or you can create upstream Kubernetes clusters through the VMware Tanzu Kubernetes Grid Service and run your applications inside these clusters.

VMware Tanzu Mission Control (TMC)

In our use case before, we have AKS and EKS for running Kubernetes clusters in the public cloud.

The VMware solution for multi-cluster Kubernetes management across clouds is called Tanzu Mission Control, which is a centralized management platform for the consistency and security the IT engineering team was looking for.

Available through VMware Cloud Services as SaaS offering, TMC provides IT operators with a single control point to provide their developers self-service access to Kubernetes clusters.

TMC also provides cluster lifecycle management for TKG clusters across environment such as vSphere, AWS and Azure.

It allows you to bring the clusters you already have in the public clouds or other environments (with Rancher or OpenShift for example) under one roof via the attachment of conformant Kubernetes clusters.

Not only do you gain global visibility across clusters, teams and clouds, but you also get centralized authentication and authorization, consistent policy management and data protection functionalities.

VMware Tanzu Observability by Wavefront (TO)

Tanzu Observability extends the basic observability provided by TMC with enterprise-grade observability and analytics.

Wavefront by VMware helps Tanzu operators, DevOps teams, and developers get metrics-driven insights into the real-time performance of their custom code, Tanzu platform and its underlying components. Wavefront proactively detects and alerts on production issues and improves agility in code releases.

TO is also a SaaS-based platform, that can handle the high-scale requirements of cloud native applications.

VMware Tanzu Service Mesh (TSM)

Tanzu Service Mesh, formerly known as NSX Service Mesh, provides consistent connectivity and security for microservices across all clouds and Kubernetes clusters. TSM can be installed in TKG clusters and third-party Kubernetes-conformant clusters.

Organizations that are using or looking at the popular Calico cloud native networking option for their Kubernetes ecosystem often consider an integration with Istio (Service Mesh) to connect services and to secure the communication between these services.

The combination of Calico and Istio can be replaced by TSM, which is built on VMware NSX for networking and that uses an Istio data plane abstraction. This version of Istio is signed and supported by VMware and is the same as the upstream version. TSM brings enterprise-grade support for Istio and a simplified installation process.

One of the primary constructs of Tanzu Service Mesh is the concept of a Global Namespace (GNS). GNS allows developers using Tanzu Service Mesh, regardless of where they are, to connect application services without having to specify (or even know) any underlying infrastructure details, as all of that is done automatically. With the power of this abstraction, your application microservices can “live” anywhere, in any cloud, allowing you to make placement decisions based on application and organizational requirements—not infrastructure constraints.

Note: On the 18th of March 2021 VMware announced the acquisition of Mesh7 and the integration of Mesh7’s contextual API behavior security solution with Tanzu Service Mesh to simplify DevSecOps.

Tanzu Editions

The VMware Tanzu portfolio comes with three different editions: Basic, Standard, Advanced

Tanzu Basic enables the straightforward implementation of Kubernetes in vSphere so that vSphere admins can leverage familiar tools used for managing VMs when managing clusters = TKGS

Tanzu Standard provides multi-cloud support, enabling Kubernetes deployment across on-premises, public cloud, and edge environments. In addition, Tanzu Standard includes a centralized multi-cluster SaaS control plane for a more consistent and efficient operation of clusters across environments = TKGS + TKGm + TMC

Tanzu Advanced builds on Tanzu Standard to simplify and secure the container lifecycle, enabling teams to accelerate the delivery of modern apps at scale across clouds. It adds a comprehensive global control plane with observability and service mesh, consolidated Kubernetes ingress services, data services, container catalog, and automated container builds = TKG (TKGS & TKGm) + TMC + TO + TSM + MUCH MORE

Tanzu Data Services

Another topic to reduce dependencies and avoid vendor lock-in would be Tanzu Data Services – a separate part of the Tanzu portfolio with on-demand caching (Tanzu Gemfire), messaging (Tanzu RabbitMQ) and database software (Tanzu SQL & Tanzu Greenplum) products.

Bringing all together

As always, I’m trying to summarize and simplify things where needed and I hope it helped you to better understand the value and capabilities of VMware Tanzu.

There are so many more products available in the Tanzu portfolio, that help you to build, run, manage, connect and protect your applications.

If you would like to know more about application and cloud transformation make sure to attend the 45 minute VMware event on March 31 (Americas) or April 1 (EMEA/APJ)!

Multi-Tenancy on VMware Cloud Foundation with vRealize Automation and Cloud Director

Multi-Tenancy on VMware Cloud Foundation with vRealize Automation and Cloud Director

In my article VMware Cloud Foundation And The Cloud Management Platform Simply Explained I wrote about why customers need a VMware Cloud Foundation technology stack and what a VMware cloud management platform is.

One of the reasons and one of the essential characteristics of a cloud computing model I mentioned is resource pooling.

By the National Institute of Standards and Technology (NIST) resource pooling is defined with the following words:

The provider’s computing resources are pooled to serve multiple
consumers using a multi-tenant model, with different physical and virtual
resources dynamically assigned and reassigned according to consumer demand.
There is a sense of location independence in that the customer generally has no
control or knowledge over the exact location of the provided resources but may be
able to specify location at a higher level of abstraction (e.g., country, state, or
data center).

This time I would like to focus on multi-tenancy and how you can achieve that on top of VMware Cloud Foundation (VCF) with Cloud Director (formerly known as vCloud Director) and vRealize Automation, which both could be part of a VMware cloud management platform (CMP).

Multi-Tenancy

There are many understandings around about multi-tenancy and different people have different definitions for it.

If we start from the top of an IT infrastructure, we will have application or software multi-tenancy with a single instance of an application serving multiple tenants. And in the past even running on the same virtual or physical server. In this case the multi-tenancy feature is built into the software, which is commonly accessed by a group of users with specific permissions. Each tenant gets a dedicated or isolated share of this application instance.

Coming from the bottom of the data center, multi-tenancy describes the isolation of resources (compute, storage) and networks to deliver applications. The best example here are (cloud) services providers.

Their goal is to create and provide virtual data centers (VDC) or a virtual private cloud (VPC) on top of the same physical data center infrastructure – for different tenants aka customers. Normally, the right VMware solution for this requirement and service providers would be Cloud Director, but this is maybe not completely true anymore with the release of vRealize Automation 8.x. 

To make it easier for all of us, I’ll call Cloud Director and vCloud Director “vCD” from now on.

VMware Cloud Director (formerly vCloud Director)

Cloud Director is a product exclusively for cloud service providers via the VMware Cloud Provider Program (VCPP). Originally released in 2010, it enables service providers (SPs) to provision SDDC (Software-Defined Data Center) services as complete virtual data centers. vCD also keeps resources from different tenants isolated from each other.

Within vCD a unit of tenancy is called Organization VDC (OrgVDC). It is defined as a set of dedicated compute (CPU, RAM), storage and network resources. A tenant can be bound to a single OrgVDC or can be composed of multiple Organization VDCs. This is typically known as Infrastructure as a Service (IaaS).

A provider virtual data center (PVDC) is a grouping of compute, storage, and network resources from a single vCenter Server instance. Multiple organizations/tenants can share provider virtual data center resources.

Cloud Director Resource Abstraction

A lot of customers and VCPP partners have now started to offer their cloud services (IaaS, PaaS, SaaS etc.) based on VMware Cloud Foundation. For private and hybrid cloud scenarios, but also in the public cloud as a managed cloud service (VMware Cloud on AWS, Azure VMware Solution, Google Cloud VMware Engine, Alibaba Cloud VMware Solution and more).

Important: I assume that you are familiar with VCF, its core components (ESXi, vSAN, NSX, SDDC Manager) and architecture models (standard as the preferred).

Cloud Director components are currently not part of the VCF lifecycle automation, but it is a roadmap item!

Cloud Director Resource Hosting Models

vCD offers multiple hosting models:

  • In the shared hosting model, multiple tenant workloads run all together on the same
    resource groups without any performance assurance
  • In the reserved hosting model, performance of workloads is assured by resource
    reservation.
  • In the physical hosting model, hardware is dedicated to a single tenant and performance
    is assured by the allocated hardware

Tenant Using Shared Hosting on VCF Workload Domain

In this use case a tenant is using shared hosting backed by a VMware Cloud Foundation workload domain. A workload domain, which is mapped to a provider VDC.

vCD VCF Shared

Tenant Using Shared Hosting and Reserved Hosting on Multiple VCF Workload Domains

This use case describes the example of customer using shared and reserved hosting backed by multiple VCD workload domains. Here each cluster has a single resource pool mapped to a single PVDC.

vCD VCF Shared Reserved

Tenant Using Physical Hosting and Central Point of Management (CPOM)

The last example shows a single customer using physical hosting. You will notice that there is also a vSphere with
Kubernetes workload domain. VMware Cloud Foundation automates the installation of vSphere with Kubernetes (Tanzu) which makes it incredibly easy to deploy and manage.

You can see that there is an “SDDC” box on top of the Kubernetes Cluster vCenter, which is attached to
the “SDDC Proxy” entity. vCD can act as an HTTP/S proxy server between tenants and the
underlying vSphere environment in VMware Cloud Foundation. An SDDC proxy is an
access point to a component from an SDDC, for example, a vCenter Server instance, an ESXi host, or
an NSX Manager instance.

The vCD becomes the central point of management (CPOM) in this case and the customer gets a complete dedicated SDDC with vCenter access.

vCD VCF Physical CPOM

Note: Since vCD 9.7 it is possible to present for example a vCenter Server instance securely to a tenant’s organization using the Cloud Director user interface. This is how you could build your own VMC-on-AWS-like cloud offering!

Cloud Director CPOM

All 3 Tenants Together

Finally, we put it all together. In the first use case we can see that different customers are sharing resources from a
single PVDC. We can also see that resources from a single vCenter can be split across different provider virtual datacenters and that we can mix and match multi-tenants workload domains and workload domains offering dedicated private cloud all together.

vCD VCF All Together

Cloud Director Service and VMware Cloud on AWS

If you don’t want to extend or operate your own data center or cloud infrastructure anymore and provide a managed service to multiple customer, there are still options for you available backed by VMware Cloud Foundation as well.

Since October 2020 you have Cloud Director Service globally available, which delivers multi-tenancy to VMware Cloud on AWS for managed service providers (MSP).

VMware sees not only new, but also existing VCPP partners moving towards a mixed-asset portfolio, where their cloud management platform consists of a VCPP and MSP (VMware SaaS offerings) contract. This allows them for example to run vCD on-premises for their current customers and the onboarding of new tenants would happen in the public cloud with CDS and VMC on AWS.

vCD CDS Mixed Mode

Enterprise Multi-Tenancy with vRealize Automation

With the release of vRealize Automation 8.1 (vRA) VMware offered support for dedicated infrastructure multi-tenancy, created and managed through vRealize Suite Lifecycle Manager. This means vRealize Automation enables customers or IT providers to set up multiple tenants or organizations within each deployment.

Providers can set up multiple tenant organizations and allocate infrastructure. Each tenant manages its own projects (team structures), resources and deployments.

Enabling tenancy creates a new Provider (default) organization. The Provider Admin will create new tenants, add tenant admins, setup directory synchronization, and add users. Tenant admins can also control directory synchronization for their tenant and will grant users access to services within their tenant. Additionally, tenant admins will configure Policies, Governance, Cloud Zones, Profiles, access to content and provisioned resources; within their tenant. A single shared SDDC or separate SDDCs can be used among tenants depending on available resources.

vRealize Automation 8.1 Multi-Tenancy

With vRealize Automation 8.2, provider administrators got the ability to share infrastructure by creating and assigning Virtual Private Zones (VPZ) to tenant organizations.

Think of VPZs as a kind of container of infrastructure capacity and services which can be defined and allocated to a Tenant. You can add unique or shared cloud accounts, with associated compute, flavors, images, storage, networking, and tags to each VPZ. Each component offers the same configuration options you would see for a standalone configuration.

vRealize Automation 8.2 Multi-Tenancy

vRealize Automation and VMware Cloud Foundation

With the pretty new multi-tenancy and VPZ capability a new consumption model on top of VCF can be built. You (provider) would map the Cloud Zones (compute resources on vSphere (or AWS for example)) to a VCF workload domain.

The provider sets these cloud zones up for their customers and provides dedicated or shared infrastructure backed by Cloud Foundation workload domains.

This combination would allow you to build an enterprise VPC construct (like AWS for example), a logically isolated section of your provider cloud.

vRealize Automation and VMware Cloud Foundation

SDDC Manager Integration and VMware Cloud Foundation (VCF) Cloud Account

Since the vRA 8.2 release customers are also able to configure a SDDC Manager integration and on-board workload domains as VMware Cloud Foundation cloud accounts into the VMware Cloud Assembly service.

VMware Cloud Director or vRealize Automation?

You wonder if vRealize Automation could replace existing vCD installations? Or if both cloud management platforms can do the same?

I can assure you, that you can provide a self-service provisioning experience with both solutions and that you can provide any technology or cloud service “as a service”. Both have in common to be backed by Cloud Foundation, have some form of integration (vRA) and can be built by a VMware Validated Design (VVD).

vCD is known to be a service provider solution, where vRA is more common in enterprise environments. VMware has VCPP partners, that use Cloud Director for their external customers and vRealize Automation for their internal IT and customers.

If you are looking for a “cloud broker” and Infrastructure as Code (IaC), because you also want to provision workloads on AWS, Azure or GCP as well, then vRealize Automation is the better solution since vCD doesn’t offer this deep integration and these deployment options yet.

Depending on your multi-tenant needs and if you for example only have chosen vCD in the past, because of the OrgVDC and resource pooling feature, vRealize Automation would be enough and could replace vCD in this case.

It is also very important to understand how your current customer onboarding process and operational model look like:

  • How do you want to create a new tenant? 
  • How do you want to onboard/migrate existing customer workloads to your provider infrastructure?
  • Do you need versioning of deployments or templates?
  • Do customers require access to the virtual infrastructure (e.g. vCenter or OrgVDC) or do you just provide SaaS or PaaS?
  • Do customers need a VPN or hybrid cloud extension into your provider cloud?
  • How would you onboard non-vSphere customers (Hyper-V, KVM) to your vSphere-based cloud?
  • Does your customer rely on other clouds like AWS or Azure?
  • How do you do billing for your vSphere-based cloud or multi-cloud environment?
  • What is your Kubernetes/container strategy?
  • And 100 other things 😉

There are so many factors and criteria to talk about, which would influence such a decision. There is no right or wrong answer to the question, if it should be VMware Cloud Director or vRealize Automation. Use what makes sense.

Which could also be a combination of both.

Multi-Cloud Load Balancing and Autoscaling with NSX Advanced Load Balancer (formerly Avi Networks)

Multi-Cloud Load Balancing and Autoscaling with NSX Advanced Load Balancer (formerly Avi Networks)

Do you want to build your private cloud like a hyperscaler is doing it? You know that VMware Cloud Foundation is becoming the new vSphere, but still wonder how you can implement software-defined load balancing (LB) or application services and features like autoscaling or predictive scaling? Then this article about multi-cloud load balancing and autoscaling with NSX Advanced Load Balancer aka Avi Networks is for you!

My Experience with a Legacy ADC

A few years ago, I was working on the customer side for an insurance company in Switzerland as a Citrix System Engineer. My daily tasks included the maintenance and operation of the Citrix environment, which included physical and virtual Citrix NetScaler Application Delivery Controller (ADC) appliances. The networking team owned a few hardware-based appliances (NetScaler SDX) with integrated virtualization capability (XenServer as hypervisor) to host multiple virtual NetScaler (VPX) instances.

The networking team had their dedicated NetScaler VPX instances (for LDAP and HTTP load balancing mostly) and deployed my appliances after I filed a change request. Today, you would call this multi-tenancy. For a Citrix architecture is was best practices to have one high availability (HA) pair for the internal and one HA pair for the external (DMZ) network access. A HA pair was running in a active/passive mode and I had to maintain the same setup for the test environment as well.

Since my virtual VPX appliances were hosted on the physical SDX appliance, I always relied on the network engineers, if I needed more resources (CPU, RAM, SSL, throughput) chips allocated to my virtual instances. Before I could upgrade to a specific firmware version, I also had to wait until they upgraded the physical NetScaler appliances and approved my change request. This meant, we had to plan changes and maintenance windows together and had to cross fingers, that their upgrade went well, that we could upgrade all our appliances after.

NetScaler SDX

It was also possible to download a VPX appliance, which could run on top of VMware vSphere. To be more independent, I decided to install four new VPX appliances (for the production and test environment) on vSphere and migrated the configuration from the appliances running on the physical SDX appliance.

Another experience I had with load balancers was when I started to work for Citrix as a consultant in Central Europe and had to perform a migration of physical NetScaler MPX appliances, which had no integrated virtualization capability. I believe I had two sites with each two of these powerful MPX appliances for tens of thousands of users. Beside the regular load balancing configuration for some of the Citrix components, I also had to configure Global Server Load Balancing (GSLB) in active/passive mode for the two sites.

NetScaler GSLB Active Passive

There were so many more features available (e.g. Web Application Firewall, Content Switching, Caching, Intrusion Detection), but I never used anything else than the NetScaler Universal Gateway for the remote access to the virtual desktop infrastructure (VDI), load balancing, HTTP to HTTPS redirections and GSLB. In all scenarios I had a HA pair where one instance was idling and doing nothing. And the active unit was in average not utilized more than 15-20%. It was common to install/buy too large or powerful instances/licenses, because you wanted to be on the safe side and have enough capacity to terminate all your SSL sessions and so on.

It (load balancing) was about distributing network traffic across multiple servers by spreading the requests and work evenly, and do add some intelligence (health monitoring) in case an application server or a service would fail or be unavailable for any reason. If one more application server was needed, I ordered a new Windows Server, installed and configured the Citrix components and added the necessary load balancing configuration on the NetScaler. These were all manual tasks. The same work has been done by the network engineers when the application team requested a new application server, which then had to be added to the load balancing configuration on their NetScaler appliances.

This was my personal experience from 2017. Since then applications became more complex and distributed. The analysts and market are focusing on containerized and portable apps running and more and more in multiple clouds. The prediction is also that the future is multi-cloud.

Multiple Clouds vs. Multi-Cloud

There are different definitions and understandings out there what multi-cloud means. In my understanding, using a private cloud, AWS, Microsoft Office 365 and Azure are a typical setup with multiple clouds. There are simple scenarios where you migrate workloads from the private to the public cloud (e.g. Azure) or having applications with services lying on the private on the public cloud. The latter would be an example of a hybrid cloud architecture.

The reasons for which services and resources are needed or distributed on multiple clouds (on-prem, Azure, AWS, GCP etc.) are various:

  • Avoid dependence on only one cloud provider
  • Consume different specific services that are not provided elsewhere
  • Optimize costs for different workloads and services
  • React to price changes by the providers

That is why we are seeing also the trend to break up big legacy applications (monoliths) in smaller pieces (segments), which is a best practice and design principle today. The goal is to move to a loosely coupled and more service-oriented architecture. This provides greater agility, more flexibility and easier scalability, because of less inter-dependencies.

And, if we take the second example from the list before, a segmented application is much easier to run in different clouds (portability). Running one application over multiple clouds is in my understanding the right definition of multi-cloud.

Multiple Clouds versus Multi-Cloud

Let’s assume that most probably all the four reasons above apply to larger enterprises. If we take another angle, we can define some business and technical requirements for multi-cloud:

  • Application or services need to be cloud vendor-agnostic
  • Provide or abstract control and management interface of multiple clouds
  • Support application portability/relocation between clouds
  • Combine IaaS and services from different clouds
  • Possibility to deploy components of applications in multiple clouds
  • (Cloud) Broker service needed
  • Policy and governance over multiple clouds
  • Network connectivity for migration scenarios with partially modernized applications
  • Automated procedures for deployments
  • Application monitoring over different clouds
  • Costs management
  • Lifecycle management of deployed applications in multiple clouds
  • Self-adaption and auto scaling features
  • Large team with various expertises needed

How can you deliver and manage the different applications services like load balancing, web application firewalling, analytics, automation and security over multiple clouds?

Another important question would be, how you want to manage the deployment on the various clouds. But cloud management or a cloud management platform is something for another article. 🙂

The requirements for the developers, operations and the business are very complex and it’s a long list (see above).

It is important, that you understand, based on the requirements for multi-cloud, that it is mandatory to implement a modern solution for your modernized application architectures. Enterprises have become more application-centric and everyone is talking about continious integration, continuous delivery and DevOps practices to automate operation and deployment tasks. A modern solution implicits a software-defined approach. Otherwise you won’t be able to be agile, adapt to changes and meet future requirements.

My past experience with Citrix’ NetScaler is a typical example that “virtualized” and “software-defined” are not the same thing. And this is very important if we want to have a future-ready solution. If we look at VMware’s software-defined data center (SDDC), beside the virtual compute, also includes software-defined storage and networking. Part of the software-defined networking portfolio is “NSX Advanced Load Balancer“, the software-defined application services platform, which was also known as “Avi Networks” before VMware bought them in June 2019.

Unlike a virtualized load-balancing appliance, a software-defined
application services platform separates the data and control planes
to deliver application services beyond load balancing, real-time
application analytics, security and monitoring, predictive autoscaling,
and end-to-end automation for Transport (Layer 4) to
Application (Layer 7) layer services. The platform supports multicloud
environments and provides software-defined application
services with infrastructure-agnostic deployments on bare metal
servers, virtual machines (VMs), and containers, in on-premises
data centers and private/public clouds.

Autoscaling became famous with AWS as it monitors your applications and automatically adjusts capacity to maintain availability and performance at the lowest possible costs. It automatically adds or removes application servers (e.g. EC2 instances), load balancers, applies the right network configuration and so on.

Can you achieve the same for your on-premises infrastructure with VMware? Yes.

Is there even a solution which can serve both worlds – on-prem and cloud? Yes.

And what about predictive scaling with real-time insights? Yes.

NSX Advanced Load Balancer (NSX ALB)

Why did VMware buy Avi? Because it follows the same architecture principles like NSX: A distributed platform with a separate control and data plane built on software-defined principles for any cloud.

Avi High Level Architecture

Traditional ADCs or load balancers are mostly configured in active/standby pairs, no matter if physical or virtual. Typically you would see around 15% utilization on the active node where the secondary standby node is just idling and doing nothing. Each pair is its own island of static capacity which shares the management, control and data plane.

You have to decide where to place the virtual IP (VIP) and how much you want to overprovision the physical or virtual appliances, because there is no capacity pooling available. This leads to operational complexity, especially when you have hundreds of such HA pairs running in different clouds. Therefore, legacy and virtualized ADCs are not the ideal choice for a multi-cloud architecture. Let’s check NSX ALB’s architecture:

Control Plane – This is the brain (single point of management) of the whole platform that can be spun up in your on-prem environment or in the cloud (also available as a managed SaaS offering), typically as a three-node cluster. Within this cluster, all configuration is done, this also where the policies reside and the decisions are made. It is the controller’s duty to place virtual services on SEs to load balance new applications or increase the capacity of running applications.

The control plane comprises the three pillars that deliver the key capabilities of the Avi platform:

  • Multi-Cloud – Consistent experience for any cloud, no lock-in
  • Intelligence – The machine learning based analytics engine enables application performance monitoring, troubleshooting, and operational insights (gathered by the SEs)
  • Automation – Elastic and predictive auto scaling & self-service without over-provisioning through a complete set of REST APIs

Data Plane –  The Service Engines (SEs) handle all data plane operations by receiving and executing instructions from the controller. The SEs perform load balancing and all client- and server-facing network interactions. It collects real-time application telemetry from application traffic flows. 

As already mentioned, NSX ALB can be deployed in multiple cloud environments like VMware vCenter, Amazon Web Services, Microsoft Azure, Google Cloud Platform, Oracle Cloud, IBM Cloud, VMC on AWS, Nutanix, OpenStack or bare-metal.

Use Cases

Most customers deploy Avi because of:

  • Load Balancer refresh
  • Multi-Cloud initiatives
  • Security including WAF, DDoS attack mitigation, achieve compliance (GDPR, PCI, HIPAA)
  • Container ingress (integrates via REST APIs with K8s ecosystems like GKE, PKS (TKGI), OpenShift, EKS, AKS, TKG)

Advanced Kubernetes Ingress Controller Avi Networks

  • Virtual Desktop Infrastructure (Citrix, VMware Horizon)

Consistent Application Services Platform (Features)

Avi/NSX ALB is an enterpise-grade solution. So, everything you would expect from a traditional ADC (e.g. F5), layer 4 to layer 7 services, SSL, DDoS, WAF etc. is built-in without the need for a special license edition. There is also no NSX license requirement even the product name would suggest it. It can be deployed as a standalone load balancer or as an integrated solution with other VMware products (e.g. VCF, vRA/vRO, Horizon, Tanzu etc.).

Avi Networks Features

Below is a list with the core features:

  • Enterprise-class load balancing – SSL termination, default gateway, GSLB, DNS, and other L4-L7 services
  • Multi-cloud load balancing – Intelligent traffic routing across multiple sites and across private or public clouds
  • Application performance monitoring – Monitor performance and record and replay network events like a Network DVR
  • Predictive autoscaling Application and load balancer scaling based on real-time traffic patterns
  • Self-service – For app developers with REST APIs to build services into applications
  • Cloud connectors – VMware Cloud on AWS, SDN/NFV controllers, OpenStack, AWS, GCP, Azure, Linux Server Cloud, OpenShift/Kubernetes
  • Distributed application security fabric – Granular app insights from distributed service proxies to secure web apps in real time
  • SSO / Client Authentication – SAML 2.0 authentication for back-end HTTP applications
  • Automation and programmability – REST API based solution for accelerated application delivery; extending automation from networking to developers
  • Application Analytics – Real-time telemetry from a distributed load balancing fabric that delivers millions of data points in real time

Load Balancing for VMware Horizon

NSX ALB can be configured for load balancing in VMware Horizon deployments, where you place SEs in front of Unified Access Gateways (UAG) or Connection Servers (CS) as required.

Avi Horizon High Level Architecture

For a multi-site architecture you can also configure GSLB if needed. With GSLB, access to resources is controlled with DNS queries and health checking.

Note: If you are using the Horizon Universal Broker, the cloud-based brokering service, there is no need for GSLB, because the Universal Broker can orchestrate connections from a higher level based on different policies.

Automation

With NSX Advanced Load Balancer there are two parts when we talk about automation. One part is about infrastructure automation, where the controller talks to the ecosystem like a vCenter, AWS or Azure to orchestrate the Service Engine. So, when you configure a new VIP, the controller would talk to vCenter to spin up a VM, put it in the right portgroup, connect the front and the back-end, download the policy and service engine, and starts receiveing traffic.

The second piece of automation focuses more on the operational automation which is through the REST API (the UI and CLI don’t offer all the configuration, 100% can be done via REST API). But, on top of that you can also run Ansible playbooks, Terraform templates, Go and Python SDKs, have integrations with Splunk or other tools like vRealize Automation. This is the built-in automation in the product.

Avi Networks Automation

VMworld 2020 Sessions

This year VMworld is going to be for free and virtual. Take this chance and register yourself and learn more about Avi aka NSX ALB:

  1. Making Your Private Cloud Network Run Like a Public Cloud – Part 2 [VCNC2918]
  2. Modern Apps and Containers: Networking and Security [VCNC2920]
  3. Prepare for the New Normal of Work from Anywhere [VCNC2919]

Expectations and Current Approaches

There is the general understanding and need for hybrid or multi-cloud architectures. Different people will tell different stories and give different advices. The result are different architectures and different approaches. Some people will tell you, that you can use a cloud serially, so moving from one cloud to another. Or, simultaneously, when using different services from different clouds.

My last article focused on hybrid cloud, the architecture with some services lying on the private infrastructure, while other services are hosted on a public cloud. A public cloud providers tells you, that you can buy all services from them and tries to give you a better discount than the competition (to avoid multiple clouds). Enterprises see the need for multiple (public) clouds to avoid a vendor lock-in instead of going all-in with just one of them.

VMware is about multi-cloud and workload mobility, with the vision, that their VCF stack is running everywhere in the future. Now, some people would now say that this is also a vendor lock-in. Depending on your strategy and technology choices and preferences (e.g. databases, AI/ML services, virtual desktops), you have to decide somewhen which (cloud) vendor, approach and operation model is the right one for you.

It may not true for every large environment, but if you go for multiple clouds, multiple technologies, management and security consoles, architecture and so on, you’ll spend a lot of time and money on engineering and keeping your environment “integrated” and functional.

VMware offers you choice. The choice to run your workloads today and tomorrow wherever you want.

If you have the same vision and strategy like VMware, then you are looking for solutions which run in or on top of every cloud. Because of that it’s very important to understand the different between multiple clouds and multi-cloud.

In this case, NSX ALB brings you multi-cloud load balancing and auto scaling features for any cloud and for multi-cloud enabled applications and services.

Don’t forget: Some people are also saying,  that multi-cloud is not needed and doesn’t exist in reality. Nobody is saying multi-cloud is a piece of cake, but VMware can definitely help you to abstract this complexity. And part of this abstraction can be handled with vRealize Automation for example, which can act as a cloud broker to deploy your application and services.

 

Cross-Cloud Mobility with VMware HCX

Cross-Cloud Mobility with VMware HCX

Update 10th September 2020: vSphere 7.0 (VDS 7.0) and NSX-T 3.0.1+ are supported since the HCX R143 release which has been made available on September 8, 2020

https://docs.vmware.com/en/VMware-HCX/services/rn/VMware-HCX-Release-Notes.html 

Most people think that VMware HCX is a only migration tool that helps you moving workloads to a vSphere based cloud like VMware Cloud on AWS, Azure VMware Solution or Google Cloud VMware Engine. But it can do so much more for you than only application or workload migrations. HCX is also designed for workload rebalancing and business continuity across data centers or VMware clouds. Why I say “across VMware clouds” and not only “clouds”?

A few years ago everyone thought or said that customers will move all their workloads to the public cloud and the majority of them don’t need local data centers anymore. But we all know that this perception has changed and that the future cloud operation is model hybrid.

A hybrid cloud environment leverages both the private and public clouds to operate. A multi-cloud environment includes two or more public cloud vendors which provide cloud-based services to a business that may or may not have a private cloud. A hybrid cloud environment might also be a multi-cloud environment.

We all know that the past perception was an illusion and we didn’t have a clue where the hyperscalers like AWS, Azure or GCP would be in the next 5 or 7 years. And I believe that even the AWS and Microsoft didn’t expect what is going to happen since we observed interesting shifts in the last few years.

Amazon Web Services (AWS) has been launched 14 years ago (2006) to provide web services from the cloud. At AWS re:Invent 2018 the CEO Andy Jassy announced AWS Outposts because their customers have been asking for an AWS option on-premises. In the end, Outpost is just an extension of an AWS region into the own data center, where you can launch EC2 instances or Amazon EBS volumes locally. AWS already had some hybrid services available (like Storage Gateway) but here we talk about infrastructure and making your own data center part of the AWS Global Infrastructure.

Microsoft Azure was released in 2010 and the first technical preview of Azure Stack has been announced in 2016. So, Microsoft also realized that the future cloud model is a hybrid approach “that provides consistency across private, hosted and public clouds”.

Google Cloud Platform (GCP) offers cloud computing services since 2008. Eleven years later (in 2019) Google introduced Anthos that you can “run an app anywhere –  simply, flexibly and securely”.

All the hyperscalers changed their cloud model to provide customers a consistent infrastructure with consistent operations and management as we understand now.

VMware is coming from the other end of this hybrid world and has the same overall goal or vision to make a hybrid or multi-cloud reality. But with one very important difference. VMware helps you to abstract the complexity of a hybrid environment and gives you the choice to run your workloads in any cloud infrastructure with a cloud-agnostic approach.

As organizations try to migrate their workloads to the public, they face multiple challenges and barriers:

  • How can I migrate my workload to the public cloud?
  • How long does it take to migrate?
  • What about application downtime?
  • Which migration options do I have?
  • Which cloud is the best destination for which workloads?
  • Do I need to refactor or develop some applications?
  • Can I do a lift and shift migration and modernize the application later?
  • How can I consistently deploy workloads and services for my multi-cloud?
  • How can I operate and monitor (visibility and observability) all the different clouds?
  • What if tomorrow one the public cloud provider is not strategic anymore? How can I move my workloads away?
  • How can I control costs over all clouds?
  • How can I maintain security?
  • What about the current tools and 3rd party software I am using now?
  • What if I want to migrate VMs back from the public cloud?
  • What if I want to move away/back somewhen from a specific cloud provider?

In summary, the challenges with a hybrid cloud are about costs, complexity, tooling and skills. Each public cloud added to your current on-premises infrastructure is in fact a new silo. If you have the extra money and time and don’t need consistent infrastructures and consistent operations and management, you’ll accept the fact that you have a heterogeneous environment with different technology formats, skill/tool sets, operational inconsistencies and security controls.

If you are interested in a more consistent platform then you should build a more unified hybrid cloud. Unified means that you provide consistent operations with the existing skills and tools (e.g. vCenter, vRealize Automation, vRealize Operations) and the same policies and security configuration over all clouds – your data center, public cloud or at the edge.

To provide such a cloud agnostic platform you need to abstract the technology format and infrastructure in the public cloud. This is why VMware built the VMware Cloud Foundation (VCF) platform that delivers a set of software-defined services for compute, storage, networking, security and cloud management for any cloud.

VMC on AWS, Azure VMware Solution, Google Cloud VMware Engine and all the other hybrid cloud offerings (IBM, Oracle, Alibaba Cloud, VCPP) are based on VMware Cloud Foundation. This is the underlying technology stack you would need if your goal is to be independent and to achieve workload mobility between clouds. With this important basic understanding we can take a closer look at VMware HCX.

VMware HCX Use Cases

HCX provides an any-to-any vSphere workload mobility service without requiring retrofit as we use the same technology stack in any cloud. 

VMware HCX Use Cases

HCX enables you to schedule application migrations of hundreds or thousands of vSphere VMs within your data centers and across public clouds without requiring a reboot or a downtime.

If you would like to change the current platform or have to modernize your current data center, HCX allows you to migrate workloads from vSphere 5.x and non-vSphere (Hyper-V and KVM) environments.

VMware HCX Migration

Workload rebalancing means providing a mobility platform across cloud regions and cloud providers to move applications and workloads at any time for any reason.

Workload mobility is cool and may be the future but is not possible today as the public cloud’s egress costs would be way too high at the moment. Let’s say you pay $0.05 per GB when you move data away from the public cloud to any external destination, this would generate costs of $2.50 for a 50GB virtual machine.

Not that much, right? If you move away 500 VMs, the bill would list $1’250 for egress costs. Evacuating VMs from one public cloud to another one is not so expensive if it happens three or four times a year. But if the rebalancing should happen at a higher cadence, the egress costs would get too high. But we can assume that this fact will change in the future as the public cloud computing prices will come down in the future. 

HCX Components and Services

HCX is available with an Advanced and Enterprise license. The Advanced license services are standard with HCX and are also included in the HCX Enterprise license. The Enterprise license is needed when you migrate non-vSphere workloads into a vSphere environment. This capability is called OS Assisted Migration (OSAM).

HCX Services

The HCX Advanced features are included in a NSX Data Center Enterprise Plus license. With a managed service like VMware Cloud on AWS or Azure VMware Solution HCX Advanced is already be included.

HCX Connector Advanced License

If you want to move workloads from a vSphere environment to a vSphere enabled public cloud, you don’t need the complete VMware Cloud Foundation stack at the source site:

  • On-premises vSphere version 5.5 and above
  • Minimum of 100 Mbps bandwidth between source and destination
  • Virtual switch based on vDS (vSphere Distributed Switch), Cisco Nexus 1000v or vSphere Standard Switch
  • Minimum of virtual machine hardware version 9
  • VMs with hard disks not larger than 2TB

Depending on the HCX license and services you need, you have to deploy some or all of the HCX components. HCX comprises components and appliances at the source and destination sites.

HCX Manager Destination

The HCX Connector services and appliances are deployed at the destination site first before you are going to deploy the virtual appliances at the source site (HCX Interconnect appliance).

HCX Interconnect Appliance Download Link

After you deployed the appliances at the source site, you can create the site pairing.

HCX Site Pairing

As soon as you have installed HCX in both sites, you can manage and configure the services within the vSphere Client.

HCX in vSphere Client

After a successful site pairing, you can start to create the HCX Service Mesh.

The Multi-Site Service mesh is used to create a secure optimized transport fabric between any two sites managed by HCX. When HCX Migration, Disaster recovery, Network Extension, and WAN Optimization services are enabled, HCX deploys Virtual Appliances in the source site and corresponding “peer” virtual appliances on the destination site. The Multi-Site Service Mesh enables the configuration, deployment, and serviceability of these Interconnect virtual appliance pairs.

HCX Service Mesh

In the HCX site-to-site architecture, there is notion of an HCX source and HCX destination environment. Depending on the environment, there is a specific HCX installer:

HCX Connector (previously HCX Enterprise) or HCX Cloud. HCX Connector is always deployed as the source. HCX Cloud is typically deployed as the destination, but it can be used as the source in cloud-to-cloud deployments. In HCX-enabled public clouds, the cloud provider deploys HCX Cloud. The public cloud tenant deploys HCX Connector on-premises.
The source and destination sites are paired together for HCX operations. 

In both the source and destination environments, HCX is deployed to the management zone, next to each site’s vCenter Server, which provides a single plane (HCX Manager) for administering VMware HCX. This HCX Manager provides a framework for deploying HCX service virtual machines across both the source and destination sites. VMware HCX administrators are authenticated, and each task authorized through the existing vSphere SSO identity sources. VMware HCX mobility, extension, protection actions can be initiated from the HCX User Interface or from within the vCenter Server Navigator screen’s context menus.

In the NSX Data Center Enterprise Plus (HCX for Private to Private deployments), the tenant deploys both source and destination HCX Managers.

The HCX-IX service appliance provides replication and vMotion-based migration capabilities over the Internet and private lines to the destination site whereas providing strong encryption, traffic engineering, and virtual machine mobility.

The VMware HCX WAN Optimization service appliance improves performance characteristics of the private lines or Internet paths by applying WAN optimization techniques like the data de-duplication and line conditioning. It makes performance closer to a LAN environment. It accelerates on-boarding to the destination site using Internet/VPN- without waiting for Direct Connect/MPLS circuits.

The VMware HCX Network Extension service appliance provides a late Performance (4–6 Gbps) Layer 2 extension capability. The extension service permits keeping the same IP and MAC addresses during a Virtual Machine migration. Network Extension with Proximity Routing provides the optimal ingress and egress connectivity for virtual machines at the destination site.

 

Using VMware HCX OS Assisted Migration (OSAM), you can migrate guest (non-vSphere) virtual machines from on-premise data centers to the cloud. The OSAM service has several components: the HCX Sentinel software that is installed on each virtual machine to be migrated, a Sentinel Gateway (SGW) appliance for connecting and forwarding guest workloads in the source environment, and a Sentinel Data Receiver (SDR) in the destination environment.

The HCX Sentinel Data Receiver (SDR) appliance works with the HCX Sentinel Gateway appliance to receive, manage, and monitor data replication operations at the destination environment.

HCX Migration Types

VMs can be moved from one HCX-enabled data center using different migration technologies or types.

HCX Migration Types

HCX cold migration uses the VMware NFC (Network File Copy) protocol and is automatically selected when the source VM is powered off.

HCX vMotion

  • This option is designed for moving a single virtual machine at a time
  • There is no service interruption during the HCX vMotion migration
  • Encrypted vMotion between legacy source and SDDC target
  • Bi-directional (Cross-CPU family compatibility without cluster EVC)
  • In-flight Optimization (deduplication/compression)
  • Compatible from vSphere 5.5+ environments (VM HW v9)

HCX Bulk Migration

  • Bulk migration uses the host-based replication (HBR) to move a virtual machine between HCX data centers
  • This option is designed for moving virtual machines in parallel (migration in waves)
  • This migration type can set to complete on a predefined schedule
  • The virtual machine runs at the source site until the failover begins. The service interruption with the bulk migration is equivalent to a reboot
  • Encrypted Replication migration between legacy source and SDDC target
  • Bi-directional (Cross-CPU family compatibility)
  • WAN Optimized (deduplication/compression)
  • VMware Tools and VM Hardware can be upgraded to the latest at the target.

HCX Replication Assisted vMotion

VMware HCX Replication Assisted vMotion (RAV) combines advantages from VMware HCX Bulk Migration (parallel operations, resiliency, and scheduling) with VMware HCX vMotion (zero downtime virtual machine state migration).

HCX OS Assisted Migration

This migration method provides for the bulk migration of guest (non-vSphere) virtual machines using OS Assisted Migration to VMware vSphere on-premise or cloud-based data centers. Enabling this service requires additional HCX licensing.

 

  • Utilizes OS assisted replication to migrate (conceptually similar to vSphere replication)
  • Source VM remains online during replication
  • Quiesce the source VM for final sync before migration
  • Source VM is powered off and the migrated VM is powered on in target site, for low downtime switchover
  • VMware tools is installed on the migrated VM

Cross-Cloud Mobility

Most customers will probably start with one public cloud first, e.g. VMC on AWS, to evaluate the hybridity and mobility HCX delivers. Cross-cloud monility is maybe not a requirement today or tomorrow but gets more important when your company has a multi-cloud strategy which becomes reality very soon.

If you want to be able to move workloads seamlessly between clouds, extend networks and protect workloads the same way across any cloud, then you should consider a VMware platform and use HCX.

HCX Cross-Cloud Mobility

Let’s take network and security as an example. How would you configure and manage all the different network, security, firewall policies etc. in your different clouds with the different infrastructure and security management consoles?

If you abstract the hyperscaler’s global infrastructure and put VMware on top, you could in this case use NSX (software-defined networking) everywhere. And because all the different policies are tied to a virtual machine, it doesn’t matter anymore if you migrate a VM from host to host or from cloud to cloud.

This is what you would call consistent operations and management which is enabled by a consistent infrastructure (across any cloud).

And how would you migrate workloads in a very cost and time efficient way without a layer 2 stretch? You would have to take care of re-IPing workloads and this involves a lot of changes and dependencies. If you have hundreds of applications then the cloud migration would be a never ending project with costs you could not justify.

In the case you need to move workload back to your own on-premises data center, HCX also gives you this advantage.

You have the choice in which cloud you want to run your applications, at any time.

 

HCX and vSphere 7

At the time of writing HCX has no official support for vSphere 7.0 yet. I tested it in my home lab and ran into an error while creating the Service Mesh. At least one other colleague had the same issue with vSphere 7 using NSX-T 3.0 and VDS 7.0.

HCX vSphere 7 Error

I would like to thank Danny Stettler for reviewing and contributing. 🙂 Big kudos to you, Danny! 🙂

I hope the article has helped you to get an overview what HCX and a hybrid cloud model really mean. Drop a comment and share your view and experience when it comes to cloud strategies and migrations.

 

Know Your Options with Citrix and VMware

Know Your Options with Citrix and VMware

No, this is not an article about Citrix vs. Horizon and which product is better. And I think that you should not compare Citrix and VMware anymore. If you are still reading and haven’t closed the tab in your browser yet, you made the right decision. The intent of this article is to help you better understand when the usage of Citrix Virtual Apps and Desktops (CVAD) makes sense, which VMware products could complement a CVAD infrastructure and the different options you have with VMware Horizon.

I think it is a very big plus that I worked for Citrix before and still have some technical knowledge. This gives me more credibility in front of the customer and I am not just someone from a vendor, who tries to blame or downplay the other competitor to sell his on stuff. In fact, I always tell my customers how good Citrix is – there is no doubt about that.

But people are still stuck in the past and have the knowledge from four or six years ago. VMware Horizon has evolved into a very mature virtual apps and desktops solution and at the same time VMware’s products evolved as well and the story and product portfolio are better than ever.

Would have asked me a few years ago, no matter if I would be still with Citrix or already with VMware, VMware Horizon had some serious (feature) gaps and differences (e.g. display protocol) compared to Citrix. But Horizon has transformed into a equal player in the market and can do almost the same as CVAD (formerly XenDesktop and XenApp).

Note: I’m not saying that VMware Horizon has reached feature parity compared to Citrix

Let’s see which enhancements or new features have been released in the last 18 months for Horizon:

  • A lot of enhancements and closed feature gaps for the Horizon HTML5 console (now default)
  • RDS Drain Mode and RDSH Load Balancing configurable from UI
  • Improved CDR (Client Drive Redirection) performance
  • Increased CPA (Cloud Pod Architecture) scale up to 250k sessions
  • Session “pre-launch”
  • Two-Factor Re-Authentication
  • Client UI redesign
  • vGPU vMotion (came with vSphere 6.7 U1)
  • VM hosted apps (published applications from Win10 desktop pools)
  • Longer Lived Instant Clones
  • Horizon Cloud Services Enhancements & WVD support for Horizon Cloud on Azure
  • VMware Skyline Log Assist
  • App Volumes 4
  • New REST APIs
  • Bandwidth savings in Blast (with Blast Codec)
  • CPU utilization by Blast has been reduced
  • Blast Extreme HEVC High Color Accuracy support
  • Automatic codec switching based on screen content
  • NSX Advanced Load Balancer (Avi LB) support

As you can see, a lot work has been done and a lot of time has been invested to make Horizon better! These improvements are one of many why I think it’s useless to compare Citrix vs. Horizon, because both can basically do the same if you ask me.

Note: Horizon 8.0 is coming very soon and the beta program for it starts in a few weeks! Stay tuned for more enhancements and innovation. 🙂

Citrix and VMware – Four Options

When I think about Citrix and VMware, there are four options which come up in my mind how a customer could move forward at any given time:

  1. Replace Citrix with Horizon
  2. Integrate Citrix with Workspace ONE
  3. Enhance Citrix with Horizon or Workspace ONE components
  4. Enhance Citrix with other VMware components
  5. Use Citrix and VMware Horizon (yes, there are customers with both!)

Replace Citrix with Horizon

The first option is the most obvious one and can happen from time to time due to various reasons. Sometimes the customer is just not happy anymore (technical or commercial) or wants to try something new because of one or more of the other listed options (integration and enhancements in place already).

A migration would be very easy on paper. StoreFront could be replaced by Workspace ONE Access (formerly vIDM), the VDA installed on RDS hosts or virtual desktops need to be replaced with the Horizon agent and on the client side the Citrix Workspace App (Citrix Receiver) gets replaced by a Horizon Client (including HTML5 client).

Caution: Even if it’s technically possible to uninstall Citrix Virtual Desktop Agenda (VDA) and install the Horizon Agent after, this is not something a good consultant would recommend normally. Do it right and rebuild a clean image and test it before going in production. 

VMware and Citrix Partnership

A replacement could also be done in parallel where you install a Horizon infrastructure beside the current Citrix environment and move the users over whenever you are ready.

If you are running your desktops on Azure together with Citrix Cloud, then the Citrix Cloud piece can be replaced with the Horizon Cloud Service on Azure. Citrix and VMware Horizon are both supported if you are looking for a connection broker for your Windows Virtual Desktops (WVD).

Integrate Citrix with Workspace ONE

The second option doesn’t come up very often. If a Citrix customer is using CVAD only and no Citrix Endpoint Management (formerly known as XenMobile) or Microsoft Intune (or MobileIron) and is considering Workspace ONE for their unified endpoint management of iOS, Android, macOS or Windows 10 clients, then mutual customers could use Workspace ONE (WS1) Access as the web portal or application catalog and single point of access for any application.

As just mentioned already, Workspace ONE users and devices access Citrix-published resources by integrating their Citrix deployment with Workspace ONE Access, which offers an application portal, single-sign on capabilities, conditional access and many other features. Citrix-published resources include applications and desktops from any CVAD infrastructure starting from XenApp 6.0.

All entitlements are still configured in Citrix Studio and you just have to sync these users and groups to the WS1 Access services from Active Directory first.

Beside WS1 Access you need one additional component called the Integration Broker, which can be installed on a Windows Server. The Integration Broker is responsible for the communication with all Citrix farms/sites. The WS1 Access connectors then communicate with the Integration Broker.

Workspace ONE Integration Broker

More information can be found here. That’s all what is needed for the integration with Workspace ONE.

Enhance Citrix with Horizon or Workspace ONE components

VMware has customers with a large Citrix footprint of several thousand users. And some of these customers are using Horizon components together with their Citrix infrastructure. The two most used Horizon components in a Citrix infrastructure are:

I am not up to date anymore what Citrix App Layering, Profile Management (UPM) and Workspace Environment Management (WEM) can do for you today. But App Volumes would replace App Layering and Dynamic User Environment (DEM) would replace UPM and WEM in a Citrix environment.

Don’t know if this still is the case, but a few years ago App Layering had very limited features, didn’t perform and the handling of layers was a pain. And WEM just didn’t scale in larger Citrix environment. Probably Citrix UPM still is doing its awesome job but is leveraging FSLogix for profile and O365 container management and I assume that WEM is also installed more nowadays.

If Citrix App Layering is in use, then probably the FSLogix Application Masking feature could be used as well to hide some components in the image, which also allows the admin to manage fewer golden images. This is something you also can do with Dynamic Environment Manager in combination with App Volumes.

Before FSLogix was available to almost every joint Citrix/VMware and Microsoft customer, it totally made sense to use something like DEM for the user environment management, as DEM has similar features as FSLogix.

To understand the integration of FSLogix and AV and DEM better, this article from VMware’s Digital Workspace TechZone is for you. 

Maybe you ask yourself now how you could get App Volumes and Dynamic Environment Manager for your Citrix environment? Well, there are a few ways and options:

  • Buy the “Horizon Enterprise” or “Horizon Apps Advanced” edition which includes AV and DEM (yes, can happen)
  • Buy the “Workspace ONE Enterprise” edition which includes “Horizon Apps Advanced”
  • Buy the “Workspace ONE Enterprise for VDI” edition which includes “Horizon Enterprise”

You have to buy another license from another vendor, yes. But, let me explain why this could make sense.

Scenario 1 – Citrix customer is buying Workspace ONE Enterprise

Let’s assume you are a Citrix customer and use CVAD to publish applications to your users, but want to manage your iOS, Android, macOS and Windows 10, IoT devices with one solution or platform. That’s the moment when you go for Workspace ONE as your Unified Endpoint Management (UEM) platform. Here’s what you get with Workspace ONE Enterprise:

  • iOS, Android, macOS, Windows 10 and IoT device management (MDM/UEM)
  • Workspace ONE Access
  • Application delivery and management (mobile and desktop)
  • Mobile SSO
  • Workspace ONE productivity apps (email, tasks, notes, content/file repository, web, card scanner)
  • Multi-Factor Authentication (MFA) with “Workspace ONE Verify” mobile application
  • Workspace ONE Intelligence (SaaS-based intelligence and automation engine including reporting)
  • Add-on: Remote Management of any device based on Workspace ONE Assist
  • Add-on: Workspace Security (Carbon Black offerings)
  • Horizon Apps Advanced

The Horizon Apps Advanced edition includes the following:

  • RDS published apps (no desktop OS, only server OS) and session-based desktops
  • ThinApp (not included with WS1 Enterprise)
  • App Volumes
  • Dynamic Environment Management
  • vSphere Desktop

As you can see, you are removing silos in your digital workspace and can use App Volumes and Dynamic Environment Management at the same time to enhance your Citrix infrastructure.

Scenario 2 – Citrix customer is buying Workspace ONE Enterprise for VDI

The difference between scenario 1 and scenario 2 is the Workspace ONE Enterprise for VDI license, which includes the following components:

  • Published desktops and apps (server OS and desktop OS incl. Linux)
  • App Volumes
  • ThinApp (not included with WS1 Enterprise for VDI)
  • Dynamic Environment Management
  • vRealize Operations for Horizon (not included with WS1 Enterprise for VDI)
  • vSphere Desktop
  • vSAN Advanced for Desktop with All-Flash

WS1 Enterprise for VDI makes it possible to have VDI based on the Windows desktop operating system (e.g. Windows 10) as well and adds the infrastructure capability to run your desktop workloads on vSAN enabled clusters! The only thing which differs from the regular standalone Horizon editions, is, that ThinApp and vRealize Operations are not part of the suite. If you have a lot of legacy apps or you need application virtualization or isolation, then take a look at ThinApp.

Applications installers such as MSI files can be packaged into a portable EXE file and can then be run on any physical or virtual Windows PC and delivered with App Volumes (RDS/VDI) or with Workspace ONE (persistent VDI desktop or physical desktop).

And you get the “vSphere for Desktop” edition in both cases which is another killer argument why you could buy Workspace ONE Enterprise (for VDI) licenses as a Citrix customer.

vSphere Desktop

I don’t have any confirmed number, but I assume that 70% of the Citrix customers are using VMware vSphere as their hypervisor. Each regular Horizon edition has vSphere Desktop included which many people are not aware of.

vSphere for Desktop is a special edition, which provides the full range of features of the vSphere Enterprise Plus edition:

  • The new image management feature to patch, update or upgrade ESXi clusters (vSphere 7.0)
  • vCenter Server profiles and update planner (vSphere 7.0)
  • Distributed vSwitch
  • Secure access and account management with ADFS (vSphere 7.0)
  • Distributed Resource Scheduler (DRS)
  • Storage DRS
  • Nvidia GRID vGPU

vSphere Desktop is licensed based on the total number of powered-on VMs and has no processor limitation. It’s available in a pack size of 100 desktop VMs with up to 100 users per pack. VERY IMPORTANT: vSphere Desktop can be used for a VDI environment only and a vCenter license is not included in vSphere for Desktop.

This is the only restriction mentioned in the vSphere Desktop FAQ:

vSphere Desktop can be used only to host a desktop
virtualization environment or desktop management and
monitoring tools. Each pack of 100 VMs can be used for
up to 100 users. You can use vSphere Desktop for desktop
management and monitoring tools in a VDI environment
only. Desktop licenses covered by this provision, however,
may not be managed by the same instance of VMware
vCenter that is being used to manage non-desktop
OS virtual machines.

So, what is considered as a “desktop virtualization environment” including monitoring tools? Normally you would separate your Citrix or Horizon infrastructure servers from the virtual machines which provide the virtual desktops and applications. But this design is more a leading practice and recommended by reference architectures and therefore it is technically possible to mix the RDS and VDI virtual machines with the infrastructure servers like:

  • Connection Server / Delivery Controller
  • Workspace ONE Access / StoreFront
  • Unified Access Gateway / NetScaler
  • Active Directory
  • Monitoring Tools (vRealize Operations / Director)
  • any “other infrastructure directly related to and exclusive to the VDI environment”

In a Citrix Virtual Apps and Desktop environment you can use vSphere Desktop to provide the virtual machines (desktops) and the underlying infrastructure. In this use case, you are licensed per virtual machine and virtual machines used to host the infrastructure servers. These two numbers will be counted against your “total powered-on VM” count. If your Citrix environment has a 100-pack of vSphere Desktop licenses and you host 85 VDI desktops and 15 VMs that host the Citrix VDI environment, then you have used up all the 100 vSphere Desktop licenses.

vSAN Advanced for Desktop

vSAN Advanced for Desktop is shipped together with Horizon Advanced, Horizon Enterprise and Workspace ONE Enterprise for VDI. This license is available for customers using vSAN exclusively for a VDI infrastructure.

Horizon Universal License

The Horizon Universal License is a single subscription-based license, which is included in the Workspace ONE Enterprise edition and serves as an entitlement for all Horizon products, namely Horizon Cloud (including Horizon Cloud Apps) and Horizon on-premises (including Horizon Apps). Thus, the universal license entitles you for the following solutions:

This universal license gives customers the choice to start with an on-premises Horizon deployment and to move to the cloud (or vice versa) without requiring a new license.

Note: Because it’s the universal license and not a regular Horizon license, which is included in the WS1 editions, vRealize Operations (vROps) is not part of this subscription bundle. If needed, vROps can be bought as a standalone license.

Thin Client Management

I thought it is worth mention it here. Keep in mind that you could use a platform like Workspace ONE to manage your thin clients. If your environment is heavily using thin clients you could “build” your own thin client based on Windows 10 IoT Enterprise and manage it via Workspace ONE.

E.g. Workspace ONE can manage Dell Wyse 5070 thin clients with Windows 10 IoT Enterprise. If needed, WS1 can configure the Unified Write Filter (UWF) feature to protect your thin client drives for any changes (saved data, setting changes or app installations). This is also helpful for increasing security for kiosk PCs in hotels, public spots, internet cafĂ©s etc. or for devices where it’s not expected to have new application frequently added.

WS1 Unified Write Filter

Enhance Citrix with other VMware components

We know that you could make your Citrix environment “better” with Horizon components like App Volumes or Dynamic Environment Manager and vSphere components like vSphere and vSAN. But there are other products and components which could make sense in a Citrix environment.

I believe, today, VMware has something which you could call a partnership and both CTOs are clearly leading the way:

Citrix Partnership VMware

 

I don’t know if it ever happened before that Citrix mentioned VMware on stage at Synergy, but the announcement from the above picture brings me to my first solution which you could use for your Citrix deployment.

VMware Cloud on AWS

What has been announced at Citrix Synergy 2019? The intent to officially support CVAD running on VMware-based clouds, starting with VMware Cloud on AWS. Many organizations are evaluating or even using a hybrid cloud approach already. This announcement should help Citrix customers, who are running their workloads on vSphere already, to seamlessly move to the cloud to experience a consistent infrastructure with consistent operations.

Because you are using the same technology stack on-prem and in the cloud, this allows you to easily bring your RDS and VDI golden images to the cloud without any a conversion.

I see two deployments options here. Either you leverage the Citrix Cloud services (use VMC as a resource location) or manually install your Citrix infrastructure like you would normally do in your on-premises environment.

VMC on AWS is Citrix-Ready

Note: VMC on AWS is citrix-ready since Q4 2018!

CVAD on VMC on AWS

If you would like to know more about running Citrix Virtual Apps and Desktops with VMC on AWS, please watch the VMworld 2019 recording of the session “Building Global Citrix Virtual Apps and Desktops with VMware Cloud on AWS (HBI2247BU)“. There’s also a recording of the US 2019 session “Building Global Citrix Virtual Apps and Desktops with VMware Cloud on AWS“, presented by Andrew Morgan and James Hsu.

Interesting facts:

  • It takes about 60-70min in average to deploy a new SDDC on VMC on AWS
  • 12min is the average time to add a new host
  • Stretched clusters give you a guaranteed SLA of 99.99%
  • Sync your VM templates with your Content Library
  • Andrew and James deployed 100 Win10 desktops in 5min only
  • PVS and MCS both work on VMC on AWS

NSX – Software-Defined Networking

Digital transformations are nothing new, but get more complex with newer technologies we have today. One very important topic which came up in 2019 and is one of the most important trends for 2020 is “cyber security” or “zero trust security”. VMware and Citrix are both pointing to a zero trust approach to protect the workforce, any app and data. VMware has defined 5 pillars of zero trust for a digital workspace and “transport/session trust” is one of them with these parameters:

  • Micro-Segmentation
  • Transport Encryption
  • Session Protection

For secure transport of a user’s session you would use appliances like the Unified Access Gateway (UAG) or Citrix NetScaler. To achieve a trusted network access within the data center and between workloads, you’ll need something like NSX and micro-segmentation. Citrix has only a SD-WAN solution to protect branch offices and branch users, but no solution for micro-segmentation. What is micro-segmentation and why is it important?

Imagine that network policies can be bound to a virtual machine or in our case to a virtual desktop and dynamically follow a virtual desktop. This is very helpful in the case of VMC on AWS for example. You can easily move the workload to the cloud and move the networking policies together with the VM, because the underlying stack on VMC on AWS (based on VMware Cloud Foundation) includes NSX and the vSphere hypervisor.

How would you secure the communication and access between desktops in the same VLAN? All desktops on a VLAN can communicate freely and one compromised desktop allows lateral movement. With NSX we can provide granular control of desktops and user/group based access control. This is micro-segmentation.

NSX Micro-Segmentation

Here are two articles about Citrix and NSX from VMware and Citrix:

If you are interested in 100% software-defined networking and are thinking to replace an existing hardware or virtual ADCs (application delivery controllers), take a look at NSX Advanced Load Balancer (formerly Load Balancer from Avi Networks).

NSX Advanced Load Balancer Architecture

Where VMware Horizon differs from Citrix

Now you know the four options you have as a Citrix customer when considering VMware products for your current and future environment. Let me explain you why you shouldn’t compare Citrix and VMware Horizon anymore. To get started, you need to understand all the different options you have and how and where you could consume VMware Horizon:

  • Horizon on-premises
  • Horizon Cloud
  • Horizon DaaS

And with the different desktop virtualization offerings there are also different management responsibilities for the customer, partner and VMware:

VMware Horizon Responisibilities

Customers have the flexibility to choose the level of control they want to have over the Horizon and data center infrastructure. If full control of the solution is needed, then you would probably implement Horizon with vSphere on-premises. For use cases where you only would like to maintain the desktop and apps only without concerning yourself about managing any infrastructure, Horizon Cloud on Azure could be one option.

Horizon On-Premises

The biggest difference for me, if you really want to compare Citrix and VMware in a better way, is to see the big picture. People need to understand that it is totally normal that one vendor sometimes is ahead or behind the competitor. The feature set from both vendors, only considering desktop virtualization, is pretty much the same.

When you start a desktop virtualization project and design the solution, you also have to think about the data center part. I’m am not only talking about Horizon and the storage or network requirements here. It’s important to understand the general strategy and vision of VMware and your employer/customer.

Today, automation is a design requirement and you ideally build your on-premises infrastructure based on public cloud principles. Companies don’t start anymore by buying hardware and think about automation later. They want to buy and build something that can be automated from day 1 like it’s done in the public cloud. Everything needs to be agile and elastic and should be able to change when any kind of change occurs.

Because of that it is essential to understand the cloud infrastructure part very well and this is the big difference between Citrix and VMware. We shouldn’t only talk about EUC (End-User Computing) only, but even consider other projects or domains of the infrastructure:

  • Does it fit in my cloud operating model?
  • Can I use an existing solution to automate it (software and hardware)?
  • How would I move my workloads to the cloud tomorrow?
  • Can I integrate existing solutions in my ecosystem (e.g. security, IPAM etc.)?
  • Can it be integrated in our existing or new platform for modern applications based on containers?
  • What about day 2 operations if I need to expand?
  • Can I reduce my silos and reduce the number of vendors and licenses somehow?

The installation of a complete Horizon (or Citrix) infrastructure can be done in a few days, normally, but larger environments require a lot of automation and integrations into the existing infrastructure. Then we talk about several months and not days or weeks anymore.

Horizon on VMware Cloud Foundation

VMware Cloud Foundation (VCF) is made for any workload and is a hybrid cloud platform which provides a set of software-defined components for compute, storage, networking, security and cloud management. VCF is an engineered solution that integrates the entire VMware stack without the need you dealing with complex interoperability matrixes.

VMware Cloud Foundation Overview

The architecture is built on VMware’s Validated Designs (VVD) to reduce the risk of misconfigurations or design failures. The VCF stack is also used with VMC on AWS or Azure VMware Solutions (AVS) for example. This is another reason that clearly shows that this technology stack is the right for any (VMware) infrastructure. If workload mobility is part of your IT strategy, then only VMware can offer this at the moment.

VCF 4.0 Bill of Materials

VMware Cloud Foundation has a “siloed” approach when it comes to the deployment. Based on different hardware resource pools you can create different so-called workload domains (WLD). Each WLD is a different SDDC instance which is managed by software-defined policies. The Horizon deployment can form one or more VDI WLDs.

VCF WLD Overview

Because it’s a standardized approach, VCF makes it very easy to scale on-demand depending on your needs. To get started you’ll need a management workload domain, which is a special-purpose workload domain dedicated for infrastructure and management components like the SDDC Manager, vCenter Servers, vRealize Suite and NSX. The SDDC Manager is responsible for the creation, update or deletion of a workload domain.

Using the regular standard architecture model for VCF, an environment starts with at least 4 physical servers for the management domain, 3 servers for the VI workload domain (Active Directory, SQL servers, any general infrastructure VM) and 3 servers for a Horizon VDI workload domain. This gives us a starting point of 10 physical servers if you build a complete IT infrastructure from scratch. Otherwise you just need the management domain and VDI workload domain with a total minimum of 7 physical servers.

There is also the option available of a consolidated architecture design for smaller environments. In this design the management and workloads run together on a shared management domain. But the consolidated architecture doesn’t support the automated deployment of Horizon yet.

For the automated deployment of Horizon on VCF you would use the SDDC Manager to deploy Connection Servers, App Volumes, Dynamic Environment Manager and Unified Access Gateways. Let me show you some part of the wizard to create a VDI WLD:

You don’t have to install the components by hand, but still need to do your homework before you can deploy the WLD.

I skipped a few steps. You need to upload the Windows server template, convert an existing VI WLD to a Horizon VDI WLD, configure the Horizon AD service account, provide a SQL server and provide information for the load balancers before you reach the step where you enter the details for the connection servers:

One more App Volumes Manager can be added as well:

If you reached the end, you’ll see a review page to do a final check and after that you can run a validation of all your inputs. The deployment of at least one Connection Server is required, but Horizon Composer Servers, UAGs, App Volumes and DEM are optional components and could be skipped.

To expand a current VDI WLD to install UAGs or just to expand the Horizon Pod (add ESXi hosts or Connection Servers) VCF gives you the option to start small and expand later. In the future it should also be possible to shrink a VDI WLD.

The lifecycle management with VCF is very easy. Available updates for all components are tested for interoperability and then bundled with the necessary logic for the proper installation order. VCF offers automated lifecycle management on a per-cluster basis (one WLD can have one or more clusters). This allows admin to target specific workloads or environments for updates independently of the rest of the environment.

VCF Lifecycle Management

For a VDI workload domain VCF delivers a nice view to see the allocated servers/resources and each component related to this workload domain. 

VCF Horizon Deployment WLD

Horizon on VCF on VxRail

So, we know now that VMware Cloud Foundation is the “easy button” for the deployment of the full vSphere stack including vSAN, NSX, vRealize Operations, vRealize Automation, vRealize Log Insight and so on. VCF on VxRail goes one step further and provides you the “one-click upgrade button” for your vSphere stack including the server hardware and firmware. VxRail bundles are pre-configured and pre-tested and therefore validated by Dell EMC and VMware.

VxRail SDDC Manager

The cool thing with VxRail is, that it gives you flexibility for your workloads and that you can choose between different series based on Dell EMC PowerEdge servers. You have multiple compute, memory, storage, network and graphics (M10, P40, T4) options available to cover your workloads and applications with the right server specifications.

VxRail Server Series

Citrix (on VCF) on VxRail

Since VxRail is an HCI appliance, it can run everything on top. I know some larger Citrix customers who are running their Citrix infrastructure on VxRail. It is also possible to run your Citrix infrastructure on VCF on VxRail on a VI workload domain. The only difference with Horizon is the missing automation and integration into the whole (VCF) stack.

Intrinsic Security

In case you missed it, VMware bought Carbon Black and has a new security business unit now. And this is one very important differentiator in this virtual cloud computing space. If VMware’s software-defined data center is your platform of choice already, it makes sense to use a security solution which can be fully integrated and provided by the same vendor.

VMware Security Solutions with Carbon Black

Imagine, that the endpoint protection agent is already integrated in the Horizon Agent and that you could deliver security from your mobile endpoints (Windows, Mac, Linux) to your workloads (VMs or container) in your data center or any cloud (AWS, Azure, GCP). Sounds too good to be true? No, this where the VMware products are heading, especially with Workspace ONE and Horizon (next-gen AV, behavioral EDR, audit and remediation)! 

Workspace ONE for Horizon

I mentioned it already, Horizon is included in the Workspace ONE Enterprise editions. I haven’t covered the case yet where you could combine Horizon and Workspace ONE. If you provide your users persistent virtual desktops based on Windows 10, then it is also possible to manage those with Workspace ONE as well. This will help if you want to move away from a traditional PC lifecycle management (PCLM) solution and move to a modern management approach. So far this only supported with Horizon on-premises installation. Take a look at the product interoperability matrix:

Workspace ONE for Horizon

For which other use cases could this be useful?

  • Physical desktops with Horizon Agent installed (Remote PC access)
  • Physical servers with Windows 10 installed (e.g. HP Moonshot)

I don’t know if the last option has been tested but Windows 10 is a supported operating system for HP Moonshot cartridges.

Horizon Cloud

The Horizon (Cloud) Service is a group of cloud-based services that deliver features for Horizon deployments. This includes the Windows Virtual Desktop (WVD) on Azure as well since the 17th March 2020. Any customer who is using a Horizon subscription license, such as the universal license, can use the Horizon Service.

Horizon Cloud Service Overview

The goal of Horizon Cloud is to provide a single-pane management UI for the delivery and management of your desktops and applications. This is the overview dashboard which shows some information about the health and capacity of all your Horizon deployments.

Horizon Cloud Dashboard

The Cloud Monitoring Service (CMS), which is one of the central services of the Horizon Service, provides data about the user’s session and issues. It can show you how many users and their user experience are impacted related to issues (latency, protocol, slow logon).

In the administration console you can configure the role-based access (RBAC) for your helpdesk admins. It allows them to log in to the admin console and use the search feature to look up users. The help desk administrator can then look up the user’s sessions and perform troubleshooting or desktop maintenance operations. 

Horizon Cloud Helpdesk

The Image Management Service (IMS) is one of the coolest feature of the Horizon Service. As the name suggests it already, it allows you to manage Horizon images from the cloud. You can create, customize, publish and even version all your different images for your Horizon pods. IMS provides a centralized catalog for your images and these can be automatically replicated across the cloud-connected Horizon pods.

Important note: The current release of Horizon Cloud only supports Windows operating systems and on-premises Horizon pods.

Universal Broker

When I joined VMware in May 2018 I was waiting for a feature like this and tried to explain some product managers (PM) that we need something like the Universal Broker. I was looking for a solution that we can avoid E/W traffic in a Horizon multi-pod deployment. I think I tried to explain it to some of our PMs using Citrix’ Optimal Gateway Routing for
Storefront & NetScaler capability. Nobody understood me, but at least we have it now. 😀

Horizon Universal Broker is the cloud-based brokering technology used to manage and allocate virtual resources from multi-cloud assignments to your end users.

These are the listed key features in the VMware Horizon Cloud Service documentation:

  • Single FQDN for all multi-cloud assignments
  • Global pod connectivity and awareness for optimal performance (no longer need for GSLB and no more E/W traffic)
  • Smart brokering (awareness of geographical sites and pod topology)

This diagram shows the Universal Broker components and how the traffic flow works:

  1. From Horizon Client, the end user requests a virtual desktop by connecting to the Horizon Universal Broker service through the brokering FQDN. The service uses the XML-API protocol to authenticate the Horizon Client user and manage the connection session.
  2. After determining that Pod 1 in Site 1 is the best available source for the desktop, the Horizon Universal Broker service sends a message to the Horizon Universal Broker client, which runs on the Horizon 7 Cloud Connector paired with Pod 1.
  3. The Horizon Universal Broker client forwards the message to the Horizon Universal Broker plugin, which runs on one of the Connection Server instances within Pod 1.
  4. The Horizon Universal Broker plugin identifies the best available desktop to deliver to the end user.
  5. The Horizon Universal Broker service returns a response to Horizon Client which includes the unique FQDN of Pod 1 (typically the FQDN of the Pod 1 load balancer). Horizon Client establishes a connection with the load balancer to request a protocol session with the desktop.
  6. After passing through the local load balancer, the request goes to the Unified Access Gateway for Pod 1. The Unified Access Gateway validates that the request is trusted and prepares the Blast Secure Gateway, PCoIP Secure Gateway, and tunnel server.
  7. The Horizon Client user receives the specified desktop and establishes a session based on the configured secondary protocol (Blast Extreme or PCoIP).

Horizon DaaS

In 2013 VMware acquired Desktone. A company that was specialized in delivering desktops and applications as a cloud service. The product got renamed during the years and kept the name “Horizon DaaS“. This is the reason that Horizon DaaS is not just another version of the classic “Horizon” or “Horizon View” since it was a different product which VMware bought. It’s important to know that there are technical differences/characteristics between Horizon and Horizon DaaS because of this history.

Horizon DaaS is the Horizon Desktop-as-a-Service platform for service providers. Not many people understand and know this specific product and you won’t find a lot of content on blogs about it.

The most recent information, beside the official Horizon DaaS documentation, can now be found here 😉 or on Johan’s blog, where he published a lightboard series about Horizon DaaS.

As a service provider you have different options to provide a “managed desktop” or “DaaS” offering:

  • Dedicated Horizon deployment hosted in your data center (licenses through VCPP rental)
  • Horizon Cloud Service (DaaS offering licensed through VCPP MSP)
  • Horizon DaaS – multi-tenant Horizon deployment hosted in your data center (VCPP rental)

Again, Horizon DaaS should be seen as something different than Horizon, it’s really just not Horizon. But the future strategy and look of the user interface will be aligned with Horizon Cloud, because VMware’s Horizon Cloud Service is powered by Horizon DaaS already.

If multi-tenancy is a key requirement for your business, you’ll have to go with Horizon DaaS. Otherwise the regular Horizon edition or the combination with Horizon Cloud are the right fit. Horizon DaaS and Horizon have common components like vCenter, Agents, UAGs etc., but there are also different appliances with Horizon DaaS which replace components of a regular Horizon deployment.

Horizon DaaS Architecture

With Horizon DaaS you are going to have “Service Provider” appliances, “Tenant” appliances and “Tenant Resource Manager” appliances, which form the DaaS back-end.

The Service Provider Appliance is the first appliance installed in a data center and provides the foundation to install the remainder of the Horizon DaaS application.

The Resource Manager abstracts the specifics about the desktop infrastructure from the tenant appliances and allows multiple Desktop Managers to communicate with their respective virtualization resources. A Resource Manager appliance integrates with the hypervisor and storage infrastructure in a given data center. A single Resource Manager appliance can be shared across multiple tenants.

The Tenant Appliance provides the tenant with both end user and administrative access to their virtual desktops. End users access and manage their individual virtual desktops via the Desktop Portal. Administrators create and manage their virtual desktops via the Enterprise Center. The Tenant Appliance includes the Desktop Manager, a per-tenant resource that manages each tenant’s virtualization resources and communicates with a tenant’s hosts (hypervisors). You associate the desktop manager with a resource manager and one or more host managers.

It’s not 100% clear from the Horizon DaaS 8.0.0 Service Center guide, but a Tenant Appliance replaces the Connection Server you would know from a regular Horizon deployment (one of the differences I was already referring to).

Use what makes sense

For me it is very important that you understand how VMware products can help and that people are aware of all the different options they would have with VMware and Horizon.

You must form your own view and opinion and I hope this article was useful to get facts from both worlds (based on my best knowledge and experience). If you understand Horizon better now, this is already fine for me.

There was no intention to lead the path to a way where you would replace Citrix. The new information should help you to make the right decision for your company, your environment, your needs and use cases. Use the products which make sense for you and make sure you understood all options.

Azure VMware Solution

Azure VMware Solution

Update May 2020:

On May 4th Microsoft announced the preview (and the “next evolution”) of Azure VMware Solution which is now a first-party offering service designed, built and supported my Microsoft and endorsed by VMware. This is an entirely new service entirely delivered and supported by Microsoft and does not replace the current AVS solution/service by CloudSimple at this time. This is truly just a Microsoft technology offering and has also nothing to do with a Virtustream branded Azure VMware Solution offering. Short: A way cleaner offering and service with a contract only between Microsoft and VMware.

— Original Text below —

Since Dell Technologies World 2019 it’s clear: VMware and Microsoft are not frenemies anymore!

Dell Technologies and Microsoft announced an expanded partnership which should help customers and provide them more choice and flexibility for their future digital workspace projects or cloud integrations.

One result and announcement of this new partnership is the still pretty new offering called “Azure VMware Solution” (AVS). Other people and websites may also call it “Azure VMware Solution by Virtustream” or “Azure VMware Solution by CloudSimple”.

AVS is a Microsoft first-party offering. Meaning, that it’s sold and supported by Microsoft, NOT VMware. This is one very important difference if you compare it with VMC on AWS. The operation, development and delivery are done by a VMware Cloud Verified and Metal-as-a-Service VCPP (VMware Cloud Provider Program) partner; CloudSimple or Virtustream (subsidiary of Dell Technologies). AVS is fully supported and verified by VMware.

VMware Metal-as-a-Service Authorized partners Virtustream and CloudSimple run the latest VMware software-defined data center technology, ensuring customers enjoy the same benefits of a consistent infrastructure and consistent operations in the cloud as they achieve in their own physical data center, while allowing customers to also access the capabilities of Microsoft Azure.

So, why would someone like Microsoft run VMware’s Cloud Foundation (VCF) stack on Azure? The answer is quite simple. VMware has over 500’000 customers and an estimated number of 70mio VMs which are mostly running on-premises. Microsoft’s doesn’t care if virtual machines (VMs) are running on vSphere, they care about Azure and the consumption in the end. AVS is just another form of Azure, Microsoft says. I would say it’s very unlikely that a customer moves on to Azure native once they are onboarded via Azure VMware Solution.

Microsoft would like to see some of the 70mio running on their platform, no matter if it’s VCF on top of their Azure servers. Customers should get the option to move to the Azure cloud, using Azure native services (e.g. Azure NetApp Files, Azure databases etc.), but give them the choice and flexibility to use their existing technology stack, ecosystem and tools (e.g. automation or operation) they are familiar with – the whole or some part of the VCF coupled with products from the vRealize Suite. Plus, other VMware 3rd party integrations they might have for data protection or backup. This is one unique specialty – Microsoft says – that there is no restricted functionality as you may experience in other VMware clouds.

Azure VMware Solutions Components

From VMware’s perspective most of our customers are already Microsoft customers as well. In addition to that VMware’s vision is to provide the freedom of choice and flexibility, same like Microsoft, but it one small difference: to be cloud and infrastructure agnostic. This vision says that VMware doesn’t care if you run your workloads on-prem, on AWS, Azure or GCP (or even at a VCPP partner’s cloud) as long it’s running on the VCF stack. Cloud is not a choice or destination anymore, it has become an operation model.

And to keep it an operation model without having a new silo and the vendor lock-in, it makes totally sense to use VMware’s VCF on top of AWS, Azure, Google Cloud, Oracle, Alibaba Cloud or any other VCPP partners. This ensures that customers have the choice and flexibility they are looking for, coupled with the new and maybe still surprising “new” or “special” public cloud. If your vision is also about workload mobility on any cloud, then VMware is the right choice and partner!

Use Cases

What are the reasons to move to Azure and use Azure VMware Solution?

If you don’t want to scale up or scale out your own infrastructure and would like to get additional capacity almost instantly, then speed is definitely one reason. Microsoft can spin up a new AVS SDDC under 60min, which is impressive. How is this possible? With automation! This proves that VMware Cloud Foundation is the new data center operating system of the future and that automation is a key design requirement. If you would like to experience nearly the same speed and work with the same principles as public cloud provider do, then VCF is the way to go.

The rest of the use cases or reasons are in general the same if we talk about cloud. If it’s not only speed, then agility, (burstable) capacity, expansion in a new geography, DRaaS or for app modernization reasons using cloud native services.

Microsoft Licenses

What I have learned from this MS Ignite recording, is, that you can bring your existing MS licenses to AVS and that you don’t have to buy them AGAIN. In any other cloud this is not the case.

This information can be found here as well:

Beginning October 1, 2019, on-premises licenses purchased without Software Assurance and mobility rights cannot be deployed with dedicated hosted cloud services offered by the following public cloud providers: Microsoft, Alibaba, Amazon (including VMware Cloud on AWS), and Google. They will be referred to as “Listed Providers”.

Regions

If you check the Azure documentation, you’ll see that AVS is only available in US East and West Azure regions, but should be available in Western Europe “in the near future”. In the YouTube video above Microsoft was showing this slide which shows their global rollout strategy and the planned availability for Q2 2020:

Azure VMware Solutions Regions 2020

According to the Azure regions website Azure VMware Solution is available at the following locations and countries in Europe:

Azure VMware Solutions by Azure RegionSo, North Europe (UK) is expected for H2 2020 and AVS is already available in the West Europe Azure region. Since no information available about the Swiss regions, even the slide from the MS Ignite recording may suggest the availability until May 2020, it’s very unlikely that AVS will be available in Zurich or Geneva before 2021.

Azure VMware Solution Components

You need at least three hosts to get started with the AVS service and you can scale up to 16 hosts per cluster with a SLA of 99.9%. More information about the available node specifications for your region can be found here. At the moment CloudSimple offers the following host types:

  • CS28 node: CPU:2x 2.2 GHz, total 28 cores, 48 HT. RAM: 256 GB. Storage: 1600 GB NVMe cache, 5760 GB data (All-Flash). Network: 4x25Gbe NIC
  • CS36 node: CPU 2x 2.3 GHz, total 36 cores, 72 HT. RAM: 512 GB. Storage: 3200 GB NVMe cache 11520 GB data (All-Flash). Network: 4x25Gbe NIC
  • CS36m node (only option for West Europe): CPU 2x 2.3 GHz, total 36 cores, 72 HT. RAM: 576 GB. Storage: 3200 GB NVMe cache 13360 GB data (All-Flash). Network: 4x25Gbe NIC

I think it’s clear that the used hypervisor is vSphere and that it’s maintained by Microsoft and not by VMware. There is no host-level access, but Microsoft gives you the possibility of a special “just in time” privileges access (root access) feature, which allows to install necessary software bits you might need – for example for 3rd party software integrations.

The storage infrastructure is based on vSAN with an all-flash persistent storage and a NVMe cache storage. More capacity can be made available by adding additional nodes or use Azure offerings which can be added to VMs directly.

Networking and security are based on NSX-T which fully supports micro segmentation.

To offer choice, Microsoft gives you the option to manage and see your AVS VMware infrastructure via vCenter or Azure Resource Manager (ARM). The ARM integration will allow you to create, start, stop and delete virtual machines and is not meant to replace existing VMware tools.

Microsoft support is your single point of contact and CloudSimple contacts VMware if needed.

Connectivity Options

CloudSimple provides the following connectivity options to connect to your AVS region network:

Depending on the connectivity option you have different ways to bring your VMs to your AVS private cloud:

How do I get started?

You have to contact your Microsoft account manager or business development manager if would like to know more. But VMware account representatives are also available to support you. If you want to learn more, check https://aka.ms/startavs.

Can I burn my existing Azure Credits?

Yes. Customers with Azure credits can use them through Azure VMware Solution.