Do you want to build your private cloud like a hyperscaler is doing it? You know that VMware Cloud Foundation is becoming the new vSphere, but still wonder how you can implement software-defined load balancing (LB) or application services and features like autoscaling or predictive scaling? Then this article about multi-cloud load balancing and autoscaling with NSX Advanced Load Balancer aka Avi Networks is for you!
My Experience with a Legacy ADC
A few years ago, I was working on the customer side for an insurance company in Switzerland as a Citrix System Engineer. My daily tasks included the maintenance and operation of the Citrix environment, which included physical and virtual Citrix NetScaler Application Delivery Controller (ADC) appliances. The networking team owned a few hardware-based appliances (NetScaler SDX) with integrated virtualization capability (XenServer as hypervisor) to host multiple virtual NetScaler (VPX) instances.
The networking team had their dedicated NetScaler VPX instances (for LDAP and HTTP load balancing mostly) and deployed my appliances after I filed a change request. Today, you would call this multi-tenancy. For a Citrix architecture it was best practices to have one high availability (HA) pair for the internal and one HA pair for the external (DMZ) network access. An HA pair was running in a active/passive mode and I had to maintain the same setup for the test environment as well.
Since my virtual VPX appliances were hosted on the physical SDX appliance, I always relied on the network engineers when I needed more resources (CPU, RAM, SSL, throughput) allocated to my virtual instances. Before I could upgrade to a specific firmware version, I also had to wait until they upgraded the physical NetScaler appliances and approved my change request. This meant, we had to plan changes and maintenance windows together and had to cross fingers, that their upgrade went well, so that we then could upgrade all our appliances after.
It was also possible to download a VPX appliance, which could run on top of VMware vSphere. To be more independent, I decided to install four new VPX appliances (for the production and test environment) on vSphere and migrated the configuration from the appliances running on the physical SDX appliance.
Another experience I had with load balancers was when I started to work for Citrix as a consultant in Central Europe and had to perform a migration of physical NetScaler MPX appliances, which had no integrated virtualization capability. I believe I had two sites with each two of these powerful MPX appliances for tens of thousands of users. Beside the regular load balancing configuration for some of the Citrix components, I also had to configure Global Server Load Balancing (GSLB) in active/passive mode for the two sites.
There were so many more features available (e.g., Web Application Firewall, Content Switching, Caching, Intrusion Detection), but I never used anything else than the NetScaler Universal Gateway for the remote access to the virtual desktop infrastructure (VDI), load balancing, HTTP to HTTPS redirections and GSLB. In all scenarios I had an HA pair where one instance was idling and doing nothing. And the active unit was in average not utilized more than 15-20%. It was common to install/buy too large or powerful instances/licenses, because you wanted to be on the safe side and have enough capacity to terminate all your SSL sessions and so on.
It (load balancing) was about distributing network traffic across multiple servers by spreading the requests and workload evenly, and to add some intelligence (health monitoring) in case an application server or a service would fail or become unavailable for any reason. If one more application server was needed, I ordered a new Windows Server, installed and configured the Citrix components and added the necessary load balancing configuration on the NetScaler. These were all manual tasks.
This was my personal experience from 2017. Since then, applications became more complex and distributed. The analysts and the market are focusing on containerized and portable apps that are running in multiple clouds. The prediction is also that the future is multi-cloud.
Multiple Clouds vs. Multi-Cloud
There are different definitions and understandings out there what multi-cloud means. In my understanding, using a private cloud, the AWS cloud, Microsoft Office 365 and Azure is a typical setup with multiple clouds. There are simple scenarios where you migrate workloads from a private to a public cloud; or having applications with services hosted on the private and on the public cloud. The latter would be an example of a hybrid cloud architecture.
There are various reasons for why services and resources (workloads) are distributed on multiple clouds (on-prem, Azure, AWS, GCP etc.):
- Avoid dependence on one cloud provider only
- Consume different specific services that are not provided elsewhere
- Optimize costs for different workloads and services
- React to price changes by the providers
That is why we are seeing also the trend to break up big legacy applications (monoliths) in smaller pieces (segments), which is a best practice and design principle today. The goal is to move to a loosely coupled and more service-oriented architecture. This provides greater agility, more flexibility and easier scalability with less inter-dependencies.
A segmented application is much easier to run in different clouds (think about portability). Running one application over multiple clouds is in my understanding the right definition of multi-cloud.
Let’s assume that most probably all the four reasons above apply to larger enterprises. If we take another angle, we can define some business and technical requirements for multi-cloud:
- Application or services need to be cloud vendor-agnostic
- Provide or abstract control and management interfaces of multiple clouds
- Support application portability (relocation between clouds)
- Combine IaaS and other services from different clouds
- Possibility to deploy components of applications in multiple clouds
- (Cloud) Broker service needed
- Policy and governance over multiple clouds
- Network connectivity for migration scenarios with partially modernized applications
- Automated procedures for deployments
- Application monitoring over different clouds
- Cost management
- Lifecycle management of deployed applications in multiple clouds
- Self-adaption and auto scaling features
- Large team with various expertise needed
How can you deliver and manage different applications services like load balancing, web application firewalling, analytics, automation and security over multiple clouds?
Another important question would be, how you want to manage the workload deployment on the various clouds. But cloud management or a cloud management platform is something for another article. 🙂
The requirements for the developers, IT operations and the business are very complex and it’s a long list (see above).
It is important, that you understand, based on the requirements for multi-cloud, that it is mandatory to implement a modern solution for your modern application architectures. Enterprises have become more application-centric and everyone is talking about continuous integration, continuous delivery and DevOps practices to automate operation and deployment tasks. A modern solution implicits a software-defined approach. Otherwise you won’t be able to be agile, adapt to changes and meet future requirements.
My past experience with Citrix’ NetScaler (Citrix ADC) is a typical example that “virtualized” and “software-defined” are not the same thing. And this is very important if we want to have a future-ready solution. If we look at VMware’s software-defined data center (SDDC), it includes virtual compute (hypervisor), but software-defined storage and networking as well. Part of the software-defined networking portfolio is “NSX Advanced Load Balancer“, the software-defined application services platform, which was also known as “Avi Networks” before VMware bought them in June 2019.
Unlike a virtualized load-balancing appliance, a software-defined
application services platform separates the data and control planes
to deliver application services beyond load balancing, real-time
application analytics, security and monitoring, predictive autoscaling,
and end-to-end automation for Transport (Layer 4) to
Application (Layer 7) layer services. The platform supports multicloud
environments and provides software-defined application
services with infrastructure-agnostic deployments on bare metal
servers, virtual machines (VMs), and containers, in on-premises
data centers and private/public clouds.
Auto scaling became famous with AWS as it monitors your applications and automatically adjusts capacity to maintain availability and performance at the lowest possible costs. It automatically adds or removes application servers (e.g. EC2 instances), load balancers, applies the right network configuration and so on.
Can you achieve the same for your on-premises infrastructure with VMware? Yes.
Is there even a solution which can serve both worlds – on-prem and cloud? Yes.
And what about predictive scaling with real-time insights? Yes.
NSX Advanced Load Balancer (NSX ALB)
Why did VMware buy Avi? Because it follows the same architecture principles like NSX: A distributed platform with a separate control and data plane built on software-defined principles for any cloud.
Traditional ADCs or load balancers are mostly configured in active/standby pairs, no matter if physical or virtual. Typically you would see around 15% utilization on the active node, while the secondary standby node is just idling and doing nothing. Each pair is its own island of static capacity which shares the management, control and data plane.
You have to decide where to place the virtual IP (VIP) and how much you want to overprovision the physical or virtual appliances, because there is no capacity pooling available. This leads to operational complexity, especially when you have hundreds of such HA pairs running in different clouds. Therefore, legacy and virtualized ADCs are not the ideal choice for a multi-cloud architecture. Let’s check NSX ALB’s architecture:
Control Plane – This is the brain (single point of management) of the whole platform that can be spun up in your on-prem environment or in the cloud (also available as a managed SaaS offering), typically as a three-node cluster. Within this cluster, all configuration is done. This also where the policies reside and the decisions are made. It is the controller’s duty to place virtual services on Service Engines to load balance new applications.
The control plane comprises the three pillars that deliver the key capabilities of the Avi platform:
- Multi-Cloud – Consistent experience for any cloud, no lock-in
- Intelligence – The machine learning based analytics engine enables application performance monitoring, troubleshooting, and operational insights (gathered by the SEs)
- Automation – Elastic and predictive auto scaling & self-service without over-provisioning through a complete set of REST APIs
Data Plane – The Service Engines (SEs) handle all data plane operations by receiving and executing instructions from the controller. The SEs perform load balancing and all client- and server-facing network interactions. It collects real-time application telemetry from application traffic flows.
As already mentioned, NSX ALB can be deployed in multiple cloud environments like VMware vCenter, Amazon Web Services, Microsoft Azure, Google Cloud Platform, Oracle Cloud, IBM Cloud, VMC on AWS, Nutanix, OpenStack or bare-metal.
Most customers deploy Avi because of:
- Load Balancer refresh
- Multi-Cloud initiatives
- Security including WAF, DDoS attack mitigation, achieving compliance (GDPR, PCI, HIPAA)
- Container ingress (integrates via REST APIs with K8s ecosystems like Tanzu Kubernetes Grid, AKS, EKS, GKE, OpenShift)
- Virtual Desktop Infrastructure (Citrix, VMware Horizon)
Consistent Application Services Platform (Features)
Avi/NSX ALB is an enterpise-grade solution. So, everything you would expect from a traditional ADC (e.g. F5); layer 4 to layer 7 services, SSL, DDoS, WAF etc. is built-in without the need for a special license or edition. There is no NSX license requirement even the product name would suggest it. It can be deployed as a standalone load balancer or as an integrated solution with other VMware products (e.g., VCF, vRA/vRO, Horizon, Tanzu etc.).
Below is a list with the core features:
- Enterprise-class load balancing – SSL termination, default gateway, GSLB, DNS, and other L4-L7 services
- Multi-cloud load balancing – Intelligent traffic routing across multiple sites and across private or public clouds
- Application performance monitoring – Monitor performance and record and replay network events like a Network DVR
- Predictive auto scaling – Application and load balancer scaling based on real-time traffic patterns
- Self-service – For app developers with REST APIs to build services into applications
- Cloud connectors – VMware Cloud on AWS, SDN/NFV controllers, OpenStack, AWS, GCP, Azure, Linux Server Cloud, OpenShift/Kubernetes
- Distributed application security fabric – Granular app insights from distributed service proxies to secure web apps in real time
- SSO / Client Authentication – SAML 2.0 authentication for back-end HTTP applications
- Automation and programmability – REST API based solution for accelerated application delivery; extending automation from networking to developers
- Application Analytics – Real-time telemetry from a distributed load balancing fabric that delivers millions of data points in real time
Load Balancing for VMware Horizon
NSX ALB can be configured for load balancing in VMware Horizon deployments, where you place Service Engines in front of Unified Access Gateways (UAG) or Connection Servers (CS) as required.
For a multi-site architecture you can also configure GSLB if needed. With GSLB, access to resources is controlled with DNS queries and health checking.
Note: If you are using the Horizon Universal Broker, the cloud-based brokering service, there is no need for GSLB, because the Universal Broker can orchestrate connections from a higher level based on different policies.
With NSX Advanced Load Balancer there are two parts when we talk about automation. One part is about infrastructure automation, where the controller talks to the ecosystem like a vCenter, AWS or Azure to orchestrate the Service Engine. So, when you configure a new VIP, the controller would talk to vCenter to spin up a VM, puts it in the right portgroup, connects the front and the back-end, downloads the policy and Service Engine, and starts receiving traffic.
The second piece of automation focuses more on the operational automation which is through the REST API. But, on top of that you can also run Ansible playbooks, Terraform templates, use Go and Python SDKs, have integrations with Splunk or other tools like vRealize Automation. These are the built-in automation capabilities of Avi.
VMworld 2020 Sessions
This year, VMworld is going to be for free and virtual. Take this chance and register yourself and learn more about Avi aka NSX ALB:
- Making Your Private Cloud Network Run Like a Public Cloud – Part 2 [VCNC2918]
- Modern Apps and Containers: Networking and Security [VCNC2920]
- Prepare for the New Normal of Work from Anywhere [VCNC2919]
on your multiple clouds multi-cloud diagram, does it mean the concept of single control plane across multi-clouds, I am sure NSX adv LB does it very well..how about the data plane, on this diagram, does it mean – provision a VS on aSE-group where multiple SE are on different fabric – on-prem / AWS /Azure ? or multiple VS on different clouds (if so, how the analytic collected to reflect single app on multiple VS ?
Yes, you would have one control plane across the multiple clouds. E.g. Your controllers could sit on-prem on a vSphere environment and your SEs would run there, but also on AWS or Azure.
The data plane is a little bit different as we usually create SE groups per cloud or data center. In this case, you would typically create different VS/VIPs for each cloud and use GSLB (same application in multiple clouds).
But in the “multiple clouds vs. multi-cloud” diagram, you would run different pieces/services of the application in multiple clouds. Then you would anyway have a dedicated VIP for each microservices in each cloud.
Avi Controllers never initiate the connection to the SEs. They are always initiated by the SEs (sitting in any cloud). Therefore, and this is normally the case, a network connection (ExpressRoute or Direct connect) exists for the metric collection.
Does this answer all your questions? 🙂