Evicted Pod Problem When Deploying App on Kubernetes

Recently, while deploying Longhorn, I encountered a frustrating issue: my pods were always showing “evicted” even though it was a freshly deployed Kubernetes Cluster. Since I am new to Kubernetes, I had no clue which commands would help me to better understand and then solve the problem. Luckily, ChatGPT makes that process much easier, but it still took me a while to figure it out.

FYI, Longhorn is a lightweight, reliable, and powerful distributed block storage system for Kubernetes.

Understanding the “Evicted” Status

In Kubernetes, eviction is a mechanism used to maintain node stability under resource pressure. When the kubelet detects that a node is running out of resources (memory, CPU, disk), it may evict pods to reclaim resources.

When you run:

kubectl get pods -A

You see something like:

instance-manager-db95e4280b535d82cde25cce5b44f97b 0/1 Evicted 0 1s

If you describe the node, you will see something like:

kubectl describe node k8s-worker1 | grep -A10 "Conditions"

Type: DiskPressure

Status: True

Message: kubelet has disk pressure

 

If you describe the pod, you will see something like:

kubectl describe pod -n longhorn-system

Status: Failed

Reason: Evicted

Message: The node was low on resource: ephemeral-storage.

 

On my node “k8s-worker1”, I then executed this command:

df -h /var/lib/kubelet

which showed:

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv 11G 8.5G 1.8G 84% /var/lib/kubelet

Conclusion: Only 1.8 GB was free, and it was already at 84% usage. Kubernetes starts evicting pods when the disk usage exceeds thresholds, typically around 85–90%, depending on your kubelet settings.

That seems to have happened in my case. I don’t know how production Kubernetes clusters are set up and which “best practices” the experts use, but you would probably at least monitor /var/lib/kubelet closely. 🙂

The Solution

I had to increase the virtual disk size in ESXi and then had to resize the partition, physical volume, and logical volume inside the Ubuntu VM. After resizing the volumes, Longhorn could deploy properly without eviction errors. My nodes now looked like this:

Filesystem Size Used Avail Use% Mounted on
/dev/mapper/ubuntu--vg-ubuntu--lv 71G 8.0G 60G 12% /var/lib/kubelet

Kubernetes stopped evicting pods, and the Longhorn UI was finally available to me and ready to provision volumes! 😀

Kubernetes Longhorn

Kubernetes Documentation – Node-pressure Eviction

Here are some snippets from the official Kubernetes documentation and node-pressure eviction.

The kubelet monitors resources like memory, disk space, and filesystem inodes on your cluster’s nodes. When one or more of these resources reach specific consumption levels, the kubelet can proactively fail one or more pods on the node to reclaim resources and prevent starvation.

You can specify custom eviction thresholds for the kubelet to use when it makes eviction decisions. You can configure soft and hard eviction thresholds.

The kubelet reports node conditions to reflect that the node is under pressure because hard or soft eviction threshold is met, independent of configured grace periods.

Private Cloud Autarky – You Are Safe Until The World Moves On

Private Cloud Autarky – You Are Safe Until The World Moves On

I believe it was 2023 when the term “autarky” was mentioned during my conversations with several customers, who maintained their own data centers and private clouds. Interestingly, this word popped up again recently at work, but I only knew it from photovoltaic systems. And it kept my mind busy for several weeks.

What is autarky?

To understand autarky in the IT world and its implications for private clouds, an analogy from the photovoltaic (solar power) system world offers a clear parallel. Just as autarky in IT means a private cloud that is fully self-sufficient, autarky in photovoltaics refers to an “off-grid” solar setup that powers a home or facility without relying on the external electrical grid or outside suppliers.

Imagine a homeowner aiming for total energy independence – an autarkic photovoltaic system. Here is what it looks like:

  • Solar Panels: The homeowner installs panels to capture sunlight and generate electricity.
  • Battery: Excess power is stored in batteries (e.g., lithium-ion) for use at night or on cloudy days.
  • Inverter: A device converts solar DC power to usable AC power for appliances.
  • Self-Maintenance: The homeowner repairs panels, replaces batteries, and manages the system without calling a utility company or buying parts. 

This setup cuts ties with the power grid – no monthly bills, no reliance on power plants. It is a self-contained energy ecosystem, much like an autarkic private cloud aims to be a self-contained digital ecosystem.

Question: Which partner (installation company) has enough spare parts and how many homeowners can repair the whole system by themselves?

Let’s align this with autarky in IT:

  • Solar Panels = Servers and Hardware: Just as panels generate power, servers (compute, storage, networking) generate the cloud’s processing capability. Theoretically, an autarkic private cloud requires the organization to build its own servers, similar to crafting custom solar panels instead of buying from any vendor.
  • Battery = Spares and Redundancy: Batteries store energy for later; spare hardware (e.g., extra servers, drives, networking equipment) keeps the cloud running when parts fail. 
  • Inverter = Software Stack: The inverter transforms raw power into usable energy, like how a software stack (OS, hypervisor) turns hardware into a functional cloud.
  • Self-Maintenance = Internal Operations: Fixing a solar system solo parallels maintaining a cloud without vendor support – both need in-house expertise to troubleshoot and repair everything.

Let me repeat it: both need in-house expertise to troubleshoot and repair everything. Everything.

The goal is self-sufficiency and independence. So, what are companies doing?

An autarkic private cloud might stockpile Dell servers or Nvidia GPUs upfront, but that first purchase ties you to external vendors. True autarky would mean mining silicon and forging chips yourself – impractical, just like growing your own silicon crystals for panels.

The problem

In practice, autarky for private clouds sounds like an extreme goal. It promises maximum control. Ideal for scenarios like military secrecy, regulatory isolation, or distrust of global supply chains but clashes with the realities of modern IT:

  • Once the last spare dies, you are done. No new tech without breaking autarky.
  • Autarky trades resilience for stagnation. Your cloud stays alive but grows irrelevant.
  • Autarky’s price tag limits it to tiny, niche clouds – not hyperscale rivals.
  • Future workloads are a guessing game. Stockpile too few servers, and you can’t expand. Too many, and you have wasted millions. A 2027 AI boom or quantum shift could make your equipment useless.

But where is this idea of self-sufficiency or sovereign operations coming from? Nowadays? Geopolitical resilience.

Sanctions or trade wars will not starve your cloud. A private (hyperscale) cloud that answers to no one, free from external risks or influence. That is the whole idea.

What is the probability of such sanctions? Who knows… but this is a number that has to be defined for each case depending on the location/country, internal and external customers, and requirements.

If it happens, is it foreseeable, and what does it force you to do? Does it trigger a cloud-exit scenario?

I just know that if there are sanctions, any hyperscaler in your country has the same problems. No matter if it is a public or dedicated region. That is the blast radius. It is not only about you and your infrastructure anymore.

What about private disconnected hyperscale clouds?

When hosting workloads in the public clouds, organizations care more about data residency, regulations, the US Cloud Act, and less about autarky.

Hyperscale clouds like Microsoft Azure and Oracle Cloud Infrastructure (OCI) are built to deliver massive scale, flexibility, and performance but they rely on complex ecosystems that make full autarky impossible. Oracle offers solutions like OCI Dedicated Region and Oracle Alloy to address sovereignty needs, giving customers more control over their data and operations. However, even these solutions fall short of true autarky and absolute sovereign operations due to practical, technical, and economic realities.

A short explanation from Microsoft gives us a hint why that is the case:

Additionally, some operational sovereignty requirements, like Autarky (for example, being able to run independently of external networks and systems) are infeasible in hyperscale cloud-computing platforms like Azure, which rely on regular platform updates to keep systems in an optimal state.

So, what are customers asking for when they are interested in hosting their own dedicated cloud region in their data centers? Disconnected hyperscale clouds.

But hosting an OCI Dedicated Region in your data center does not change the underlying architecture of Oracle Cloud Infrastructure (OCI). Nor does it change the upgrade or patching process, or the whole operating model.

Hyperscale clouds do not exist in a vacuum. They lean on a web of external and internal dependencies to work:

  • Hardware Suppliers. For example, most public clouds use Nvidia’s GPUs for AI workloads. Without these vendors, hyperscalers could not keep up with the demand.
  • Global Internet Infrastructure. Hyperscalers need massive bandwidth to connect users worldwide. They rely on telecom giants and undersea cables for internet backbone, plus partnerships with content delivery networks (CDNs) like Akamai to speed things up.
  • Software Ecosystems. Open-source tools like Linux and Kubernetes are part of the backbone of hyperscale operations.
  • Operations. Think about telemetry data and external health monitoring.

Innovation depends on ecosystems

The tech world moves fast. Open-source software and industry standards let hyperscalers innovate without reinventing the wheel. OCI’s adoption of Linux or Azure’s use of Kubernetes shows they thrive by tapping into shared knowledge, not isolating themselves. Going it alone would skyrocket costs. Designing custom chips, giving away or sharing operational control or skipping partnerships would drain billions – money better spent on new features, services or lower prices.

Hyperscale clouds are global by nature, this includes Oracle Dedicated Region and Alloy. In return you get:

  • Innovation
  • Scalability
  • Cybersecurity
  • Agility
  • Reliability
  • Integration and Partnerships

Again, by nature and design, hyperscale clouds – even those hosted in your data center as private Clouds (OCI Dedicated Region and Alloy) – are still tied to a hyperscaler’s software repositories, third-party hardware, operations personnel, and global infrastructure.

Sovereignty is real, autarky is a dream

Autarky sounds appealing: a hyperscale cloud that answers to no one, free from external risks or influence. Imagine OCI Dedicated Region or Oracle Alloy as self-contained kingdoms, untouchable by global chaos.

Autarky sacrifices expertise for control, and the result would be a weaker, slower and probably less secure cloud. Self-sufficiency is not cheap. Hyperscalers spend billions of dollars yearly on infrastructure, leaning on economies of scale and vendor deals. Tech moves at lightning speed. New GPUs drop yearly, software patches roll out daily (think about 1’000 updates/patches a month). Autarky means falling behind. It would turn your hyperscale cloud into a relic.

Please note, there are other solutions like air-gapped isolated cloud regions, but those are for a specific industry and set of customers.

VMware vSphere Foundation and VMware Cloud Foundation Overview

VMware vSphere Foundation and VMware Cloud Foundation Overview

As some of you already know, VMware by Broadcom is moving forward to primary offers only: VMware vSphere Foundation (VVF) and VMware Cloud Foundation (VCF). If you have missed this announcement, have a look at my blog A New Era: Broadcom’s Streamlined Approach to VMware’s Product Lineup and Licensing.

Since a lot of solutions and different editions from before are included, I thought it might be helpful to summarize in a little bit more detail what is known to partners, analysts, and some customers already. I am also adding some screenshots from VMware websites and presentations, which should help everyone get a better understanding of VVF and VCF.

vSphere Foundation and VMware Cloud Foundation

I have included the vSphere editions for smaller use cases and projects as well.

Please note that ROBO licenses are not available anymore and I expect the edge division at VMware by Broadcom to come up with additional bundles in the future.

VVF and VCF Products

More details about the different products and the features included in the Aria suites can be found here: VMware Aria Suite Editions and Products

If you are looking for more information about the Aria Operations management packs (formerly known as True Visibility Suite or Aria Operations for Integrations), have a look here: VMware Aria Operations for Integrations Documentation

Add-ons for VVF and VCF

The table below gives you an overview of which add-ons are available at the time of writing this blog.

Make sure to contact your VMware representative to understand which add-ons are available for VVF and VCF.

Note: Available add-ons are the text in bold.

VVF and VCF Add-ons

Tanzu Guardrails (formerly VMware Aria Guardrails)

Looking at the official Tanzu Guardrails product website we can learn the following:

Tanzu Guardrails Editions

Note: It seems that Tanzu Hub is part of Tanzu Guardrails Advanced and Enterprise

SRE Services for VMware Cloud Foundation

VMware Site Reliability Engineering (SRE) Services for VMware Cloud Foundation provide VMware expertise to create highly reliable and scalable cloud environments. The services provide a range of capabilities from patching and upgrades to security hardening to automated management and operations.

The SRE Services for VCF datasheet can be found here.

How to count cores for VVF/VCF and TiBs for vSAN add-on?

Please have a look at this updated knowledgebase article: KB95927

What about VMwara Aria SaaS?

Customers have no more option to buy VMware Aria products as standalone products or as SaaS: https://blogs.vmware.com/management/2024/01/dramatic-simplification-of-vmware-aria-as-part-of-vmware-cloud-foundation.html

The Aria cloud management capabilities are available only as components of VMware vSphere Foundation and VMware Cloud Foundation, which are sold for deployment on-premises or on certain public cloud providers including VMware Cloud on AWS. Existing Aria SaaS subscriptions will continue through the end of their term. At time of renewal, customers should purchase VMware vSphere Foundation and VMware Cloud Foundation.

Use this KB96168 to understand which products are impacted by this new policy.

What about Tanzu products?

For some of us, it seems that the Tanzu products are only available as VVF/VCF add-ons, which is not true.

Based on the different comments on various social media platforms and the interviews we have seen from VMware by Broadcom executives, we can say the following:

  • vSphere with Tanzu (aka TKGs) with its Supervisor architecture is going to be the long-term strategy (part of VVF and VCF)
  • Heavy focus on Tanzu Application Platform (TAP) and Tanzu For Kubernetes Operations (TKO)
  • We can expect continued support for TKGm and TKGi

Tanzu Portofolio and Strategy Recap

At VMware Explore 2023, VMware presented the “develop, operate, optimize” approach when they talk about platform engineering:

  1. Develop – Secure paths to production
  2. Operate – Deploy, managed and scale applications seamlessly
  3. Optimize – Continuously tune cost, performance and security of applications at runtime

We learned that VMware (by Broadcom) is going to invest in TAP, Spring, TKO and data services. What’s the difference between TAS and TAP again?

  • Tanzu Application Service – Opinionated platform built on Cloud Foundry
  • Tanzu Application Platform – Modular and portable PaaS for any conformant Kubernetes

Tanzu Portfolio Jan 2024

Tanzu for Kubernetes Operations Refresher

TKO comes in two different editions:

  • Tanzu for Kubernetes Operations Foundation (TKO-F)
    • Tanzu Mission Control (includes TMC self-managed)
    • Tanzu Service Mesh
  • Tanzu for Kubernetes Operations (TKO)
    • Tanzu Mission Control (includes TMC self-managed)
    • Tanzu Service Mesh
    • Tanzu Observability (aka Aria Operations for Apps, formerly Wavefront)
    • Antrea (CNI)
    • TKGm
    • Harbor, HA Proxy, Calico, FluentBit, Contour, Prometheus, Grafana
    • Avi Essentials (NSX ALB)

Note: NSX Advanced Load Balancer (aka NSX ALB) is no longer part of TKO since NSX ALB can be purchased as an add-on

Last Comments

While I was waiting to publish this blog, William Lam wrote a more detailed blog about VMware vSphere Foundation and VMware Cloud Foundation as well.

It is still early days and we can expect more updates from VMware by Broadcom soon. 🙂

 

A New Era: Broadcom’s Streamlined Approach to VMware’s Product Lineup and Licensing

A New Era: Broadcom’s Streamlined Approach to VMware’s Product Lineup and Licensing

Krish Prasad, VMware Cloud Foundation division General Manager, published this blog a few hours ago: https://news.vmware.com/company/vmware-by-broadcom-business-transformation

He announced a massive simplification of the VCF division’s product portfolio, which should help customers get more value for their investment in VMware solutions. To summarize Krish’s announcement:

  • There are going to be two primary standardized offers only from now on: vSphere Foundation and VMware Cloud Foundation (VCF)
  • End of sale of perpetual licenses and Support and Subscription (SnS) renewals
  • Bring-your-own-subscription license (BYOL) option, which provides portability to VMware-validated hybrid clouds running VMware Cloud Foundation

Disclaimer: This blog should provide customers and partners with a summary and some additional information from Krish’s announcement. There is also a chance that some of my understanding is incomplete or wrong. This article reflects my personal opinion and understanding, not Broadcom’s.

I will update the blog over the next few days and weeks.

vSphere Editions

The primary vSphere edition moving forward is called “vSphere Foundation”:

The new VMware vSphere Foundation delivers a more simplified enterprise-grade workload platform for our mid-sized to smaller customers. This solution integrates vSphere with our intelligent operations management to provide the best performance, availability, and efficiency with greater visibility and insights.

In other words, from now on vSphere customers get Aria Operations Advanced (formerly known as vRealize Operations) and Aria Operations for Logs (formerly known as vRealize Log Insight) together with vSphere (which has vCenter and Tanzu Kubernetes Grid included).

vSphere Essentials Plus Kit

Components:

  • vSphere Essentials Plus
  • vCenter Essentials
  • Per 96-core Kit (3 host max. / 6 CPU limit / 192-core limit; a host can have either 1x 64 cores or 2x 32 cores max.)
  • Includes Production Support

vSphere Standard

Components:

  • vSphere Standard
  • vCenter Standard
  • Includes Production Support

vSphere Foundation (VVF)

Components:

  • vSphere Enterprise Plus
  • vCenter Standard
  • Tanzu Kubernetes Grid (TKG)
  • vSAN Enterprise 100 GiB free (see FAQ below)
  • Aria Suite Term Standard
    • Includes Aria Operations Advanced and Aria Operations for Logs
  • Includes Production Support
  • Plus Available Add-ons

Add-on Offerings

VMware vSAN Enterprise (add-on for VCF and vSphere Foundation only)

VMware Cloud Disaster Recovery / Ransomware Recovery  (add-on for VCF and vSphere Foundation only)

Site Recovery Manager Enterprise

VMware Advanced Load Balancer (aka Avi)

VMware Firewall (add-on for VCF only), aka NSX Distributed Firewall

VMware Firewall with Advanced Threat Protection (ATP) (add-on for VCF only)

Tanzu Intelligence (details not clear yet)

Tanzu Mission Control (SaaS and self-managed)

Tanzu Application Platform / Spring Runtime (details not clear yet)

More add-ons coming in future (for example Private AI Foundation)

VMware Cloud Customer Journey

Existing VCF or future vSphere Foundation customers would go for the new VMware Cloud Foundation now, which can now be considered a true full-stack private and hybrid cloud stack:

VMware Cloud Foundation, our flagship enterprise-class hybrid cloud solution for customers to run their business critical and modern applications – in a secure, resilient and cost efficient manner. To allow more customers to benefit from this solution, we’ve reduced the previous subscription list price by half and added higher support service levels including enhanced support for activating the solution and lifecycle management.

Very important in case you missed that from the vSphere Foundation section above: Moving forward, VMware Cloud Foundation only includes NSX for network virtualization (overlay), with no more micro-segmentation or distributed firewalling (DFW) capabilities.

In other words, customers who need NSX’s DFW (VMware Firewall add-on) capabilities, need a VCF subscription first, which includes NSX.

Note: Currently, all new licensing bundles are coming in a “disconnected” fashion (no VMware Cloud connectivity)

While Krish mentioned BYOL and license portability in the future, there seem to be no immediate changes about the VMware Cloud and other hyperscaler offerings.

VMware Cloud Foundation (VCF)

Components:

  • vSphere Enterprise Plus
    • Includes TKG and vCenter Standard
  • vSAN Enterprise per TiB per X amount of cores
  • Aria Suite Enterprise
    • Aria Operations Enterprise (includes Aria Operations for Logs)
    • Aria Automation
  • NSX Networking for VCF
  • HCX Enterprise
  • Aria Operations for Networks Enterprise (formerly known as vRealize Network Insight)
  • SDDC Manager
  • Includes Select Support
    • Includes SRE (customers must deploy SDDC Manager to be entitled to SRE)
  • Add-ons: See above

Note: Going forward, standalone offerings are being EOA-ed (end of availability) – no more VCF components “a la carte”

VMware Cloud on X

We can expect that long-term these VMC or hyperscaler subscription offerings will converge to VCF.

Final Comments

Look, I do not have all the answers and information yet, because it is a lot to unpack. That is all I can share with you right now. Be patient. 🙂

But, this announcement from Broadcom (Krish) was an unexpected surprise since almost everyone was expecting price increases after the acquisition. Instead, Broadcom is cutting the subscription pricing in half!

An update of the above content can be found here: VMware vSphere Foundation and VMware Cloud Foundation Overview

Additional Resources

Unofficial Licensing FAQ

  • Q: What if I am a vSphere Essentials customer?
    • A: I would recommend the vSphere Essentials Plus Kit
  • Q: What if I only need vSphere Enterprise Plus?
    • A: Your best option is vSphere Foundation. There are no more standalone products.
  • Q: Do I need VCF if I want NSX distributed firewall?
    • A: Yes, at the moment this seems to be the case that the “VMware Firewall” (distributed firewall aka micro-segmentation) add-on cannot be subscribed as a vSphere Foundation customer. The same is true of other features like security or gateway firewall.
  • Q: What if have/need vSphere for Desktop?
    • A: The recommended solution is vSphere Foundation
  • Q: What if I am a vCloud Suite customer?
    • A: VMware Cloud Foundation makes sense for vCloud Suite Enterprise and Advanced editions. If you have vCloud Suite Standard I recommend vSphere Foundation going forward.
  • Q: How many vCenters are included?
    • A: To my knowledge, it is one vCenter per core. So, one could say that this means “unlimited”.
  • Q: What happens to the Avi (NSX ALB) Basic edition?
    • A: It seems there will be no Avi Basic anymore. Customers need to go for the add-on.
  • Q: Do customers from now on need to deploy SDDC Manager as part of VMware Cloud Foundation?
    • A: No, they do not. But to be entitled to SRE, you need to deploy the full stack.
  • Q: How is Site Recovery Manager (SRM) Enterprise licensed?
    • A: Per protected 25-VMs
  • Q: What about the True Visibility Suite (TVS)?
    • A: These management packs will be enabled as part of the Aria Suites which are included in vSphere Foundation and VMware Cloud Foundation.
  • Q: What about (vSAN) ROBO licenses?
    • A: ROBO licenses are EOA as well, but all vSphere Foundation customers will receive vSAN Enterprise 100 GiB (for free) for every core purchased.
  • Q: What is included in the Tanzu Intelligence add-on?
    • A: Tanzu Guardrails (Advanced or Enterprise), Aria Operations for Apps (formerly known as Tanzu Observability (aka Wavefront)), Tanzu Application Catalog, Tanzu CloudHealth Enterprise, Tanzu Insights
  • Q: What happened to SaltStack Cionfig and SecOps?
    • A: Both are part of the Tanzu Guardrails Enterprise add-on
  • Q: Can customers mix perpetual and new offerings?
    • A: In general yes.
  • Q: Can you tell me more about the vSAN free tier included with vSphere Foundation?
    • A: It seems you are going to be entitled to a maximum of 100GiB per core in the vSAN storage cluster. Example: 4 hosts with 32 cores each * 100GiB = 12.8TiB (without paying for any vSAN add-on!).
      • Important: This feature will be available in one of the upcoming releases. Hopefully in vSphere 8.0U3 🙂
VMware Cloud Foundation 5.1 – Technical Overview

VMware Cloud Foundation 5.1 – Technical Overview

This technical overview supersedes this version, which was based on VMware Cloud Foundation 5.0, and now covers all capabilities and enhancements that were delivered with VCF 5.1.

What is VMware Cloud Foundation (VCF)?

VMware Cloud Foundation is a multi-cloud platform that provides a full-stack hyperconverged infrastructure (HCI) that is made for modernizing data centers and deploying modern container-based applications. VCF is based on different components like vSphere (compute), vSAN (storage), NSX (networking), and some parts of the Aria Suite (formerly vRealize Suite). The idea of VCF follows a standardized, automated, and validated approach that simplifies the management of all the needed software-defined infrastructure resources.

This stack provides customers with consistent infrastructure and operations in a cloud operating model that can be deployed on-premises, at the edge, or in the public cloud.

What software is being delivered in VMware Cloud Foundation?

Update February 16th, 2024: Please have a look at this article to understand the current VCF licensing. I will publish an updated version of this blog as soon as VMware Cloud Foundation 5.2 has been released.

The BoM (bill of materials) is changing with each VCF release. With VCF 5.1 the following components and software versions are included:

 

Software Component

Version

Date

Build Number

Cloud Builder VM

5.1

07 NOV 2023

22688368

SDDC Manager

5.1

07 NOV 2023

22688368

VMware vCenter Server Appliance

8.0 Update 2a

26 OCT 2023

22617221

VMware ESXi

8.0 Update 2

21 SEP 2023

22380479

VMware vSAN Witness Appliance

8.0 Update 2

21 SEP 2023

22385739

VMware NSX

4.1.2.1

7 NOV 2023

22667789

VMware Aria Suite Lifecycle

8.14

19 OCT 2023

22630473

  • VMware vSAN is included in the VMware ESXi bundle.
  • You can use VMware Aria Suite Lifecycle to deploy VMware Aria Automation, VMware Aria Operations, VMware Aria Operations for Logs, and Workspace ONE Access. VMware Aria Suite Lifecycle determines which versions of these products are compatible and only allows you to install/upgrade to supported versions.
  • VMware Aria Operations for Logs content packs are installed when you deploy VMware Aria Operations for Logs.
  • The VMware Aria Operations management pack is installed when you deploy VMware Aria Operations.
  • You can access the latest versions of the content packs for VMware Aria Operations for Logs from the VMware Solution Exchange and the VMware Aria Operations for Logs in-product marketplace store.

What’s new with VCF 5.1?

Important changes mentioned in the release notes:

  • Support for vSAN ESA.vSAN ESA is an alternative, single-tier architecture designed ground-up for NVMe-based platforms to deliver higher performance with more predictable I/O latencies, higher space efficiency, per-object based data services, and native, high-performant snapshots.
    VCF 5.1 vSAN ESA
  • vSphere Distributed Services engine for Ready nodes. AMD-Pensando and NVIDIA BlueField-2 DPUs are now supported. Offloading the Virtual Distributed Switch (VDS) and NSX network and security functions to the hardware provides significant performance improvements for low latency and high bandwidth applications. NSX distributed firewall processing is also offloaded from the server CPUs to the network silicon.
  • Mixed-mode Support for Workload Domains​. A VCF instance can exist in a mixed BOM state where the workload domains are on different VCF 5.x versions. Note: The management domain should be on the highest version in the instance.
    VCF 5.1 Mixed Mode
  • Support for mixed license deployment. A combination of keyed and keyless licenses can be used within the same VCF instance.
  • VMware vRealize rebranding. VMware recently renamed vRealize Suite of products to VMware Aria Suite. See the Aria Naming Updates blog post for more details.
  • Increased GPU scale. VMware Cloud Foundation 5.1 provides increased support for VMs to be configured with up to 16 GPU devices.
    VCF 5.1 GPU Scale

What are the VMware Cloud Foundation components?

To manage the logical infrastructure in the private cloud, VMware Cloud Foundation augments the VMware virtualization and management components with VMware Cloud Builder and VMware Cloud Foundation SDDC Manager.

VMware Cloud Foundation Component Description
VMware Cloud Builder VMware Cloud Builder automates the deployment of the software-defined stack, creating the first software-defined unit known as the management domain.
SDDC Manager

SDDC Manager automates the entire system life cycle, that is, from configuration and provisioning to upgrades and patching including host firmware, and simplifies day-to-day management and operations. From this interface, the virtual infrastructure administrator or cloud administrator can provision new private cloud resources, monitor changes to the logical infrastructure, and manage life cycle and other operational activities.

VMware Cloud Foundation SDDC Manager Dashboard

vSphere

vSphere uses virtualization to transform individual data centers into aggregated computing infrastructures that include CPU, storage, and networking resources. VMware vSphere manages these infrastructures as a unified operating environment and provides you with the tools to administer the data centers that participate in that environment.

The two core components of vSphere are ESXi and vCenter Server. ESXi is the virtualization platform where you create and run virtual machines and virtual appliances. vCenter Server is the service through which you manage multiple hosts connected in a network and pool host resources.

vSAN

vSAN aggregates local or direct-attached data storage devices to create a single storage pool that is shared across all hosts in the vSAN cluster. Using vSAN removes the need for external shared storage, and simplifies storage configuration and virtual machine provisioning. Built-in policies allow for flexibility in data availability.

NSX NSX is focused on providing networking, security, automation, and operational simplicity for emerging application frameworks and architectures that have heterogeneous endpoint environments and technology stacks. NSX supports cloud-native applications, bare-metal workloads, multi-hypervisor environments, public clouds, and multiple clouds.
vSphere with Tanzu By using the integration between VMware Tanzu and VMware Cloud Foundation, you can deploy and operate the compute, networking, and storage infrastructure for vSphere with Tanzu, also called Workload Management. vSphere with Tanzu transforms vSphere to a platform for running Kubernetes workloads natively on the hypervisor layer. When enabled on a vSphere cluster, vSphere with Tanzu provides the capability to run Kubernetes workloads directly on ESXi hosts and to create upstream Kubernetes clusters within dedicated resource pools.
VMware Aria Suite

VMware Cloud Foundation supports automated deployment of VMware Aria Suite Lifecycle. You can then deploy and manage the life cycle of Workspace ONE Access and the VMware Aria Suite products (VMware Aria Operations for Logs, VMware Aria Automation, and VMware Aria Operations) by using VMware Aria Suite Lifecycle.

VMware Aria Suite is a purpose-built management solution for the heterogeneous data center and the hybrid cloud. It is designed to deliver and manage infrastructure and applications to increase business agility while maintaining IT control. It provides the most comprehensive management stack for private and public clouds, multiple hypervisors, and physical infrastructure.

VMware Cloud Foundation Architecture

VCF is made for greenfield deployments (brownfield not supported) and supports two different architecture models:

  • Standard Architecture
  • Consolidated Architecture

VMware Cloud Foundation Deployment Options

The standard architecture separates management workloads and lets them run on a dedicated management workload domain. Customer workloads are deployed on a separate virtual infrastructure workload domain (VI workload domain). Each workload domain is managed by a separate vCenter Server instance, which allows autonomous licensing and lifecycle management.

VMware Cloud Foundation Single Site Deployment

Note: The standard architecture is the recommended model because it separates management workloads from customer workloads.

Customers with a small environment (or a PoC) can start with a consolidated architecture. This allows you to run customer and management workloads together on the same workload domain (WLD).

Management Domain

The management domain is created during the bring-up process by VMware Cloud Builder and contains the VMware Cloud Foundation management components as follows:

  • Minimum four ESXi hosts

  • An instance of vCenter Server

  • A three-node NSX Manager cluster

  • SDDC Manager

  • vSAN datastore
  • One or more vSphere clusters each of which can scale up to the vSphere maximum of 64

VI Workload Domains

You create VI workload domains to run customer workloads. For each VI workload domain, you can choose the storage option – vSAN, NFS, vVols, or VMFS on FC.

VMware Cloud Foundation Storage Options

A VI workload domain consists of one or more vSphere clusters. Each cluster starts with a minimum of three hosts and can scale up to the vSphere maximum of 64 hosts. SDDC Manager automates the creation of the VI workload domain and the underlying vSphere clusters.

For the first VI workload domain in your environment, SDDC Manager deploys a vCenter Server instance and a three-node NSX Manager cluster in the management domain. For each subsequent VI workload domain, SDDC Manager deploys an additional vCenter Server instance. New VI workload domains can share the same NSX Manager cluster with an existing VI workload domain or you can deploy a new NSX Manager cluster. VI workload domains cannot use the NSX Manager cluster for the management domain.

What is a vSAN Stretched Cluster?

vSAN stretched clusters extend a vSAN cluster from a single site to two sites for a higher level of availability and inter-site load balancing.

VMware Cloud Foundation Stretched Cluster

Does VCF provide flexible workload domain sizing?

Yes, that’s possible. You can license the WLDs based on your needs and use the editions that make the most sense depending on your use cases.

VMware Cloud Foundation Flexible Licensing

How many physical nodes are required to deploy VMware Cloud Foundation?

A minimum of four physical nodes is required to start in a consolidated architecture or to build your management workload domain. Four nodes are required to ensure that the environment can tolerate a failure while another node is being updated.

VI workload domains require a minimum of three nodes.

Can I mix vSAN ReadyNodes and Dell EMC VxRail deployments?

No. This is not possible.

What about edge/remote use cases?

When you would like to deploy VMware Cloud Foundation workload domains at a remote site, you can deploy so-called “VCF Remote Clusters”. Those remote workload domains are managed by the VCF instance at the central site and you can perform the same full-stack lifecycle management for the remote sites from the central SDDC Manager.

VMware Cloud Foundation Remote Cluster

Prerequisites to deploy remote clusters can be found here.

Note: If vSAN is used, VCF only supports a minimum of 3 nodes and a maximum of 4 nodes per VCF Remote Cluster. If NFS, vVOLs or Fiber Channel is used as principal storage, then VCF supports a minimum of 2 and a maximum of 4 nodes.

Important: Remote clusters and remote workload domains are not supported when VCF+ is enabled.

How many resources does the VCF management WLD need during the bring-up process?

We know that VCF includes vSphere (ESXi and vCenter), vSAN, SDDC Manager, NSX and eventually some components of the vRealize Suite. The following table should give you an idea what the resource requirements look like to get VCF up and running:

VMware Cloud Foundation Resource Requirements

If you are interested to know how many resources the Aria Suite (formerly vRealize Suite) will consume of the management workload domain, have a look at this table:

VMware Cloud Foundation Resource Requirements vRealize

Does VCF support HCI Mesh?

Yes. VMware Cloud Foundation 4.2 and later supports sharing remote datastores with HCI Mesh for VI workload domains.

HCI Mesh is a software-based approach for disaggregation of compute and storage resources in vSAN. HCI Mesh brings together multiple independent vSAN clusters by enabling cross-cluster utilization of remote datastore capacity within vCenter Server. HCI Mesh enables you to efficiently utilize and consume data center resources, which provides simple storage management at scale.

Note: At this time, HCI Mesh is not supported with VCF ROBO.

Important: HCI Mesh can be configured with vSAN OSA or ESA. HCI Mesh is not supported between a mix of
vSAN OSA and ESA clusters.

Does VMware Cloud Foundation support vSAN Max?

At the time of writing, no.

How is VMware Cloud Foundation licensed?

Currently, VCF is sold as part of VMware Cloud editions.

How can I migrate my workloads from a non-VCF environment to a new VCF deployment?

VMware HCX provides a path to modernize from a legacy data center architecture by migrating to VMware Cloud Foundation.

VMware Cloud Foundation HCX

Can I install VCF in my home lab?

Yes, you can. With the VLC Lab Constructor, you can deploy an automated VCF instance in a nested configuration. There is also a Slack VLC community for support.

VCF Lab Constructor

Note: Please have a look at “VCF Holodeck” if you would like to create a smaller “sandbox” for testing or training purposes

VCF Holodeck Toolkit 

Where can I find more information about VCF?

Please consult the VMware Cloud Foundation FAQ for more information.

VMware Explore 2023 – Major Announcements

VMware Explore 2023 – Major Announcements

After Las Vegas, I was lucky enough to attend VMware Explore 2023 in Barcelona as well. This article gives you an overview of some of the major announcements. In case you missed the announcements from Las Vegas in August 2023, have a look here: https://www.cloud13.ch/2023/08/22/vmware-explore-2023-us-day-1-announcements/

VMware Sovereign Cloud

Today, VMware announced new innovations and technology partnerships that will help accelerate sovereign digital innovation and enhance security for customers around the world. Today, more than 50 VMware Sovereign Cloud providers in 33 countries are part of a powerful, interconnected, and diverse ecosystem that supports customers’ sovereign cloud requirements. Together, VMware and VMware Sovereign Cloud partners are helping organizations unlock the innovative power of their data while remaining compliant with data privacy regulations.

Details can be found here: https://news.vmware.com/releases/vmware-explore-2023-barcelona-sovereign-cloud 

Software-Defined Data Center

Since VMware announced vSphere 8.0 U2 and vSAN 8.0 U2 in Las Vegas, and NSX 4.1.2 in October 2023, we only heard about the future VMware Cloud Foundation 5.1 release in Barcelona.

VMware Cloud Foundation (VCF)

VMware announced VCF 5.1 with the following BOM:

VMware Cloud Foundation 5.1 BOM

Here is a list with other enhancements:

  • VCF Support for vSAN Express Storage Architecture (ESA)
  • Networking and Security Enhancements
  • vSphere Distributed Services Engine for VCF environments (support for DPUs)
  • GPU Enhancements for Performance and Scale (up to 16 GPUs/vGPUs per VM)
  • Mixed-mode Support for Workload Domains (run VCF 5.x workload domains of different versions)
  • Terraform Provider for VMware Cloud Foundation
  • Run VCF 5.x workload domains of different versions (key-based & keyless licensing options for brownfield deployments)

VCF Mixed Mode

Note: VMware Cloud Foundation 5.1 is now available for perpetual deployments, subscription environments will be supported with VCF 5.1 at a later date.

Ransomware

VMware announced the upcoming launch of VMware Live Recovery, a new solution that provides protection against ransomware as well as disaster recovery across VMware Cloud in one unified console. VMware Live Recovery is designed to help organizations protect their VMware-based applications and data from a wide variety of threats, including ransomware attacks, infrastructure failure, human error, and more. By bringing together the functions of established products VMware Site Recovery Manager and VMware Cloud Disaster Recovery with Ransomware Recovery – and combining them under a unified, flexible, and SaaS-based console – customers can realize comprehensive enterprise protection within a single solution.

VMware Live Recovery provides:

  • Secure Cyber Recovery. VMware Live Recovery will enable organizations to recover from ransomware attacks confidently and quickly.
  • Unified Protection. VMware Live Recovery will provide a single console to manage ransomware and disaster recovery functionality, simplifying administration across the enterprise.
  • Simplified Consumption. VMware Live Recovery will offer flexible licensing across use cases and clouds, making it easy for organizations to get the protection they need.

Modern Applications

Have a look at this blog to understand the enhancements made to Tanzu Hub and Tanzu Intelligence Services: https://tanzu.vmware.com/content/blog/tanzu-hub-integrations-explore-2023 

Tanzu Application Platform 1.7

If you are interested in TAP, have a look at this blog: https://tanzu.vmware.com/content/blog/tanzu-application-platform-1-7-release-news

Tanzu Application Service 5.0

Should you be looking for the TAS 5.0 announcement, have a look at this blog: https://tanzu.vmware.com/content/blog/tanzu-application-service-5-release-news 

Tanzu Application Catalog

After Tanzu Application Catalog was renamed to VMware Application Catalog in November 2021, it now seems to be called Tanzu Application Catalog again. 😀

I think the same is true for Tanzu Data Services, which was renamed to VMware Data Services, which now seems to be known as Tanzu Data Services.

Tanzu Data Services

VMware announced the next major release for Data Services Manager (DSM) and two new partnerships.

Data Services Manager 2.0 (DSM)

VMware vision is to make data easy to store, manage, and consume on any VMware cloud.

The upcoming release of Data Services Manager will be tightly integrated with VMware Cloud Foundation, with built-in database visibility and resource controls accessible through vSphere UI and APIs. Data Services Manager 2.0 will also support enhanced automation for data services lifecycle management, including non-disruptive patching and upgrades. Finally, it will deliver cloud-native self-service capabilities for application teams through tools of their choice, with support for VMware Aria Automation and Kubernetes. Last but not least, with version 2.0, Data Services Manager will become a platform for managing different data services, based not only on data engines that come from VMware (currently Tanzu SQL), but also engines from 3rd parties.

That is why they announced the initial key partnerships with Google Cloud for AlloyDB Omni and MinIO for Object Storage on VMware Cloud Foundation. All managed by DSM with the same features, workflows, and user experience. This includes deeper integration with vSphere as well as VMware vSAN with data path optimizations and the use of features like snapshots and clones.

VMware Data Services Manager 2.0

Note: Expected GA date is January 2024

Anywhere Workspace

The end-user computing related announcements are summarized in this blog: https://blogs.vmware.com/euc/2023/11/whats-new-in-anywhere-workspace-at-vmware-explore-barcelona-2023.html