← Back to blog
ProxmoxPrivate CloudVMware

Private Cloud on Proxmox: A Cost-Effective VMware Alternative

· 8 min read

VMware’s acquisition by Broadcom reshaped the virtualization market overnight. License changes, price increases, and the elimination of perpetual licenses pushed many organizations to re-evaluate their hypervisor strategy. The question is no longer “should we consider alternatives?” but “which alternative fits our needs?”

For many mid-size companies, the answer is Proxmox VE — an open-source virtualization platform based on KVM and LXC that delivers enterprise-grade features without enterprise-grade licensing costs. In this post, we walk through building a production private cloud on Proxmox, from cluster design to Kubernetes deployment.

The Cost Reality

Let us start with numbers, because this is often the deciding factor.

VMware vSphere (post-Broadcom):

  • vSphere Foundation: ~$250/core/year (minimum 16 cores per CPU)
  • A 3-node cluster with dual 32-core CPUs: ~$48,000/year in licensing alone
  • vSAN licensing adds more on top

Proxmox VE:

  • Community edition: Free (open-source, AGPL)
  • Enterprise subscription (recommended for production): ~$110/CPU socket/year
  • Same 3-node cluster: ~$660/year

That is not a typo. The licensing delta is roughly $47,000/year for a modest cluster. Over three years, you are looking at $140,000+ in savings — enough to hire an engineer or fund an entire infrastructure refresh.

The trade-off is ecosystem maturity. VMware has decades of enterprise tooling, certified integrations, and a massive knowledge base. Proxmox requires more hands-on operational knowledge. But for teams already comfortable with Linux administration (which most DevOps teams are), this is not a significant barrier.

graph TD
    subgraph Proxmox Cluster
        N1[Node 1 - 256GB]
        N2[Node 2 - 256GB]
        N3[Node 3 - 256GB]
    end
    N1 --- CEPH[(Ceph Replicated Pool)]
    N2 --- CEPH
    N3 --- CEPH
    N1 --> K8S[Kubernetes VMs]
    N2 --> APP[Application VMs]
    N3 --> DB[Database VMs]

Cluster Design

A production Proxmox cluster requires a minimum of three nodes for quorum. Here is our recommended starting configuration:

Hardware per Node

ComponentSpecification
CPUAMD EPYC 9354 (32 cores) or similar
RAM256 GB DDR5 ECC
OS disk2x 480 GB NVMe (ZFS mirror)
Ceph OSD disks4x 3.84 TB NVMe
Network2x 25 GbE (Ceph/storage), 2x 10 GbE (VM traffic), 1x 1 GbE (management)

Network Architecture

Network design is critical in a Proxmox cluster. We separate traffic into dedicated networks, as discussed in our network engineering post:

┌─────────────────────────────────────────────┐
│              Management Network               │
│          VLAN 10 — 10.10.0.0/24              │
│  Proxmox Web UI, SSH, Corosync cluster       │
├─────────────────────────────────────────────┤
│              VM / Production Network          │
│          VLAN 20 — 10.20.0.0/22              │
│  VM traffic, bonded 10 GbE                   │
├─────────────────────────────────────────────┤
│              Ceph Storage Network             │
│          VLAN 30 — 10.30.0.0/24              │
│  Ceph OSD replication, bonded 25 GbE         │
├─────────────────────────────────────────────┤
│              Ceph Public Network              │
│          VLAN 31 — 10.31.0.0/24              │
│  Ceph client access from VMs                 │
└─────────────────────────────────────────────┘

Separate the Ceph replication traffic from client traffic. Ceph replication during recovery can saturate links, and you do not want that affecting VM disk I/O.

Ceph Storage: Your Software-Defined SAN

Ceph replaces the need for a dedicated SAN or VMware vSAN. It provides distributed, replicated block storage using the NVMe drives in each Proxmox node.

Setting Up Ceph

Proxmox integrates Ceph directly. After installing Ceph through the Proxmox UI or CLI:

# On each node, create OSDs from NVMe drives
pveceph osd create /dev/nvme1n1
pveceph osd create /dev/nvme2n1
pveceph osd create /dev/nvme3n1
pveceph osd create /dev/nvme4n1

# Create a storage pool with replication factor 3
pveceph pool create vm-storage --size 3 --min_size 2 --pg_autoscale_mode on

Tuning for Production

Default Ceph settings are conservative. For NVMe-backed clusters:

# Increase OSD memory target for better caching
ceph config set osd osd_memory_target 8589934592  # 8 GB per OSD

# Enable bluestore compression for mixed workloads
ceph config set osd bluestore_compression_algorithm zstd
ceph config set osd bluestore_compression_mode aggressive

# Tune recovery to be less disruptive
ceph config set osd osd_recovery_max_active 3
ceph config set osd osd_max_backfills 1

With 12 NVMe OSDs across three nodes (replication factor 3), you get roughly 15 TB of usable, highly available storage with excellent IOPS performance.

VM Templating with Packer

Manual VM creation does not scale. We use Packer to build golden images that are imported into Proxmox as templates:

# ubuntu-server.pkr.hcl
source "proxmox-iso" "ubuntu" {
  proxmox_url              = "https://pve1.internal:8006/api2/json"
  username                 = "root@pam"
  token                    = var.proxmox_api_token
  node                     = "pve1"
  insecure_skip_tls_verify = false

  iso_file         = "local:iso/ubuntu-24.04-live-server-amd64.iso"
  iso_checksum     = "sha256:..."
  unmount_iso      = true

  vm_id            = 9000
  vm_name          = "ubuntu-24.04-template"
  template_name    = "ubuntu-24.04-template"

  cores            = 2
  memory           = 2048
  scsi_controller  = "virtio-scsi-single"

  disks {
    disk_size    = "20G"
    storage_pool = "vm-storage"
    type         = "scsi"
    iothread     = true
  }

  network_adapters {
    bridge   = "vmbr1"
    model    = "virtio"
    vlan_tag = 20
  }

  cloud_init              = true
  cloud_init_storage_pool = "vm-storage"

  ssh_username = "ubuntu"
  ssh_timeout  = "20m"
}

build {
  sources = ["source.proxmox-iso.ubuntu"]

  provisioner "shell" {
    inline = [
      "sudo apt-get update",
      "sudo apt-get upgrade -y",
      "sudo apt-get install -y qemu-guest-agent cloud-init",
      "sudo systemctl enable qemu-guest-agent",
      "sudo cloud-init clean",
    ]
  }
}

Infrastructure as Code with Terraform

Once you have templates, deploy VMs with the Proxmox Terraform provider:

terraform {
  required_providers {
    proxmox = {
      source  = "bpg/proxmox"
      version = "~> 0.60"
    }
  }
}

resource "proxmox_virtual_environment_vm" "k8s_worker" {
  count     = 3
  name      = "k8s-worker-${count.index + 1}"
  node_name = element(["pve1", "pve2", "pve3"], count.index)

  clone {
    vm_id = 9000  # Our Packer template
  }

  cpu {
    cores = 8
    type  = "host"
  }

  memory {
    dedicated = 32768
  }

  disk {
    interface    = "scsi0"
    size         = 100
    datastore_id = "vm-storage"
    iothread     = true
  }

  network_device {
    bridge  = "vmbr1"
    vlan_id = 20
  }

  initialization {
    ip_config {
      ipv4 {
        address = "10.20.0.${10 + count.index}/22"
        gateway = "10.20.0.1"
      }
    }

    user_account {
      keys     = [var.ssh_public_key]
      username = "ubuntu"
    }
  }
}

This gives you the same declarative, reproducible infrastructure workflow you would have in AWS or Azure. The provider is mature and actively maintained.

Running Kubernetes on Proxmox

Proxmox is an excellent platform for self-managed Kubernetes clusters. We typically deploy using one of these approaches:

  1. Kubeadm on Terraform-provisioned VMs — Maximum control, good for teams who want to understand every component
  2. Talos Linux — Immutable, API-driven Kubernetes OS. Excellent security posture, no SSH access to nodes
  3. K3s — Lightweight Kubernetes, good for smaller clusters or edge deployments

For production, we lean toward Talos Linux. Its immutable design eliminates configuration drift and reduces the attack surface — concepts we explore further in our Kubernetes security hardening post.

The storage integration uses the Ceph CSI driver, giving Kubernetes pods access to Ceph RBD volumes:

apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
  name: ceph-rbd
provisioner: rbd.csi.ceph.com
parameters:
  clusterID: proxmox-ceph
  pool: kubernetes-storage
  imageFeatures: layering
  csi.storage.k8s.io/provisioner-secret-name: csi-rbd-secret
  csi.storage.k8s.io/provisioner-secret-namespace: ceph-system
  csi.storage.k8s.io/node-stage-secret-name: csi-rbd-secret
  csi.storage.k8s.io/node-stage-secret-namespace: ceph-system
reclaimPolicy: Delete
allowVolumeExpansion: true

Migration from VMware

If you are migrating from VMware, the process is straightforward but requires planning:

  1. Export VMs from VMware as OVA/OVF files
  2. Convert disks from VMDK to QCOW2: qemu-img convert -f vmdk -O qcow2 disk.vmdk disk.qcow2
  3. Import into Proxmox using qm importdisk
  4. Adjust drivers — Replace VMware Tools with QEMU Guest Agent, switch virtual NIC to VirtIO for performance

Plan for a parallel-run period. Run both platforms simultaneously while validating that workloads perform correctly on Proxmox before decommissioning VMware.

When Proxmox Is Not the Right Choice

Proxmox is not a universal answer. Consider staying with VMware or moving to cloud if:

  • You rely heavily on VMware-specific features like vMotion across datacenters (Proxmox live migration works within a cluster but not across sites without additional tooling)
  • Your compliance framework specifically mandates VMware certification
  • Your team has zero Linux administration experience and no capacity to learn
  • You need vendor support SLAs beyond what Proxmox’s enterprise subscription offers

For most DevOps-capable teams, though, Proxmox delivers the functionality you need at a price that frees budget for what actually matters — better hardware, more engineering time, or expanding into a hybrid cloud architecture.

Conclusion

Building a private cloud on Proxmox is not a compromise. It is a deliberate choice to invest in open-source infrastructure that your team fully controls, at a cost that makes financial sense. The tooling ecosystem — Terraform, Packer, Ceph, Kubernetes — is mature enough for production workloads.

At robto, we have deployed and operated Proxmox clusters for clients ranging from small startups to mid-size enterprises. If you are evaluating your options post-VMware, we are happy to run a technical assessment and help you plan the transition.