CloudStack VM Deployment - Complete Flow

Understanding the sync and async phases of VM deployment. SYNC PHASE (~250ms) Client blocks here. Must complete before job ID is returned. ┌─────────────────────────────────────────────────────────────────────────┐ │ SYNC PHASE (37.044 - 37.296) ~250ms │ ├─────────────────────────────────────────────────────────────────────────┤ │ │ │ HTTP POST /deployVirtualMachine │ │ │ │ │ ▼ │ │ ┌─────────────────┐ │ │ │ Authentication │ 2FA check, CIDR validation │ │ │ Authorization │ Role check, resource permissions │ │ └────────┬────────┘ │ │ ▼ │ │ ┌─────────────────┐ │ │ │ DB Allocation │ vm_instance (Stopped), nics, volumes (Allocated) │ │ │ │ resource_count incremented │ │ └────────┬────────┘ │ │ ▼ │ │ ┌─────────────────┐ │ │ │ Submit Job │ AsyncJobManager creates job-43 │ │ └────────┬────────┘ │ │ ▼ │ │ Return: { jobid: "5dc96a21-....

November 29, 2025 · 9 min · Daman Arora

CloudStack Hypervisor Network Topology Debugging

This guide covers how to map and debug CloudStack VM and Virtual Router (VR) network topology using tools like brctl, virsh, ip, tcpdump, and iptables — all from the hypervisor (host) level. Mapping Network Topology Tools That Work Together Tool What it Tells You brctl show Shows which interfaces are connected to which Linux bridges virsh domiflist <vm> Maps VMs/VRs to their tap devices and bridges virsh dumpxml <vm> Shows MACs, interface order, bridge assignments (detailed view) Example Use Step 1: See which interfaces are on a bridge sudo brctl show Example:...

October 22, 2025 · 2 min · Daman Arora

CloudStack KVM Host Add Failure - Part 1

Error encountered while attempting to add a KVM host to CloudStack: Failed to setup keystore on the KVM host: 192.168.122.10 Stack trace from management-server.log: 2025-10-03 22:36:14,828 DEBUG [c.c.h.k.d.KvmServerDiscoverer] (qtp1438988851-25:[ctx-99e0fc72, ctx-e969ec93]) (logid:a67f40bb) can't setup agent, due to com.cloud.utils.exception.CloudRuntimeException: Failed to setup keystore on the KVM host: 192.168.122.10 - Failed to setup keystore on the KVM host: 192.168.122.10 com.cloud.utils.exception.CloudRuntimeException: Failed to setup keystore on the KVM host: 192.168.122.10 at com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.setupAgentSecurity(LibvirtServerDiscoverer.java:196) at com.cloud.hypervisor.kvm.discoverer.LibvirtServerDiscoverer.find(LibvirtServerDiscoverer.java:339) at com....

October 22, 2025 · 2 min · Daman Arora

CloudStack KVM Host Add Failure - Part 2

This document captures the logs and analysis of a failure while attempting to add a KVM host to a CloudStack zone, due to issues setting up the agent certificate keystore. Error Summary DiscoveryException: Could not add host at [http://192.168.122.10] ... Failed to setup certificate in the KVM agent's keystore file, please see logs and configure manually! 2025-10-04 23:41:58,838 INFO [c.c.a.ApiServer] (qtp1438988851-22:[ctx-b8b55cff, ctx-2b748765]) (logid:046ec553) Could not add host at [http://192.168.122.10] with zone [Zone {"id": "2", "name": "MyAdvancedZone", "uuid": "30d1810a-9680-49c7-8926-75fcf5e4dfb8"}], pod [HostPod {"id":1,"name":"Pod1","uuid":"996b281e-a200-4a64-9cd2-9f7fc8d0d55a"}] and cluster [Cluster {id: "1", name: "Cluster1", uuid: "c6981176-f03d-45dd-aba8-bbe3ab38e6f4"}] due to: [ can't setup agent, due to com....

October 22, 2025 · 3 min · Daman Arora

CloudStack Logging Guide

1. Where CloudStack Logs Live CloudStack has three main components — each with its own logs: Component Purpose Log Path Management Server UI, API, core business logic /var/log/cloudstack/management/management-server.log KVM Agent (if used) Talks to libvirt/hypervisor /var/log/cloudstack/agent/agent.log Usage Server Tracks usage for billing/stats /var/log/cloudstack/usage/usage.log Worked a lot with: /var/log/cloudstack/management/management-server.log 2. How to Explore Logs Like a Pro Read logs live (tailing) tail -f /var/log/cloudstack/management/management-server.log Use grep to filter: tail -f management-server.log | grep -i error Or generate fresh new logs (rotated):...

October 22, 2025 · 2 min · Daman Arora