Open-sourcing the Virtainer guest agent

We are releasing the Virtainer guest agent as open source under Apache-2.0. It is a QEMU Guest Agent (QGA) compatible agent for x86_64 Linux VMs on Cloud Hypervisor: one static binary of about 1 MB with no runtime dependencies, which the host reaches over vsock. It installs itself on systemd and OpenRC guests.

The code is at github.com/virtainer/virtainer-guest-agent, and the first public release, v0.2.0, is on the Releases page.

Why not just qemu-ga

A guest agent is how a host asks a running VM for its addresses, freezes its filesystems for a consistent snapshot, or sets its clock. The usual answer is to install qemu-guest-agent in every guest. On Cloud Hypervisor that turns into a list of chores:

A package per distribution. The package name and its unit files differ, and the install needs internet access at first boot, which a VM without a route out does not have.
A transport override. Cloud Hypervisor’s virtio-console has no named virtio-serial ports, which is what qemu-ga expects, so the agent has to be pointed at vsock instead.
Unit and SELinux work. Overrides for the unit, and policy for the guests that enforce it.

There is one more gap that has nothing to do with packaging. The kernel does not online a hot-added CPU by itself; that is normally a udev rule’s job, and most images do not ship one for virtual machines. Of the images our end-to-end suite covers, only RHEL, AlmaLinux and Rocky 9 and 10, and openSUSE and SLES, ship rules that online a hot-added CPU. Ubuntu’s rule matches only Hyper-V and Xen DMI vendors; Debian, Fedora, Arch and Alpine have none. On those, a live resize without an agent leaves the new vCPUs offline.

What it is

The agent speaks QGA’s wire format and matches qemu-ga’s observable behaviour, so a host-side QGA client does not need to change. It listens on AF_VSOCK port 100 and accepts connections only from the host (CID 2); local and nested-VM callers are dropped. It implements every Linux command of QEMU 11.1’s QGA schema except the three guest-suspend-* commands.

How far “compatible” goes is worth stating precisely. The end-to-end suite diffs ten read-only commands against the qemu-ga that each test image ships, with the known qemu-ga differences normalised. The rest is checked against the QGA documentation, not against a reference run.

There are a few deliberate differences from qemu-ga, as observed with 10.2.2:

It serves up to 16 connections at once. qemu-ga serves one.
guest-set-time works without hwclock, which the Ubuntu 24.04 and Debian 13 cloud images do not ship, and with an explicit time it works without an RTC too.
Files created through guest-file-open get mode 0644, not 0666.
IPv6 route metric and flags come back as the kernel’s unsigned values instead of sign-extended ones.
There is no guest-suspend-*. On Cloud Hypervisor v53 with Fedora 44, qemu-ga 10.2.2 reported success for guest-suspend-ram and never woke, and for guest-suspend-disk it wrote the hibernation image to zram, so the next start was a cold boot. That is one setup, not a claim about every guest or hypervisor version.

What it adds

An interactive Shell. A command named __io.virtainer_shell turns the connection into a terminal: the chosen user’s login shell on a pseudo-terminal, with live resize and up to 8 sessions at once, separate from the 16 QGA connections. The host asks for it like any other command:

{"execute": "__io.virtainer_shell",
 "arguments": {"user": "debian", "rows": 40, "cols": 120, "term": "xterm-256color"}}

On success the connection carries length-prefixed frames instead of QGA messages: raw terminal bytes in one direction, and small JSON control objects such as a resize, or the exit code or signal when the shell ends. The frame layout is documented in the README. It is not a PAM login: no password, no logind session, no utmp entry. Closing the connection hangs the shell up, like closing an SSH session.

Automatic onlining of hot-added CPUs and memory. The agent listens for the kernel’s add uevents and onlines the new CPU or memory block itself. Devices present when it starts are left alone. This part is adapted from Kata Containers.

An auto-thaw watchdog. If no thaw arrives within freeze-timeout (10 minutes by default), the agent thaws the filesystems itself, so a guest never stays frozen forever. A host comparing freeze and thaw counts sees them differ, so an early thaw cannot pass for a quiesced snapshot.

Security posture

The agent runs as root, and its command set is root-equivalent. Whoever can reach the host’s vsock socket for a VM has root in that guest, and the Shell adds nothing beyond what guest-exec already allows. Protect that socket the way you would protect root SSH access.

A guest that must not give the host that reach can say so in /etc/virtainer-guest-agent.conf, with the same semantics as qemu-ga’s block-rpcs and allow-rpcs. Blocking guest-exec blocks the Shell too. The README warns that blocking only some root-equivalent commands is not a boundary, since guest-file-open can rewrite /etc/passwd.

If you deliver the agent with the recommended ISO and cloud-init boothook, that ISO runs as root at every boot, so treat it as trusted code: build it yourself or verify its checksum against one you obtained separately.

To report a vulnerability, please do not open a public issue. Use Security → Report a vulnerability on the repository; the details are in SECURITY.md.

Using it

In Virtainer you do nothing. The agent is placed on each classic VM’s cloud-init seed disk and installed or upgraded from there at every boot, with no download.

On any other Cloud Hypervisor setup, the README’s recommended delivery is to build the ISO with tools/build-iso.sh, attach it read-only to every VM, and add a short #cloud-boothook to the NoCloud vendor data that mounts it and runs virtainer-guest-agent install. The install copies the binary into place, writes a systemd unit or an OpenRC script, and does nothing on a re-run unless the binary or unit changed, so the same call upgrades. Without cloud-init, mount the ISO and run install once by hand.

License and contributions

The code is under the Apache License 2.0. One file, src/autoonline.rs, is adapted from Kata Containers, which is also Apache-2.0, and its notice ships with the binary. No QEMU code is included, and none may be added: the protocol is implemented from the published QGA documentation and from observing qemu-ga.

Issues and pull requests are welcome; CONTRIBUTING.md in the repository covers the build and test commands. The end-to-end suite boots real cloud images under Cloud Hypervisor, and the README lists the images it covers. It does not publish per-distribution results, so if an image misbehaves, an issue with the guest’s distribution and version is the most useful thing to send.