Creating custom container disk images for kubevirt VMs#

You may need to create a custom container disk image for kubevirt VMs if the distro you want to run, isn’t available on https://quay.io/organization/containerdisks. To do that, you need:

  1. A docker registry. (You can use DockerHub, Github, or you can host one yourself on Gitea).
  2. A .qcow2 image of the distro you want to run.

Installation (ArchLinux)#

  1. Grab the latest .qcow2 archlinux file from https://fastly.mirror.pkgbuild.com/images/latest/Arch-Linux-x86_64-basic.qcow2

  2. On paper the steps are pretty simple. Create the following Dockerfile

    FROM scratch
ADD --chown=107:107 Arch-Linux-x86_64-basic.qcow2 /disk/

  3. Build and Push:

    docker buildx build -t registry/username/arch-containerdisk:latest .
    docker push registry/username/arch-containerdisk:latest

  4. But, when you pull and try to use this image, CDI will try to import a 40Gi filesystem into your PVC. Which means, you’ll need a 40Gi+ PVC. This is because, the .qcow2 image arch provides, has a 40Gi filesystem. You can shrink it using these steps:

  5. Install the libguestfs-tools package on your local system. The package is called:

  • Fedora- guestfs-tools
  • Debian- libguestfs-tools
  • ArchLinux- libguestfs

  1. Resize the image to ~ 8Gi:

    guestfish -a Arch-Linux-x86_64-basic.qcow2 <<'EOF'
run
mount /dev/sda3 /
btrfs-filesystem-resize / size:7000M
umount /
EOF

  2. Create a blank image with the desired size:

    qemu-img create -f qcow2 arch-8g.qcow2 8G

  3. Copy the contents of the modified image to the new image:

    virt-resize --shrink /dev/sda3 Arch-Linux-x86_64-basic.qcow2 arch-8g.qcow2

  4. Verify the new image:

    virt-filesystems --long --parts --blkdevs -h -a arch-8g.qcow2
guestfish --ro -a arch-8g.qcow2 -i <<< 'll /; df-h'

  5. Now you can just point to this new arch-8g.qcow2 image in your Dockerfile:

    FROM scratch
ADD --chown=107:107 arch-8g.qcow2 /disk/

  6. Build and push

  7. Refer to this in your new vm like so:

    spec:
  dataVolumeTemplates:
    - metadata:
        name: arch-vm-rootdisk
      spec:
        storage:
          storageClassName: longhorn
          accessModes:
            - ReadWriteOnce
          resources:
            requests:
              storage: 8Gi
        source:
          registry:
            url: "docker://registry/username/arch-containerdisk:latest"     # HERE