QEMU#

Use declarative configuration to configure and run virtual machines on a server via QEMU

Running#

Vedi anche

A collection of scripts I have written and/or adapted that I currently use on my systems as automated tasks 1

Basic steps#

install these packages

apt-get install qemu-system-x86 openssh-client python3-yaml

install fpyutils. See reference

create a new user

useradd --system -s /bin/bash -U qvm
passwd qvm
usermod -aG jobs qvm

create the jobs directories. See reference

mkdir -p /home/jobs/{scripts,services}/by-user/qvm

create the qvm script

/home/jobs/scripts/by-user/qvm/qvm.py#

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# qvm.py
#
# Copyright (C) 2021,2023 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
#
# Original license header:
#
# qvm - Trivial management of 64 bit virtual machines with qemu.
#
# Written in 2016 by Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the public
# domain worldwide. This software is distributed without any warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.
r"""Run virtual machines."""

import shlex
import sys

import fpyutils
import yaml


def build_remote_command(prf: dict) -> str:
    if prf['system']['display']['enabled']:
        # See https://unix.stackexchange.com/a/83812
        # See also the 'TCP FORWARDING' section in man 1 ssh.
        ssh = '-f -p ' + prf['system']['network']['ports']['host'][
            'ssh'] + ' -L ' + prf['system']['network']['ports']['local'][
                'vnc'] + ':127.0.0.1:' + prf['system']['network']['ports'][
                    'host']['vnc'] + ' -l ' + prf['system']['users'][
                        'host'] + ' ' + prf['system']['network']['addresses'][
                            'host']
        ssh += ' sleep 10; vncviewer 127.0.0.1::' + prf['system']['network'][
            'ports']['local']['vnc']
    else:
        ssh = '-p ' + prf['system']['network']['ports']['guest'][
            'ssh'] + ' -l ' + prf['system']['users']['guest'] + ' ' + prf[
                'system']['network']['addresses']['host']

    return (prf['executables']['ssh'] + ' ' + ssh)


def build_local_command(prf: dict) -> str:
    head = str()

    # Generic options.
    generic_options: str = '-enable-kvm'

    # Memory.
    memory = '-m ' + prf['system']['memory']

    # CPU.
    cpu = '-smp ' + prf['system']['cpu']['cores'] + ' -cpu ' + prf['system'][
        'cpu']['type']

    # Display.
    if prf['system']['display']['enabled']:
        if prf['system']['display']['vnc']['enabled']:
            display_number = int(
                prf['system']['display']['vnc']['port']) - 5900
            display = '-display none -monitor pty -vnc 127.0.0.1:' + str(
                display_number)
        else:
            display = '-display gtk'
    else:
        display = '-display none'

    # Audio.
    if prf['system']['audio']['enabled']:
        audio = '-device ' + prf['system']['audio']['device']
        head += 'export QEMU_AUDIO_DRV=alsa;'
    else:
        audio = str()

    # Network.
    user_net: str = str()
    for i in prf['system']['network']['ports']:
        if i['type'] == 'user':
            # User Networking (SLIRP).
            if i['enabled']:
                user_net += ',hostfwd=tcp::' + i['host'] + '-:' + i['guest']
        else:
            # TAP, VDE, socket.
            print('error: not implemented. setting empty network')

    user_net = '-netdev user,id=user.0' + user_net + ' -device virtio-net-pci,netdev=user.0'

    # nw = user_nw + ' ' + tap_nw + ' ' + vde_nw + ' ' + socker_nw
    net: str = user_net

    # CD-ROM.
    if prf['system']['cdrom']['enabled']:
        cdrom = '-cdrom ' + prf['system']['cdrom']['device'] + ' -boot order=d'
    else:
        cdrom = str()

    # Mounts.
    mnt = str()
    for i in prf['system']['mount']:
        if i['enabled']:
            if i['type'] == 'virtfs':
                mnt += ' -virtfs local,path=' + i[
                    'path'] + ',security_model=passthrough,mount_tag=' + i[
                        'ount_tag']
            elif i['type'] == 'virtiofs':
                mnt += (
                    '-object memory-backend-file,id=mem,size=' +
                    prf['system']['memory'] + ',mem-path=/dev/shm,share=on' +
                    ' -numa node,memdev=mem' +
                    ' -chardev socket,id=char0,path=' + i['path'] +
                    ' -device vhost-user-fs-pci,queue-size=1024,chardev=char0,tag='
                    + i['mount_tag'])
            else:
                print('error: not implemented. cannot set mount')

    # Mass memory.
    hdd = str()
    for drive in prf['system']['drives']:
        hdd += ' -drive file=' + drive

    return (head + ' ' + prf['executables']['qemu'] + ' ' +
            prf['options']['start'] + ' ' + generic_options + ' ' + memory +
            ' ' + cpu + ' ' + display + ' ' + net + ' ' + cdrom + ' ' + audio +
            ' ' + mnt + ' ' + prf['options']['end'] + ' ' + hdd)


if __name__ == '__main__':
    configuration_file = shlex.quote(sys.argv[1])
    config = yaml.load(open(configuration_file, 'r'), Loader=yaml.SafeLoader)
    type = shlex.quote(sys.argv[2])
    profile = shlex.quote(sys.argv[3])

    prf = config[type][profile]
    if prf['enabled']:
        if type == 'local':
            command = build_local_command(prf)
        elif type == 'remote':
            command = build_remote_command(prf)

    fpyutils.shell.execute_command_live_output(command, dry_run=False)

create the configuration for qvm

/home/jobs/scripts/by-user/qvm/qvm.yaml#

#
# qvm.yaml
#
# Copyright (C) 2021-2023 Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
#
# This program is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program.  If not, see <http://www.gnu.org/licenses/>.
#
#
# Original license header:
#
# qvm - Trivial management of 64 bit virtual machines with qemu.
#
# Written in 2016 by Franco Masotti (franco \D\o\T masotti {-A-T-} tutanota \D\o\T com)
#
# To the extent possible under law, the author(s) have dedicated all
# copyright and related and neighboring rights to this software to the public
# domain worldwide. This software is distributed without any warranty.
#
# You should have received a copy of the CC0 Public Domain Dedication along
# with this software. If not, see
# <http://creativecommons.org/publicdomain/zero/1.0/>.

local:
    test:
        enabled: true

        executables:
            qemu: '/usr/bin/qemu-system-x86_64'

        options:
            start: ''
            end: ''

        system:
            memory: '12G'
            cpu:
                cores: '6'

                # See
                # $ qemu -cpu help
                type: 'host'

            # Mass memory. Use device name with options.
            drives:
                # IMPORTANT: enable the following during the setup.
                # - '/home/user/qvm/development.qcow2'

                # Enable the following after the setup.
                - '/home/user/qvm/development.qcow2.mod'

               # Mass memory. Use device name with options.
               # - '/dev/sdx,format=raw'
               # - '/dev/sdy,format=raw'

            # Enable this for maintenance or installation.
            cdrom:
                # IMPORTANT: set the following to true during the setup.
                # enabled: true

                # Set the following to true after the setup.
                enabled: false

                # Device or file name.
                device: debian.iso

            # Shared data using virtfs or virtiofs.
            mount:
                # Enable this if you need to mount a directory using 9p.
                - name:         'shared'
                  enabled:      false
                  type:         'virtfs'
                  path:         '/home/qvm/shares/test'
                  mount_tag:    'shared'
                # This type of mounting requires more configuration
                - name:         'shared_performance'
                  enabled:      false
                  type:         'virtiofs'
                  path:         '/var/run/qemu-vm-001.sock'
                  mount_tag:    'shared'

            # If enabled is false never show a display.
            display:
                enabled: true
                vnc:
                    enabled: true
                    port: 5900

            audio:
                enabled: true
                device: 'AC97'

            network:
                # TCP ports only.
                ports:
                    - name:      'ssh'
                      type:      'user'
                      enabled:    true
                      host:      '2222'
                      guest:     '22'
                    - name:      'other_0'
                      type:      'user'
                      enabled:    true
                      host:      '5555'
                      guest:     '3050'
                    - name:      'other_1'
                      type:      'user'
                      enabled:    true
                      host:      '5556'
                      guest:     '3051'

create the Systemd service unit file for qvm

/home/jobs/services/by-user/qvm/qvm.local_test.service#

[Unit]
Description=Run qvm local test
Requires=network-online.target
# Requires=qvm-virtiofs.local_test-qemu-vm-001.service
After=network-online.target
# After=qvm-virtiofs.local_test-qemu-vm-001.service

[Service]
Type=simple
# ExecStartPre=/usr/bin/sleep 4
ExecStart=/home/jobs/scripts/by-user/qvm/qvm.py /home/jobs/scripts/by-user/qvm/qvm.yaml local test
ExecStop=/usr/bin/bash -c '/usr/bin/ssh -p 2222 powermanager@127.0.0.1 sudo poweroff; sleep 30'
User=qvm
Group=qvm

[Install]
WantedBy=multi-user.target

fix the permissions

chmod 700 -R /home/jobs/{scripts,services}/by-user/qvm
chown -R qvm:qvm /home/jobs/{scripts,services}/by-user/qvm

create a new virtual hard disk

sudo -i -u qvm
qemu-img create -f qcow2 development.qcow2 64G

modify the configuration to point to development.qcow2, the virtual hdd. See the commented local.test.system.drives[0] key

Also set the cdrom device and enabled values in local.test.cdrom
run the installation
```
./qvm ./qvm.py local development
```
once the installation is finished power down the machine

create a backup virtual hard disk

qemu-img create -f qcow2 -b development.qcow2 development.qcow2.mod

set local.test.system.cdrom.enabled to false. Set local.test.system.drives[0] to development.qcow2.mod
run the virual machine
```
./qvm ./qvm.py local development
```
quit the virtual machine and go back to the root user
```
exit
```
run the deploy script
if you are using iptables rules on the host machine modify the rules to let data through the shared ports
continue with the client configuration

Automatic shutdown#

Vedi anche

SSH config host match port 2

To be able to shutdown automatically when the Systemd service is stopped follow these instructions

connect to the guest machine

create a new user

sudo -i
useradd -m -s /bin/bash -U powermanager
passwd powermanager

change the sudoers file

visudo

Add this line

powermanager ALL=(ALL) NOPASSWD:/sbin/poweroff

go back on the host machine and create an SSH key so that the qvm host user can connect to the powermanager guest user. Do not encrypt the key with a passphrase
```
sudo -i -u qvm
ssh-keygen -t rsa -b 16384 -C "qvm@host-2022-01-01"
```
Save the keys as ~/.ssh/powermanager_test.

Have a look at the ExecStop command in the Systemd service unit file

in the host machine, configure the SSH config file like this

/home/qvm/.ssh/config#

Match host 127.0.0.1 user powermanager exec "test %p = 2222"
    IdentitiesOnly yes
    IdentityFile ~/.ssh/powermanager_test

copy the content of /home/qvm/.ssh/powermanager_test.pub into /home/powermanager/.ssh/authorized_keys of the guest machine

Using physical partitions#

Instead of using QCOW2 disk files you can use existing phisical partitions and filesystems.

Avvertimento

Remember NOT to mount the partitions while running because data loss occurs in that case.

uncomment and edit the highlighted lines and comment the original drive line

/home/jobs/scripts/by-user/qvm/qvm.yaml#

            # Mass memory. Use device name with options.
            drives:
                # IMPORTANT: enable the following during the setup.
                # - '/home/user/qvm/development.qcow2'

                # Enable the following after the setup.
                - '/home/user/qvm/development.qcow2.mod'

               # Mass memory. Use device name with options.
               # - '/dev/sdx,format=raw'

run the deploy script

Resize disks#

Vedi anche

How to Expand QCOW2 3
Provider::VMonG5K — EnOSlib 8.0.0a12 8.0.0a12 documentation 4

stop the virtual machine
install this package
```
apt-get install libguestfs-tools
```
Have a look at this bug report if you have problem installing
resize for example by increasing to 40G more, where virtual_hard_disk is a backing file. I always set the backing file ending in .mod.
```
qemu-img resize ${virtual_hard_disk} +40G
```

make a backup

cp -aR image.qcow2.mod image.qcow2.mod.bak

get the partition name you want to expand. Usually partition names start from /dev/sda1 (note: these partition names are not the same as the host system ones!)
```
virt-filesystems --long --human-readable --all --add ${virtual_hard_disk}
```
execute the actual resize operation on the virtual partition and filesystem. You can use sda1 for example as partition_name
```
virt-resize --expand ${patition_name} ${virtual_hard_disk}.bak ${virtual_hard_disk}
```
start the virtual machine
if everything works remove the ${virtual_hard_disk}.bak file

Rename disk files#

Vedi anche

Qemu-img Cheatsheet | Programster’s Blog 5

Rename backing file#

TODO

FSCK#

You must run FSCK while the virtual machine is off if you want to fix the root partition.

Vedi anche

kvm virtualization - How to run fsck on guest VMs from KVM - Server Fault 6

identify the troubling filesystem by running the virtual machine in «display» mode, not via SSH: if the broken partition is root your virtual machine might not get to load SSHD.
stop the virtual machine
load the virtual hard disk
```
guestfish -a ${virtual_hard_disk}
run
list-filesystems
```
You get a list of partitions with the last command, for example /dev/sda1.

You can run various fsck commands such as

e2fsck-f ${partition_name}
e2fsck ${partition_name} /dev/sda1 forceall:true
e2fsck ${partition_name} correct:true
fsck ${partition_name}

Quit the program and start the virtual machine
```
exit
```

Footnotes

1: https://software.franco.net.eu.org/frnmst/automated-tasks GNU GPLv3+, copyright (c) 2019-2022, Franco Masotti
2: https://superuser.com/a/870918 CC BY-SA 3.0, (c) 2015, Kenster (at superuser.com)
5: https://blog.programster.org/qemu-img-cheatsheet unknown license,
6: https://serverfault.com/a/380210 CC BY-SA 3.0, Copyright (c) 2012 mrc (at serverfault.com)
7: https://virtio-fs.gitlab.io/howto-qemu.html, Copyright (c) virtio-fs
9: https://wiki.archlinux.org/title/QEMU#Using_filesystem_passthrough_and_VirtFS GNU Free Documentation License 1.3 or later, Copyright (c) ArchWiki contributors

ftutorials documentazione

QEMU

Contents

QEMU#

Running#

Basic steps#

Automatic shutdown#

Using physical partitions#

Resize disks#

Rename disk files#

Rename backing file#

FSCK#

ftutorials documentazione

QEMU

Contents

QEMU#

Running#

Basic steps#

Automatic shutdown#

Using physical partitions#

Share a directory#

9p filesystem#

virtiofs#

Resize disks#

Rename disk files#

Rename backing file#

FSCK#