You are not logged in.

#1 2020-03-26 11:14:56

Muata
Member
Registered: 2020-03-26
Posts: 7

[SOLVED] QEMU CPU Isolating hooks script doesn't work

Hey, I tried to use this: https://wiki.archlinux.org/index.php/PC … ating_CPUs
I've created the hooks folders here:
#   - /etc/libvirt/hooks/qemu.d/win10/prepare/begin/cset.sh
#   - /etc/libvirt/hooks/qemu.d/win10/release/end/cset.sh
And nothing happens, I tried to find some logs, but there is nothing about hooks.
Then I read the libvirt https://www.libvirt.org/hooks.html so I did the same but in /etc/libvirt/hooks/qemu/cset.sh and again nothing.

Maybe someone knows how to run hooks under libvirt?

I tried with these locations:

/etc/libvirt/hooks
/etc/libvirt/hooks/cset.sh
/etc/libvirt/hooks/qemu
/etc/libvirt/hooks/qemu.d
/etc/libvirt/hooks/qemu/cset.sh
/etc/libvirt/hooks/qemu/win10
/etc/libvirt/hooks/qemu/win10/prepare
/etc/libvirt/hooks/qemu/win10/release
/etc/libvirt/hooks/qemu/win10/prepare/begin
/etc/libvirt/hooks/qemu/win10/prepare/begin/cset.sh
/etc/libvirt/hooks/qemu/win10/release/end
/etc/libvirt/hooks/qemu/win10/release/end/cset.sh
/etc/libvirt/hooks/qemu.d/win10
/etc/libvirt/hooks/qemu.d/win10/prepare
/etc/libvirt/hooks/qemu.d/win10/release
/etc/libvirt/hooks/qemu.d/win10/prepare/begin
/etc/libvirt/hooks/qemu.d/win10/prepare/begin/cset.sh
/etc/libvirt/hooks/qemu.d/win10/release/end
/etc/libvirt/hooks/qemu.d/win10/release/end/cset.sh

The script which I use:

#!/usr/bin/env bash

#
# Original author: Rokas Kupstys <rokups@zoho.com>
# Heavily modified by: Danny Lin <danny@kdrag0n.dev>
#
# This hook uses the `cset` tool to dynamically isolate and unisolate CPUs using
# the kernel's cgroup cpusets feature. While it's not as effective as
# full kernel-level scheduler and timekeeping isolation, it still does wonders
# for VM latency as compared to not isolating CPUs at all. Note that vCPU thread
# affinity is a must for this to work properly.
#
# Original source: https://rokups.github.io/#!pages/gaming-vm-performance.md
#
# Target file locations:
#   - $SYSCONFDIR/hooks/qemu.d/vm_name/prepare/begin/cset.sh
#   - $SYSCONFDIR/hooks/qemu.d/vm_name/release/end/cset.sh
# $SYSCONFDIR is usually /etc/libvirt.
#

TOTAL_CORES='0-7'
TOTAL_CORES_MASK=FF             # 0-7, bitmask 0b11111111
HOST_CORES='0,4'                # Cores reserved for host
HOST_CORES_MASK=11              # 0,4, bitmask 0b00010001
VIRT_CORES='1-3,5-7'            # Cores reserved for virtual machine(s)

VM_NAME="$1"
VM_ACTION="$2/$3"

function shield_vm() {
    cset -m set -c $TOTAL_CORES -s machine.slice
    cset -m shield --kthread on --cpu $VIRT_CORES
}

function unshield_vm() {
    cset -m shield --reset
}

# For convenient manual invocation
if [[ "$VM_NAME" == "shield" ]]; then
    shield_vm
    exit
elif [[ "$VM_NAME" == "unshield" ]]; then
    unshield_vm
    exit
fi

if [[ "$VM_ACTION" == "prepare/begin" ]]; then
    echo "libvirt-qemu cset: Reserving CPUs $VIRT_CORES for VM $VM_NAME" > /dev/kmsg 2>&1
    shield_vm > /dev/kmsg 2>&1

    # the kernel's dirty page writeback mechanism uses kthread workers. They introduce
    # massive arbitrary latencies when doing disk writes on the host and aren't
    # migrated by cset. Restrict the workqueue to use only cpu 0.
    echo $HOST_CORES_MASK > /sys/bus/workqueue/devices/writeback/cpumask
    echo 0 > /sys/bus/workqueue/devices/writeback/numa

    echo "libvirt-qemu cset: Successfully reserved CPUs $VIRT_CORES" > /dev/kmsg 2>&1
elif [[ "$VM_ACTION" == "release/end" ]]; then
    echo "libvirt-qemu cset: Releasing CPUs $VIRT_CORES from VM $VM_NAME" > /dev/kmsg 2>&1
    unshield_vm > /dev/kmsg 2>&1

    # Revert changes made to the writeback workqueue
    echo $TOTAL_CORES_MASK > /sys/bus/workqueue/devices/writeback/cpumask
    echo 1 > /sys/bus/workqueue/devices/writeback/numa

    echo "libvirt-qemu cset: Successfully released CPUs $VIRT_CORES" > /dev/kmsg 2>&1
fi
 

Last edited by Muata (2020-03-26 13:47:44)

Offline

#2 2020-03-26 11:46:25

Lone_Wolf
Member
From: Netherlands, Europe
Registered: 2005-10-04
Posts: 11,868

Re: [SOLVED] QEMU CPU Isolating hooks script doesn't work

You did build & install cpuset or cpuset-git aur package ?


Disliking systemd intensely, but not satisfied with alternatives so focusing on taming systemd.


(A works at time B)  && (time C > time B ) ≠  (A works at time C)

Offline

#3 2020-03-26 11:59:54

Muata
Member
Registered: 2020-03-26
Posts: 7

Re: [SOLVED] QEMU CPU Isolating hooks script doesn't work

Yes, cpuset-git

 cset: Cpuset (cset) 1.6 

Last edited by Muata (2020-03-26 12:00:07)

Offline

#4 2020-03-26 12:07:39

Muata
Member
Registered: 2020-03-26
Posts: 7

Re: [SOLVED] QEMU CPU Isolating hooks script doesn't work

Ok, I've fixed by myself, maybe someone will have the same problem so:

I've changed the name of the script  from cset.sh to 'qemu'(before qemu was a folder) on the /etc/libvirt/hooks and now is working.

Last edited by Muata (2020-03-27 16:32:54)

Offline

#5 2020-04-07 05:47:57

endotronic
Member
Registered: 2019-05-22
Posts: 7

Re: [SOLVED] QEMU CPU Isolating hooks script doesn't work

I've been trying to do the exact same thing as you for the past couple of days. How did you get cset working? I've installed the same package as you, but "cset shield" fails for me always. I've seen various patches and tried them all with no luck.

I'm also curious why you decided to omit the hugepages setup from the original script.

EDIT: I found a solution, which I posted here.

Last edited by endotronic (2020-04-14 18:16:58)

Offline

Board footer

Powered by FluxBB