Jump to content

Gamberz

Members
 View Profile  See their activity

  • Posts

    9

  • Joined

  • Last visited

 Content Type 

Posts posted by Gamberz

    [Support] Flame

    in Docker Containers

    Posted

    18 minutes ago, paululibro said:


    Hi. You can try one of these three solutions : Corrupted Database File. I think you will have to open db file with sqlite browser and drop additional tables. 

    Thanks, got it working. Was able to open the corrupted one in an sqlite browser alright. Backed up the corrupted file, reloaded the plugin and then inserted the old data into the new db.

    [Support] Flame

    in Docker Containers

    Posted

    Been using this fine for a few days, but did a few restarts today and now I'm getting 

    [2021-09-01 08:48:08.411 UTC-7] [INFO] Connected to database
[2021-09-01 08:48:08.412 UTC-7] [INFO] Starting model synchronization
[2021-09-01 08:48:09.261 UTC-7] [ERROR] Unable to connect to the database: Validation error

    I presume something has been corrupted. I don't have any backups, is there any steps to try and recover it, or is something else the issue?

    VM start/stop events (SOLVED)

    in Programming

    Posted · Edited by Gamberz

    20 hours ago, Squid said:

    What you'd need to do is have 2 separate XML's for the same container being installed.  One with the GPU changes, and one without.

     

    Then when you need to stop one and start the other you would do a simple docker stop nameOfContainer and then start the other one via https://forums.unraid.net/topic/40016-start-docker-template-via-command-line/?tab=comments#comment-1022006

     

     

    Thanks for the suggestion. I ended up modifying the config.v2.json for the container and was able to get it working, this is a bit more dynamic which is good.

     

    Here's the source incase anybody else finds it useful. I'd like to make it work with multiple VMs at once, but this is fine for the moment. 

    #!/bin/bash

CONTAINER_NAME="aio-miner"
CONTAINER_ID=$(docker inspect --format="{{.Id}}" $CONTAINER_NAME)
INPUT_VM=$1
INPUT_EVENT=$2

update_container() {
    local devices="$(echo "$1" | cut -d, -f"2" | xargs | sed -e 's/ /,/g')"
    local devices_escaped="$(echo $devices | sed 's/\-/\\-/g')"

    echo "cd /Container/Config[@Name='NVIDIA_VISIBLE_DEVICES']
    set $devices
    cd /Container/Environment/Variable[2]/Value
    set $devices
    save" | xmllint --shell /boot/config/plugins/dockerMan/templates-user/my-$CONTAINER_NAME.xml

    docker stop $CONTAINER_ID
    sed -i "s/\"NVIDIA_VISIBLE_DEVICES=[A-Za-z0-9,\-]*\"/\"NVIDIA_VISIBLE_DEVICES=$devices_escaped\"/g" /var/lib/docker/containers/$CONTAINER_ID/config.v2.json
    /etc/rc.d/rc.docker restart
}

get_vm_buses() {
    local buses=""

    while read -r bus; do
        bus=${bus#*'"'}; bus=${bus%'"'*}
        buses+="$bus|"
    done <<< $(xmllint --xpath '//domain/devices/hostdev[@type="pci"]/source/address/@bus' --nowarning /etc/libvirt/qemu/$INPUT_VM.xml | sort | uniq)
    
    echo "${buses%?}"
}

if [[ "$INPUT_EVENT" == "prepare" ]]; then
    buses="$(get_vm_buses)"

    if [ -z "$buses" ]; then
        exit 0
    fi

    devices="$(nvidia-smi --query-gpu="pci.bus,uuid" --format=csv,noheader | grep -Ev "$buses")"
    update_container "$devices"
fi

if [[ "$INPUT_EVENT" == "release" ]]; then
    buses="$(get_vm_buses)"
    
    if [ -z "$buses" ]; then
        exit 0
    fi

    devices="$(nvidia-smi --query-gpu="pci.bus,uuid" --format=csv,noheader)"
    update_container "$devices"
fi

(sleep 30s && nvidia-smi -pl 280) &

     

    VM start/stop events (SOLVED)

    in Programming

    Posted · Edited by Gamberz

    3 hours ago, SimonF said:

    I use QEMU hooks file in my USB Manager plugin to connect USB devices at VM startup etc.

     

    https://github.com/SimonFair/USB_Manager

     

    With 6.10 there is an easier way to use hooks files, pre 6.10 you have to modify the existing hooks file, happy to asist and provide advice as required.

     

    You may not need to write a plugin but create hooks files which are persistance in /etc/libvir/hooks/

     

    Here is my code for  installing QEMU updates.

     

      Reveal hidden contents 
    #!/bin/bash

function version { echo "$@" | awk -F. '{ printf("%d%03d%03d%03d\n", $1,$2,$3,$4); }'; }

QEMU=/etc/libvirt/hooks/qemu
QEMUDFILE=/etc/libvirt/hooks/qemu.d/USB_Manager

VERSION=$(sed -n 's/.*version *= *\"\([^ ]*.*\)\"/\1/p' < /etc/unraid-version)



if [ $(version $VERSION) -ge $(version "6.9.9") ]; then
    echo "Version is for qemu.d $VERSION"

# Process OS Version > 6.9.9
# Remove Previous embedded code from old QEMU File

if  ( grep -q "usb_manager/scripts/rc.usb_manager" "${QEMU}" ); then    
START=$(grep -n "begin USB_MANAGER" "${QEMU}" | cut -f1 -d:)
END=$(grep -n "end USB_MANAGER" "${QEMU}" | cut -f1 -d:)
[[ -n ${START} ]] && [[ -n ${END} ]] && sed -i "${START},${END}d" "${QEMU}"
fi

sed -i "s@^[^#]\(.*rc.usb_manager\)@#\1@" "${QEMU}"

# Check qemu.d exists if not create.

[ ! -d "/etc/libvirt/hooks/qemu.d" ] && mkdir /etc/libvirt/hooks/qemu.d

# Create USB_Manager File.

cat << EOF > $QEMUDFILE
#!/usr/bin/env php
<?php
#begin USB_MANAGER
if (\$argv[2] == 'prepare' || \$argv[2] == 'stopped'){
      shell_exec("/usr/local/emhttp/plugins/usb_manager/scripts/rc.usb_manager vm_action '{\$argv[1]}' {\$argv[2]} {\$argv[3]} {\$argv[4]}  >/dev/null 2>&1 & disown") ;
}
#end USB_MANAGER
?>
EOF

chmod +x $QEMUDFILE

# Copy the rules file
cp /usr/local/emhttp/plugins/usb_manager/99_persistent_usb_manager6.10.rules /etc/udev/rules.d/99_persistent_usb_manager.rules
chmod 644 -R /etc/udev/rules.d/99_persistent_usb_manager.rules 2>/dev/null

else

# Process OS Version < 6.9.9

echo "Version is not for qemu.d $VERSION"

if ! ( grep -q "usb_manager/scripts/rc.usb_manager" "${QEMU}" ); then

FINDLINE=\<\?php
NEWLINE=$(cat<<'END_HEREDOC'
#begin USB_MANAGER\nif ($argv[2] == 'prepare' || $argv[2] == 'stopped'){\n  shell_exec("/usr/local/emhttp/plugins/usb_manager/scripts/rc.usb_manager vm_action '{$argv[1]}' {$argv[2]} {$argv[3]} {$argv[4]}  >/dev/null 2>&1 & disown") ;\n}\n#end USB_MANAGER
END_HEREDOC
)
sed -i "/${FINDLINE}/a ${NEWLINE}" "${QEMU}"

fi
# Copy the rules file
cp /usr/local/emhttp/plugins/usb_manager/99_persistent_usb_manager.rules /etc/udev/rules.d/99_persistent_usb_manager.rules
chmod 644 -R /etc/udev/rules.d/99_persistent_usb_manager.rules 2>/dev/null
fi

     

     

    Hey so I got the hook working thanks to your file. I'm stuck on actually restarting the docker container with the new config though, if you have any suggestions.

    I'm updating the template config here 

    /boot/config/plugins/dockerMan/templates-user/my-miner.xml 

    The changes seem to get reflected in the dashboard. If I simply restart the container though, the changes obviously don't take affect. Only if I modify another config option and apply it in the dashboard does it take effect. I could technically stop, delete and start manually like 

    /usr/local/emhttp/plugins/dynamix.docker.manager/scripts/docker run -d --name='miner' --net='bridge' --cpuset-cpus='0' -e TZ="America/Los_Angeles" -e HOST_OS="Unraid" -e 'NVIDIA_DRIVER_CAPABILITIES'='all' -e 'NVIDIA_VISIBLE_DEVICES'='REDACTED' -l net.unraid.docker.managed=dockerman --runtime=nvidia REDACTED_IMAGE REDACTED_ARGS

    But I'd like to avoid that, with all the params I'd need to fetch and the fact it could break at some point. 

    My other idea was to trigger the API call 

    http://192.168.2.156/Docker/UpdateContainer?xmlTemplate=edit:/boot/config/plugins/dockerMan/templates-user/my-miner-gpu.xml

    But that's also not as clean as I would like. Do you know if there's a way to trigger a container to recreate itself?

     

    EDIT: I'm going to try updating 

    /var/lib/docker/containers/containerid/config.v2.json

    and restarting the container. If you've any other suggestions though that'd be great.

    VM start/stop events (SOLVED)

    in Programming

    Posted

    19 minutes ago, SimonF said:

    I use QEMU hooks file in my USB Manager plugin to connect USB devices at VM startup etc.

     

    https://github.com/SimonFair/USB_Manager

     

    With 6.10 there is an easier way to use hooks files, pre 6.10 you have to modify the existing hooks file, happy to asist and provide advice as required.

     

    You may not need to write a plugin but create hooks files which are persistance in /etc/libvir/hooks/

     

    Here is my code for  installing QEMU updates.

     

      Reveal hidden contents 
    #!/bin/bash

function version { echo "$@" | awk -F. '{ printf("%d%03d%03d%03d\n", $1,$2,$3,$4); }'; }

QEMU=/etc/libvirt/hooks/qemu
QEMUDFILE=/etc/libvirt/hooks/qemu.d/USB_Manager

VERSION=$(sed -n 's/.*version *= *\"\([^ ]*.*\)\"/\1/p' < /etc/unraid-version)



if [ $(version $VERSION) -ge $(version "6.9.9") ]; then
    echo "Version is for qemu.d $VERSION"

# Process OS Version > 6.9.9
# Remove Previous embedded code from old QEMU File

if  ( grep -q "usb_manager/scripts/rc.usb_manager" "${QEMU}" ); then    
START=$(grep -n "begin USB_MANAGER" "${QEMU}" | cut -f1 -d:)
END=$(grep -n "end USB_MANAGER" "${QEMU}" | cut -f1 -d:)
[[ -n ${START} ]] && [[ -n ${END} ]] && sed -i "${START},${END}d" "${QEMU}"
fi

sed -i "s@^[^#]\(.*rc.usb_manager\)@#\1@" "${QEMU}"

# Check qemu.d exists if not create.

[ ! -d "/etc/libvirt/hooks/qemu.d" ] && mkdir /etc/libvirt/hooks/qemu.d

# Create USB_Manager File.

cat << EOF > $QEMUDFILE
#!/usr/bin/env php
<?php
#begin USB_MANAGER
if (\$argv[2] == 'prepare' || \$argv[2] == 'stopped'){
      shell_exec("/usr/local/emhttp/plugins/usb_manager/scripts/rc.usb_manager vm_action '{\$argv[1]}' {\$argv[2]} {\$argv[3]} {\$argv[4]}  >/dev/null 2>&1 & disown") ;
}
#end USB_MANAGER
?>
EOF

chmod +x $QEMUDFILE

# Copy the rules file
cp /usr/local/emhttp/plugins/usb_manager/99_persistent_usb_manager6.10.rules /etc/udev/rules.d/99_persistent_usb_manager.rules
chmod 644 -R /etc/udev/rules.d/99_persistent_usb_manager.rules 2>/dev/null

else

# Process OS Version < 6.9.9

echo "Version is not for qemu.d $VERSION"

if ! ( grep -q "usb_manager/scripts/rc.usb_manager" "${QEMU}" ); then

FINDLINE=\<\?php
NEWLINE=$(cat<<'END_HEREDOC'
#begin USB_MANAGER\nif ($argv[2] == 'prepare' || $argv[2] == 'stopped'){\n  shell_exec("/usr/local/emhttp/plugins/usb_manager/scripts/rc.usb_manager vm_action '{$argv[1]}' {$argv[2]} {$argv[3]} {$argv[4]}  >/dev/null 2>&1 & disown") ;\n}\n#end USB_MANAGER
END_HEREDOC
)
sed -i "/${FINDLINE}/a ${NEWLINE}" "${QEMU}"

fi
# Copy the rules file
cp /usr/local/emhttp/plugins/usb_manager/99_persistent_usb_manager.rules /etc/udev/rules.d/99_persistent_usb_manager.rules
chmod 644 -R /etc/udev/rules.d/99_persistent_usb_manager.rules 2>/dev/null
fi

     

     

    That's exactly what I needed, thanks a million! I've upgraded to 6.10 and am testing out the method in your install.sh file. 

    VM start/stop events (SOLVED)

    in Programming

    Posted · Edited by Gamberz
    solved

    I'd like to create a plugin that monitors VM start/stop events and then changes the allocated GPUs on a docker container.

     

    I haven't done any plugins in Unraid yet, but I've been looking through the community apps to see if there's a plugin that might be monitoring VM events. I haven't found any, but if anybody knows of any or how to monitor when a VM is started, please share.

     

    The reason I'm doing this is that I would like to mine crypto in a docker container while no VMs are in use, and manually resizing the container each time I start/stop a VM is quite annoying.

     

    Any pointers or links to plugins that implement some of these features that I can learn off would be really helpful. Thanks.

    [Support] Tailscale Support Thread

    in Docker Containers

    Posted

    Having a lot of issues getting this working if anybody could help. Once enabled (as "host" network), my other containers on "bridge" network don't have internet access. If under "host" they're fine. They don't respond to ping requests to IPs, so not a DNS issue. When I first installed the plugin I had to change my unraid DNS settings because once the tailscale container is running, I can't access local addresses and DNS requests were failing. I finally got that fixed, and I can ping domains from the host now, but other containers can't. I've tried enabling "preserve custom networks" and "Host access to custom networks" but no joy.

×
×
  • Create New...