I'm curious if anyone else has any numbers on some modern GPUs when used with unraid. I've been testing this out the past couple days with my limited selection of cards. I've found a couple weird things, like my nvidia cards seem to use more power when the VM is shutdown than at the desktop. But my AMD cards seem to be at their lowest when they're unloaded, albeit not by much. Any observations are of interest.

 

I'm testing with a killawatt and using subtraction with different cards swapped in. The idle power is a little noisy on this system but I'm pretty confident in the read +/-1watt

This is with a gold Antec power supply.

 

My GT710 seems to idle at 5watt, although I'm using it as a host card so perhaps it would do better with a driver loaded.

 

My R7 250 DDR3 seems to idle at 6watt

 

My 1060 3GB seems to idle at 14watt which seems high.

 

Has anyone gotten zerocore on AMD to work in a satisfactory way? It didn't seem to make much difference to be versus shutting down the VM. It appears to be broken in newer drivers as well, the fan only shut down when I loaded an old 14.4 driver in Windows 7. There are forum posts indicating it doesn't even work in Windows 10.

Hi scorcho99,

 

i am facing the exact same issue with my build. I am running a nvidia 1080 for my daily driver and gaming windows 10 vm. I have also loaded the "unraid nvidia" plugin, which provides the nvidia driver so the card can be used in docker containers as well. So I can issue the nvidia-smi command on the unraid cli. So there i can see, that the card alone hast an power draw of about 40-45 watt because they sit at Power State 0 (P0), which is the performance powerstate). So the card alone makes 50% of the hole power consumption of the entire system. When i run my windows vm and i issue the same command in the windows cli i can see that the card is in Power State 8 and draws only 12-14 watt, which is not good but okish with two monitors plugged in.

 

Unfurtunaltely i haven't found a way to bring the nvidia card to power state 8 during idling without running a windows vm. So this is really unsatisfing. So any hints / tipps are really desirable.

 

 

On 10/11/2019 at 12:15 PM, ph0b0s101 said:

Unfurtunaltely i haven't found a way to bring the nvidia card to power state 8 during idling without running a windows vm. So this is really unsatisfing. So any hints / tipps are really desirable.

 

 

I've found that forcing persistance mode does the trick. Open a terminal and "nvidia-smi -pm 1". Why? Dunno but it works in my 1060

Hi Octa,

thanks for that tipp. So have you also a Windows VM with GPU passthrough? If yes, did the vm work with this setting active?

Yes it does. But in order to get the power state 8 once you power off the VM you have to execute the command again.

 

As far as i can tell, the bahaviour is like the win VM sets the graphics card in power state 0 when is shutting down (of course if it wasn't a VM this doesn't matter), and then the host leaves it that way. I don't know if there is a way for QEMU to catch a shutting down machine and automatically set the persistance mode, i haven't researched that far.

Edited October 20, 20196 yr by Octa

Hi Octa,

 

so it seems that it helps to save some watts in idle without passthrough the gpu to vm. So I try to find a possibility to script the command after shutting down the vm. When I am succesful with that, I let you know.

Thank you, much appreciated.

 

Btw, worth noting that this also spins down tha fan/s, which was my main concern.

Ah thanks for that peace of information. Didn't noticed it at all, but my system is really quiet

On 10/22/2019 at 9:17 AM, ph0b0s101 said:

Hi Octa,

 

so it seems that it helps to save some watts in idle without passthrough the gpu to vm. So I try to find a possibility to script the command after shutting down the vm. When I am succesful with that, I let you know.

Hey, did you find a possibility to automate to power down the graphics card after shutting down the VM?

 

Greetings,

ViproXX

17 hours ago, ViproXX said:

Hey, did you find a possibility to automate to power down the graphics card after shutting down the VM?

 

Greetings,

ViproXX

Unfortunately not yet. I am a little bit busy at the moment so I haven't found the time to do some research 😞

I used the Plug-In "User Scripts" to put the drives only the VM uses to standby and to issue the nvidia-smi -pm 1 command. This is quite handy and works, although I have to trigger the script manually (in the Unraid-GUI) after shutting down the VM. One could set a schedule with cron for that, thou.

 

Stay healthy!

Hi ViproXX,

could you share your script for issuing the nvidia-smi -pm 1 command, particularly as I have two identical GTX 1660 super videocards used for two separate gaming VMs (for in-home streaming) and a normal VM (but not always running). Does one use its UUID or how does one send the command to the correct GPU?

 

Note: As at least one GPU is used for one gaming VM or the normal VM, I already have scripts (to run in the background) to shutdown one VM, wait for 30 secs and then start another VM. This works perfectly. And then it would be nice to have in the same script add the -pm 1 command.

 

Perfect, "nvidia-smi -pm 1" also helped my Quadro to get higher pstates. My card is only used in connection with docker containers.

So I wrote a script for the "User Script" plugin which in my case is only executed once after starting the server.

See appendix:

Have fun saving energy!

nvidia-smi_persistence_mode.zip

it might be possible to check kvm about a specific vm is running and if vm is not running set lower power state.

user script run on hourly schedule in scheduler.

Dot have the script for it yet, but i was considering something like this.

tmp=$(virsh list --all | grep " vmtest " | awk '{ print $3}')
if ([ "x$tmp" == "x" ] || [ "x$tmp" != "xrunning" ])
then
    echo "VM does not exist or is shut down!"
    nvidia-smi -pm 1
else
    echo "VM is running!"
fi

similar to this....  Example only i google'd 🙂

Edited September 21, 20205 yr by mdsloop

I edited the script provided by @mdsloop, now he working smooth, with cron run every 2 min.

 

VMnametocheck="Windows 10 Gaming"
stateofvm=$(virsh list --all | grep "$VMnametocheck" | awk 'NF{ print $NF }')
if [[ "$stateofvm" == "running"  ||  "$stateofvm" == "idle"  ||  "$stateofvm" == "paused" ]]; then
	echo "The $VMnametocheck VM is in $stateofvm state"
else
	echo "The $VMnametocheck VM is Shutted down! And now Nvidia GPU will go to power save state P8"
	nvidia-smi -pm 1
fi

 

I've tried this, however I always get the following error:

 

NVIDIA-SMI has failed because it couldn't communicate with the NVIDIA driver. Make sure that the latest NVIDIA driver is installed and running.

 

I've bound the graphics/audio part of the card to my VM, but that's required to boot the VM. Any hints to get this working?

Old Topic but maybe someone is interested in my Code:

 

# Get the number of GPUs
num_gpus=$(nvidia-smi --query-gpu=count --format=csv,noheader | awk 'NR==1 {sum+=$1} END {print sum}')

echo "Found $num_gpus GPUs."

# Iterate over the GPUs
for ((i=0; i< $num_gpus; i++)); do
  # Check if there are any processes running on GPU $i
  if nvidia-smi -i $i --query-compute-apps=pid --format=csv,noheader | grep -q '^[0-9]'; then
    echo "There are processes running on GPU $i."
  else
    echo "There are no processes running on GPU $i. Setting GPU $i to power save mode."
    nvidia-smi -i $i -pm 1
  fi
done

 

It first checks how many GPUS are installed and then if there are running processes. If not, Power save mode will be activated.

19 hours ago, MeisterPilaf said:

Old Topic but maybe someone is interested in my Code:

 

# Get the number of GPUs
num_gpus=$(nvidia-smi --query-gpu=count --format=csv,noheader | awk 'NR==1 {sum+=$1} END {print sum}')

echo "Found $num_gpus GPUs."

# Iterate over the GPUs
for ((i=0; i< $num_gpus; i++)); do
  # Check if there are any processes running on GPU $i
  if nvidia-smi -i $i --query-compute-apps=pid --format=csv,noheader | grep -q '^[0-9]'; then
    echo "There are processes running on GPU $i."
  else
    echo "There are no processes running on GPU $i. Setting GPU $i to power save mode."
    nvidia-smi -i $i -pm 1
  fi
done

 

It first checks how many GPUS are installed and then if there are running processes. If not, Power save mode will be activated.

Thank you for sharing your solution. This is only working I use the nvidia gpu driver and did not pass through the gpu to any vm, right?

Thank you for sharing your solution. This is only working I use the nvidia gpu driver and did not pass through the gpu to any vm, right?

Yes. But you can use the GPU for your VM‘s as long no docker container using it.


Gesendet von iPhone mit Tapatalk

On 1/3/2023 at 3:16 PM, MeisterPilaf said:

Yes. But you can use the GPU for your VM‘s as long no docker container using it.


Gesendet von iPhone mit Tapatalk

At the moment, I did not use the gpu in any docker. Only in the VM but did not have installed any nvidia drivers over the application store. Furthermore the gpu is bind via the tools system devices page in unraid. Is this a showstopper?

At the moment, I did not use the gpu in any docker. Only in the VM but did not have installed any nvidia drivers over the application store. Furthermore the gpu is bind via the tools system devices page in unraid. Is this a showstopper?

Yes it is. You have to undo it and then download the nvidia driver from the store. In this case you have full control with unraid and the script above and also for you vm. Just start the vm. Stop the vm and the script is working again. I setup a cronjob and the script is running every 5 minutes.


Gesendet von iPhone mit Tapatalk

At the moment, I did not use the gpu in any docker. Only in the VM but did not have installed any nvidia drivers over the application store. Furthermore the gpu is bind via the tools system devices page in unraid. Is this a showstopper?

Yes it is. You have to undo it and then download the nvidia driver from the store. In this case you have full control with unraid and the script above and also for you vm. Just start the vm. Stop the vm and the script is working again. I setup a cronjob and the script is running every 5 minutes.


Gesendet von iPhone mit Tapatalk

On 1/4/2023 at 3:31 PM, MeisterPilaf said:

Yes it is. You have to undo it and then download the nvidia driver from the store. In this case you have full control with unraid and the script above and also for you vm. Just start the vm. Stop the vm and the script is working again. I setup a cronjob and the script is running every 5 minutes.


Gesendet von iPhone mit Tapatalk

Sorry for my late feedback but it works like a charm. Thank you so much for sharing your solution. Great!

Sorry for my late feedback but it works like a charm. Thank you so much for sharing your solution. Great!

No problem mate. Thank you for the feedback.


Gesendet von iPhone mit Tapatalk