4 minutes ago, sittingmongoose said:

I’m pretty sure you can only give Plex 1 gpu.  I think you can select different ones in emby though but I haven’t tried it yet.  You’re saying you had artifacts too?  Was it one every stream?  I only noticed it when I had multiple hevc streams going.

 

Also, I have a 9900k, a P2000 and I am going to get a 1660ti.  I am going to compare performance between cpu, igpu, p2000 and turing.  Probably going to do this next month.  Currently unraid doesn’t support the 9th gen Igpus, so I need to wait for them to add that.  Which might be a while

I'm pretty sure you are correct. That's mostly why I haven't spent the cycles looking into it. Artifacts are visible on any transcoded stream. I tried multiple formats and multiple devices. I discovered the issue immediately after setting up the GPU. I wanted to see how many streams the card would support before things started to choke. I got up to 7 before I ran out of client devices, and the card had hardly broken a sweat. I used multiple source files including 4K h264 and HEVC, as well as 1080p. On the clients, I used various quality profiles from 1080p down to 480p using both wired and wireless connections. I have more network capacity than most office buildings, so that wasn't the issue. All transcoded streams showed artifacts, even if I was only running a single session.

 

FWIW, I've only heard complaints from 1 or 2 remote users, and their client device (Apple TV) is notoriously bad at format support anyway. I can't verifiably say that the artifacts are happening on all remote transcode streams, but it definitely happened on all local transcode streams.

14 minutes ago, JasonM said:

I got up to 7 before I ran out of client devices, and the card had hardly broken a sweat.

Can't you open up multiple clients in browser tabs and give them various quality settings, or would that not accomplish what you're after?

 

2 minutes ago, Cessquill said:

Can't you open up multiple clients in browser tabs and give them various quality settings, or would that not accomplish what you're after?

I certainly could have on the devices that were computers, but even then, you could only do maybe 2 or 3 before the client device became the bottleneck. I think I was close to the limit. The GPU wasn't maxed out, but the 8GB RAM in the card was just about tapped. I might have been able to get one more.

21 minutes ago, JasonM said:

 

I certainly could have on the devices that were computers, but even then, you could only do maybe 2 or 3 before the client device became the bottleneck. I think I was close to the limit. The GPU wasn't maxed out, but the 8GB RAM in the card was just about tapped. I might have been able to get one more.

I’ll have to test 1080p again and look for artifacts.

5 hours ago, gfjardim said:

The power state should be between P8 and P15. The P0 state is for maximum 3D performance.

https://docs.nvidia.com/gameworks/content/gameworkslibrary/coresdk/nvapi/group__gpupstate.html

Enviado de meu SM-G973F usando o Tapatalk
 

Is there any way to determine why the power state is like this? Why would this be at P0? 

 

I bought this card used. Is it possible to permanently set the power state until manually reset? Maybe the previous owner did that? If so, wondering how I can reset it. I did try some reset commands through nvidia-smi but I got errors along the lines of (paraphrasing) "You can't do this because this is your only video card" 

Edited May 7, 20197 yr by Chad Kunsman

5 hours ago, Chad Kunsman said:

Is there any way to determine why the power state is like this? Why would this be at P0? 

 

I bought this card used. Is it possible to permanently set the power state until manually reset? Maybe the previous owner did that? If so, wondering how I can reset it. I did try some reset commands through nvidia-smi but I got errors along the lines of (paraphrasing) "You can't do this because this is your only video card" 

Just for comparison, I checked nvidia-smi on my 750ti to see if the high idle was a driver issue but mine is currently idling at 1w:

13 hours ago, JasonM said:

 

 

Same here with multiple formats on the P4000. Since it only affects remote users, I've told them to "deal with it" until I can sort it out. My proc supports QuickSync. I haven't had time to determine if I can configure things such that Plex will use that for decode and the GPU for encode.

 

If anyone has done this, I'd be curious as to how. I just haven't had time to mess with it.

Use the older wrapper script.

Without the marap filter, just 

`exec Plex Transcoder2 -hwaccel qsv "$@"`

 

Hwaccel should only affect the decoder.

Encoder is set with the encoder parameter later in the string, so this should work to enable quicksync for decode, and nvenc for encode.

 

Ymmv.

16 hours ago, gfjardim said:

The power state should be between P8 and P15. The P0 state is for maximum 3D performance.

https://docs.nvidia.com/gameworks/content/gameworkslibrary/coresdk/nvapi/group__gpupstate.html

Enviado de meu SM-G973F usando o Tapatalk
 

 

6 hours ago, j0nnymoe said:

Just for comparison, I checked nvidia-smi on my 750ti to see if the high idle was a driver issue but mine is currently idling at 1w:

The Nvidia Linux blobs only support manipulating the power states through the use of powermizer and coolbits flags for X11. Since the driver isn't being used to initialize a display, the card does it's own default power management and software has little control.

You MAY be able to force lower power states by echoing values to sysfs nodes for the card, which can be found in some DRM subfolder of the /sys directory. This probably won't persist, and could lead to instability. 

36 minutes ago, Xaero said:

 

The Nvidia Linux blobs only support manipulating the power states through the use of powermizer and coolbits flags for X11. Since the driver isn't being used to initialize a display, the card does it's own default power management and software has little control.

You MAY be able to force lower power states by echoing values to sysfs nodes for the card, which can be found in some DRM subfolder of the /sys directory. This probably won't persist, and could lead to instability. 

That's pretty disappointing. I'd like to pursue another angle, then. Is it possible the card is not clocking down due to something on the system side keeping it busy/locked? Is there any way to search for things from that angle? Instead of assuming it doesn't know how to idle, let me assume something is keeping it from being idle. But I don't know how to search for that. SMI command maybe?

 

Additionally does this card have some sort of NVRAM on it that would permanently save settings applied to it even across power cycles or being removed from the computer? I'm trying to find out/understand if there is some sort of permanent setting that might have been applied to the card that I could undo if I moved it to a different OS for example. 

Edited May 8, 20197 yr by Chad Kunsman

Hey guys

So I got this up and running finally, even though the P2000 is on an X4 PCI-E 3.0 lane. Should I have any bottlenecks with this in terms of using the p2000 to the fullest being that it's on an x4 lane? I have an m1015 HBA on the X16 lane (my motherboard make the second x16 slots X4). 

 

I installed the CA User Scripts, and copied and pasted the most recent script listed here to patch. When I check on nvidia-smi it spikes my p2000 up to about 56% when transcoding just starts or if I fas forward in the movie, for an x265 1080p file. Is that normal? I thought it's supposed to barely use any of the  GPU for 1 stream?

 

Edited May 8, 20197 yr by maxse

7 hours ago, maxse said:

Hey guys

So I got this up and running finally, even though the P2000 is on an X4 PCI-E 3.0 lane. Should I have any bottlenecks with this in terms of using the p2000 to the fullest being that it's on an x4 lane? I have an m1015 HBA on the X16 lane (my motherboard make the second x16 slots X4). 

 

I installed the CA User Scripts, and copied and pasted the most recent script listed here to patch. When I check on nvidia-smi it spikes my p2000 up to about 56% when transcoding just starts or if I fas forward in the movie, for an x265 1080p file. Is that normal? I thought it's supposed to barely use any of the  GPU for 1 stream?

 

If it has hw twice then it is.  But there is a command like watch nvidia smi.....I forget it though and I can’t remember where I found it lol sorry

right, I know it's working. I just dont get how it spike to 56% when I irst run the stream or fast forward while plex fills it's buffer I guess. If it goes tht high with one stream, how will I be able to use multiple streams? I thought one stream is supposed to barely affect the p2000. Is this normal?

3 hours ago, sittingmongoose said:

If it has hw twice then it is.  But there is a command like watch nvidia smi.....I forget it though and I can’t remember where I found it lol sorry

nvidia-smi dmon

19 hours ago, Chad Kunsman said:

That's pretty disappointing. I'd like to pursue another angle, then. Is it possible the card is not clocking down due to something on the system side keeping it busy/locked? Is there any way to search for things from that angle? Instead of assuming it doesn't know how to idle, let me assume something is keeping it from being idle. But I don't know how to search for that. SMI command maybe?

 

Additionally does this card have some sort of NVRAM on it that would permanently save settings applied to it even across power cycles or being removed from the computer? I'm trying to find out/understand if there is some sort of permanent setting that might have been applied to the card that I could undo if I moved it to a different OS for example. 

If you purchased this second hand, it's possible that someone flashed a second hand BIOS designed for overclocking or cryptocurrency mining. As far as un-doing that, you'd want to find the OEM firmware for that card and flash it. Know that flashing a card ALWAYS poses the risk of bricking the card. It's also possible that the fan motor is using a decent amount of power as it seems to be idling at 40%, where it could turn the fan completely off with a lower temperature. It's also a higher wattage GPU, and I'm not certain what the idle power consumption claims from the manufacturer are, or how accurate those claims are.

9 hours ago, maxse said:

right, I know it's working. I just dont get how it spike to 56% when I irst run the stream or fast forward while plex fills it's buffer I guess. If it goes tht high with one stream, how will I be able to use multiple streams? I thought one stream is supposed to barely affect the p2000. Is this normal?

Where are you looking to see the utilization?
the default nvidia-smi screen shows fan speed, GPU core and GPU error correcting percentages, which aren't applicable to the nvenc/nvdec pipelines.
You'll want to use nvidia-smi dmon to get the columns of percentages, and pay attention in particular to the last two columns. If you watch them for the duration of a short video, you should see how it works. It fills a buffer rapidly with video, and then idles. With multiple streams it will simply use the idle durations from the other streams to buffer a new stream. You'd have to have about a dozen or more streams for them to need to double up the duty cycle, and that's where you MIGHT start seeing decreased performance. Of course, this also means you would need to be able to DECODE fast enough for 12+ streams simultaneously encoding. Which is probably more of a problem.

1 hour ago, Xaero said:

If you purchased this second hand, it's possible that someone flashed a second hand BIOS designed for overclocking or cryptocurrency mining. As far as un-doing that, you'd want to find the OEM firmware for that card and flash it. Know that flashing a card ALWAYS poses the risk of bricking the card. It's also possible that the fan motor is using a decent amount of power as it seems to be idling at 40%, where it could turn the fan completely off with a lower temperature. It's also a higher wattage GPU, and I'm not certain what the idle power consumption claims from the manufacturer are, or how accurate those claims are.

 

By the way, I have half solved this problem after more testing. 

 

The card can and will clock itself down while idle UNTIL it has to perform transcoding work. Once the transcode job is finished, it will never clock itself down again until I restart the Plex docker container. 

 

See the moment I restarted the container here: http://i.imgur.com/zeXKiFZ.png

 

See a thread from another user who seems to have the exact same issue here: https://forums.plex.tv/t/stuck-in-p-state-p0-after-transcode-finished-on-nvidia/387685/2

 

I really hope there's a solution as this is about $40 a year down the drain in extra power usage just due to the card not idling as it should. 

5 hours ago, Xaero said:

Where are you looking to see the utilization?
the default nvidia-smi screen shows fan speed, GPU core and GPU error correcting percentages, which aren't applicable to the nvenc/nvdec pipelines.
You'll want to use nvidia-smi dmon to get the columns of percentages, and pay attention in particular to the last two columns. If you watch them for the duration of a short video, you should see how it works. It fills a buffer rapidly with video, and then idles. With multiple streams it will simply use the idle durations from the other streams to buffer a new stream. You'd have to have about a dozen or more streams for them to need to double up the duty cycle, and that's where you MIGHT start seeing decreased performance. Of course, this also means you would need to be able to DECODE fast enough for 12+ streams simultaneously encoding. Which is probably more of a problem.

I have used watch nvidia-smi and looked under GPU-Util on the right side of the box. Right below that is a percentage. Is that something else? 

18 hours ago, maxse said:

I have used watch nvidia-smi and looked under GPU-Util on the right side of the box. Right below that is a percentage. Is that something else? 

That's the GPU core utilization. It shouldn't really increase with nvenc or nvdec usage. That's the whole point of nvenc and nvdec - giving gamers a way to accelerate video transcoding for streaming. For a streaming site like Netflix, it doesn't really make sense to transcode video data live for each stream, just store the video pre-encoded for transcoding. For a gamer to stream to twitch, they need a way to encode video on the fly without impacting rendering performance, enter nvenc and nvdec. The spike you are seeing is kind of suspicious, but the important metric to look at is the enc and dec columns of
nvidia-smi dmon

On 5/8/2019 at 8:07 PM, Chad Kunsman said:

 

By the way, I have half solved this problem after more testing. 

 

The card can and will clock itself down while idle UNTIL it has to perform transcoding work. Once the transcode job is finished, it will never clock itself down again until I restart the Plex docker container. 

 

See the moment I restarted the container here: http://i.imgur.com/zeXKiFZ.png

 

See a thread from another user who seems to have the exact same issue here: https://forums.plex.tv/t/stuck-in-p-state-p0-after-transcode-finished-on-nvidia/387685/2

 

I really hope there's a solution as this is about $40 a year down the drain in extra power usage just due to the card not idling as it should. 

This information could prove useful for creating an (albeit, hacky) solution - if need be.
I don't currently have any hardware set up where I can test anything (my server is in boxes, currently)

One thing you may try is enabling persistence mode:
nvidia-smi -pm 1

This will result in a couple of watts of idle usage, but will force the drivers to stay loaded, even when no job is running. It's possible the drivers are exiting as soon as the transcode jobs finish, and not changing the power state back to idle.

 

Or, if it's already enabled, you could try disabling it:
nvidia-smi -pm 0



A hacky, scripted solution would be to monitor the nvidia smi output for a condition of both LOW GPU, NVENC and NVDEC performance and a HIGH power state, and issue the nvidia-smi --gpu-reset which would reset the GPU allowing it to idle again. Both of these are hacky workarounds.
I too would echo the Plex team on this in posting on the nvidia developer forums with this information, particularly point out the use of the new nvidia docker blobs, as it could quite possibly be an issue there. Once I have my server up and running again, I'll have a poke at seeing if I can replicate and/or resolve this issue.
https://devtalk.nvidia.com/

Edited May 10, 20197 yr by Xaero

14 hours ago, Xaero said:

This information could prove useful for creating an (albeit, hacky) solution - if need be.
I don't currently have any hardware set up where I can test anything (my server is in boxes, currently)

One thing you may try is enabling persistence mode:
nvidia-smi -pm 1

This will result in a couple of watts of idle usage, but will force the drivers to stay loaded, even when no job is running. It's possible the drivers are exiting as soon as the transcode jobs finish, and not changing the power state back to idle.

 

Or, if it's already enabled, you could try disabling it:
nvidia-smi -pm 0



A hacky, scripted solution would be to monitor the nvidia smi output for a condition of both LOW GPU, NVENC and NVDEC performance and a HIGH power state, and issue the nvidia-smi --gpu-reset which would reset the GPU allowing it to idle again. Both of these are hacky workarounds.
I too would echo the Plex team on this in posting on the nvidia developer forums with this information, particularly point out the use of the new nvidia docker blobs, as it could quite possibly be an issue there. Once I have my server up and running again, I'll have a poke at seeing if I can replicate and/or resolve this issue.
https://devtalk.nvidia.com/

"GPU Reset couldn't run because GPU 00000000:01:00.0 is the primary GPU." is the message I get when I try to reset. 

 

Persistence mode on or off makes no difference. Only resetting the Plex docker container causes it to downclock. 

 

The hacky solution for scripting may come into play if nothing else works. My 'something else' I'm trying is going to be to give a P400 a shot to see if behavior is any different. I know for a fact its general power usage will be less. Maybe it will also not be affected by this bug. If it is, I'll give the script a try and definitely echo these concerns to the Plex team. 

 

Thank you!

 

 

6 hours ago, Chad Kunsman said:

"GPU Reset couldn't run because GPU 00000000:01:00.0 is the primary GPU." is the message I get when I try to reset. 

 

Persistence mode on or off makes no difference. Only resetting the Plex docker container causes it to downclock. 

 

The hacky solution for scripting may come into play if nothing else works. My 'something else' I'm trying is going to be to give a P400 a shot to see if behavior is any different. I know for a fact its general power usage will be less. Maybe it will also not be affected by this bug. If it is, I'll give the script a try and definitely echo these concerns to the Plex team. 

 

Thank you!

 

 

I think the concerns are best directed at the nvidia team, specifically the nvidia docker team. Plex can't really do anything about what the driver or kernel decides to do with the card when it's done using it. The kernel, or driver, should be telling the card to enter a different PState when it's idle and that's not happening.

I recently installed this "plugin". Now 6.7 official is out.  I tried searching it but couldn't find it. How do i upgrade the unraid OS in the future? I assume through the plugin, but 6.7 isn't available.  If that's the case is there a turn around time typically for updates?

1 hour ago, mattcoughlin said:

I recently installed this "plugin". Now 6.7 official is out.  I tried searching it but couldn't find it. How do i upgrade the unraid OS in the future? I assume through the plugin, but 6.7 isn't available.  If that's the case is there a turn around time typically for updates?

We have to sleep.  Turnaround time is when I get to it, and if it builds successfully, which at the moment, it is not.....

 

ETA:  God only knows.....

 

Please remember that this is not official and relies on someone (me) manually running the builds, waiting for it to compile, (Typically around 30 mins using 32GB RAM and 24 cores) and uploading it to the server.  Delays can come from my other commitments, family, job, other life things, and as it states in the original post, we rely on a lot of upstream work, so there are no guarantees that we will release any given version.  Very dependent on things just working as we have no control over the upstream.

1 minute ago, CHBMB said:

God only knows.....

how many times you will have to answer this same question.

Just now, trurl said:

how many times you will have to answer this same question.

I'm not going to answer it again.  If people can't be bothered to read a couple of posts before they post, then I'm not going to be bothered answering....

 

Community can self police this one.

 

That was the first person to ask though, I make some allowances as I honestly believe a lot of people have no real idea how much work goes into stuff like this.