Xaero
-
Posts
413 -
Joined
-
Last visited
-
Days Won
3
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by Xaero
-
-
25 minutes ago, Marshalleq said:
I actually hadn't edited it. I use VI and Nano so I don't think there's much chance of it coming from my end? - are you saying it was published to this site with windows line endings and I need to change it? In which case I probably would need to do that with something (I run Mac) but this would be a first in 20 years!
The forum may be mucking it up for us, let me post a link from a sane paste site.
https://paste.ee/p/wcwWV
Some quick notes for the curious for some decisions:
Each call of echo actually eats CPU and I/O scheduler time, even the ones that only write to the screen. It's better to produce many lines of output with a single echo or cat statement generally than to use many. I left some of these in just to avoid frustration with implementing a single echo that did the same thing.
In some places I've opted to use cat. This happens in two circumstances:
1.) Where there's identations. A CR escaped echo `echo "\` prints identations literally. You could use a heredoc with echo, but it can be unpredictable depending on what implementation of echo is present on the target machine. It's easier, and safer in this circumstance to use `cat`
2.) Where there's bash sentinel characters `*` for similar reasons as above, if you use a cr escaped echo to print a block that contains a bash sentinel, it will expand those sentinels. To work around this, you can temporarily turn off this processing, and then turn it back on with `set -f` and `set +f` respectively. This is 3 separate calls to do what a single cat handles with no problem. cat can be measurably slower in certain circumstances, but its generally negligible, especially since we've reduced the number of calls 3 fold.
You'll also notice I use two different heredoc sentinels with cat, `<< EOF` and `<<-EOF` the former includes whitespace to the left of any text, and requires that terminating EOF sentinel to rest on the first character and be the only character on the entire line at the end of the heredoc. While the latter truncates anything whitespace located to the left of the terminating EOF sentinel, and allows whitespace to preceed the terminating EOF sentinel. In this manner I can retain identation on appropriate text blocks without including it in the resulting output.The other changes are not preferential but agreed upon by most of the industry as "proper syntax" ( `$()` for command substitution in lieu of ``, double quotes around variables to prevent globbing, etc.)
I didn't bother messing with the logic as I wanted the script to retain its original functionality, and just have more longevity. Currently no errors should be output during a FULLAUTO run, from my test earlier today. If you run into an error message ping me here and I'll see what I can do for now. I don't plan on adopting this project, just don't like seeing hard work lost or error messages on my terminal windows.
(I suppose I could just wrap everything with a 2>/dev/null)PS: I've been very ill, so I've made a substantial number of typographical errors in my notes and comments in the script. The script runs okay, as I had to test it - but trust me, it didn't run the first time I tried after making all of the changes. Not even close. Hopefully my brain starts to pick up speed again over the next several days.
-
1
-
-
Attached is a debugged version of this script modified by me.
I've eliminated almost all of the extraneous echo calls.Many of the block outputs have been replaced with heredocs.
All of the references to md_write_limit have either been commented out or removed outright.
All legacy command substitution has been replaced with modern command substitution
The script locates mdcmd on it's own.
-
3
-
-
Good evening!
I'd like to not only request this as a feature, but explain how one could implement this on their own!
Basically, the idea is that all user customization done via configuration files located in "/root" are lost on each boot.
I know this is intentional, but there's an "easy" way to implement this with clever failsafe mechanics.
I also know that one can work around this by adding a couple of lines to /boot/config/go, and storing the configuration files on the flash drive. This isn't as desirable as Fat32 doesn't properly handle Linux permissions, and can require other manual edits to the go file down the road.
Enter OverlayFS (a feature built into the Linux kernel for eons)First we create the container for our data. I use the truncate command as it is safe and "quick" (note: we are writing over USB so this step will take time no matter which option we use)
truncate -s 4000M /boot/config/root.persist
I chose to go with 4000M as it is close to the Fat32 ceiling of "4gb" (note: if you specify 4G you will receive an error)
Next we format that image, and set up some important directories within it:
mkfs.ext4 /boot/config/root.persist mkdir /tmp/overlay mount /boot/config/root.persist /tmp/overlay mkdir /tmp/overlay/upper mkdir /tmp/overlay/workdir
Finally the special sauce that overlays the image we created on top of the normal unraid /root/ directory:
mount -t overlay -o lowerdir=/root,upperdir=/tmp/overlay/upper,workdir=/tmp/overlay/workdir none /root
Anything written to /root/ after this command is run will actually be writting to /tmp/overlay/upperdir, and permanently stored there. The lowerdir will never be modified in this situation as it isn't addressable since we are placing the overlay on top of lowerdir.
And to make it persistent, we add this block to /boot/config/go:
if [ -f /boot/config/root.persist ]; then mkdir /tmp/overlay mount /boot/config/root.persist /tmp/overlay mount -t overlay -o lowerdir=/root,upperdir=/tmp/overlay/upper,workdir=/tmp/overlay/workdir none /root fi
A couple of notes:
The if statement above makes sure that we don't try doing anything if there isn't a persistent image for the root folder. It's kind of redundant (the first and second mount commands will just fail and regurgitate errors if the file isn't there) but I prefer a clean console log.
If the image becomes corrupt, or unusable you can safely discard it this way.Safe mode shouldn't use /boot/config/go so if anything goes wrong safe mode will undo any of the changes contained in the image. Meaning you can boot into safe mode, manually mount the image, and undo whatever you did in upperdir and be back up and running. I'm not sure what you could do to cause those sorts of things.
This also allows for:
Persistent bash history (forget that command you ran before you rebooted? No more.)
Persistent config file storage (tmux preferences, terminal colors, and htop profiles? Oh my.)
Persistent KNOWN_HOSTS and AUTHORIZED_KEYS for ssh.
Anything you would normally want a home directory to be useful for in LinuxLand.
-
3
-
4
-
-
2 hours ago, aptalca said:
I use both (paid versions too). Soon after I started trying emby out, I completely switched over to it for myself. The main reason was that Plex apps kept transcoding all the time (every time I turn on srt subtitles, if there is multi channel audio, Plex transcodes). Emby apps direct play or direct stream on the same hardware with subtitles.
I still have family and friends streaming off of plex. I figured it would be too hard to switch some of them over as they are not very technical. So I still run both, but emby for me, Plex for the legacy folk
I kind of regret paying for a plex pass, honestly. Emby seems like the development team is not only more active, but more responsive.
-
1
-
-
On 7/8/2019 at 5:59 AM, jonathanm said:
How are you handling permissions? FAT32 doesn't support normal linux permissions, and causes issues if not dealt with.
You could use a persistence file, I forgot about this issue. We've dealt with this in USB Linux Land for a long time.
Basically,
dd if=/dev/zero of=/boot/config/persist.img bs=1M count=512
(For a 512mb persist file)
mkfs.ext4 /boot/config/persist.img
And then change the overlay mount to mount that persist image rather than a folder.
Max size in this case would be 4gb due to fat32 ceiling.
EDIT:
This was from memory - there's a few steps involved. I'll document this in a separate thread and then link that thread here, so as to not derail this OP any further. I may also make it a feature suggestion, since it wouldn't be too outlandish to offer this as an option in the WebUI by default, and have safe mode just "NOT" use the home folder overlay. -
3 hours ago, Bob1215 said:
I just downloaded this plugin after learning about it here:
I'd like to be able to preserve/repopulate .bashrc with some aliases I use often. I'm not really sure how to go about doing this. Any pointers?
A little OT,
What I do is use "heredocs" in the go file;
Like so:
cat << 'EOF' > ~/.bashrc
Your
Full
Bashrc
Goes
Here
EOF
You can do this for any files you'd like.
You can also store them in a "overlay" folder, which can be more convenient depending on circumstances (I'm going to switch to this as I have several user customized files now)
You would do this by creating an overlay folder in the same directory as the "go" script on your boot drive.
Put your .bashrc, .bash_profile etc into there.
At the end of your "go" file put:
mount -t overlay overlay -o lowerdir=~/,upperdir=./overlay ~/
This "should" (I have not tested it yet) make the original ~/ created by unraid read only, and mount the "overlay" directory from the flash drive as read write over the top of it. Anything written to the home directory of root ~/, /root/ will be written to the flash drive and persistent across reboots.
This makes for a good failsafe, if something breaks you can simply comment out the mount line in the go file and it will be reverted. I'm kind of surprised a persistent home directory hasn't already been added to unraidnin this manner.
-
1 hour ago, Dazog said:
Card is always at P0
Stream or no stream.I will wait to see if someone creates a docker or script to fake the desktop to trick nvidia's cards into letting us not having to run P0 24/7
Run fuser -vk /dev/nvidia* and see if the p state changes.
It should boot to a p8 state even with no X server, from playing with this. Chances are some process is not ending properly
fuser -v lists open handles on the specified file(s) since devices are treated as files in Linux land, it can be used to see what process has a device open.
You can then either kill listed PIDs individually, or just specify the -k option to have fuser kill the processes on its own.
-
26 minutes ago, Dazog said:
Any idea about us emby users?
I was under the impression that Emby doesn't have this behavior?
To see if it does, and part of "why" run
watch nvidia-smi
Check the power state is not latched to P0.
Start a stream, verify the power state latches to P0
Stop the stream, verify the process has ended, and that the power state is stuck in P0
Run fuser -v /dev/nvidia*
To see what processes are still using the driver.
-
1
-
-
5 hours ago, Max said:
Could anyone guide what's the difference between stock unraid builds and nvidia unraid builds.
Sorry for asking stupid questions😅
The stock builds are released by limetech and are official, but have only the open source graphics drivers, which are missing features from the proprietary driversm
The unraid Nvidia builds have the Nvidia proprietary driver packages added in,.which enable.things like CUDA, nvenc and nvdec.
By using the card inside of Dockers you can take advantage of this for things like hardware accelerated transcoding and GPGPU computing.
-
Edit:
This is not a problem with an easy solution at all.
I can monitor the transcode processes and make sure that everything is killed - but the only solution is to kill Plex:
https://forums.plex.tv/t/stuck-in-p-state-p0-after-transcode-finished-on-nvidia/387685/24
I can user fuser -vk /dev/nvidia* and it will immediately switch to a P8 state. The only process using the card when this is run is "Plex Media Server"
It's not hard to write a script that will only do this if:
There are no processes using the card and the card is in a P0 state. I just don't know if there are any undesirable side-effects of doing it this way.
Here is such a script:#!/bin/bash while true; do cur_pstate=$(nvidia-smi --query-gpu=pstate --format=csv,noheader) running_processes=$(ps --no-headers "$(nvidia-smi |tail -n +16 | head -n -1 | sed 's/\s\s*/ /g' | cut -d' ' -f3)" | wc -l) 2>/dev/null if [[ $cur_pstate = "P0" && $running_processes -eq 0 ]]; then # if we got here, the card is only running the Xorg process and is in the P0 state, let's fix that. fuser -kv /dev/nvidia* echo "Reset Power State" fi #sleep so we aren't blocking a thread constantly. sleep 1 done
Starting the X server on Unraid does allow one to open nvidia settings; to do this you can use a script like this to start the X server (note, that since chvt and fgconsole aren't available, you will have to switch back to VT7 by pressing Ctrl+Alt+F7):
#!/bin/bash ##This will only work on single GPU systems: GPUID=$(nvidia-xconfig --query-gpu-info | grep BusID | sed 's/^[^:]*: //') #Now that we know the PCI BusID of the card we can create the X server with a fake display: nvidia-xconfig -s -a --allow-empty-initial-configuration --use-display-device=None --virtual=640x480 --busid "$GPUID" -o /dev/stdout | X :99 -config /dev/stdin&
Once you have that server running, you can return to the default unraid GUI and run:
nvidia-settings -c :99
To open nvidia-settings on the card. You could also store an xorg configuration file and use that for the virtual X display, and to set persistent nvidia settings.The only way I can think of to fix this properly is to figure out why the Plex process is claiming the card and prevent that from happening. I'll look into it some more, but this needs to be fixed properly by Plex/nVidia. The linked thread at the Plex forums has more information.
I may be able to detach the Plex Transcoder process with the wrapper script, making it it's own entity, and then trapping the SIGINT/SIGKILL in the wrapper and using it to kill the transcoder, effectively using the wrapper script to separate the Plex Media Server process from the Plex Transcoder process. It's pretty kludgy, but might work.
Oh Boy:
We're in idle P-State while transcoding territory!
-
2
-
-
Give me a bit here, there are some problems with this - for one, I think persistencemode may need to be forced on, it seems like the driver is releasing the card when the rendering finishes, which sets it back to a P0 state, restarting the X server isn't enough to get the p-state back. I'll work on it a bit, and make sure it's ready for prime time before I push anything out. It is the first time my card has dropped back from the P0 state since it was installed in the server though, so it's definitely progress in the right direction
-
1
-
-
1 hour ago, pappaq said:
Did you get any further with that? I would love to see the card going back to P8 state after transcoding with plex. I don't really understand what you mean with the Xorg server. I know it's a display server but don't get how this two things are connected to each other. Could you explain that? Thanks.
Basically, the Nvidia Linux drivers are designed such that they aren't fully independent of the display server. Parts of the driver aren't active until a display server like X11 or Wayland hooks the driver resources.
As a result of there not being an active display in the unraid environment, nvidia-settings can't be called to change driver settings. In the example above ":0" is shorthand for: "localhost:0.0" which is the first screen, on the first display server, on the local machine. That display doesn't exist, so the n idia settings application just tells you it can't do what you asked.
Normally, in Linux land, we have "sysfs" nodes for driver and hardware settings. "Sysfs" is the /sys/ folder on Linux systems. All of the driver flags, power states, temperature sensors, etc live in this folder as files. Nvidia, for whatever reason, has avoided embracing both KMS (kernel mode setting, which let's the kernel make decisions about what the display should be doing during boot) and sysfs nodes.
This honestly is something Nvidia should be fixing. We might be able to band-aid it by running a display server in a docker with the Nvidia settings application and using it to manage the power state. It also might fail horribly.
It might be better to create a fake second X server on unraid itself using a userscript as the temporary solution.
EDIT2:
Just tested my above theory with a fake X server on unraid and it works perfectly. Let me write a userscript to create one of these fake environments on the fly.
-
2
-
-
https://sourceforge.net/projects/avf/
I'd like to request AVFS - it allows one to mount various archive and network protocols as virtual filesystems.
There are slackbuilds available for it:
https://slackbuilds.org/repository/14.2/system/avfs/ -
With Nvidia and some extra legwork you can use moonlight. With moonlight you could add individual applications or the entire desktop for a streamed remote session.
-
13 hours ago, IamSpartacus said:
If I'm reading this post correctly, you are running Unraid in some type of production environment? If that's the case, why in the world would you be running a 3rd party non-officially supported OS version? That just can't happen in a production environment IMO.
Just a suggestion - but a couple of conditionals and a mail daemon could automatically detect, and notify of backup failures.
-
8 hours ago, Auxilium said:
Thank you both, that explains now for what area that tool is specifically for.
The reason I was seeking nvidia-settings is to see/change GPU performance settings, I have Quadro P4000 that is consuming 27W/150W while system is idle and docker is not running, so was trying to find a a ways to poke it other than nvidia-smi to see if I can reduce it's idle power consumption. Thanks!
@CHBMB I too see this high power consumption. I know why it's happening, too.
Basically, the nvidia driver doesn't initialize power management until an Xorg server is running. The only way to force a power profile on Linux currently is to use nvidia-smi like so:
nvidia-settings --ctrl-display :0 -a "[gpu:0]/GPUPowerMizerMode=2"
Which requires a running Xorg display. I've been trying to dig around in sysfs to see if there is another place that this value is stored, but there doesn't seem to be. It looks like the cards are locked into performance mode... Perhaps this is worth bringing up to nvidia?
In the meantime, I'm going to continue digging to see if I can find a way (perhaps an nvidia-settings docker?) to force the power state.-
3
-
-
1 hour ago, Auxilium said:
Is it by design that libXxf86vm.so.1 is missing?
It prevents use of nvidia-settings
nvidia-settings: error while loading shared libraries: libXxf86vm.so.1: cannot open shared object file: No such file or directory
You should be providing that library in your docker container if you need nvidia-settings.
If you are running nvidia-settings in unraid itself (not sure why you would need to) then you would need to grab slackware packages for that library. That library is part of the Xorg project, and has nothing to do with Nvidia driver function, outside of the gui tool.
-
I've moved my script from gist to a dedicated github repository. I was trying to avoid this as it shouldn't be necessary for this script to exist in the near future, and it's a single source file with a pretty basic function. I've also created a dedicated thread on the forum for this script and any issues that result from using it. @CHBMB, please feel free to point people to either this repository or this forum thread for issues you believe to be caused by the wrapper script.
Anybody using the wrapper script originally posted to Reddit, the Plex forums, or earlier in this thread, be advised that you *should* move to the newer script written by Revr3nd and facilitated by my User Script. The nvidia decoder doesn't like certain formats, and those aren't filtered by earlier versions of the script. If you use those formats, and the decode script without filtering - you WILL have problems. This is likely the reason that Plex has been so reluctant to enable hardware decoding on Linux for nvidia. Emby is likely already filtering what content is transcoded by the GPU out of the box, though I have not taken a look under the hood. -
This post is under construction
Plex nvdec wrapper script
Please use this thread, to report issues, or discuss the use of the nvdec wrapper script. DO NOT report issues resulting from the use of this script to Unraid, LS.IO, Revr3nd, Plex, or nvidia.
What is this?
This is a wrapper script to enable nvidia based hardware decoding in Plex dockers running on unraid-nvidia. You must be running an unraid-nvidia build and have a working transcode environment using your nvidia card for this script to do anything. To find out more about the unraid-nvidia project and install it yourself, see this post.How do I get it?
Reporting issues
At a minimum, please provide a brief description of the issue you are facing, and a copy of the System Log from the current boot WHILE the issue is present. Do not reboot until after you have copied the log. To post the log, please use a paste service, like paste.ubuntu.com, rather than pasting into the forum directly. Full diagnostic zips may be required, but are generally overkill for troubleshooting what's happening with pci-e devices.Specifically, this advice applies to anyone reporting an issue of the use of this script causing their card to "drop" from unraid nvidia, as this may be something more major that needs to be submitted to the proper channels.
Please feel free to also mention any improvements for this OP.-
1
-
-
18 hours ago, CHBMB said:
OK everyone, the Plex HW Decode script seems to be causing more issues than the Unraid Nvidia build as far as I can tell.
From this point on, to reduce the unnecessary noise and confusion on this thread, I'm going to request whoever is looking after, documenting or willing to support the Plex HW Decode scripts spins off their own thread.
We will only be answering any support questions on people not using the script. If your post is regarding Plex and you do not EXPLICITLY state that you are not using the Plex HW Decode script then it will be ignored.
I know some of you may think this is unreasonable but it's creating a lot of additional work/time commitments for something I never intended to support and something I don't use (Not being a Plex user)
May I suggest respectfully, that one of you steps forward to create a thread, document it, and support it in it's own support place. I think we need to decouple issues with the work we've done versus issues with a currently unsupported script. I believe @Xaero may be the best person for this job, but do not let that stop any of you who are willing to step up to the plate.
Thanks.
I've mentioned, at the top of my script to post any issues using it on the gist itself, rather than reporting them to Plex, Unraid, LSIO, or Reverend (the guy who's wrapper my script downloads.) So that I can filter issues that are caused by error on my part, or the end user's part. My apologies since this is apparently insufficient.
I'll go ahead and make a thread, though my ability to provide support will be limited as I am currently without a PC until its unpacked.
-
I'm not at any of my normal computers - but knowing the codes for these hidden characters, can be useful when diagnosing this type of problem. It's possible, that it's something simple like not consistently using UTF-8 (i.e. it's set as a meta tag, so the client will render using it, but the server itself isn't aware of the character set being used, or content being submitted is being submitted using a different character set)
-
6 hours ago, Chad Kunsman said:
"GPU Reset couldn't run because GPU 00000000:01:00.0 is the primary GPU." is the message I get when I try to reset.
Persistence mode on or off makes no difference. Only resetting the Plex docker container causes it to downclock.
The hacky solution for scripting may come into play if nothing else works. My 'something else' I'm trying is going to be to give a P400 a shot to see if behavior is any different. I know for a fact its general power usage will be less. Maybe it will also not be affected by this bug. If it is, I'll give the script a try and definitely echo these concerns to the Plex team.
Thank you!
I think the concerns are best directed at the nvidia team, specifically the nvidia docker team. Plex can't really do anything about what the driver or kernel decides to do with the card when it's done using it. The kernel, or driver, should be telling the card to enter a different PState when it's idle and that's not happening.
-
1
-
-
On 5/8/2019 at 8:07 PM, Chad Kunsman said:
By the way, I have half solved this problem after more testing.
The card can and will clock itself down while idle UNTIL it has to perform transcoding work. Once the transcode job is finished, it will never clock itself down again until I restart the Plex docker container.
See the moment I restarted the container here: http://i.imgur.com/zeXKiFZ.png
See a thread from another user who seems to have the exact same issue here: https://forums.plex.tv/t/stuck-in-p-state-p0-after-transcode-finished-on-nvidia/387685/2
I really hope there's a solution as this is about $40 a year down the drain in extra power usage just due to the card not idling as it should.
This information could prove useful for creating an (albeit, hacky) solution - if need be.
I don't currently have any hardware set up where I can test anything (my server is in boxes, currently)
One thing you may try is enabling persistence mode:
nvidia-smi -pm 1This will result in a couple of watts of idle usage, but will force the drivers to stay loaded, even when no job is running. It's possible the drivers are exiting as soon as the transcode jobs finish, and not changing the power state back to idle.
Or, if it's already enabled, you could try disabling it:
nvidia-smi -pm 0
A hacky, scripted solution would be to monitor the nvidia smi output for a condition of both LOW GPU, NVENC and NVDEC performance and a HIGH power state, and issue the nvidia-smi --gpu-reset which would reset the GPU allowing it to idle again. Both of these are hacky workarounds.
I too would echo the Plex team on this in posting on the nvidia developer forums with this information, particularly point out the use of the new nvidia docker blobs, as it could quite possibly be an issue there. Once I have my server up and running again, I'll have a poke at seeing if I can replicate and/or resolve this issue.
https://devtalk.nvidia.com/-
1
-
-
18 hours ago, maxse said:
I have used watch nvidia-smi and looked under GPU-Util on the right side of the box. Right below that is a percentage. Is that something else?
That's the GPU core utilization. It shouldn't really increase with nvenc or nvdec usage. That's the whole point of nvenc and nvdec - giving gamers a way to accelerate video transcoding for streaming. For a streaming site like Netflix, it doesn't really make sense to transcode video data live for each stream, just store the video pre-encoded for transcoding. For a gamer to stream to twitch, they need a way to encode video on the fly without impacting rendering performance, enter nvenc and nvdec. The spike you are seeing is kind of suspicious, but the important metric to look at is the enc and dec columns of
nvidia-smi dmon
unraid-tunables-tester.sh - A New Utility to Optimize unRAID md_* Tunables
in User Customizations
Posted · Edited by Xaero
You aren't running my script.
If you do a ctrl-f on my script and type "/root" you'll see there is no hard coded reference to that location.
Your output is also showing:
"unRAID Tunables Tester v2.2 by Pauven"
I've added to the end of that and it should read:
"unRAID Tunables Tester v2.2 by Pauven - MOD by Xaero"
Both of these changes were intentional. One was to help differentiate, the other was because we shouldn't be using a hard coded nonstandard path like /root.