Jump to content
Squid

Additional Scripts For User.Scripts Plugin

313 posts in this topic Last Reply

Recommended Posts

I am getting ready to shrink two drives out of my array and was reading through the wiki and I have a question.

 

this is the first step.

 

Quote

Make sure that the drive you are removing has been removed from any inclusions or exclusions for all shares, including in the global share settings.

I am constantly having new data written to my server how do I prevent new data from being written to the drive I am trying to clear and remove?

 

Because one of the later steps says

 

Quote

While the script is running, the Main screen may show invalid numbers for the drive, ignore them. Important! Do not try to access the drive, at all!

I am removing two 6TB drives in total, I saw that you can only remove one at a time witch is fine.  My issue is that a clear takes a very long time to clear and I can't have my server down for that amount of time.

 

What do I need to do to make sure the dive that I am clearing does not get accessed or written to?

Share this post


Link to post
10 minutes ago, sgibbers17 said:

I am getting ready to shrink two drives out of my array

The method you are using is very long and has some risky aspects if you happen to mistype a command.

 

I strongly recommend doing it the "easy" way and rebuild parity with a new config. That way you can simply remove the drives as is, and if you find out that you accidentally left some data on them, it's easy enough to mount them with the Unassigned Devices plugin and copy the data back to the array.

 

Any time you do major reconfigurations to the server you are taking on risk. Do you have backups of any data you can't afford to lose? Are all your drives healthy?

Share this post


Link to post

Thanks.  I was trying to avoid rebuilding the parity, I don't have anything really important on the server I'll just make sure I have a backup of the stuff I'd rather not lose just incase.

 

Too bad there isn't a just a button to remove a drive an unraid would just redistribute the data for you then remove it.  I've been wanting that since v4.7

Share this post


Link to post
51 minutes ago, sgibbers17 said:

Too bad there isn't a just a button to remove a drive an unraid would just redistribute the data for you then remove it.  I've been wanting that since v4.7

Well a button wouldn't quite be enough to suit some people in the details regarding the redistribution. And of course it would still take a long time to either clear disks or rebuild parity.

 

As for redistributing data, have you looked at the Unbalance plugin?

Share this post


Link to post

Script to spin up all drives at certain times of day...

 

#!/bin/bash
startTime="20:00"
endTime="23:00"

begin=$(date --date=$startTime +%s)
end=$(date --date=$endTime +%s)
now=$(date +%s)

if [ "$begin" -le "$now" -a "$now" -le "$end" ]; then
    disks=$(ls /dev/md* | sed "sX/dev/mdXX");
    for disknum in $disks; do
        /usr/local/sbin/mdcmd spinup $disknum;
    done
fi

Instructions...

  • Set appropriate start and end times (example above between 8pm and 11pm)
  • Set it to run with a custom cron schedule of - say - every 10 minutes (or lower than your drive's spin down timings...
*/10 * * * *

Notes...

  • This checks every ten minutes to see whether it's within the window.  If so, issue a spin up command on all data drives (could potentially check its state beforehand?)
  • There might be a way to cron it so it only runs during the window, but I don't know enough about that
  • More complicated schedules (eg, longer times at weekends) would need more complex logic, or duplication of the script
  • It was written from bits and pieces I found.  If I'm doing something very bad, please do let me know.  Somebody raised it in a post and I thought "this sounds like a job for User Scripts"
  • Testing it seems to work fine.  Drives appear to spin up straight away, but don't have temperature readings for about 4-5 minutes (19 data drives)

 

 

Edit: after looking more into cron, you can simplify it by making the schedule decide when the script runs, and having the script just run spinup.  New script...

#!/bin/bash
disks=$(ls /dev/md* | sed "sX/dev/mdXX");
for disknum in $disks; do
    /usr/local/sbin/mdcmd spinup $disknum;
done

with the schedule command as...

*/10 20-22 * * *

If I understand correctly, this will trigger the script at 8pm, and then re-run it every 10 minutes until 22:50.  Change these values to suit.

 

I've kept both for now in case one is stupid.  Plus some snippets might come in handy for other people's projects.

 

If you want to have your drives up for a different duration at the weekends, duplicate the script and give them both different schedules...

Every 10 minutes between midday and 11pm, Saturday & Sunday
*/10 12-23 * * 6-7

Every 10 minutes between 8-11pm, Monday - Friday
*/10 20-23 * * 1-5

 

Edited by Cessquill
Thought of a better way

Share this post


Link to post

Great thanks ive been replying to you on you tube, im a total beginner in every sense of the word, ive installed plugib so do i just drop this in 

 

9 hours ago, Cessquill said:

#!/bin/bash disks=$(ls /dev/md* | sed "sX/dev/mdXX"); for disknum in $disks; do /usr/local/sbin/mdcmd spinup $disknum; done

 

Share this post


Link to post
1 hour ago, coblck said:

Great thanks ive been replying to you on you tube, im a total beginner in every sense of the word, ive installed plugib so do i just drop this in

Pretty much, yes.  I'm not at my machine at the moment, so it might be a little vague, but in the plugin...

  • Click the Add Script button
  • Give it a title (say, Spin Up All Drives, or whatever you want)
  • Save it
  • Hover over the new script entry that's been created and click on it.  The popup that appears will allow you to "Edit Script".  Click on this
  • In the box that appears at the top, enter...
#!/bin/bash
disks=$(ls /dev/md* | sed "sX/dev/mdXX");
for disknum in $disks; do
    /usr/local/sbin/mdcmd spinup $disknum;
done
  • Click the Save button.  Your script will now be created
  • To test it, you can click the Run Script button next to the title.  A window will pop up.  If its contents have any errors, we'll need to look into them.  If not, close the popup and switch to the home page and check that all drives have spun up (note, mine didn't have temperature readings for a few minutes)
  • If it's working as expected, you can now schedule it to run during your peak times.  Go back to the User Scripts plugin.  To the right of your script you've got a schedule dropdown.  From it, select "custom"
  • In the text box to the right of that, enter/paste...
*/10 20-22 * * *
  • Click apply at the bottom.  This is a cron schedule that basically tells User Scripts to execute your script every 10 minutes between 20:00 and 22:00 (actually at the end of 22:00, so in this case 22:50), every day.  Adjust these hours to suit.  I have to relearn the formatting every time I do something like this, but fortunately there are a lot of online tools to build the schedule code for you
  • Once the script has run on the schedule, you'll get three buttons to the right of it - one of them allows you to view the log.  You can use this to check for errors and make sure it's running at the correct times
  • I've assumed that your disks' spin-down times are 15 minutes (default?).  If not, you can adjust the 10 at the start to something that's just before when they'd normally spin down.  Technically you could set the "10" to "14"
  • As mentioned, if you need multiple schedules, that's possible with duplicating the script and adjusting the schedules of each to suit

What I haven't thought about is whether running the spinup command on a drive that's already spun up will reset its spin-down timer.  That is, if it's been up for 10 minutes, scheduled to spin down in 5 and you run a spin-up command - will it still spin down in 5, or has it been reset to 15?  I'm afraid I don't know enough about the innards of Unraid, but it could be tested (and somebody would know).

Share this post


Link to post

hello,

 

can i use user scripts for scanning my nextcloud files?

 

i want to insert this code:

docker exec -it mynextcloudserver sudo -u abc php /config/www/nextcloud/occ files:scan --all 

the code should be executed daily.

Thx for your replies.

 

Share this post


Link to post

Possible problems  with "clear_array_drive" script and xfs formatted disks resulting in extremely low speed

 

I initially ran the script on one "reiserFS" formatted disk and it worked beautifully. 

 

Then I decided to change my disks from "reiserFS to xfs using the "mirror" strategy and ended with one unused drive (the smallest one). By default my disks are set to use xfs and when I formatted it in the last step it did use the xfs.

I saw that the disk was set as "MBR unaligned" and I zeroed the first 512 bytes. After another formatting it was all OK and I proceed to zero the disk using the script above.

It started with over 200MB/s but quickly dropped to 1.2MB/s and later to 600-700kB/s. In a few hours it was zeroed only 1.5GB but it resulted in tens of millions of "writes" on the main page. Nothing wrong in the log file... I tried to kill the "dd" PID, then used Ctrl-C but was not successful and had to stop the array.

It took maybe 10-20-30 minutes but finally it stopped and I could power down the server.

 

I inspected the cables and the powered-up the server , formatted the disk again and then let it run overnight. In the morning it was barely zeroed 15GB, speed was around 500kB/s and again tens of millions of "writes " om the main page.

 

Repeated the stopping/shut-down procedure again but this time after the power-up I formatted it with the older "reiserFS" and then gave it another go with the script.

BINGO  - it works as it should - in the 2 and half hours it did zeroed 700GB+ and the "writes" are around 1.65 million

 

I believe someone should investigate it further

reiserFS formatted disks - works fine

xfs formatted disks - read elsewhere that the array should be in "maintenance mode" (not tested by me)

btrfs formatted disks - no idea!!!

 

Just for the kick I decided to  format the drive with btrfs and surprisingly the script refused to run claiming that there is not a single  empty hard drive !!!

==============================================================================================

To summarize my tests for the "clear_array_drive" script:

unRAID v 6.6.6

1TB WD green disk with no SMART problems.

 

reiserFS format - 33.6MB used space - script runs as intended

 

btrfs format - 17.2MB used space - script refused to run claiming no empty hard drive found

 

xfs format - 1.03GB used space - script runs painfully slow with tens of millions of "writes" to the disk.

The test was cancelled by me as the estimated time could be weeks or even month!!!

Another post here claims OK but with array running in "maintenance mode"  (this I did not test)

=============================================================================================

 

 

 

 

 

 

Edited by bcbgboy13
Update on the additional tests

Share this post


Link to post

Enable Hardware Decoding in Plex

#!/bin/bash

con="plex"

echo ""
echo "<font color='red'><b>Applying hardware decode patch...</b></font>"
echo "<hr>"

docker exec -i $con mv "/usr/lib/plexmediaserver/Plex Transcoder" "/usr/lib/plexmediaserver/Plex Transcoder2"
docker exec -i $con /bin/sh -c 'printf "#!/bin/sh\nexec /usr/lib/plexmediaserver/Plex\ Transcoder2 -hwaccel nvdec "\""\$@"\""" > "/usr/lib/plexmediaserver/Plex Transcoder";'
docker exec -i $con chmod +x "/usr/lib/plexmediaserver/Plex Transcoder"
docker exec -i $con chmod +x "/usr/lib/plexmediaserver/Plex Transcoder2"
docker restart $con

echo ""
echo "<font color='red'><b>Done!</b></font>"

 

Description: Translation of manual steps required to patch Plex docker to enable hardware decoding if you're running an Nvidia version of Unraid.

 

Quick Start: Set up and run as a script every time Plex updates.  If your container is not called "plex", change the "con" variable (see notes)

 

Disclaimer: If it can be improved (or if it's dangerously wrong), please let me know.

 

Notes:

  • Should be run when Plex is installed/updated
  • From the command line, run "docker ps" to see what your plex container is called.  Set that as the "con" variable in your script (mine is "plex")
  • This script is only required until Plex officially supports hardware decoding
  • It preforms the same as recommended in the NVidia plugin support thread here (where it was originally published), namely...
    • Renames the file "Plex Transcoder" to "Plex Transcoder2"
    • Creates a new "Plex Transcoder" file with the suggested contents
    • Changes permissions on both "Plex Transcoder" and "Plex Transcoder2" files (not sure it's required on Transcoder2 - seemed to work for me without)
    • Restarts the Plex container (not sure if required, but doing it anyhow)
  • Probably best nothing is playing whilst the script is run
  • You'll need to have Plex running for the script to work.  Would require different code if stopped (would probably be safer to stop the container first, make the changes then start again, but here we are)
  • Run "nvidia-smi dmon -s u" from the terminal (not within Plex container) to check whether the decoding is working.  Set a video to play in a transcoded state, and the 3rd and 4th columns from the end should be non-zero
  • This includes the "exec" addition to the Plex Transcoder file contents

Good luck!

Share this post


Link to post

Hey all, 

 

I'm looking to execute a script after the array is completely started. I've tried selecting the "At Startup of Array" schedule, but the array doesn't completely start until the script is finished. Any ideas?

Share this post


Link to post
13 hours ago, rotorhead8 said:

Hey all, 

 

I'm looking to execute a script after the array is completely started. I've tried selecting the "At Startup of Array" schedule, but the array doesn't completely start until the script is finished. Any ideas?

By design.  Schedule at start or stop of array does this.  At first start only, it will run in the background and not tie up the boot process.

 

You can get around this by having your script simply

 

/path/to/real/script | at NOW -M 

 

Edited by Squid

Share this post


Link to post
2 hours ago, nuhll said:

add a timer, add a "wait if mtn/user is there" condition.

Thank you for the input! This might be my best option it seems like. What would that condition look like? *noob-ish to scripts*

 

Here is what the script looks like:

ssh ubuntu sudo ./mntservershares.sh 

 

30 minutes ago, Squid said:

By design.  Schedule at start or stop of array does this.  At first start only, it will run in the background and not tie up the boot process.

 

You can get around this by having your script simply

 


/path/to/real/script | at NOW -M 

 

Thank you for the help! So would I just add 

| at NOW -M

to the end of the script just like this?

ssh ubuntu sudo ./mntservershares.sh | at NOW -M

 

Also, does this option work at array start or only on the first array start?

Edited by rotorhead8

Share this post


Link to post

I dont know anything about that "at NOW".

 

just add a "sleep 500" after that time the mount should be long finished.

Share this post


Link to post
Posted (edited)
On 2/25/2019 at 11:12 AM, Cessquill said:

Enable Hardware Decoding in Plex


#!/bin/bash

con="plex"

echo ""
echo "<font color='red'><b>Applying hardware decode patch...</b></font>"
echo "<hr>"

docker exec -i $con mv "/usr/lib/plexmediaserver/Plex Transcoder" "/usr/lib/plexmediaserver/Plex Transcoder2"
docker exec -i $con /bin/sh -c 'printf "#!/bin/sh\nexec /usr/lib/plexmediaserver/Plex\ Transcoder2 -hwaccel nvdec "\""\$@"\""" > "/usr/lib/plexmediaserver/Plex Transcoder";'
docker exec -i $con chmod +x "/usr/lib/plexmediaserver/Plex Transcoder"
docker exec -i $con chmod +x "/usr/lib/plexmediaserver/Plex Transcoder2"
docker restart $con

echo ""
echo "<font color='red'><b>Done!</b></font>"

 

Description: Translation of manual steps required to patch Plex docker to enable hardware decoding if you're running an Nvidia version of Unraid.

 

Quick Start: Set up and run as a script every time Plex updates.  If your container is not called "plex", change the "con" variable (see notes)

 

Disclaimer: If it can be improved (or if it's dangerously wrong), please let me know.

 

Notes:

  • Should be run when Plex is installed/updated
  • From the command line, run "docker ps" to see what your plex container is called.  Set that as the "con" variable in your script (mine is "plex")
  • This script is only required until Plex officially supports hardware decoding
  • It preforms the same as recommended in the NVidia plugin support thread here (where it was originally published), namely...
    • Renames the file "Plex Transcoder" to "Plex Transcoder2"
    • Creates a new "Plex Transcoder" file with the suggested contents
    • Changes permissions on both "Plex Transcoder" and "Plex Transcoder2" files (not sure it's required on Transcoder2 - seemed to work for me without)
    • Restarts the Plex container (not sure if required, but doing it anyhow)
  • Probably best nothing is playing whilst the script is run
  • You'll need to have Plex running for the script to work.  Would require different code if stopped (would probably be safer to stop the container first, make the changes then start again, but here we are)
  • Run "nvidia-smi dmon -s u" from the terminal (not within Plex container) to check whether the decoding is working.  Set a video to play in a transcoded state, and the 3rd and 4th columns from the end should be non-zero
  • This includes the "exec" addition to the Plex Transcoder file contents

Good luck!

I just tossed a 1050Ti into my server to see if I could get this working, and after running the script (with the right container name) it doesn't seem to be working. It shows that it's encoding, but the decoding still sits at 0% and I'm not sure why.

 

Nevermind, I seem to have found out why. It looks like this script only works on version 1.15+ and I'm just running 1.14 so I need to wait until the alpha is pushed to the main channel before it will work.

Edited by zacharyd3
solved

Share this post


Link to post

 

On 3/9/2019 at 3:36 PM, zacharyd3 said:

I just tossed a 1050Ti into my server to see if I could get this working, and after running the script (with the right container name) it doesn't seem to be working. It shows that it's encoding, but the decoding still sits at 0% and I'm not sure why.

 

Nevermind, I seem to have found out why. It looks like this script only works on version 1.15+ and I'm just running 1.14 so I need to wait until the alpha is pushed to the main channel before it will work.

I am on 1.15+ and I can't get video to play.  It shows that it is working but no sound/video plays on the player.  Really really weird.  Fixed once I remove the script and play again.

Share this post


Link to post

Wondering if I can get some assistance creating an ExifTool script. I installed exiftool from the nerd pack plugin, and created a script using CA user scripts. 

 

I am getting the following error:

/tmp/user.scripts/tmpScripts/exiftool/script: /usr/bin/exiftool: /usr/bin/perl: bad interpreter: No such file or directory
Script Finished Sun, 17 Mar 2019 21:21:55 -0500

Full logs for this script are available at /tmp/user.scripts/tmpScripts/exiftool/log.txt

 

My script is as follows:

#!/bin/bash
exiftool -progress "-FileName<FileModifyDate" "-FileName<DateCreated" "-FileName<CreateDate" -d "/mnt/user/media/photos/exiftool_test/%%e/%Y/%b/%Y-%b-%d-(%%c).%%e" -r -v /mnt/user/media/photos/photo_drop

 

In an ideal word I would like to have my files named:

 

Year\Month\Mon.Date HH:MM:SS

Share this post


Link to post
Posted (edited)

The Nerdpack plugin also has the perl package so you need to install that.

I don't know if there any dependencies afterwards.

Edited by ken-ji

Share this post


Link to post
Posted (edited)

Hi, im also helpless on a script i would like to create

 

usecase, i do my commercial cuts manually (as comskip aint so reliable) with avidemux, so i have a .ts file, after cutting i have a .idx2 file and my final .mkv file.

 

now, what i would like to reach

 

look for .ts file, check if .mkv exist, if so run a curl command including the file name, remove the ts and idx2 file.

 

after looking for some kind of commands i guess its currently over my head, may someone has some hints where to start like finding the dupes and catch the filepath from location /Media instead /mnt/user/Media/ (needed for update TVHeadend curl command).

 

for any tipps thanks ahead

 

### update, i guess i have it finally, may someone could take a look if its reliable or if theres something i may didnt checked out ...

### like this its updating live TVH recording and removing the .ts and .idx2 file, cause they need to be there for updating TVH.

 

#!/bin/bash

startpath="/mnt/cache/Media/"   ### main place where to look for
searchpattern="*.ts"            ### what type of files to look for
matchpattern=".mkv"             ### compare if exist - work done ?
extradelpattern=".idx2"         ### extra delete type - sample, avidemux file
mediafolder="/Media"            ### add to path beginning for TVHeadend update
waittime=30                     ### time to wait in case script runs while processing file

tvhuser="user"
tvhpass="pass"
tvhip="192.168.1.2"

cd $startpath

find . -mindepth 2 -iname $searchpattern|sed "s|^\./||"|while read fname; do
  echo "$fname"
  sourcefile=$(basename "$fname")
  filenameonly="${sourcefile%.*}"
  endfile="$filenameonly$matchpattern"
  extrafile="$sourcefile$extradelpattern"
  filepath=$(dirname "$fname")

  cd "$startpath/$filepath"

  if [ -f "$endfile" ]
    then
      echo "$endfile found, processing ..."
      curl -s "http://"$tvhuser":"$tvhpass"@"$tvhip":9981/api/dvr/entry/filemoved" --data-urlencode "src=$mediafolder/$filepath/$sourcefile" --data-urlencode "dst=$mediafolder/$filepath/$endfile"
      sleep $waittime
      rm "$sourcefile"
      rm "$extrafile"
    else
      echo "$endfile not found."
    fi
done

Edited by alturismo

Share this post


Link to post

Does anyone have a User Script for scheduling the startup/shutdown of a container at certain parts of the day?  Would love to do this with my Handbrake container so it's only encoding overnight.

Share this post


Link to post

2 basic Scripts with a custom schedule configured should work. @IamSpartacusLets say

docker start handbrake

at 9:30

30 9 * * *

and

docker stop handbrake

at 12:30

30 12 * * *

 

Share this post


Link to post
10 minutes ago, bastl said:

2 basic Scripts with a custom schedule configured should work. @IamSpartacusLets say


docker start handbrake

at 9:30


30 9 * * *

and


docker stop handbrake

at 12:30


30 12 * * *

 

 

Oh wow, super easy.  Thanks @bastl.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.