Additional Scripts For User.Scripts Plugin


Recommended Posts

I have made a script to delete all cached images in Plex's PhotoTranscoder folder that are older than 7 days. See below:

 

Quote

#!/bin/bash
echo "Searching for (and deleting) cached files"
find /mnt/user/appdata/plex/Library/Application\ Support/Plex\ Media\ Server/Cache/PhotoTranscoder/* -maxdepth 9999 -mtime +5 -exec rm -vf {} \; | wc -l
echo " files were deleted"

 

in the output the number of files deleted is on one line and then the "files were deleted" text is on the line under. I have tried to make the text appear directly after the count but can't seem to make it work. any ideas?

  • Like 1
Link to comment
51 minutes ago, aussie_huddo said:

I have tried to make the text appear directly after the count but can't seem to make it work. any ideas?

 

find * -maxdepth 9999 -mtime +5 -exec rm -vf {} \; -exec /bin/echo {} \; | wc -l | { read count; echo "Done. $count files deleted."; }

 

Grabbed this syntax from a thread on stackoverflow, works good. 

  • Like 1
  • Thanks 1
Link to comment
On 7/23/2016 at 4:00 PM, Squid said:

Run mover at a certain threshold of cache drive utilization.

 

Adjust the value to move at within the script.  Really only makes sense to use this script as a scheduled operation, and would have to be set to a frequency (hourly?) more often than how often mover itself runs normally.

 

 



#!/usr/bin/php
<?PHP

$moveAt = 70;    # Adjust this value to suit.

$diskTotal = disk_total_space("/mnt/cache");
$diskFree = disk_free_space("/mnt/cache");
$percent = ($diskTotal - $diskFree) / $diskTotal * 100;

if ( $percent > $moveAt ) {
  exec("/usr/local/sbin/mover");
}
?>
 

 

 

run_mover_at_threshold.zip 717 B · 130 downloads

Perhaps a bit... superfluous, but hereby the same script, but then in bash:

#!/bin/bash
thresh=70;

used=$(df --output=used /mnt/cache/ | awk '{print $1}' | tail -n 1);
size=$(df --output=size /mnt/cache/ | awk '{print $1}' | tail -n 1);
percent=$((100 * used / size));

if (( $percent > $thresh )); then
    sh /usr/local/sbin/mover;
fi

 

Edited by sgraaf
Fixed a small bug
Link to comment
40 minutes ago, sgraaf said:

Perhaps a bit... superfluous, but hereby the same script, but then in bash:


#!/bin/bash
thresh=70;

used=$(df --output=used /mnt/cache/ | awk '{print $1}' | tail -n 1);
avail=$(df --output=avail /mnt/cache/ | awk '{print $1}' | tail -n 1);
percent=$((100 * used / avail));

if (( $percent > $thresh )); then
    sh /usr/local/sbin/mover;
fi

 

Would also be helpful to note that ( used / avail ) is not quite what you want as while used increases, avail decreases. You want used / size

#!/bin/bash
thresh=70
stats=( $( df --output=used,size /mnt/cache | tail -n 1) )
percent=$[ 100 * ${stats[0]} / ${stats[1]} ]
[ $percent >  $thresh ] && sh /usr/local/sbin/mover

 

Link to comment
25 minutes ago, ken-ji said:

Would also be helpful to note that ( used / avail ) is not quite what you want as while used increases, avail decreases. You want used / size


#!/bin/bash
thresh=70
stats=( $( df --output=used,size /mnt/cache | tail -n 1) )
percent=$[ 100 * ${stats[0]} / ${stats[1]} ]
[ $percent >  $thresh ] && sh /usr/local/sbin/mover

 

Thanks; fixed it in my original post / reply.

Link to comment

Easiest is to use a lockfile (or semaphore)

Near the start of the script

if [ -e /var/lock/script.lock ]; then
  echo "Still running"
  exit
fi
touch /var/lock/script.lock

Then at the end of your script

rm -f /var/lock/script.lock

 

Edited by ken-ji
  • Like 1
Link to comment

Regarding the the "Clear an unRAID array data drive" script. The script doesn't verify that the desired drive is unmounted, I tried running the script last night and realized this morning that it was still running. The 'sync' progress had a hold on the drive. Could be an easy validation step to add to the script.

Link to comment
On 11/27/2017 at 6:07 PM, Squid said:

Check plugin integrity

 

Prior to inclusion in FCP, I'd like to get this script out to see if I'm just opening a can of worms and/or don't have a clue about what I'm doing  :D

 

This script will confirm the files downloaded by a plugin and stored on the flash drive actually match what the author intended (since unRaid only checks the files when they are actually downloaded, and upon a reboot it assumes that the file(s) stored on the flash are correct and does not confirm the checksums again when reinstalling at boot)

 

This script does require Community Applications to be installed, and if adding the script via Add Script in user.scripts, ensure you delete the opening #!/bin/bash that user scripts automatically pops into the script for you.

 

script 1.31 kB · 27 downloads

 

Hey @Squid - I couldn't find if this had already been addressed elsewhere or not (if so, my google-fu sucks):

 

As of at least 6.9, the top of the script needs a tiny edit, from:
 

require_once("/usr/local/emhttp/plugins/community.applications/include/xmlHelpers.php");

 

To:

require_once("/usr/local/emhttp/plugins/community.applications/include/helpers.php");

 

For anyone getting something about an error at line three, calling blah/blah/xmlHelpers.php when they run the plugin integrity check script, the above is the quick fix (hoping to catch some search keywords with this sentence to avoid repeat requests where possible 👍

Link to comment
5 minutes ago, Squid said:

Yup.  That script was posted over 3 years ago and things have been re-organized within CA since then.

Totally understood! Moreso posting to try to help any others that might end up seeing issues and come in with such a question 👍

 

Would it be worthwhile to have an open access GitHub page or something where all of the 'known' scripts could be stored, then folks can just edit to update them as/when needed? Or is that something that'd be discouraged at this point?

Link to comment
On 3/9/2021 at 11:07 AM, ken-ji said:

Easiest is to use a lockfile (or semaphore)

Near the start of the script


if [ -e /var/lock/scrip.lock ]; then
  echo "Still running"
  exit
fi
touch /var/lock/script.lock

Then at the end of your script


rm -f /var/lock/script.lock

 

 

could you explain this a little more. I do not understand what is happening? I just paste that exact code into the top of the script and the other at the end?

Link to comment

This is just sample code, but it can go into your script at line 2 (first possible code line)

what the 1st line does is check if there exists a file /var/lock/script.lock (note the typo) and if it does, it prints "Still running" for the user reference and exits the script (lines 2 and 3)

If the script still keeps running beyond the check (ie the file doesn't exist) it will then create the file and the script can do whatever you want.

 

Then at the end of the script, the command I showed will forcibly delete any existing /var/lock/script.lock.

 

All in all this means that when the script starts up, it checks if the lock file exists and will abort if it exists. this means a previously running script is still running (or crashed/aborted) and the script will no longer run until the lock file has been deleted. When the script does run, the last thing it will do is delete the lock file, thus allowing the script to run again.

Link to comment
  • 1 month later...
On 9/3/2016 at 8:06 PM, RobJ said:

Clear an unRAID array data drive  (for the Shrink array wiki page)

 

This script is for use in clearing a drive that you want to remove from the array, while maintaining parity protection.  I've added a set of instructions within the Shrink array wiki page for it.  It is designed to be as safe as possible, and will not run unless specific conditions are met -

- The drive must be a data drive that is a part of an unRAID array

- It must be a good drive, mounted in the array, capable of every sector being zeroed (no bad sectors)

- The drive must be completely empty, no data at all left on it.  This is tested for!

- The drive should have a single root folder named clear-me - exactly 8 characters, 7 lowercase and 1 hyphen.  This is tested for!

 

Because the User.Scripts plugin does not allow interactivity (yet!), some kludges had to be used, one being the clear-me folder, and the other being a 60 second wait before execution to allow the user to abort.  I actually like the clear-me kludge, because it means the user cannot possibly make a mistake and lose data.  The user *has* to empty the drive first, then add this odd folder.

 

 


#!/bin/bash
# A script to clear an unRAID array drive.  It first checks the drive is completely empty,
# except for a marker indicating that the user desires to clear the drive.  The marker is
# that the drive is completely empty except for a single folder named 'clear-me'.
#
# Array must be started, and drive mounted.  There's no other way to verify it's empty.
# Without knowing which file system it's formatted with, I can't mount it.
#
# Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder.
#
# 1.0  first draft
# 1.1  add logging, improve comments
# 1.2  adapt for User.Scripts, extend wait to 60 seconds
# 1.3  add progress display; confirm by key (no wait) if standalone; fix logger
# 1.4  only add progress display if unRAID version >= 6.2

version="1.4"
marker="clear-me"
found=0
wait=60
p=${0%%$P}              # dirname of program
p=${p:0:18}
q="/tmp/user.scripts/"

echo -e "*** Clear an unRAID array data drive ***  v$version\n"

# Check if array is started
ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null
if [ $? -ne 0 ]
then
   echo "ERROR:  Array must be started before using this script"
   exit
fi

# Look for array drive to clear
n=0
echo -n "Checking all array data drives (may need to spin them up) ... "
if [ "$p" == "$q" ] # running in User.Scripts
then
   echo -e "\n"
   c="<font color=blue>"
   c0="</font>"
else #set color teal
   c="\x1b[36;01m"
   c0="\x1b[39;49;00m"
fi

for d in /mnt/disk[1-9]*
do
   x=`ls -A $d`
   z=`du -s $d`
   y=${z:0:1}
#   echo -e "d:"$d "x:"${x:0:20} "y:"$y "z:"$z

   # the test for marker and emptiness
   if [ "$x" == "$marker" -a "$y" == "0" ]
   then
      found=1
      break
   fi
   let n=n+1
done

#echo -e "found:"$found "d:"$d "marker:"$marker "z:"$z "n:"$n

# No drives found to clear
if [ $found == "0" ]
then
   echo -e "\rChecked $n drives, did not find an empty drive ready and marked for clearing!\n"
   echo "To use this script, the drive must be completely empty first, no files"
   echo "or folders left on it.  Then a single folder should be created on it"
   echo "with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen."
   echo "This script is only for clearing unRAID data drives, in preparation for"
   echo "removing them from the array.  It does not add a Preclear signature."
   exit
fi

# check unRAID version
v1=`cat /etc/unraid-version`
# v1 is 'version="6.2.0-rc5"' (fixme if 6.10.* happens)
v2="${v1:9:1}${v1:11:1}"
if [[ $v2 -ge 62 ]]
then
   v=" status=progress"
else
   v=""
fi
#echo -e "v1=$v1  v2=$v2  v=$v\n"

# First, warn about the clearing, and give them a chance to abort
echo -e "\rFound a marked and empty drive to clear: $c Disk ${d:9} $c0 ( $d ) "
echo -e "* Disk ${d:9} will be unmounted first."
echo "* Then zeroes will be written to the entire drive."
echo "* Parity will be preserved throughout."
echo "* Clearing while updating Parity takes a VERY long time!"
echo "* The progress of the clearing will not be visible until it's done!"
echo "* When complete, Disk ${d:9} will be ready for removal from array."
echo -e "* Commands to be executed:\n***** $c umount $d $c0\n***** $c dd bs=1M if=/dev/zero of=/dev/md${d:9} $v $c0\n"
if [ "$p" == "$q" ] # running in User.Scripts
then
   echo -e "You have $wait seconds to cancel this script (click the red X, top right)\n"
   sleep $wait
else
   echo -n "Press ! to proceed. Any other key aborts, with no changes made. "
   ch=""
   read -n 1 ch
   echo -e -n "\r                                                                  \r"
   if [ "$ch" != "!" ];
   then
      exit
   fi
fi

# Perform the clearing
logger -tclear_array_drive "Clear an unRAID array data drive  v$version"
echo -e "\rUnmounting Disk ${d:9} ..."
logger -tclear_array_drive "Unmounting Disk ${d:9}  (command: umount $d ) ..."
umount $d
echo -e "Clearing   Disk ${d:9} ..."
logger -tclear_array_drive "Clearing Disk ${d:9}  (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} $v ) ..."
dd bs=1M if=/dev/zero of=/dev/md${d:9} $v
#logger -tclear_array_drive "Clearing Disk ${d:9}  (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000 ) ..."
#dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000

# Done
logger -tclear_array_drive "Clearing Disk ${d:9} is complete"
echo -e "\nA message saying \"error writing ... no space left\" is expected, NOT an error.\n"
echo -e "Unless errors appeared, the drive is now cleared!"
echo -e "Because the drive is now unmountable, the array should be stopped,"
echo -e "and the drive removed (or reformatted)."
exit
 

 

 

The attached zip is 'clear an array drive.zip', containing both the User.Scripts folder and files, but also the script named clear_array_drive (same script) for standalone use.  Either extract the files for User.Scripts, or extract clear_array_drive into the root of the flash, and run it from there.

 

Also attached is 'clear an array drive (test only).zip', for playing with this, testing it.  It contains exactly the same scripts, but writing is turned off, so no changes at all will happen.  It is designed for those afraid of clearing the wrong thing, or not trusting these scripts yet.  You can try it in various conditions, and see what happens, and it will pretend to do the work, but no changes at all will be made.

 

I do welcome examination by bash shell script experts, to ensure I made no mistakes.  It's passed my own testing, but I'm not an expert.  Rather, a very frustrated bash user, who lost many hours with the picky syntax!  I really don't understand why people like type-less languages!  It only *looks* easier.

 

After a while, you'll be frustrated with the 60 second wait (when run in User Scripts).  I did have it at 30 seconds, but decided 60 was better for new users, for now.  I'll add interactivity later, for standalone command line use.  It also really needs a way to provide progress info while it's clearing.  I have ideas for that.

 

The included 'clear_array_drive' script can now be run at the command line within any unRAID v6, and possibly unRAID v5, but is not tested there.  (Procedures for removing a drive are different in v5.)  Progress display is only available in 6.2 or later.  In 6.1 or earlier, it's done when it's done.

 

Update 1.3 - add display of progress; confirm by key '!' (no wait) if standalone; fix logger; add a bit of color

  Really appreciate the tip on 'status=progress', looks pretty good.  Lots of numbers presented, the ones of interest are the second and the last.

Update 1.4 - make progress display conditional for 6.2 or later; hopefully now, the script can be run in any v6, possibly v5

clear_an_array_drive.zip 4.37 kB · 1299 downloads

clear_an_array_drive_test_only.zip 4.61 kB · 117 downloads

Is there a log kept of this somewhere? I let it run overnight. It seems that my Unraid browser window disconnected though. It doesn't seem that dd is running anymore. Obviously I need to know if it completed successfully though before pulling the drive.

Link to comment
1 hour ago, jonathanm said:

In my experience, that script takes multiple days to run, even on a small drive. You are probably much better off following the normal method and recalculating parity.

I can't just restart dd?

 

edit: That's what I did yesterday anyway. It took about 24 hours to zero out an 8TB drive. 

Edited by flyize
Link to comment
2 minutes ago, jonathanm said:

Be sure to do a parity check immediately after completing the removal process to be sure everything worked properly and is in sync.

I'll be adding a second parity drive after I remove this data drive. Is it safe to assume that parity is checked when parity2 is added?

Link to comment
41 minutes ago, flyize said:

I'll be adding a second parity drive after I remove this data drive. Is it safe to assume that parity is checked when parity2 is added?

No. Adding parity2 only writes the generated data to the parity2 drive, it doesn't touch or verify the information on parity1.

Sorry, you are correct, parity1 is checked.

Edited by jonathanm
Bad info
Link to comment
2 minutes ago, jonathanm said:

No. Adding parity2 only writes the generated data to the parity2 drive, it doesn't touch or verify the information on parity1.

I know we're getting way off topic here, but doesn't it seem like it should? If parity1 is wrong, then won't parity2 be calculated incorrectly as well?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.