Majyk Oyster Posted August 4, 2022 Share Posted August 4, 2022 (edited) 11 hours ago, hernandito said: Has anyone created a script that renews the LetsEncrypt certificate? Would love to implement. thank you. Do you need the certificate updated for the Unraid server or for a hosted VM/container ? Edited August 4, 2022 by Majyk Oyster Quote Link to comment
hernandito Posted August 5, 2022 Share Posted August 5, 2022 4 hours ago, Majyk Oyster said: Do you need the certificate updated for the Unraid server or for a hosted VM/container ? I would like it for the unRAID server. I'm sure at some point I will forget about it and then it may expire. Thank you! H. Quote Link to comment
AverageMarcus Posted August 7, 2022 Share Posted August 7, 2022 I made a small update to the `clear_array_drive` script to support setting `status=progress` for unRAID >= 6.10. #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0 first draft # 1.1 add logging, improve comments # 1.2 adapt for User.Scripts, extend wait to 60 seconds # 1.3 add progress display; confirm by key (no wait) if standalone; fix logger # 1.4 only add progress display if unRAID version >= 6.2 # 1.5 fix progress display if unRAID version >= 6.10 version="1.5" marker="clear-me" found=0 wait=60 p=${0%%$P} # dirname of program p=${p:0:18} q="/tmp/user.scripts/" echo -e "*** Clear an unRAID array data drive *** v$version\n" # Check if array is started ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null if [ $? -ne 0 ] then echo "ERROR: Array must be started before using this script" exit fi # Look for array drive to clear n=0 echo -n "Checking all array data drives (may need to spin them up) ... " if [ "$p" == "$q" ] # running in User.Scripts then echo -e "\n" c="<font color=blue>" c0="</font>" else #set color teal c="\x1b[36;01m" c0="\x1b[39;49;00m" fi for d in /mnt/disk[1-9]* do x=`ls -A $d` z=`du -s $d` y=${z:0:1} # echo -e "d:"$d "x:"${x:0:20} "y:"$y "z:"$z # the test for marker and emptiness if [ "$x" == "$marker" -a "$y" == "0" ] then found=1 break fi let n=n+1 done #echo -e "found:"$found "d:"$d "marker:"$marker "z:"$z "n:"$n # No drives found to clear if [ $found == "0" ] then echo -e "\rChecked $n drives, did not find an empty drive ready and marked for clearing!\n" echo "To use this script, the drive must be completely empty first, no files" echo "or folders left on it. Then a single folder should be created on it" echo "with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen." echo "This script is only for clearing unRAID data drives, in preparation for" echo "removing them from the array. It does not add a Preclear signature." exit fi # check unRAID version v1=(`cat /etc/unraid-version | awk -F'"' '{ print $2 }' | tr '.' ' '`) if [[ ${v1[0]} -eq 6 ]] && [[ ${v1[1]} -ge 2 ]] then v=" status=progress" else v="" fi # First, warn about the clearing, and give them a chance to abort echo -e "\rFound a marked and empty drive to clear: $c Disk ${d:9} $c0 ( $d ) " echo -e "* Disk ${d:9} will be unmounted first." echo "* Then zeroes will be written to the entire drive." echo "* Parity will be preserved throughout." echo "* Clearing while updating Parity takes a VERY long time!" echo "* The progress of the clearing will not be visible until it's done!" echo "* When complete, Disk ${d:9} will be ready for removal from array." echo -e "* Commands to be executed:\n***** $c umount $d $c0\n***** $c dd bs=1M if=/dev/zero of=/dev/md${d:9} $v $c0\n" if [ "$p" == "$q" ] # running in User.Scripts then echo -e "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else echo -n "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi # Perform the clearing logger -tclear_array_drive "Clear an unRAID array data drive v$version" echo -e "\rUnmounting Disk ${d:9} ..." logger -tclear_array_drive "Unmounting Disk ${d:9} (command: umount $d ) ..." umount $d echo -e "Clearing Disk ${d:9} ..." logger -tclear_array_drive "Clearing Disk ${d:9} (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} $v ) ..." dd bs=1M if=/dev/zero of=/dev/md${d:9} $v # Done logger -tclear_array_drive "Clearing Disk ${d:9} is complete" echo -e "\nA message saying \"error writing ... no space left\" is expected, NOT an error.\n" echo -e "Unless errors appeared, the drive is now cleared!" echo -e "Because the drive is now unmountable, the array should be stopped," echo -e "and the drive removed (or reformatted)." exit Quote Link to comment
kri kri Posted August 7, 2022 Share Posted August 7, 2022 (edited) I am looking for a way to move data around my server, similar to how Hazel works on my mac. I would like it to be based on file type (video) and part of a filename. So basically look inside a folder, and its subfolders to see if the syntax matches and then move to another folder on my server. I have everything under /data per trash guides so I would imagine it would be pretty quick, but I don't really care about performance since I will just probably run it once a day or so. Anyone have something like this? Edited August 7, 2022 by kri kri Quote Link to comment
TheLargeFries Posted August 8, 2022 Share Posted August 8, 2022 On 8/4/2022 at 7:43 AM, hernandito said: Has anyone created a script that renews the LetsEncrypt certificate? Would love to implement. thank you. I just pull my wildcard cert from my nginx proxy manager docker (the container handles the actual renewal with lets encrypt). then I use the scripts plugin to run this weekly: #!/bin/bash # Check if array is started ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null if [ $? -ne 0 ] then echo $(date -u) " LE SCRIPT - ERROR: Array must be started before using this script" exit else echo "Moving cert from NGINX Proxy Manager docker" cp /mnt/cache/appdata/Nginx-Proxy-Manager-Official/letsencrypt/live/npm-1/fullchain.pem /boot/config/ssl/certs/unas_unraid_bundle.pem cat /mnt/cache/appdata/Nginx-Proxy-Manager-Official/letsencrypt/live/npm-1/privkey.pem >> /boot/config/ssl/certs/unas_unraid_bundle.pem echo "Reloading nginx service" /etc/rc.d/rc.nginx reload echo $(date -u) " LE SCRIPT - COMPLETE" fi Quote Link to comment
Majyk Oyster Posted August 8, 2022 Share Posted August 8, 2022 On 8/7/2022 at 2:03 PM, AverageMarcus said: I made a small update to the `clear_array_drive` script to support setting `status=progress` for unRAID >= 6.10. I did that and more changes a few does ago, you can see it on the previous page. There's so many links pointing at RobJ's initial post, that would be nice if there was a way to mention updates. 1 Quote Link to comment
Digital Shamans Posted August 30, 2022 Share Posted August 30, 2022 On 7/23/2016 at 7:00 PM, Squid said: Run mover at a certain threshold of cache drive utilization. Adjust the value to move at within the script. Really only makes sense to use this script as a scheduled operation, and would have to be set to a frequency (hourly?) more often than how often mover itself runs normally. #!/usr/bin/php <?PHP $moveAt = 70; # Adjust this value to suit. $diskTotal = disk_total_space("/mnt/cache"); $diskFree = disk_free_space("/mnt/cache"); $percent = ($diskTotal - $diskFree) / $diskTotal * 100; if ( $percent > $moveAt ) { exec("/usr/local/sbin/mover"); } ?> On 7/23/2016 at 7:00 PM, Squid said: run_mover_at_threshold.zip 717 B · 168 downloads Thanks for the script! Do you have any suggestion, how to avoid these error when run the script? Script location: /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script Note that closing this window will abort the execution of this script Warning: disk_total_space(): No such file or directory in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 6 Warning: disk_free_space(): No such file or directory in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 7 Warning: Division by zero in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 8 Quote Link to comment
kuchta Posted September 4, 2022 Share Posted September 4, 2022 On 7/29/2022 at 10:20 PM, Majyk Oyster said: Hi there. I needed to use RobJ's "Clear An unRaid Data Drive" script, and noticed it wasn't compatible with Unraid 6.10 (version verification checked only 1 digit). So I dived into the script and found a few things I could update or do differently. Since RobJ doesn't seem to be around anymore, I took the liberty to rewrite most of it. There seems to be a pretty major bug here. I tried to run it and it told me it was going to clear the wrong disk. I looked at the code a little closer and it seems like you opted to continue looking for additional disks that were ready to be cleared instead of breaking out of the loop. Unfortunately, this means that if only one disk is found ready to be cleared, it will think it's the last one in the array since the previously identified disk isn't remembered. I "fixed" this by adding a break command at line 113 (after the "((found++))" command). Oddly, the script still didn't work for me even though it identified the correct device. I get an EOF error right when it is about to unmount the drive. I didn't see any errors in the code to cause this, so I'm wondering if it's a copy-and-paste issue or something. I've attached my log in case it's helpful. For reference, disk7 is the one I was trying to clear and disk8 is the one it incorrectly tried to clear first. I don't need the script anymore as I just created my own script with the 2 action lines, but wanted to make sure I reported this issue so that no one accidentally clears the wrong drive. clear_array_drive_error.txt Quote Link to comment
Nanobug Posted September 14, 2022 Share Posted September 14, 2022 On 2/25/2017 at 9:41 PM, SpaceInvaderOne said: As nested vms have been disabled by default in 6.3 due to the issue with avast and windows vms when enabled. I have made 2 scripts to turn it on and off. In script set cpu type 1 for intel 2 for amd. Download link for both scripts https://www.dropbox.com/s/0b1tvotvl6y80uy/nested vms scripts.zip?dl=0 nested vm on script #!/bin/bash #set whether your cpu is Intel or AMD [1-Intel] [2-Amd] cputype="1" #Do not edit below this line #turn nested on if [[ "$cputype" =~ ^(1|2)$ ]]; then if [ "$cputype" -eq 1 ]; then modprobe -r kvm_intel modprobe kvm_intel nested=1 echo "Nested vms are enabled for intel cpus" elif [ "$pushnotifications" -eq 2 ]; then modprobe -r kvm_amd modprobe kvm_amd nested=1 echo "Nested vms are enabled for AMD cpus" fi else echo "invalid cpu type set please check config" fi sleep 4 exit and nested vm off script #!/bin/bash #set whether your cpu is Intel or AMD [1-Intel] [2-Amd] cputype="1" #Do not edit below this line #turn nested off if [[ "$cputype" =~ ^(1|2)$ ]]; then if [ "$cputype" -eq 1 ]; then modprobe -r kvm_intel modprobe kvm_intel nested=0 echo "Nested vms are disabled for intel cpus" elif [ "$pushnotifications" -eq 2 ]; then modprobe -r kvm_amd modprobe kvm_amd nested=0 echo "Nested vms are disabled for AMD cpus" fi else echo "invalid cpu type set please check config" fi sleep 4 exit I get this error when I run the script: /tmp/user.scripts/tmpScripts/Nested VM on/script: line 20: [: : integer expression expected I tried looking for an updated version, but wasn't able to find one. Line 20 should be: elif [ "$pushnotifications" -eq 2 ]; then Can anyone help? Quote Link to comment
inh Posted September 15, 2022 Share Posted September 15, 2022 (edited) On 7/29/2022 at 4:20 PM, Majyk Oyster said: Hi there. I needed to use RobJ's "Clear An unRaid Data Drive" script, and noticed it wasn't compatible with Unraid 6.10 (version verification checked only 1 digit). So I dived into the script and found a few things I could update or do differently. Since RobJ doesn't seem to be around anymore, I took the liberty to rewrite most of it. Change log : Version check adapted for Unraid 6.10 and up. Check if unmount successful before clearing. Send mail notification when clearing finished. Script adapted for more then 9 drives. Dead code removed. Code simplified and optimized. Ambiguous variables renamed. Progression of the dd command sent to the GUI log Error message more explicit if more than 1 drive is prepared for clearing. #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0first draft # 1.1add logging, improve comments # 1.2adapt for User.Scripts, extend wait to 60 seconds # 1.3add progress display; confirm by key (no wait) if standalone; fix logger # 1.4only add progress display if unRAID version >= 6.2 # 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net. #Change log : #Version check adapted for Unraid 6.10 and up. #Check if unmount successful before clearing. #Send mail notification when clearing finished. #Script adapted for more then 9 drives. #Dead code removed. #Code simplified and optimized. #Ambiguous variables renamed. #Progression of the dd command sent to the GUI log ############# # Functions # ############# funDisplay () { echo -e "\n`date +%d/%m" "%T` : $1\n" logger -t$tag "$2" } funMail() { /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2" } ############# # Variables # ############# scriptVersion="2.0" marker="clear-me" tag="clear_array_drive" started=0 ddArg="" #used for dd command found=0 wait=60 scriptDir=$(dirname "$0") userscriptstDir="/tmp/user.scripts" # Colors management if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors. colorStart="<font color=blue>" colorFinish="</font>" else # running from CLI. Sets shell colors colorStart="\x1b[36;01m" colorFinish="\x1b[39;49;00m" fi ########## # Checks # ########## funDisplay "Clear an unRAID array data drive v$scriptVersion." # check unRAID version unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2` majorVersion=`echo $unraidVersion | cut -d "." -f 1` minorVersion=`echo $unraidVersion | cut -d "." -f 2` if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then ddArg="status=progress" else funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)." if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI echo "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi fi # checks if array is started disks=`ls -d /mnt/* | grep disk | grep -v disks` drivesNumber=`echo "$disks" | wc -l` if [ $drivesNumber == 0 ]; then funDisplay "ERROR: Array must be started before using this script. Exiting." exit fi # checks if a disk is ready for clearing funDisplay "Checking all array data drives (may need to spin them up)..." for disk in $disks; do itemsList=`ls -A $disk 2> /dev/null` itemsNumber=`echo "$itemsList" | wc -l` # test for marker and emptiness if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then itemsSize=`du -s $disk | awk '{print $1}'` if [ $itemsSize -eq 0 ]; then ((found++)) fi fi done # No drive or multiple drives found to clear if [ $found -eq 0 ]; then funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit elif [ $found -ge 2 ]; then funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit else deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- ` device="/dev/md$deviceNumber" funDisplay "Device found : $colorStart $device $colorFinish" fi ############ # Warnings # ############ # First, warn about the clearing, and give them a chance to abort funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device ) * Disk $disk will be unmounted first. * Then zeroes will be written to the entire drive. * Parity will be preserved throughout. * Clearing while updating Parity takes a VERY long time! * The progress of the clearing will not be visible until it's done! * When complete, Disk $disk will be ready for removal from array. * Commands to be executed: \t* $colorStart umount $disk $colorFinish \t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish" if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI funDisplay "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi ############# # Preparing # ############# funDisplay "Unmounting $disk ... \r" funDisplay "Unmounting Disk $disk (command: umount $disk ) ..." if `umount $disk`; then # unmount success funDisplay "Disk $disk unmounted successfully." else # unmount failure funDisplay "Disk $disk unmount failed. Exiting." exit #fi ############ # Clearing # ############ funDisplay "Clearing Disk $disk (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..." startTime="`date +%c`" if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then logFile="ls $userscriptstDir/tmpScripts/*/log.txt" dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile else dd bs=1M if=/dev/zero of=$device $ddArg fi endTime="`date +%c`" ######## # Done # ######## funDisplay "Clearing Disk $disk is complete" funDisplay "Started @ $startTime\nFinished @ $endTime" echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error. Unless errors appeared, the drive is now cleared! Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)." funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime" This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3 Edit: Here is my fixed version: #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0first draft # 1.1add logging, improve comments # 1.2adapt for User.Scripts, extend wait to 60 seconds # 1.3add progress display; confirm by key (no wait) if standalone; fix logger # 1.4only add progress display if unRAID version >= 6.2 # 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net. #Change log : #Version check adapted for Unraid 6.10 and up. #Check if unmount successful before clearing. #Send mail notification when clearing finished. #Script adapted for more then 9 drives. #Dead code removed. #Code simplified and optimized. #Ambiguous variables renamed. #Progression of the dd command sent to the GUI log # 2.1 - Fix to clear the found disk ############# # Functions # ############# funDisplay () { echo -e "\n`date +%d/%m" "%T` : $1\n" logger -t$tag "$2" } funMail() { /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2" } ############# # Variables # ############# scriptVersion="2.0" marker="clear-me" tag="clear_array_drive" started=0 ddArg="" #used for dd command found=0 wait=60 scriptDir=$(dirname "$0") userscriptstDir="/tmp/user.scripts" # Colors management if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors. colorStart="<font color=blue>" colorFinish="</font>" else # running from CLI. Sets shell colors colorStart="\x1b[36;01m" colorFinish="\x1b[39;49;00m" fi ########## # Checks # ########## funDisplay "Clear an unRAID array data drive v$scriptVersion." # check unRAID version unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2` majorVersion=`echo $unraidVersion | cut -d "." -f 1` minorVersion=`echo $unraidVersion | cut -d "." -f 2` if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then ddArg="status=progress" else funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)." if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI echo "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi fi # checks if array is started disks=`ls -d /mnt/* | grep disk | grep -v disks` drivesNumber=`echo "$disks" | wc -l` if [ $drivesNumber == 0 ]; then funDisplay "ERROR: Array must be started before using this script. Exiting." exit fi # checks if a disk is ready for clearing funDisplay "Checking all array data drives (may need to spin them up)..." for disk in $disks; do itemsList=`ls -A $disk 2> /dev/null` itemsNumber=`echo "$itemsList" | wc -l` # test for marker and emptiness if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then itemsSize=`du -s $disk | awk '{print $1}'` if [ $itemsSize -eq 0 ]; then foundDisk=$disk ((found++)) fi fi done # No drive or multiple drives found to clear if [ $found -eq 0 ]; then funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit elif [ $found -ge 2 ]; then funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit else disk=$foundDisk deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- ` device="/dev/md$deviceNumber" funDisplay "Device found : $colorStart $device $colorFinish" fi ############ # Warnings # ############ # First, warn about the clearing, and give them a chance to abort funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device ) * Disk $disk will be unmounted first. * Then zeroes will be written to the entire drive. * Parity will be preserved throughout. * Clearing while updating Parity takes a VERY long time! * The progress of the clearing will not be visible until it's done! * When complete, Disk $disk will be ready for removal from array. * Commands to be executed: \t* $colorStart umount $disk $colorFinish \t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish" if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI funDisplay "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi ############# # Preparing # ############# funDisplay "Unmounting $disk ... \r" funDisplay "Unmounting Disk $disk (command: umount $disk ) ..." if `umount $disk`; then # unmount success funDisplay "Disk $disk unmounted successfully." else # unmount failure funDisplay "Disk $disk unmount failed. Exiting." exit fi ############ # Clearing # ############ funDisplay "Clearing Disk $disk (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..." startTime="`date +%c`" if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then logFile="ls $userscriptstDir/tmpScripts/*/log.txt" dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile else dd bs=1M if=/dev/zero of=$device $ddArg fi endTime="`date +%c`" ######## # Done # ######## funDisplay "Clearing Disk $disk is complete" funDisplay "Started @ $startTime\nFinished @ $endTime" echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error. Unless errors appeared, the drive is now cleared! Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)." funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime" Edited September 15, 2022 by inh add fixed script 1 Quote Link to comment
flyize Posted September 23, 2022 Share Posted September 23, 2022 On 9/15/2022 at 10:14 AM, inh said: This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3 Edit: Here is my fixed version: #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0first draft # 1.1add logging, improve comments # 1.2adapt for User.Scripts, extend wait to 60 seconds # 1.3add progress display; confirm by key (no wait) if standalone; fix logger # 1.4only add progress display if unRAID version >= 6.2 # 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net. #Change log : #Version check adapted for Unraid 6.10 and up. #Check if unmount successful before clearing. #Send mail notification when clearing finished. #Script adapted for more then 9 drives. #Dead code removed. #Code simplified and optimized. #Ambiguous variables renamed. #Progression of the dd command sent to the GUI log # 2.1 - Fix to clear the found disk ############# # Functions # ############# funDisplay () { echo -e "\n`date +%d/%m" "%T` : $1\n" logger -t$tag "$2" } funMail() { /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2" } ############# # Variables # ############# scriptVersion="2.0" marker="clear-me" tag="clear_array_drive" started=0 ddArg="" #used for dd command found=0 wait=60 scriptDir=$(dirname "$0") userscriptstDir="/tmp/user.scripts" # Colors management if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors. colorStart="<font color=blue>" colorFinish="</font>" else # running from CLI. Sets shell colors colorStart="\x1b[36;01m" colorFinish="\x1b[39;49;00m" fi ########## # Checks # ########## funDisplay "Clear an unRAID array data drive v$scriptVersion." # check unRAID version unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2` majorVersion=`echo $unraidVersion | cut -d "." -f 1` minorVersion=`echo $unraidVersion | cut -d "." -f 2` if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then ddArg="status=progress" else funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)." if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI echo "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi fi # checks if array is started disks=`ls -d /mnt/* | grep disk | grep -v disks` drivesNumber=`echo "$disks" | wc -l` if [ $drivesNumber == 0 ]; then funDisplay "ERROR: Array must be started before using this script. Exiting." exit fi # checks if a disk is ready for clearing funDisplay "Checking all array data drives (may need to spin them up)..." for disk in $disks; do itemsList=`ls -A $disk 2> /dev/null` itemsNumber=`echo "$itemsList" | wc -l` # test for marker and emptiness if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then itemsSize=`du -s $disk | awk '{print $1}'` if [ $itemsSize -eq 0 ]; then foundDisk=$disk ((found++)) fi fi done # No drive or multiple drives found to clear if [ $found -eq 0 ]; then funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit elif [ $found -ge 2 ]; then funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit else disk=$foundDisk deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- ` device="/dev/md$deviceNumber" funDisplay "Device found : $colorStart $device $colorFinish" fi ############ # Warnings # ############ # First, warn about the clearing, and give them a chance to abort funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device ) * Disk $disk will be unmounted first. * Then zeroes will be written to the entire drive. * Parity will be preserved throughout. * Clearing while updating Parity takes a VERY long time! * The progress of the clearing will not be visible until it's done! * When complete, Disk $disk will be ready for removal from array. * Commands to be executed: \t* $colorStart umount $disk $colorFinish \t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish" if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI funDisplay "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi ############# # Preparing # ############# funDisplay "Unmounting $disk ... \r" funDisplay "Unmounting Disk $disk (command: umount $disk ) ..." if `umount $disk`; then # unmount success funDisplay "Disk $disk unmounted successfully." else # unmount failure funDisplay "Disk $disk unmount failed. Exiting." exit fi ############ # Clearing # ############ funDisplay "Clearing Disk $disk (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..." startTime="`date +%c`" if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then logFile="ls $userscriptstDir/tmpScripts/*/log.txt" dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile else dd bs=1M if=/dev/zero of=$device $ddArg fi endTime="`date +%c`" ######## # Done # ######## funDisplay "Clearing Disk $disk is complete" funDisplay "Started @ $startTime\nFinished @ $endTime" echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error. Unless errors appeared, the drive is now cleared! Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)." funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime" Is there a way to run this script in the background, without having to keep the tab open? A quick Google says I need NerdPack to install screen, which is no longer supported. So I'd like to make sure this continues without my input and/or laptop being connected to the Unraid server. Quote Link to comment
fritolays Posted September 28, 2022 Share Posted September 28, 2022 (edited) On 9/23/2022 at 1:37 PM, flyize said: Is there a way to run this script in the background, without having to keep the tab open? A quick Google says I need NerdPack to install screen, which is no longer supported. So I'd like to make sure this continues without my input and/or laptop being connected to the Unraid server. Check out tmux as an alternative to screen. Its available to install via the Nerd Pack plugin. Github - tmux wiki tmux cheat sheet Create new named session tmux new -s sessionname Close current session (same as exit command) Ctrl+D Detach from current session (run in background) Ctrl+B then D List current sessions tmux ls Attach to last session tmux a Attach to specific session tmux a -t sessionname Edited September 28, 2022 by fritolays added basic tmux commands Quote Link to comment
Majyk Oyster Posted October 3, 2022 Share Posted October 3, 2022 On 9/4/2022 at 7:28 PM, kuchta said: There seems to be a pretty major bug here. I tried to run it and it told me it was going to clear the wrong disk. I looked at the code a little closer and it seems like you opted to continue looking for additional disks that were ready to be cleared instead of breaking out of the loop. Unfortunately, this means that if only one disk is found ready to be cleared, it will think it's the last one in the array since the previously identified disk isn't remembered. I "fixed" this by adding a break command at line 113 (after the "((found++))" command). Oddly, the script still didn't work for me even though it identified the correct device. I get an EOF error right when it is about to unmount the drive. I didn't see any errors in the code to cause this, so I'm wondering if it's a copy-and-paste issue or something. I've attached my log in case it's helpful. For reference, disk7 is the one I was trying to clear and disk8 is the one it incorrectly tried to clear first. I don't need the script anymore as I just created my own script with the 2 action lines, but wanted to make sure I reported this issue so that no one accidentally clears the wrong drive. On 9/15/2022 at 4:14 PM, inh said: This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3 Edit: Here is my fixed version: Thanks for your inputs. Seems like I had a brain fart. I guess I debugged it when clearing the last disk of my array. I'm updating my previous post. 1 Quote Link to comment
flyize Posted October 3, 2022 Share Posted October 3, 2022 (edited) On 9/28/2022 at 9:53 AM, fritolays said: Check out tmux as an alternative to screen. Its available to install via the Nerd Pack plugin. Github - tmux wiki tmux cheat sheet Create new named session tmux new -s sessionname Close current session (same as exit command) Ctrl+D Detach from current session (run in background) Ctrl+B then D List current sessions tmux ls Attach to last session tmux a Attach to specific session tmux a -t sessionname That's my problem though. Nerd Pack is no longer supported or available. edit: OH! It seems that tmux is bundled with Unraid now. Edited October 3, 2022 by flyize Quote Link to comment
JorgeB Posted October 3, 2022 Share Posted October 3, 2022 7 minutes ago, flyize said: edit: OH! It seems that tmux is bundled with Unraid now. It's not, but it is with the UD preclear disk plugin. Quote Link to comment
flyize Posted October 3, 2022 Share Posted October 3, 2022 Well then that explains it. Thanks! Quote Link to comment
droopie Posted October 12, 2022 Share Posted October 12, 2022 On 12/7/2016 at 9:20 AM, ClunkClunk said: I started this a few months ago, but never got around to really optimizing it and rotating out old listings or compressing them. But hopefully it'll give you a starting point! #!/bin/sh DISKS=8 COUNTER=1 THEDATE=`date +"%Y-%m-%d_%H-%M-%S"` LOCATION="/boot/trees" echo " " echo " " echo "Starting tree scanning" while [ $COUNTER -le $DISKS ] do echo "Scanning tree for disk$COUNTER/Movies" tree -h /mnt/disk$COUNTER/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for disk$COUNTER/TV" tree -h /mnt/disk$COUNTER/TV >> "$LOCATION/$THEDATE.tv.log" echo "Incrementing disk counter" COUNTER=$[$COUNTER+1] done echo "Scanning tree for cache/Movies" tree -h /mnt/cache/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for cache/TV" tree -h /mnt/cache/TV >> "$LOCATION/$THEDATE.tv.log" echo "All done scanning disks. Beginning scanning shares." echo "Scanning tree for user/Movies" tree -h /mnt/user/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for user/TV" tree -h /mnt/user/TV >> "$LOCATION/$THEDATE.tv.log" echo " " echo " " echo "Complete! Logs stored in $LOCATION" wow! this would have been a great script to have when i lost 3 hard drives. dont know what data got lost. Quote Link to comment
fritolays Posted October 31, 2022 Share Posted October 31, 2022 On 10/3/2022 at 1:51 PM, flyize said: That's my problem though. Nerd Pack is no longer supported or available. edit: OH! It seems that tmux is bundled with Unraid now. My error, new plugin is called NerdTools... but the same idea. Quote Link to comment
Iceman24 Posted November 7, 2022 Share Posted November 7, 2022 On 12/7/2016 at 10:20 AM, ClunkClunk said: I started this a few months ago, but never got around to really optimizing it and rotating out old listings or compressing them. But hopefully it'll give you a starting point! #!/bin/sh DISKS=8 COUNTER=1 THEDATE=`date +"%Y-%m-%d_%H-%M-%S"` LOCATION="/boot/trees" echo " " echo " " echo "Starting tree scanning" while [ $COUNTER -le $DISKS ] do echo "Scanning tree for disk$COUNTER/Movies" tree -h /mnt/disk$COUNTER/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for disk$COUNTER/TV" tree -h /mnt/disk$COUNTER/TV >> "$LOCATION/$THEDATE.tv.log" echo "Incrementing disk counter" COUNTER=$[$COUNTER+1] done echo "Scanning tree for cache/Movies" tree -h /mnt/cache/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for cache/TV" tree -h /mnt/cache/TV >> "$LOCATION/$THEDATE.tv.log" echo "All done scanning disks. Beginning scanning shares." echo "Scanning tree for user/Movies" tree -h /mnt/user/Movies >> "$LOCATION/$THEDATE.movies.log" echo "Scanning tree for user/TV" tree -h /mnt/user/TV >> "$LOCATION/$THEDATE.tv.log" echo " " echo " " echo "Complete! Logs stored in $LOCATION" Thanks for this, although I could use some help understanding how to modify it to use. I have 2 disks I want to do this to. I don't quite understand the COUNTER part. It seems to be going through all the disks, but only one disk is there for me and the other is the cache drive. Also, how it is saving I'm not quite sure about. I just want to let it run on disk1 and cache at directories of my choosing. Quote Link to comment
MwA83 Posted November 23, 2022 Share Posted November 23, 2022 (edited) Hi folks, I'm in trouble with the "clear an array disk" script by RobJ. I used the original one. I followed the shrink array docs instructions closely. This morning I started the script, logged of from GUI and went to work. After work I'd like to verify the clearing process but my server isn't reachable anymore. My router shows unRaid is offline. Was it a mistake to log off from GUI? What should I do now? Power the server off and restart? I'm a bit scary to do this. Sorry, for my English in advance. I'm German. Greetings Marcel Late edit 04.12.22: I powered my server off and forgot about using a script to zeroing out a disk. I let build a new parity. Next time I'm going to use the script in an updated version. Edited December 4, 2022 by MwA83 Quote Link to comment
JoeUnraidUser Posted December 25, 2022 Share Posted December 25, 2022 Script to trim Docker logs to 1 Megabyte. #!/bin/bash size=1M function trimLog { file=$1 temp="$file.$(date +%s%N).tmp" time=$(date --rfc-3339='seconds') before=$(du -sh "$file" | cut -f1) echo -n "$time: $file: $before=>" tail --bytes=$size "$file" > "$temp" chown $(stat -c '%U' "$file"):$(stat -c '%G' "$file") "$temp" chmod $(stat -c "%a" "$file") "$temp" mv "$temp" "$file" after=$(du -sh "$file" | cut -f1) echo "$after" } find "/var/lib/docker/containers" -name "*.log" -size +$size 2>/dev/null | sort -f |\ while read file; do trimLog "$file"; done trimDockerLogs Quote Link to comment
JoeUnraidUser Posted December 25, 2022 Share Posted December 25, 2022 Script to trim white space from the end of each line of text files. #!/bin/bash # Trim white space from the end of each line of text files. if [ $# -eq 0 ] || [ "$1" == "--help" ] then printf "Usage: trimWhite <files>...\n" exit 0 fi for file in "${@}" do printf "$file\n" perl -pi -e 's/\s+$/\n/' "$file" done trimWhite Quote Link to comment
glave Posted December 26, 2022 Share Posted December 26, 2022 Has anyone created a script that can parse the log file looking for a string and then send a notification using unraid's notification system? I'd like to search for "No space left on device" in the log file, just wondering if something similar may already exist before I reinvent the wheel. Quote Link to comment
trurl Posted December 26, 2022 Share Posted December 26, 2022 9 hours ago, glave said: "No space left on device" You can already get Notifications about how full disks are. Disk Settings has utilization thresholds (%) for all disks, and you can override that for individual disks in the Settings for each disk. If you wait for "No space left on device" it is already too late. Quote Link to comment
glave Posted December 26, 2022 Share Posted December 26, 2022 1 hour ago, trurl said: You can already get Notifications about how full disks are. Disk Settings has utilization thresholds (%) for all disks, and you can override that for individual disks in the Settings for each disk. If you wait for "No space left on device" it is already too late. Unfortunately they don't work in some situations, although I haven't tracked down the specifics yet. I have a 50GB minimum set on each share, yet I still have a few disks than continue to fill up past that without triggering warnings. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.