Sherk Posted May 4, 2022 Share Posted May 4, 2022 Hello Everyone, thanks for helping the community with great script. however I have a small request, I have one docker container that always fail or crash for no reason,I tried to solve this issue by making a script that will start container every hour, some time it crash between that time and I lose container function so I need a script to monitor docker container and if it crash/stopped it will directly start again in anytime. Thank you all in advance for your time and effort! Quote Link to comment
Meles Meles Posted May 4, 2022 Share Posted May 4, 2022 3 hours ago, Sherk said: Hello Everyone, thanks for helping the community with great script. however I have a small request, I have one docker container that always fail or crash for no reason,I tried to solve this issue by making a script that will start container every hour, some time it crash between that time and I lose container function so I need a script to monitor docker container and if it crash/stopped it will directly start again in anytime. Thank you all in advance for your time and effort! Quote Link to comment
Sherk Posted May 4, 2022 Share Posted May 4, 2022 (edited) 5 hours ago, Meles Meles said: thanks for your help Edited May 4, 2022 by Sherk Quote Link to comment
Sherk Posted May 4, 2022 Share Posted May 4, 2022 47 minutes ago, Sherk said: thanks for your help I follow you example but it didn't work for me, zigbee2mqtt is the container that I have the issue with it, could you help Quote Link to comment
simo241 Posted May 10, 2022 Share Posted May 10, 2022 Hello Everyone, thank you for helping with this great script. I need a script to copy in automatic all file from an external disk to a folder on unraid. The script need to mount the disk when detected synchronize all file from the mounted disk to a specific folder and then unmount the disk. If same file on the external disk have been deleted , delete as well the one on unraid . I was using till now next cloud but to do that i need to switch on another computer and i would like to avoid it Thank you for your support Quote Link to comment
trurl Posted May 13, 2022 Share Posted May 13, 2022 On 5/10/2022 at 6:48 AM, simo241 said: script to copy in automatic all file from an external disk to a folder on unraid You can probably adapt one of the example scripts for Unassigned Devices. Quote Link to comment
simo241 Posted May 13, 2022 Share Posted May 13, 2022 Can you please indicate the link with all script Quote Link to comment
wgstarks Posted May 13, 2022 Share Posted May 13, 2022 35 minutes ago, simo241 said: Can you please indicate the link with all script https://forums.unraid.net/topic/92462-unassigned-devices-managing-disk-drives-and-remote-shares-outside-of-the-unraid-array/ scroll down to the Scripts section. Quote Link to comment
timrettop Posted June 17, 2022 Share Posted June 17, 2022 On 9/3/2016 at 5:06 PM, RobJ said: #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0 first draft # 1.1 add logging, improve comments # 1.2 adapt for User.Scripts, extend wait to 60 seconds # 1.3 add progress display; confirm by key (no wait) if standalone; fix logger # 1.4 only add progress display if unRAID version >= 6.2 version="1.4" marker="clear-me" found=0 wait=60 p=${0%%$P} # dirname of program p=${p:0:18} q="/tmp/user.scripts/" echo -e "*** Clear an unRAID array data drive *** v$version\n" # Check if array is started ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null if [ $? -ne 0 ] then echo "ERROR: Array must be started before using this script" exit fi # Look for array drive to clear n=0 echo -n "Checking all array data drives (may need to spin them up) ... " if [ "$p" == "$q" ] # running in User.Scripts then echo -e "\n" c="<font color=blue>" c0="</font>" else #set color teal c="\x1b[36;01m" c0="\x1b[39;49;00m" fi for d in /mnt/disk[1-9]* do x=`ls -A $d` z=`du -s $d` y=${z:0:1} # echo -e "d:"$d "x:"${x:0:20} "y:"$y "z:"$z # the test for marker and emptiness if [ "$x" == "$marker" -a "$y" == "0" ] then found=1 break fi let n=n+1 done #echo -e "found:"$found "d:"$d "marker:"$marker "z:"$z "n:"$n # No drives found to clear if [ $found == "0" ] then echo -e "\rChecked $n drives, did not find an empty drive ready and marked for clearing!\n" echo "To use this script, the drive must be completely empty first, no files" echo "or folders left on it. Then a single folder should be created on it" echo "with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen." echo "This script is only for clearing unRAID data drives, in preparation for" echo "removing them from the array. It does not add a Preclear signature." exit fi # check unRAID version v1=`cat /etc/unraid-version` # v1 is 'version="6.2.0-rc5"' (fixme if 6.10.* happens) v2="${v1:9:1}${v1:11:1}" if [[ $v2 -ge 62 ]] then v=" status=progress" else v="" fi #echo -e "v1=$v1 v2=$v2 v=$v\n" # First, warn about the clearing, and give them a chance to abort echo -e "\rFound a marked and empty drive to clear: $c Disk ${d:9} $c0 ( $d ) " echo -e "* Disk ${d:9} will be unmounted first." echo "* Then zeroes will be written to the entire drive." echo "* Parity will be preserved throughout." echo "* Clearing while updating Parity takes a VERY long time!" echo "* The progress of the clearing will not be visible until it's done!" echo "* When complete, Disk ${d:9} will be ready for removal from array." echo -e "* Commands to be executed:\n***** $c umount $d $c0\n***** $c dd bs=1M if=/dev/zero of=/dev/md${d:9} $v $c0\n" if [ "$p" == "$q" ] # running in User.Scripts then echo -e "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else echo -n "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi # Perform the clearing logger -tclear_array_drive "Clear an unRAID array data drive v$version" echo -e "\rUnmounting Disk ${d:9} ..." logger -tclear_array_drive "Unmounting Disk ${d:9} (command: umount $d ) ..." umount $d echo -e "Clearing Disk ${d:9} ..." logger -tclear_array_drive "Clearing Disk ${d:9} (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} $v ) ..." dd bs=1M if=/dev/zero of=/dev/md${d:9} $v #logger -tclear_array_drive "Clearing Disk ${d:9} (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000 ) ..." #dd bs=1M if=/dev/zero of=/dev/md${d:9} status=progress count=1000 seek=1000 # Done logger -tclear_array_drive "Clearing Disk ${d:9} is complete" echo -e "\nA message saying \"error writing ... no space left\" is expected, NOT an error.\n" echo -e "Unless errors appeared, the drive is now cleared!" echo -e "Because the drive is now unmountable, the array should be stopped," echo -e "and the drive removed (or reformatted)." exit @RobJ The version check fails with 6.10 (As you predicted when you created this. This seems to be a valid alternative version check from a variety of version number tests I've tried. It pulls the major and minor version numbers and then tests with those values to determine whether to add the string. I initialized $v as well to simplify it slightly. v="" #Initialize $v # check unRAID version v1=`cat /etc/unraid-version` #v1="version=\"7.0.5\"" # For Testing v2="$(echo $v1 | cut -d \" -f2)" v2M="$(echo $v2 | cut -d '.' -f 1)" v2m="$(echo $v2 | cut -d '.' -f 2)" if [[ $v2M == 6 ]] then if [[ $v2m -ge 2 ]] then v=" status=progress" fi elif [[ $v2M -gt 6 ]] then v=" status=progress" fi #echo -e "v1=$v1 v2=$v2 v2M=$v2M v2m=$v2m v=$v\n" Quote Link to comment
ss1gohan13 Posted July 2, 2022 Share Posted July 2, 2022 On 9/3/2016 at 7:06 PM, RobJ said: Update 1.3 - add display of progress; confirm by key '!' (no wait) if standalone; fix logger; add a bit of color Really appreciate the tip on 'status=progress', looks pretty good. Lots of numbers presented, the ones of interest are the second and the last. If I have closed my browser, is there a way to still see the status of the progress? Quote Link to comment
Darksurf Posted July 11, 2022 Share Posted July 11, 2022 What is up with the zero drive script? It immediately gives up. *** Clear an unRAID array data drive *** v1.4 Checking all array data drives (may need to spin them up) ... Checked 10 drives, did not find an empty drive ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature. Script Finished Jul 10, 2022 19:21.37 Full logs for this script are available at /tmp/user.scripts/tmpScripts/ZeroDisks_ShrinkArray/log.txt ^C [email protected]:~# ls -al /mnt/disk* /mnt/disk1: total 16 drwxrwxrwx 1 nobody users 84 Jul 10 04:30 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx+ 1 nobody users 190 Jul 10 18:28 Docker/ drwxrwxrwx 1 nobody users 14 Jul 4 22:49 Downloads/ drwxrwxrwx 1 nobody users 60 Jul 6 03:26 ZDRIVE/ drwxrwxrwx 1 nobody users 0 Jul 20 2021 appdata/ drwxrwxrwx 1 nobody users 16 Apr 16 2021 home/ drwxrwxrwx 1 nobody users 1884 Jul 9 04:40 system/ drwxrwxrwx 1 nobody users 138 Dec 31 2017 tftp/ /mnt/disk10: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:33 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:33 clear-me/ /mnt/disk2: total 16 drwxrwxrwx 1 nobody users 12 Jul 10 04:30 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx+ 1 nobody users 260 Jul 9 23:38 Docker/ /mnt/disk3: total 16 drwxrwxrwx 1 nobody users 84 Jul 10 04:30 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx+ 1 nobody users 188 Jul 9 23:38 Docker/ drwxrwxrwx 1 nobody users 0 Jul 6 22:31 Downloads/ drwxr-xr-x 1 nobody users 0 May 9 09:08 ISOs/ drwxrwxrwx 1 nobody users 32 Jul 6 22:28 ZDRIVE/ drwxrwxrwx 1 nobody users 0 Jul 20 2021 appdata/ drwxrwxrwx 1 nobody users 16 Jul 6 21:50 home/ drwxrwxrwx 1 nobody users 394 Jul 6 22:31 system/ /mnt/disk4: total 16 drwxrwxrwx 1 nobody users 66 Jul 10 04:30 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx+ 1 nobody users 170 Jul 6 12:48 Docker/ drwxrwxrwx 1 nobody users 8 Jun 5 2021 ZDRIVE/ drwxrwxrwx 1 nobody users 0 Jul 20 2021 appdata/ drwxrwxrwx 1 nobody users 38 Jul 6 12:47 home/ drwxrwxrwx 1 nobody users 96 Jul 6 12:48 system/ drwxrwxrwx 1 nobody users 0 Dec 31 2017 tftp/ /mnt/disk5: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:35 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:35 clear-me/ /mnt/disk6: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:35 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:35 clear-me/ /mnt/disk7: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:34 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:34 clear-me/ /mnt/disk8: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:34 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:34 clear-me/ /mnt/disk9: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:33 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:33 clear-me/ /mnt/disks: total 0 drwxrwxrwt 2 nobody users 40 Jul 10 00:11 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ Quote Link to comment
kizer Posted July 12, 2022 Share Posted July 12, 2022 @Darksurf Did you make sure the target drive was completely empty with the exception of a single folder clear-me Quote Link to comment
Darksurf Posted July 12, 2022 Share Posted July 12, 2022 (edited) 2 hours ago, kizer said: @Darksurf Did you make sure the target drive was completely empty with the exception of a single folder clear-me Yes. If you check my post above you'll see ls -al of disks5-10 look like this: Quote /mnt/disk5: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:35 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:35 clear-me/ /mnt/disk6: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:35 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:35 clear-me/ /mnt/disk7: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:34 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:34 clear-me/ /mnt/disk8: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:34 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:34 clear-me/ /mnt/disk9: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:33 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:33 clear-me/ /mnt/disk10: total 16 drwxrwxrwx 1 nobody users 16 Jul 10 18:33 ./ drwxr-xr-x 18 root root 360 Jul 10 00:12 ../ drwxrwxrwx 1 nobody users 0 Jul 10 18:33 clear-me/ I ran unbalance on these drives 3 times to be sure, checked there were no files left, then did a full rm -rf /mnt/disk#/* on each drive I planned to wipe and then mkdir -p /mnt/disk#/clear-me on every disk I planned to wipe. I'm 100% positive the drives were empty besides the clear-me folder. I doubt its a problem, but these drives are all formatted BTRFS not XFS. It could also be some incompatibility with 6.10.3, not sure. I ended up just removing them from the machine and creating a new config for the array and rebuilding the parity drives. Edited July 12, 2022 by Darksurf Quote Link to comment
Majyk Oyster Posted July 30, 2022 Share Posted July 30, 2022 (edited) Hi there. I needed to use RobJ's "Clear An unRaid Data Drive" script, and noticed it wasn't compatible with Unraid 6.10 (version verification checked only 1 digit). So I dived into the script and found a few things I could update or do differently. Since RobJ doesn't seem to be around anymore, I took the liberty to rewrite most of it. Change log : Version check adapted for Unraid 6.10 and up. Check if unmount successful before clearing. Send mail notification when clearing finished. Script adapted for more then 9 drives. Dead code removed. Code simplified and optimized. Ambiguous variables renamed. Progression of the dd command sent to the GUI log Error message more explicit if more than 1 drive is prepared for clearing. #!/bin/bash # A script to clear an unRAID array drive. It first checks the drive is completely empty, # except for a marker indicating that the user desires to clear the drive. The marker is # that the drive is completely empty except for a single folder named 'clear-me'. # # Array must be started, and drive mounted. There's no other way to verify it's empty. # Without knowing which file system it's formatted with, I can't mount it. # # Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder. # # 1.0first draft # 1.1add logging, improve comments # 1.2adapt for User.Scripts, extend wait to 60 seconds # 1.3add progress display; confirm by key (no wait) if standalone; fix logger # 1.4only add progress display if unRAID version >= 6.2 # 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net. #Change log : #Version check adapted for Unraid 6.10 and up. #Check if unmount successful before clearing. #Send mail notification when clearing finished. #Script adapted for more then 9 drives. #Dead code removed. #Code simplified and optimized. #Ambiguous variables renamed. #Progression of the dd command sent to the GUI log # 2.1 - Fix to clear the found disk ############# # Functions # ############# funDisplay () { echo -e "\n`date +%d/%m" "%T` : $1\n" logger -t$tag "$2" } funMail() { /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2" } ############# # Variables # ############# scriptVersion="2.0" marker="clear-me" tag="clear_array_drive" started=0 ddArg="" #used for dd command found=0 wait=60 scriptDir=$(dirname "$0") userscriptstDir="/tmp/user.scripts" # Colors management if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors. colorStart="<font color=blue>" colorFinish="</font>" else # running from CLI. Sets shell colors colorStart="\x1b[36;01m" colorFinish="\x1b[39;49;00m" fi ########## # Checks # ########## funDisplay "Clear an unRAID array data drive v$scriptVersion." # check unRAID version unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2` majorVersion=`echo $unraidVersion | cut -d "." -f 1` minorVersion=`echo $unraidVersion | cut -d "." -f 2` if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then ddArg="status=progress" else funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)." if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI echo "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi fi # checks if array is started disks=`ls -d /mnt/* | grep disk | grep -v disks` drivesNumber=`echo "$disks" | wc -l` if [ $drivesNumber == 0 ]; then funDisplay "ERROR: Array must be started before using this script. Exiting." exit fi # checks if a disk is ready for clearing funDisplay "Checking all array data drives (may need to spin them up)..." for disk in $disks; do itemsList=`ls -A $disk 2> /dev/null` itemsNumber=`echo "$itemsList" | wc -l` # test for marker and emptiness if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then itemsSize=`du -s $disk | awk '{print $1}'` if [ $itemsSize -eq 0 ]; then foundDisk=$disk ((found++)) fi fi done # No drive or multiple drives found to clear if [ $found -eq 0 ]; then funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit elif [ $found -ge 2 ]; then funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing! To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature." exit else disk=$foundDisk deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- ` device="/dev/md$deviceNumber" funDisplay "Device found : $colorStart $device $colorFinish" fi ############ # Warnings # ############ # First, warn about the clearing, and give them a chance to abort funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device ) * Disk $disk will be unmounted first. * Then zeroes will be written to the entire drive. * Parity will be preserved throughout. * Clearing while updating Parity takes a VERY long time! * The progress of the clearing will not be visible until it's done! * When complete, Disk $disk will be ready for removal from array. * Commands to be executed: \t* $colorStart umount $disk $colorFinish \t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish" if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI funDisplay "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi ############# # Preparing # ############# funDisplay "Unmounting $disk ... \r" funDisplay "Unmounting Disk $disk (command: umount $disk ) ..." if `umount $disk`; then # unmount success funDisplay "Disk $disk unmounted successfully." else # unmount failure funDisplay "Disk $disk unmount failed. Exiting." exit fi ############ # Clearing # ############ funDisplay "Clearing Disk $disk (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..." startTime="`date +%c`" if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then logFile="ls $userscriptstDir/tmpScripts/*/log.txt" dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile else dd bs=1M if=/dev/zero of=$device $ddArg fi endTime="`date +%c`" ######## # Done # ######## funDisplay "Clearing Disk $disk is complete" funDisplay "Started @ $startTime\nFinished @ $endTime" echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error. Unless errors appeared, the drive is now cleared! Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)." funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime" Edited October 3, 2022 by Majyk Oyster 1 1 Quote Link to comment
Majyk Oyster Posted July 30, 2022 Share Posted July 30, 2022 On 7/11/2022 at 2:27 AM, Darksurf said: What is up with the zero drive script? It immediately gives up. Check my post above, I think it might help. Quote Link to comment
Majyk Oyster Posted July 31, 2022 Share Posted July 31, 2022 (edited) After removing devices from the array, I needed to shred any trace of previous data to sell the HDDs, so I made this. #!/bin/bash # The purpose of this script is to remove every trace of the data that was once on a device (aka data shredding), so it can be discarded or sold. Multiple devices can be shredded simultaneously (not one after another). # How the script works : # - Using the Unassigned devices plugin, format and mount (if needed) the devices you want to shred. # - If any data remains on the devices, remove it completely. # - Create a "shred-me" folder on each device you want to shred. Any other device will be ignored. # - Launch the script via the "User Scripts" plugin (preferably in background) or via CLI. # - A list of the devices correctly prepared for shredding will be displayed. # - Devices to shred will be unmounted, and won't be remounted after shredding. # - Shredding will then start on all relevant devices simultaneously. Progression will be visible in real time for each device. # - A notification will be sent on completion, using Unraid notification system. # # Version 1.0 - This is a script by user Majyk Oyster on forums.unraid.net. Some elements of RobJ's Clear An unRaid Data Drive were used. ############# # Functions # ############# funDisplay () { echo -e "\n`date +%d/%m" "%T` : $1\n" logger -t$tag "$2" } funMail() { /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2" } ############# # Variables # ############# # Sets the number of passes. 3 is standard, you can increase the value for even safer data removal. passes=3 scriptVersion="1.0" marker="shred-me" tag="shred_unassigned_device" started=0 found=0 wait=60 scriptDir=$(dirname "$0") userscriptstDir="/tmp/user.scripts" # Colors management if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors. colorStart="<font color=blue>" colorFinish="</font>" else # running from CLI. Sets shell colors colorStart="\x1b[36;01m" colorFinish="\x1b[39;49;00m" fi ########## # Checks # ########## funDisplay "Shredder v$scriptVersion." funDisplay "The purpose of this script is to remove every trace of the data that was once on a device (aka data shredding), so it can be discarded or sold. Multiple devices can be shredded simultaneously." # checks if a disk is ready for shredding disks=`ls -d /mnt/disks/* 2> /dev/null` devicesToShred="" funDisplay "Checking all unassigned devices..." for disk in $disks; do itemsList=`ls -A $disk 2> /dev/null` itemsNumber=`echo "$itemsList" | wc -l` # test for marker and emptiness if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then itemsSize=`du -s $disk | awk '{print $1}'` if [ $itemsSize -eq 0 ]; then disksToShred="$disksToShred $disk" device=`df | grep "$disk" | awk '{print $1}' | sed 's/.$//'` devicesToShred="$devicesToShred $device" ((found++)) fi fi done # No drive or multiple drives found to clear if [ $found -eq 0 ] || [ -z "${devicesToShred// }" ]; then funDisplay "Did not find an empty drive ready and marked for shredding. To use this script, the drive must be completely empty first, no files or folders left on it. Then a single folder should be created on it with the name 'shred-me', exactly 8 characters, 7 lowercase and 1 hyphen. This script is only for safely removing all data from unassigned devices, typically after removing them from the array." exit else funDisplay "Device(s) found : $colorStart $devicesToShred $colorFinish" fi ############ # Warnings # ############ funDisplay "Found marked and empty drive(s) to shred. \t* Device(s) will be unmounted first. \t* Then random data will be written to the entire device(s) $passes time(s). \t* When complete, all data from device(s) will be cleanly erased." echo -e "\n Commands to be executed:" for diskToShred in $disksToShred; do echo -e "\t* $colorStart umount $diskToShred $colorFinish" done for deviceToShred in $devicesToShred; do echo -e "\t* $colorStart shred -vf $deviceToShred $colorFinish" done if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n" sleep $wait else #running from CLI funDisplay "Press ! to proceed. Any other key aborts, with no changes made. " ch="" read -n 1 ch echo -e -n "\r \r" if [ "$ch" != "!" ]; then exit fi fi ############# # Preparing # ############# funDisplay "Unmounting device(s)... \r" for diskToShred in $disksToShred; do if `umount $diskToShred`; then # unmount success funDisplay "Disk $diskToShred unmounted successfully." else # unmount failure funDisplay "Disk $diskToShred unmount failed. Exiting." exit fi done ############# # Shredding # ############# funDisplay "Shredding Device(s)..." startTime="`date +%c`" if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Shredding unassigned devices"`]; then logFile="ls $userscriptstDir/tmpScripts/*/log.txt" else logFile="/tmp/shredLog.txt" touch $logFile fi for deviceToShred in $devicesToShred; do shred -vf $deviceToShred | tee -a $logFile & done wait endTime="`date +%c`" ######## # Done # ######## funDisplay "Shredding is complete" funDisplay "Started @ $startTime\nFinished @ $endTime" funDisplay "Unless errors appeared, the device(s) is/are now shredded!" funMail "Shredding complete on device(s) $devicesToShred." "Shredding complete on following device(s): $devicesToShred Started @ $startTime Finished @ $endTime " Your devices to shred should look something like that before using the script (formated, mounted) : Here's how the script looks like in User Scripts : Edited July 31, 2022 by Majyk Oyster Quote Link to comment
JonathanM Posted July 31, 2022 Share Posted July 31, 2022 10 hours ago, Majyk Oyster said: I needed to shred any trace of previous data to sell the HDDs, so I made this. The existing preclear plugin for Unassigned devices does a good job of erasing drives, with the added benefit of ensuring that the final pass actually was written correctly. Quote Link to comment
Majyk Oyster Posted July 31, 2022 Share Posted July 31, 2022 (edited) 4 hours ago, JonathanM said: The existing preclear plugin for Unassigned devices does a good job of erasing drives, with the added benefit of ensuring that the final pass actually was written correctly. I checked the documentation on that plugin, but if it actually renders the previous data unrecoverable the info eluded me. Zeroing a device doesn't prevent a potentially malicious person from using data recovery tools to obtain potentially private data. The previous magnetic state of the device has to be randomly altered several times to obtain a satisfactory protection. Edited July 31, 2022 by Majyk Oyster Quote Link to comment
JonathanM Posted July 31, 2022 Share Posted July 31, 2022 10 minutes ago, Majyk Oyster said: Zeroing a device doesn't prevent a potentially malicious person from using data recovery tools to obtain potentially private data. Quote The Erase All the Disk option uses random data to wipe out the drive; the resulting drive can't be quickly added to the array. If you want to add if after erase, you must select Erase and Clear the Disk. Quote Link to comment
JonathanM Posted July 31, 2022 Share Posted July 31, 2022 14 minutes ago, Majyk Oyster said: The previous magnetic state of the device has to be randomly altered several times to obtain a satisfactory protection. You can select multiple erase passes with the preclear plugin that write random data to the drive. As many passes as you feel safe with, and as an added bonus you can do a standard clear with verification, which gives you confidence that the final write was actually honored and not a placebo where the write was accepted but the subsequent read still returned old data. Quote Link to comment
Majyk Oyster Posted July 31, 2022 Share Posted July 31, 2022 I just watched Spaceinvader One video about the plugin, and I indeed saw that there was an option for multiple cycles (couldn't find it mentioned in the plugin thread). I don't want to sound like I'm arguing for the sake of it, but I just noticed Squid mentioned on the Preclear plugin thread that it was incompatible with Unraid versions 6.9.0+ and should be uninstalled. So there's that. As for the final check for the actual destruction of the data, the fact that "successful" shred commands could actually fail to destruct the data beyond recognition didn't occur to me. I'd have to look into that. Quote Link to comment
JonathanM Posted July 31, 2022 Share Posted July 31, 2022 31 minutes ago, Majyk Oyster said: I just noticed Squid mentioned on the Preclear plugin thread that it was incompatible with Unraid versions 6.9.0+ and should be uninstalled. This is the replacement option, it remains compatible using mostly the same base code. Read the first post. 1 Quote Link to comment
JonathanM Posted July 31, 2022 Share Posted July 31, 2022 34 minutes ago, Majyk Oyster said: As for the final check for the actual destruction of the data, the fact that "successful" shred commands could actually fail to destruct the data beyond recognition didn't occur to me. I'd have to look into that. If the disk is working properly, it won't happen. However... the situations that call for permanent erasure are often when a disk is no longer working reliably. If the software based erasure isn't reliable, the procedure must be escalated to physical destruction. If you like playing with powerful magnets, orderly disassembly is recommended, followed by shattering the platters. If you don't have time for that, 5.65 or larger orifices through the platters is satisfying and quite reliable. Quote Link to comment
Majyk Oyster Posted July 31, 2022 Share Posted July 31, 2022 (edited) 14 minutes ago, JonathanM said: If the disk is working properly, it won't happen. However... the situations that call for permanent erasure are often when a disk is no longer working reliably. If the software based erasure isn't reliable, the procedure must be escalated to physical destruction. If you like playing with powerful magnets, orderly disassembly is recommended, followed by shattering the platters. If you don't have time for that, 5.65 or larger orifices through the platters is satisfying and quite reliable. I actually have access to an actual certified disk shredder. I work for a managed security services provider, specialized in hosting critical data (for hospitals, health insurances, ministry of defense...). I haven't worked as a sys admin for a while, but my colleagues in the datacenters can turn my drives into coarse powder if need be. Edited July 31, 2022 by Majyk Oyster 1 Quote Link to comment
kizer Posted August 1, 2022 Share Posted August 1, 2022 Us Poor Folk would just use a drill or a hammer. lol 1 Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.