Additional Scripts For User.Scripts Plugin

Majyk Oyster · August 4, 2022

11 hours ago, hernandito said:

Has anyone created a script that renews the LetsEncrypt certificate? Would love to implement.

thank you.

Do you need the certificate updated for the Unraid server or for a hosted VM/container ?

Edited August 4, 2022 by Majyk Oyster

hernandito · August 5, 2022

4 hours ago, Majyk Oyster said:

Do you need the certificate updated for the Unraid server or for a hosted VM/container ?

I would like it for the unRAID server. I'm sure at some point I will forget about it and then it may expire.

Thank you!

H.

AverageMarcus · August 7, 2022

I made a small update to the `clear_array_drive` script to support setting `status=progress` for unRAID >= 6.10.

#!/bin/bash
# A script to clear an unRAID array drive.  It first checks the drive is completely empty,
# except for a marker indicating that the user desires to clear the drive.  The marker is
# that the drive is completely empty except for a single folder named 'clear-me'.
#
# Array must be started, and drive mounted.  There's no other way to verify it's empty.
# Without knowing which file system it's formatted with, I can't mount it.
#
# Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder.
#
# 1.0  first draft
# 1.1  add logging, improve comments
# 1.2  adapt for User.Scripts, extend wait to 60 seconds
# 1.3  add progress display; confirm by key (no wait) if standalone; fix logger
# 1.4  only add progress display if unRAID version >= 6.2
# 1.5  fix progress display if unRAID version >= 6.10

version="1.5"
marker="clear-me"
found=0
wait=60
p=${0%%$P}              # dirname of program
p=${p:0:18}
q="/tmp/user.scripts/"

echo -e "*** Clear an unRAID array data drive ***  v$version\n"

# Check if array is started
ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null
if [ $? -ne 0 ]
then
   echo "ERROR:  Array must be started before using this script"
   exit
fi

# Look for array drive to clear
n=0
echo -n "Checking all array data drives (may need to spin them up) ... "
if [ "$p" == "$q" ] # running in User.Scripts
then
   echo -e "\n"
   c="<font color=blue>"
   c0="</font>"
else #set color teal
   c="\x1b[36;01m"
   c0="\x1b[39;49;00m"
fi

for d in /mnt/disk[1-9]*
do
   x=`ls -A $d`
   z=`du -s $d`
   y=${z:0:1}
#   echo -e "d:"$d "x:"${x:0:20} "y:"$y "z:"$z

   # the test for marker and emptiness
   if [ "$x" == "$marker" -a "$y" == "0" ]
   then
      found=1
      break
   fi
   let n=n+1
done

#echo -e "found:"$found "d:"$d "marker:"$marker "z:"$z "n:"$n

# No drives found to clear
if [ $found == "0" ]
then
   echo -e "\rChecked $n drives, did not find an empty drive ready and marked for clearing!\n"
   echo "To use this script, the drive must be completely empty first, no files"
   echo "or folders left on it.  Then a single folder should be created on it"
   echo "with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen."
   echo "This script is only for clearing unRAID data drives, in preparation for"
   echo "removing them from the array.  It does not add a Preclear signature."
   exit
fi

# check unRAID version
v1=(`cat /etc/unraid-version | awk -F'"' '{ print $2 }' | tr '.' ' '`)
if [[ ${v1[0]} -eq 6 ]] && [[ ${v1[1]} -ge 2 ]]
then
   v=" status=progress"
else
   v=""
fi

# First, warn about the clearing, and give them a chance to abort
echo -e "\rFound a marked and empty drive to clear: $c Disk ${d:9} $c0 ( $d ) "
echo -e "* Disk ${d:9} will be unmounted first."
echo "* Then zeroes will be written to the entire drive."
echo "* Parity will be preserved throughout."
echo "* Clearing while updating Parity takes a VERY long time!"
echo "* The progress of the clearing will not be visible until it's done!"
echo "* When complete, Disk ${d:9} will be ready for removal from array."
echo -e "* Commands to be executed:\n***** $c umount $d $c0\n***** $c dd bs=1M if=/dev/zero of=/dev/md${d:9} $v $c0\n"
if [ "$p" == "$q" ] # running in User.Scripts
then
   echo -e "You have $wait seconds to cancel this script (click the red X, top right)\n"
   sleep $wait
else
   echo -n "Press ! to proceed. Any other key aborts, with no changes made. "
   ch=""
   read -n 1 ch
   echo -e -n "\r                                                                  \r"
   if [ "$ch" != "!" ];
   then
      exit
   fi
fi

# Perform the clearing
logger -tclear_array_drive "Clear an unRAID array data drive  v$version"
echo -e "\rUnmounting Disk ${d:9} ..."
logger -tclear_array_drive "Unmounting Disk ${d:9}  (command: umount $d ) ..."
umount $d
echo -e "Clearing   Disk ${d:9} ..."
logger -tclear_array_drive "Clearing Disk ${d:9}  (command: dd bs=1M if=/dev/zero of=/dev/md${d:9} $v ) ..."
dd bs=1M if=/dev/zero of=/dev/md${d:9} $v

# Done
logger -tclear_array_drive "Clearing Disk ${d:9} is complete"
echo -e "\nA message saying \"error writing ... no space left\" is expected, NOT an error.\n"
echo -e "Unless errors appeared, the drive is now cleared!"
echo -e "Because the drive is now unmountable, the array should be stopped,"
echo -e "and the drive removed (or reformatted)."
exit

kri kri · August 7, 2022

I am looking for a way to move data around my server, similar to how Hazel works on my mac. I would like it to be based on file type (video) and part of a filename. So basically look inside a folder, and its subfolders to see if the syntax matches and then move to another folder on my server. I have everything under /data per trash guides so I would imagine it would be pretty quick, but I don't really care about performance since I will just probably run it once a day or so.

Anyone have something like this?

Edited August 7, 2022 by kri kri

TheLargeFries · August 8, 2022

On 8/4/2022 at 7:43 AM, hernandito said:

Has anyone created a script that renews the LetsEncrypt certificate? Would love to implement.

thank you.

I just pull my wildcard cert from my nginx proxy manager docker (the container handles the actual renewal with lets encrypt). then I use the scripts plugin to run this weekly:

#!/bin/bash
# Check if array is started
ls /mnt/disk[1-9]* 1>/dev/null 2>/dev/null
if [ $? -ne 0 ]
then
   echo $(date -u) " LE SCRIPT - ERROR: Array must be started before using this script"
   exit
else
   echo "Moving cert from NGINX Proxy Manager docker"
   cp /mnt/cache/appdata/Nginx-Proxy-Manager-Official/letsencrypt/live/npm-1/fullchain.pem /boot/config/ssl/certs/unas_unraid_bundle.pem
   cat /mnt/cache/appdata/Nginx-Proxy-Manager-Official/letsencrypt/live/npm-1/privkey.pem >> /boot/config/ssl/certs/unas_unraid_bundle.pem
   echo "Reloading nginx service"
   /etc/rc.d/rc.nginx reload
   echo $(date -u) " LE SCRIPT - COMPLETE"
fi

Majyk Oyster · August 8, 2022

On 8/7/2022 at 2:03 PM, AverageMarcus said:

I made a small update to the `clear_array_drive` script to support setting `status=progress` for unRAID >= 6.10.

I did that and more changes a few does ago, you can see it on the previous page.

There's so many links pointing at RobJ's initial post, that would be nice if there was a way to mention updates.

Digital Shamans · August 30, 2022

On 7/23/2016 at 7:00 PM, Squid said:
Run mover at a certain threshold of cache drive utilization.

Adjust the value to move at within the script. Really only makes sense to use this script as a scheduled operation, and would have to be set to a frequency (hourly?) more often than how often mover itself runs normally.
#!/usr/bin/php
<?PHP

$moveAt = 70;    # Adjust this value to suit.

$diskTotal = disk_total_space("/mnt/cache");
$diskFree = disk_free_space("/mnt/cache");
$percent = ($diskTotal - $diskFree) / $diskTotal * 100;

if ( $percent > $moveAt ) {
  exec("/usr/local/sbin/mover");
}
?>

On 7/23/2016 at 7:00 PM, Squid said:

run_mover_at_threshold.zip 717 B · 168 downloads

Thanks for the script!

Do you have any suggestion, how to avoid these error when run the script?

Script location: /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script
Note that closing this window will abort the execution of this script

Warning: disk_total_space(): No such file or directory in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 6
Warning: disk_free_space(): No such file or directory in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 7
Warning: Division by zero in /tmp/user.scripts/tmpScripts/Run-Mover-At-Disk-Utilization-Threshold/script on line 8

kuchta · September 4, 2022

On 7/29/2022 at 10:20 PM, Majyk Oyster said:

Hi there.

I needed to use RobJ's "Clear An unRaid Data Drive" script, and noticed it wasn't compatible with Unraid 6.10 (version verification checked only 1 digit). So I dived into the script and found a few things I could update or do differently.

Since RobJ doesn't seem to be around anymore, I took the liberty to rewrite most of it.

There seems to be a pretty major bug here. I tried to run it and it told me it was going to clear the wrong disk. I looked at the code a little closer and it seems like you opted to continue looking for additional disks that were ready to be cleared instead of breaking out of the loop. Unfortunately, this means that if only one disk is found ready to be cleared, it will think it's the last one in the array since the previously identified disk isn't remembered. I "fixed" this by adding a break command at line 113 (after the "((found++))" command). Oddly, the script still didn't work for me even though it identified the correct device. I get an EOF error right when it is about to unmount the drive. I didn't see any errors in the code to cause this, so I'm wondering if it's a copy-and-paste issue or something. I've attached my log in case it's helpful. For reference, disk7 is the one I was trying to clear and disk8 is the one it incorrectly tried to clear first.

I don't need the script anymore as I just created my own script with the 2 action lines, but wanted to make sure I reported this issue so that no one accidentally clears the wrong drive.

clear_array_drive_error.txt

Nanobug · September 14, 2022

On 2/25/2017 at 9:41 PM, SpaceInvaderOne said:

As nested vms have been disabled by default in 6.3 due to the issue with avast and windows vms when enabled.

I have made 2 scripts to turn it on and off. In script set cpu type 1 for intel 2 for amd.

Download link for both scripts https://www.dropbox.com/s/0b1tvotvl6y80uy/nested vms scripts.zip?dl=0

nested vm on script

#!/bin/bash

#set whether your cpu is Intel or AMD  [1-Intel] [2-Amd] 
cputype="1"

#Do not edit below this line
#turn nested on


	if [[ "$cputype" =~ ^(1|2)$ ]]; then

		if [ "$cputype" -eq 1 ]; then
			
			modprobe -r kvm_intel
				modprobe kvm_intel nested=1
				
				echo "Nested vms are enabled for intel cpus"


			elif [ "$pushnotifications" -eq 2 ]; then

			modprobe -r kvm_amd
					modprobe kvm_amd nested=1
				
					echo "Nested vms are enabled for AMD cpus"			

			fi

		else
			
				echo "invalid cpu type set please check config"
			
			fi


sleep 4
exit

and nested vm off script

#!/bin/bash
#set whether your cpu is Intel or AMD  [1-Intel] [2-Amd] 
cputype="1"
#Do not edit below this line
#turn nested off

    if [[ "$cputype" =~ ^(1|2)$ ]]; then
        if [ "$cputype" -eq 1 ]; then
            modprobe -r kvm_intel
                modprobe kvm_intel nested=0
                
                echo "Nested vms are disabled for intel cpus"

            elif [ "$pushnotifications" -eq 2 ]; then
            modprobe -r kvm_amd
                    modprobe kvm_amd nested=0
                
                    echo "Nested vms are disabled for AMD cpus"            
            fi
        else
            
                echo "invalid cpu type set please check config"
            
            fi

sleep 4
exit

I get this error when I run the script:

/tmp/user.scripts/tmpScripts/Nested VM on/script: line 20: [: : integer expression expected

I tried looking for an updated version, but wasn't able to find one.

Line 20 should be:

			elif [ "$pushnotifications" -eq 2 ]; then

Can anyone help?

inh · September 15, 2022

On 7/29/2022 at 4:20 PM, Majyk Oyster said:

Hi there.

I needed to use RobJ's "Clear An unRaid Data Drive" script, and noticed it wasn't compatible with Unraid 6.10 (version verification checked only 1 digit). So I dived into the script and found a few things I could update or do differently.

Since RobJ doesn't seem to be around anymore, I took the liberty to rewrite most of it.

Change log :
Version check adapted for Unraid 6.10 and up.
Check if unmount successful before clearing.
Send mail notification when clearing finished.
Script adapted for more then 9 drives.
Dead code removed.
Code simplified and optimized.
Ambiguous variables renamed.
Progression of the dd command sent to the GUI log

Error message more explicit if more than 1 drive is prepared for clearing.

#!/bin/bash
# A script to clear an unRAID array drive.  It first checks the drive is completely empty,
# except for a marker indicating that the user desires to clear the drive.  The marker is
# that the drive is completely empty except for a single folder named 'clear-me'.
#
# Array must be started, and drive mounted.  There's no other way to verify it's empty.
# Without knowing which file system it's formatted with, I can't mount it.
#
# Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder.
#
# 1.0first draft
# 1.1add logging, improve comments
# 1.2adapt for User.Scripts, extend wait to 60 seconds
# 1.3add progress display; confirm by key (no wait) if standalone; fix logger
# 1.4only add progress display if unRAID version >= 6.2
# 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net.
	#Change log :
	#Version check adapted for Unraid  6.10 and up.
	#Check if unmount successful before clearing.
	#Send mail notification when clearing finished.
	#Script adapted for more then 9 drives.
	#Dead code removed.
	#Code simplified and optimized.
	#Ambiguous variables renamed.
	#Progression of the dd command sent to the GUI log 

  #############
 # Functions #
#############

funDisplay () {
	echo -e "\n`date +%d/%m" "%T` : $1\n"
	logger -t$tag "$2"
}

funMail() {
     /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2"
}

  #############
 # Variables #
#############

scriptVersion="2.0"
marker="clear-me"
tag="clear_array_drive"
started=0
ddArg="" #used for dd command
found=0
wait=60
scriptDir=$(dirname "$0")
userscriptstDir="/tmp/user.scripts"
# Colors management
if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors.
	colorStart="<font color=blue>"
	colorFinish="</font>"
else # running from CLI. Sets shell colors
	colorStart="\x1b[36;01m"
	colorFinish="\x1b[39;49;00m"
fi

  ##########
 # Checks #
##########

funDisplay "Clear an unRAID array data drive v$scriptVersion."

# check unRAID version
unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2`
majorVersion=`echo $unraidVersion | cut -d "." -f 1`
minorVersion=`echo $unraidVersion | cut -d "." -f 2`

if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then
	ddArg="status=progress"
else
	funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)."
	if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
		funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
		sleep $wait
	else #running from CLI
		echo  "Press ! to proceed. Any other key aborts, with no changes made. "
		ch=""
		read -n 1 ch
		echo -e -n "\r                                                                  \r"
		if [ "$ch" != "!" ]; then
			exit
		fi
	fi
fi

# checks if array is started
disks=`ls -d /mnt/* | grep disk | grep -v disks`
drivesNumber=`echo "$disks" | wc -l`

if [ $drivesNumber == 0 ]; then
	funDisplay "ERROR:  Array must be started before using this script. Exiting."
	exit
fi

# checks if a disk is ready for clearing

funDisplay "Checking all array data drives (may need to spin them up)..."

for disk in $disks; do
	itemsList=`ls -A $disk 2> /dev/null`
	itemsNumber=`echo "$itemsList" | wc -l`

	# test for marker and emptiness
	if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then
		itemsSize=`du -s $disk | awk '{print $1}'`
		if [ $itemsSize -eq 0 ]; then
			((found++))
		fi
	fi
done

# No drive or multiple drives found to clear
if [ $found -eq 0 ]; then
	funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it.
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
elif [ $found -ge 2 ]; then
	funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it. 
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
else
	deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- `
	device="/dev/md$deviceNumber"
	funDisplay "Device found : $colorStart $device $colorFinish"
fi


  ############
 # Warnings #
############

# First, warn about the clearing, and give them a chance to abort
funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device )
* Disk $disk will be unmounted first.
* Then zeroes will be written to the entire drive.
* Parity will be preserved throughout.
* Clearing while updating Parity takes a VERY long time!
* The progress of the clearing will not be visible until it's done!
* When complete, Disk $disk will be ready for removal from array.
* Commands to be executed:
\t* $colorStart umount $disk $colorFinish
\t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish"

if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
	funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
	sleep $wait
else #running from CLI
	funDisplay "Press ! to proceed. Any other key aborts, with no changes made. "
	ch=""
	read -n 1 ch
	echo -e -n "\r                                                                  \r"
	if [ "$ch" != "!" ]; then
		exit
	fi
fi

  #############
 # Preparing #
#############

funDisplay "Unmounting $disk ... \r"

funDisplay "Unmounting Disk $disk (command: umount $disk ) ..."
if `umount $disk`; then # unmount success
	funDisplay "Disk $disk unmounted successfully."
else # unmount failure
	funDisplay "Disk $disk unmount failed. Exiting."
	exit
#fi

  ############
 # Clearing #
############

funDisplay "Clearing Disk $disk  (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..."
startTime="`date +%c`"

if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then 
	logFile="ls $userscriptstDir/tmpScripts/*/log.txt"
	dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile
else
	dd bs=1M if=/dev/zero of=$device $ddArg
fi

endTime="`date +%c`"

  ########
 # Done #
########

funDisplay "Clearing Disk $disk is complete"
funDisplay "Started @ $startTime\nFinished @ $endTime"
echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error.
Unless errors appeared, the drive is now cleared!
Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)."

funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime"

This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3

Edit:
Here is my fixed version:

#!/bin/bash
# A script to clear an unRAID array drive.  It first checks the drive is completely empty,
# except for a marker indicating that the user desires to clear the drive.  The marker is
# that the drive is completely empty except for a single folder named 'clear-me'.
#
# Array must be started, and drive mounted.  There's no other way to verify it's empty.
# Without knowing which file system it's formatted with, I can't mount it.
#
# Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder.
#
# 1.0first draft
# 1.1add logging, improve comments
# 1.2adapt for User.Scripts, extend wait to 60 seconds
# 1.3add progress display; confirm by key (no wait) if standalone; fix logger
# 1.4only add progress display if unRAID version >= 6.2
# 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net.
	#Change log :
	#Version check adapted for Unraid  6.10 and up.
	#Check if unmount successful before clearing.
	#Send mail notification when clearing finished.
	#Script adapted for more then 9 drives.
	#Dead code removed.
	#Code simplified and optimized.
	#Ambiguous variables renamed.
	#Progression of the dd command sent to the GUI log 
# 2.1 - Fix to clear the found disk

  #############
 # Functions #
#############

funDisplay () {
	echo -e "\n`date +%d/%m" "%T` : $1\n"
	logger -t$tag "$2"
}

funMail() {
     /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2"
}

  #############
 # Variables #
#############

scriptVersion="2.0"
marker="clear-me"
tag="clear_array_drive"
started=0
ddArg="" #used for dd command
found=0
wait=60
scriptDir=$(dirname "$0")
userscriptstDir="/tmp/user.scripts"
# Colors management
if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors.
	colorStart="<font color=blue>"
	colorFinish="</font>"
else # running from CLI. Sets shell colors
	colorStart="\x1b[36;01m"
	colorFinish="\x1b[39;49;00m"
fi

  ##########
 # Checks #
##########

funDisplay "Clear an unRAID array data drive v$scriptVersion."

# check unRAID version
unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2`
majorVersion=`echo $unraidVersion | cut -d "." -f 1`
minorVersion=`echo $unraidVersion | cut -d "." -f 2`

if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then
	ddArg="status=progress"
else
	funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)."
	if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
		funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
		sleep $wait
	else #running from CLI
		echo  "Press ! to proceed. Any other key aborts, with no changes made. "
		ch=""
		read -n 1 ch
		echo -e -n "\r                                                                  \r"
		if [ "$ch" != "!" ]; then
			exit
		fi
	fi
fi

# checks if array is started
disks=`ls -d /mnt/* | grep disk | grep -v disks`
drivesNumber=`echo "$disks" | wc -l`

if [ $drivesNumber == 0 ]; then
	funDisplay "ERROR:  Array must be started before using this script. Exiting."
	exit
fi

# checks if a disk is ready for clearing

funDisplay "Checking all array data drives (may need to spin them up)..."

for disk in $disks; do
	itemsList=`ls -A $disk 2> /dev/null`
	itemsNumber=`echo "$itemsList" | wc -l`

	# test for marker and emptiness
	if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then
		itemsSize=`du -s $disk | awk '{print $1}'`
		if [ $itemsSize -eq 0 ]; then
    		foundDisk=$disk
			((found++))
		fi
	fi
done

# No drive or multiple drives found to clear
if [ $found -eq 0 ]; then
	funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it.
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
elif [ $found -ge 2 ]; then
	funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it. 
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
else
    disk=$foundDisk
	deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- `
	device="/dev/md$deviceNumber"
	funDisplay "Device found : $colorStart $device $colorFinish"
fi


  ############
 # Warnings #
############

# First, warn about the clearing, and give them a chance to abort
funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device )
* Disk $disk will be unmounted first.
* Then zeroes will be written to the entire drive.
* Parity will be preserved throughout.
* Clearing while updating Parity takes a VERY long time!
* The progress of the clearing will not be visible until it's done!
* When complete, Disk $disk will be ready for removal from array.
* Commands to be executed:
\t* $colorStart umount $disk $colorFinish
\t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish"

if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
	funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
	sleep $wait
else #running from CLI
	funDisplay "Press ! to proceed. Any other key aborts, with no changes made. "
	ch=""
	read -n 1 ch
	echo -e -n "\r                                                                  \r"
	if [ "$ch" != "!" ]; then
		exit
	fi
fi

  #############
 # Preparing #
#############

funDisplay "Unmounting $disk ... \r"

funDisplay "Unmounting Disk $disk (command: umount $disk ) ..."
if `umount $disk`; then # unmount success
	funDisplay "Disk $disk unmounted successfully."
else # unmount failure
	funDisplay "Disk $disk unmount failed. Exiting."
	exit
fi

  ############
 # Clearing #
############

funDisplay "Clearing Disk $disk  (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..."
startTime="`date +%c`"

if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then 
	logFile="ls $userscriptstDir/tmpScripts/*/log.txt"
	dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile
else
	dd bs=1M if=/dev/zero of=$device $ddArg
fi

endTime="`date +%c`"

  ########
 # Done #
########

funDisplay "Clearing Disk $disk is complete"
funDisplay "Started @ $startTime\nFinished @ $endTime"
echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error.
Unless errors appeared, the drive is now cleared!
Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)."

funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime"

Edited September 15, 2022 by inh
add fixed script

flyize · September 23, 2022

On 9/15/2022 at 10:14 AM, inh said:

This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3

Edit:
Here is my fixed version:

#!/bin/bash
# A script to clear an unRAID array drive.  It first checks the drive is completely empty,
# except for a marker indicating that the user desires to clear the drive.  The marker is
# that the drive is completely empty except for a single folder named 'clear-me'.
#
# Array must be started, and drive mounted.  There's no other way to verify it's empty.
# Without knowing which file system it's formatted with, I can't mount it.
#
# Quick way to prep drive: format with ReiserFS, then add 'clear-me' folder.
#
# 1.0first draft
# 1.1add logging, improve comments
# 1.2adapt for User.Scripts, extend wait to 60 seconds
# 1.3add progress display; confirm by key (no wait) if standalone; fix logger
# 1.4only add progress display if unRAID version >= 6.2
# 2.0 - This is an update/fork of RobJ script by user Majyk Oyster on forums.unraid.net.
	#Change log :
	#Version check adapted for Unraid  6.10 and up.
	#Check if unmount successful before clearing.
	#Send mail notification when clearing finished.
	#Script adapted for more then 9 drives.
	#Dead code removed.
	#Code simplified and optimized.
	#Ambiguous variables renamed.
	#Progression of the dd command sent to the GUI log 
# 2.1 - Fix to clear the found disk

  #############
 # Functions #
#############

funDisplay () {
	echo -e "\n`date +%d/%m" "%T` : $1\n"
	logger -t$tag "$2"
}

funMail() {
     /usr/local/emhttp/webGui/scripts/notify -i normal -s "$1" -d "$2"
}

  #############
 # Variables #
#############

scriptVersion="2.0"
marker="clear-me"
tag="clear_array_drive"
started=0
ddArg="" #used for dd command
found=0
wait=60
scriptDir=$(dirname "$0")
userscriptstDir="/tmp/user.scripts"
# Colors management
if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running from User.Scripts GUI. Sets html colors.
	colorStart="<font color=blue>"
	colorFinish="</font>"
else # running from CLI. Sets shell colors
	colorStart="\x1b[36;01m"
	colorFinish="\x1b[39;49;00m"
fi

  ##########
 # Checks #
##########

funDisplay "Clear an unRAID array data drive v$scriptVersion."

# check unRAID version
unraidVersion=`cat /etc/unraid-version | cut -d '"' -f 2`
majorVersion=`echo $unraidVersion | cut -d "." -f 1`
minorVersion=`echo $unraidVersion | cut -d "." -f 2`

if [ $majorVersion -eq 6 -a $minorVersion -ge 2 ]; then
	ddArg="status=progress"
else
	funDisplay "This script was not validated for this version of Unraid ($majorVersion.$minorVersion.x)."
	if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
		funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
		sleep $wait
	else #running from CLI
		echo  "Press ! to proceed. Any other key aborts, with no changes made. "
		ch=""
		read -n 1 ch
		echo -e -n "\r                                                                  \r"
		if [ "$ch" != "!" ]; then
			exit
		fi
	fi
fi

# checks if array is started
disks=`ls -d /mnt/* | grep disk | grep -v disks`
drivesNumber=`echo "$disks" | wc -l`

if [ $drivesNumber == 0 ]; then
	funDisplay "ERROR:  Array must be started before using this script. Exiting."
	exit
fi

# checks if a disk is ready for clearing

funDisplay "Checking all array data drives (may need to spin them up)..."

for disk in $disks; do
	itemsList=`ls -A $disk 2> /dev/null`
	itemsNumber=`echo "$itemsList" | wc -l`

	# test for marker and emptiness
	if [ $itemsNumber -eq 1 -a "$itemsList" == "$marker" ]; then
		itemsSize=`du -s $disk | awk '{print $1}'`
		if [ $itemsSize -eq 0 ]; then
    		foundDisk=$disk
			((found++))
		fi
	fi
done

# No drive or multiple drives found to clear
if [ $found -eq 0 ]; then
	funDisplay "Checked $drivesNumber drives, did not find an empty drive ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it.
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
elif [ $found -ge 2 ]; then
	funDisplay "Checked $drivesNumber drives, found multiple empty drives ready and marked for clearing!
	To use this script, the drive must be completely empty first, no files or folders left on it. 
	Then a single folder should be created on it with the name 'clear-me', exactly 8 characters, 7 lowercase and 1 hyphen.
	This script is only for clearing unRAID data drives, in preparation for removing them from the array. It does not add a Preclear signature."
	exit
else
    disk=$foundDisk
	deviceNumber=`echo $disk | cut -d "/" -f 3 | cut -c 5- `
	device="/dev/md$deviceNumber"
	funDisplay "Device found : $colorStart $device $colorFinish"
fi


  ############
 # Warnings #
############

# First, warn about the clearing, and give them a chance to abort
funDisplay "Found a marked and empty drive to clear: $colorStart Disk $deviceNumber $colorFinish ( $device )
* Disk $disk will be unmounted first.
* Then zeroes will be written to the entire drive.
* Parity will be preserved throughout.
* Clearing while updating Parity takes a VERY long time!
* The progress of the clearing will not be visible until it's done!
* When complete, Disk $disk will be ready for removal from array.
* Commands to be executed:
\t* $colorStart umount $disk $colorFinish
\t* $colorStart dd bs=1M if=/dev/zero of=$device $ddArg $colorFinish"

if [[ "$scriptDir" == "$userscriptstDir"* ]]; then # running in User.Scripts
	funDisplay "You have $wait seconds to cancel this script (click the red X, top right)\n"
	sleep $wait
else #running from CLI
	funDisplay "Press ! to proceed. Any other key aborts, with no changes made. "
	ch=""
	read -n 1 ch
	echo -e -n "\r                                                                  \r"
	if [ "$ch" != "!" ]; then
		exit
	fi
fi

  #############
 # Preparing #
#############

funDisplay "Unmounting $disk ... \r"

funDisplay "Unmounting Disk $disk (command: umount $disk ) ..."
if `umount $disk`; then # unmount success
	funDisplay "Disk $disk unmounted successfully."
else # unmount failure
	funDisplay "Disk $disk unmount failed. Exiting."
	exit
fi

  ############
 # Clearing #
############

funDisplay "Clearing Disk $disk  (command: dd bs=1M if=/dev/zero of=$device $ddArg ) ..."
startTime="`date +%c`"

if [ ! -z `cat $userscriptstDir/tmpScripts/*/log.txt 2> /dev/null | grep "Clear an unRAID array data drive"`]; then 
	logFile="ls $userscriptstDir/tmpScripts/*/log.txt"
	dd bs=1M if=/dev/zero of=$device $ddArg >> $logFile
else
	dd bs=1M if=/dev/zero of=$device $ddArg
fi

endTime="`date +%c`"

  ########
 # Done #
########

funDisplay "Clearing Disk $disk is complete"
funDisplay "Started @ $startTime\nFinished @ $endTime"
echo -e "A message saying \"error writing ... no space left\" is expected, NOT an error.
Unless errors appeared, the drive is now cleared!
Because the drive is now unmountable, the array should be stopped, and the drive removed (or reformatted)."

funMail "Clearing complete on $device." "Clearing complete on $device.\nStarted @ $startTime\nFinished @ $endTime"

Is there a way to run this script in the background, without having to keep the tab open? A quick Google says I need NerdPack to install screen, which is no longer supported. So I'd like to make sure this continues without my input and/or laptop being connected to the Unraid server.

fritolays · September 28, 2022

On 9/23/2022 at 1:37 PM, flyize said:

Is there a way to run this script in the background, without having to keep the tab open? A quick Google says I need NerdPack to install screen, which is no longer supported. So I'd like to make sure this continues without my input and/or laptop being connected to the Unraid server.

Check out tmux as an alternative to screen.

Its available to install via the Nerd Pack plugin.

Github - tmux wiki

tmux cheat sheet

Create new named session
  tmux new -s sessionname

Close current session (same as exit command)
  Ctrl+D

Detach from current session (run in background)
  Ctrl+B then D

List current sessions
  tmux ls

Attach to last session
  tmux a

Attach to specific session
  tmux a -t sessionname

Edited September 28, 2022 by fritolays
added basic tmux commands

Majyk Oyster · October 3, 2022

On 9/4/2022 at 7:28 PM, kuchta said:

There seems to be a pretty major bug here. I tried to run it and it told me it was going to clear the wrong disk. I looked at the code a little closer and it seems like you opted to continue looking for additional disks that were ready to be cleared instead of breaking out of the loop. Unfortunately, this means that if only one disk is found ready to be cleared, it will think it's the last one in the array since the previously identified disk isn't remembered. I "fixed" this by adding a break command at line 113 (after the "((found++))" command). Oddly, the script still didn't work for me even though it identified the correct device. I get an EOF error right when it is about to unmount the drive. I didn't see any errors in the code to cause this, so I'm wondering if it's a copy-and-paste issue or something. I've attached my log in case it's helpful. For reference, disk7 is the one I was trying to clear and disk8 is the one it incorrectly tried to clear first.

I don't need the script anymore as I just created my own script with the 2 action lines, but wanted to make sure I reported this issue so that no one accidentally clears the wrong drive.

On 9/15/2022 at 4:14 PM, inh said:

This script looks good but it still loops through every drive, so it always wants to clear my highest numbered data drive. Very scary. This is on 6.10.3

Edit:
Here is my fixed version:

Thanks for your inputs. Seems like I had a brain fart.

I guess I debugged it when clearing the last disk of my array. I'm updating my previous post.

flyize · October 3, 2022

On 9/28/2022 at 9:53 AM, fritolays said:
Check out tmux as an alternative to screen.

Its available to install via the Nerd Pack plugin.

Github - tmux wiki

tmux cheat sheet
Create new named session
  tmux new -s sessionname

Close current session (same as exit command)
  Ctrl+D

Detach from current session (run in background)
  Ctrl+B then D

List current sessions
  tmux ls

Attach to last session
  tmux a

Attach to specific session
  tmux a -t sessionname

That's my problem though. Nerd Pack is no longer supported or available.

edit: OH! It seems that tmux is bundled with Unraid now.

Edited October 3, 2022 by flyize

JorgeB · October 3, 2022

7 minutes ago, flyize said:

edit: OH! It seems that tmux is bundled with Unraid now.

It's not, but it is with the UD preclear disk plugin.

flyize · October 3, 2022

Well then that explains it. Thanks!

droopie · October 12, 2022

On 12/7/2016 at 9:20 AM, ClunkClunk said:

I started this a few months ago, but never got around to really optimizing it and rotating out old listings or compressing them. But hopefully it'll give you a starting point!

#!/bin/sh

DISKS=8
COUNTER=1
THEDATE=`date +"%Y-%m-%d_%H-%M-%S"`
LOCATION="/boot/trees"

echo " "
echo " "
echo "Starting tree scanning"

while [ $COUNTER -le $DISKS ]
do
echo "Scanning tree for disk$COUNTER/Movies"
tree -h /mnt/disk$COUNTER/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for disk$COUNTER/TV"
tree -h /mnt/disk$COUNTER/TV >> "$LOCATION/$THEDATE.tv.log"
echo "Incrementing disk counter"
COUNTER=$[$COUNTER+1]
done
echo "Scanning tree for cache/Movies"
tree -h /mnt/cache/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for cache/TV"
tree -h /mnt/cache/TV >> "$LOCATION/$THEDATE.tv.log"
echo "All done scanning disks. Beginning scanning shares."
echo "Scanning tree for user/Movies"
tree -h /mnt/user/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for user/TV"
tree -h /mnt/user/TV >> "$LOCATION/$THEDATE.tv.log"
echo " "
echo " "
echo "Complete! Logs stored in $LOCATION"

wow! this would have been a great script to have when i lost 3 hard drives. dont know what data got lost.

fritolays · October 31, 2022

On 10/3/2022 at 1:51 PM, flyize said:

That's my problem though. Nerd Pack is no longer supported or available.

edit: OH! It seems that tmux is bundled with Unraid now.

My error, new plugin is called NerdTools... but the same idea.

Iceman24 · November 7, 2022

On 12/7/2016 at 10:20 AM, ClunkClunk said:

I started this a few months ago, but never got around to really optimizing it and rotating out old listings or compressing them. But hopefully it'll give you a starting point!

#!/bin/sh

DISKS=8
COUNTER=1
THEDATE=`date +"%Y-%m-%d_%H-%M-%S"`
LOCATION="/boot/trees"

echo " "
echo " "
echo "Starting tree scanning"

while [ $COUNTER -le $DISKS ]
do
echo "Scanning tree for disk$COUNTER/Movies"
tree -h /mnt/disk$COUNTER/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for disk$COUNTER/TV"
tree -h /mnt/disk$COUNTER/TV >> "$LOCATION/$THEDATE.tv.log"
echo "Incrementing disk counter"
COUNTER=$[$COUNTER+1]
done
echo "Scanning tree for cache/Movies"
tree -h /mnt/cache/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for cache/TV"
tree -h /mnt/cache/TV >> "$LOCATION/$THEDATE.tv.log"
echo "All done scanning disks. Beginning scanning shares."
echo "Scanning tree for user/Movies"
tree -h /mnt/user/Movies >> "$LOCATION/$THEDATE.movies.log"
echo "Scanning tree for user/TV"
tree -h /mnt/user/TV >> "$LOCATION/$THEDATE.tv.log"
echo " "
echo " "
echo "Complete! Logs stored in $LOCATION"

Thanks for this, although I could use some help understanding how to modify it to use. I have 2 disks I want to do this to. I don't quite understand the COUNTER part. It seems to be going through all the disks, but only one disk is there for me and the other is the cache drive. Also, how it is saving I'm not quite sure about. I just want to let it run on disk1 and cache at directories of my choosing.

MwA83 · November 23, 2022

Hi folks,

I'm in trouble with the "clear an array disk" script by RobJ. I used the original one. I followed the shrink array docs instructions closely. This morning I started the script, logged of from GUI and went to work. After work I'd like to verify the clearing process but my server isn't reachable anymore. My router shows unRaid is offline.

Was it a mistake to log off from GUI? What should I do now? Power the server off and restart? I'm a bit scary to do this.

Sorry, for my English in advance. I'm German.

Greetings

Marcel

Late edit 04.12.22:

I powered my server off and forgot about using a script to zeroing out a disk. I let build a new parity. Next time I'm going to use the script in an updated version.

Edited December 4, 2022 by MwA83

JoeUnraidUser · December 25, 2022

Script to trim Docker logs to 1 Megabyte.

#!/bin/bash

size=1M

function trimLog
{
	file=$1
	temp="$file.$(date +%s%N).tmp"
	time=$(date --rfc-3339='seconds')
	before=$(du -sh "$file" | cut -f1)
	echo -n "$time: $file: $before=>"

	tail --bytes=$size "$file" > "$temp"
	chown $(stat -c '%U' "$file"):$(stat -c '%G' "$file") "$temp"
	chmod $(stat -c "%a" "$file") "$temp"
	mv "$temp" "$file"

	after=$(du -sh "$file" | cut -f1)
	echo "$after"
}

find "/var/lib/docker/containers" -name "*.log" -size +$size 2>/dev/null | sort -f |\
while read file; do trimLog "$file"; done

trimDockerLogs

JoeUnraidUser · December 25, 2022

Script to trim white space from the end of each line of text files.

#!/bin/bash

# Trim white space from the end of each line of text files.

if [ $# -eq 0 ] || [ "$1" == "--help" ]
then
	printf "Usage: trimWhite <files>...\n"
	exit 0
fi

for file in "${@}"
do
	printf "$file\n"
	perl -pi -e 's/\s+$/\n/' "$file"
done

trimWhite

glave · December 26, 2022

Has anyone created a script that can parse the log file looking for a string and then send a notification using unraid's notification system? I'd like to search for "No space left on device" in the log file, just wondering if something similar may already exist before I reinvent the wheel.

trurl · December 26, 2022

9 hours ago, glave said:

"No space left on device"

You can already get Notifications about how full disks are. Disk Settings has utilization thresholds (%) for all disks, and you can override that for individual disks in the Settings for each disk.

If you wait for "No space left on device" it is already too late.

glave · December 26, 2022

1 hour ago, trurl said:

You can already get Notifications about how full disks are. Disk Settings has utilization thresholds (%) for all disks, and you can override that for individual disks in the Settings for each disk.

If you wait for "No space left on device" it is already too late.

Unfortunately they don't work in some situations, although I haven't tracked down the specifics yet. I have a 50GB minimum set on each share, yet I still have a few disks than continue to fill up past that without triggering warnings.

Additional Scripts For User.Scripts Plugin

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

Squid

RobJ

SpaceInvaderOne

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation