[Plugin] CA User Scripts

[itimpi]

2 hours ago, Leseratte10 said:

Great Plugin, I started to use this to automatically create backups of my Docker container data (lives on the cache drive) to the main array. 

 

When I started using it I added the "noParity" flag because I didn't want to have backups or other intensive tasks running on the array during a parity check or data rebuild - and then a month later wondered why it didn't run the backup for two days in a row ...

 

Would it be possible to add another flag that auto-pauses a Parity Check or Data Rebuild if it's currently running? So, a flag like "pauseParity" that checks if a Parity operation is currently running before it starts the userscript, and if so, pause the operation, run the userscript, then start it again? Or would that cause any issues like when running multiple scripts in parallel?

Have you thought of instead of a custom script using the CA backup plugin to do the backups and also have the Parity Check Tuning plugin installed (and have the option to pause array operations if the backup is running enabled in its settings) to handle the pause/resume of array operations.

[Gex2501]

I have a script I'd like to start and stop with the array. Obviously I can tell it to run on array start, but how can I stop it? Is there a command I can execute in another script on array stop to abort the first script? I've been search and digging through the plugin code to no avail...

[Gex2501]

Another conundrum...when I click the abort script button to end a script running in the background, is it sending a kill -9? It seems my trap call isn't being processed when the script dies.

[mattw]

Odd permissions question... I have a script that takes a curl backup of my pfsense router, works great and has been for a good while.  The problem, the backup file permissions are only rw for the user.  I need rw,r,r or rw,rw,rw.  Is that something I can correct in the plugin?

[Squid]

2 hours ago, mattw said:

Odd permissions question... I have a script that takes a curl backup of my pfsense router, works great and has been for a good while.  The problem, the backup file permissions are only rw for the user.  I need rw,r,r or rw,rw,rw.  Is that something I can correct in the plugin?

chmod command on the backup created?

[mattw]

That is what I have been doing in order to download and use a specific backup file.  I am not a coder... I am using this script and it very faithfully makes my backups and they work.  https://blogmotion.fr/systeme/script-backup-pfsense-configuration-16496

 

I assume that I could somewhere add a chmod to the script?

[diedrichg]

Illegal characters is still an issue in v2023.03.29. The apply button will not function if an illegal character is used upon script creation (folder name) - ex: run_mover_at_85% . The user is unable to press Apply when changing the schedule combo box for a script.

 

mv run run_mover_at_85%/ run_mover_at_85

fixed the issue for me

 

reference:

 

[orangeisnon]

I just made a new simple script to force reboot a VM (to try to get around the VirtioFS issue until a solution is found) but I can't seem to get Unraid to save a schedule. I have tried using the built-in Daily schedule from the drop down as well as Cron via the Custom entry, but when I hit "Apply" nothing happens and it reverts on the next page load. Tried on Chrome and Safari. Is this a known issue? I have done some searching but only come up with illegal character issues, which shouldn't be the case as my scripts only contain alphanumeric characters in their names (besides the couple of built in scripts with underscores and "at" symbol)

Edit: It seems that I cannot edit the scheduling of pre-existing scripts now either. I was definitely able to do this previously, but I cannot remember how long ago I last did it. Might have been prior to updating to 6.11.5.

Edited June 13, 2023 by orangeisnon
More testing

[mattw]

I am running the "clear an array drive" script on a 1.5tb drive.  I am only writing at around 5.2 MB/s, this has been running about 12 hours so far and appears that it will be running for a good while.  I see that it is not the fastest script, but is this about right?  BTW, running 6.11.5.

[wgstarks]

56 minutes ago, mattw said:

I am running the "clear an array drive" script on a 1.5tb drive.  I am only writing at around 5.2 MB/s, this has been running about 12 hours so far and appears that it will be running for a good while.  I see that it is not the fastest script, but is this about right?  BTW, running 6.11.5.

I tried that script once and it was even slower. I wouldn’t recommend it for erasing a drive. Using the preclear docker or plugin would be much faster.

[mattw]

37 minutes ago, wgstarks said:

I tried that script once and it was even slower. I wouldn’t recommend it for erasing a drive. Using the preclear docker or plugin would be much faster.

Well crap, I wanted to shrink my array and maintain parity.  This seems to be the best option, what is a better idea? 

[JonathanM]

11 minutes ago, mattw said:

Well crap, I wanted to shrink my array and maintain parity.  This seems to be the best option, what is a better idea? 

If you are comfortable with the command line you can do what the script does manually, but that removes all the safety features that try to prevent you from erasing a drive you really didn't want to erase.

 

The speed issue seems to go away if the drive is unmountable when the array starts, which means the script won't work as it looks for a mountable drive with the specifically named file / folder.

[mattw]

6 minutes ago, JonathanM said:

If you are comfortable with the command line you can do what the script does manually, but that removes all the safety features that try to prevent you from erasing a drive you really didn't want to erase.

 

The speed issue seems to go away if the drive is unmountable when the array starts, which means the script won't work as it looks for a mountable drive with the specifically named file / folder.

I am ok with command line linux.  I would be willing to give that a go... if this does not finish today.  Not sure why I am really in a hurry, but I just want to remove an very old drive with some error.

[mattw]

So, if I am reading my screen correctly... the script that zeros a drive is not slow, it is almost not worth trying.  I have been running almost 48 hours at this point and if I am reading correctly I have only cleard 200 mb of 1.5 tb.  Am I correct?  If so, I think I will just rip the empty drive out and unassign it and do the darn parity rebuild as all of my remaining drives are in good condition.

 

[JonathanM]

1 hour ago, mattw said:

the script that zeros a drive is not slow, it is almost not worth trying.

We tried to tell you.

 

If you stop the array and start it again, the drive will show as unmountable, and the dd command issued at the console will go at full speed, probably several hours at most to finish.

 

If you want to do that, post back with a screenshot of the main array started and showing the drive assignments, and your proposed command line from the "alternate" section of the shrink array wiki page. That way we can double check that you are indeed going to clear the correct drive and not permanently erase one of your other drives, which would be very sad. Note that since the drive will already be unmounted when you restart the array you can omit the umount part and just do the dd line.

 

https://docs.unraid.net/legacy/FAQ/shrink-array#alternate-procedure-for-linux-proficient-users

[mattw]

2 hours ago, JonathanM said:

We tried to tell you.

 

If you stop the array and start it again, the drive will show as unmountable, and the dd command issued at the console will go at full speed, probably several hours at most to finish.

 

If you want to do that, post back with a screenshot of the main array started and showing the drive assignments, and your proposed command line from the "alternate" section of the shrink array wiki page. That way we can double check that you are indeed going to clear the correct drive and not permanently erase one of your other drives, which would be very sad. Note that since the drive will already be unmounted when you restart the array you can omit the umount part and just do the dd line.

 

https://docs.unraid.net/legacy/FAQ/shrink-array#alternate-procedure-for-linux-proficient-users

It appears that the following command would be issued from the cli since disk 2 is the one I want to remove:

dd bs=1M if=/dev/zero of=/dev/md2 status=progress

 

 

Thanks

[JonathanM]

2 minutes ago, mattw said:

It appears that the following command would be issued from the cli since disk 2 is the one I want to remove:

dd bs=1M if=/dev/zero of=/dev/md2 status=progress

 

 

Thanks

Looks good, assuming after the array is stopped and started again disk2 shows unmountable as predicted.

[mattw]

1 hour ago, JonathanM said:

Looks good, assuming after the array is stopped and started again disk2 shows unmountable as predicted.

So, how long should it say stopping array, sync filesystems. ?

 

[JonathanM]

8 minutes ago, mattw said:

So, how long should it say stopping array, sync filesystems. ?

 

Did you kill the original script? You may need to manually kill it using htop at the console.

[mattw]

6 minutes ago, JonathanM said:

Did you kill the original script? You may need to manually kill it using htop at the console.

I did kill the zero script in the gui, and it said it aborted.  Will look at htop from the shell.

 

[mattw]

So with ps -aux | grep zero from the shell I see this...

 

root@Tower:~# ps -aux | grep zero
root      6211  0.0  0.0   4048  2232 pts/0    S+   20:56   0:00 grep zero
root     21434  0.1  0.0   3636  2460 ?        D    Jun12   4:50 dd bs=1M if=/dev/zero of=/dev/md2

 

If I am not mistaken the "D" indicates that the process is dead but not gone?

 

should I kill process 21434?

tower-diagnostics-20230614-2101.zip

Edited June 15, 2023 by mattw
added diag.

[mattw]

So kill -9 21434 allowed the stopping array to progress to stopping array - unmounting disks. 

 

Current dead processes...

root@Tower:~# ps -aux | grep D
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root        51  0.0  0.0      0     0 ?        S    Jun11   0:00 [irq/24-AMD-Vi]
root     11701  2.3  0.0      0     0 ?        D    Jun12  68:59 [kworker/u8:0+flush-9:2]
root     19703  0.0  0.0   2676   116 ?        S    14:42   0:00 /usr/sbin/avahi-dnsconfd -D
root     28751  0.0  0.0   4048  2296 pts/0    S+   21:25   0:00 grep D
root     30160  0.0  0.0   3296  1148 ?        D    21:11   0:00 umount /mnt/disk2

 

My gut now tells me to kill 30160.

 

From system logs:

Jun 14 21:11:33 Tower emhttpd: Unmounting disks...

Jun 14 21:11:33 Tower emhttpd: shcmd (680076): umount /mnt/disk1

Jun 14 21:11:34 Tower kernel: XFS (md1): Unmounting Filesystem

Jun 14 21:11:37 Tower emhttpd: shcmd (680077): rmdir /mnt/disk1

Jun 14 21:11:37 Tower emhttpd: shcmd (680078): umount /mnt/disk2

 

It appears to be stuck unmounting disk 2.  That was the last log entry short of me opening an ssh session.

 

Open files plugin reports no open files.

Edited June 15, 2023 by mattw
Fix formatting

[mattw]

Ok, I feel like a dumb-ass!  I only clicked done in New Config the first time... done has always seemed to me that it should do everything in one shot, it does not!  The array is up and the disk I wanted to remove can now be remove. 

 

Thanks much @JonathanM, I always get nervous when my array is in an odd state.

[orangeisnon]

On 6/13/2023 at 1:23 AM, orangeisnon said:

I just made a new simple script to force reboot a VM (to try to get around the VirtioFS issue until a solution is found) but I can't seem to get Unraid to save a schedule. I have tried using the built-in Daily schedule from the drop down as well as Cron via the Custom entry, but when I hit "Apply" nothing happens and it reverts on the next page load. Tried on Chrome and Safari. Is this a known issue? I have done some searching but only come up with illegal character issues, which shouldn't be the case as my scripts only contain alphanumeric characters in their names (besides the couple of built in scripts with underscores and "at" symbol)

Edit: It seems that I cannot edit the scheduling of pre-existing scripts now either. I was definitely able to do this previously, but I cannot remember how long ago I last did it. Might have been prior to updating to 6.11.5.

Turns out it was an illegal character issue, but with a completely different script. I had erroneously thought that a script I had for deleting thumbnails was a built-in script because the name/description matched up so well with those of the built-in scripts. I had to delete it with the terminal because the settings button for that script also was not working.

Edited June 15, 2023 by orangeisnon

[JonathanM]

7 hours ago, mattw said:

The array is up and the disk I wanted to remove can now be remove. 

You let parity rebuild, correct?