(solved)User script to run multiple user script backups daily without interference/overlap

[RaidUnnewb]

Hello, I am relatively new to scripting (I use other people's scripts, not creating my own) and would love help creating a script to run other scripts in order.

 

I have 5 instances of SpaceInvadors 'snap and replication' script for my various zfs cache drives (appdata, system, vm's, iso storage, ect) running daily backups to a single zfs drive I have on my array. These are creating daily/weekly/monthly backups for all of the datasets and storing them on my drive, which then gets parity protection. Works great so far.

 

But, I am worried that having so many scripts running timer'd is

1) Inefficient (I want my disk to spin down instead of kept active for extra hours), and

2) I am worried that if I do a big change to one of my folders (adding gigabytes of LLM files or something) that one of the backups could take long enough to interfere with the timing of another, and stop it from running, or catch my server/house/me on fire or something.

 

I haven't been able to find (and dont want to test) what happens with trying to run multiple scripts at once.

Does anyone know to write a script that will do what this reddit post suggests? Apparently it should be simple, I just cant find an example to copy/alter haha.

Solution is to have a script that will start each script right after the previous one finishes. The 'master script' will be set to run daily and all other scripts will have their schedule disabled.

 

Thank you!

 

 

Edited July 20 by RaidUnnewb
solved!

[foo_fighter]

It's exactly like the post says. You create 1 user script that calls the other 5:

 

#!/bin/bash

/boot/config/plugins/user.scripts/scripts/script1.sh

/boot/config/plugins/user.scripts/scripts/script2.sh

/boot/config/plugins/user.scripts/scripts/script3.sh

/boot/config/plugins/user.scripts/scripts/script4.sh

/boot/config/plugins/user.scripts/scripts/script5.sh

 

That's it. 

[RaidUnnewb]

Thanks!

Was thinking it was more complicated with waits and whatnots (closest to coding I've come is editing macros in keyboard/mouse software). If the script waits for the previous line to be done before moving onto the next, then it solves itself. Awesome!

[RaidUnnewb]

56 minutes ago, RaidUnnewb said:

Thanks!

Was thinking it was more complicated with waits and whatnots (closest to coding I've come is editing macros in keyboard/mouse software). If the script waits for the previous line to be done before moving onto the next, then it solves itself. Awesome!

Actually, I'm stuck again.
Been trying different options for the last 30 minutes. Keep getting errors trying to run the script.

 

 

 

So far my script is:

#!/bin/bash
/boot/config/plugins/user.scripts/scripts/'00001'
/boot/config/plugins/user.scripts/scripts/"0 isos snap and replication"
/boot/config/plugins/user.scripts/scripts/"0 system snap and replication"/script.sh
/boot/config/plugins/user.scripts/scripts/"0 VMs snap and replication"/script.sh
/boot/config/plugins/user.scripts/scripts/"appdata snap and replication"/script.sh

 

 

Ive tried using quotes, apostrophes, no punctuation, adding and removing .sh to the end.
In each script directory, there are only 3 files: description, name, and script. 
I assume that I want to call out the script file to run, but I don't know the proper syntax. 
(also future note to anyone searching for this, the script files are on the flash drive)

 

Edit: after some more process of elimination:

 

Looks like a few of them are seeing the script, but getting permission denied...?
Doesnt seem to make a difference between " and '  for the below scripts. But need to have one of them, and a /script at the end.

 

#!/bin/bash
/boot/config/plugins/user.scripts/scripts/00001/script
/boot/config/plugins/user.scripts/scripts/0 isos snap and replication/script.sh
/boot/config/plugins/user.scripts/scripts/"0 system snap and replication".sh
/boot/config/plugins/user.scripts/scripts/'0 VMs snap and replication'/script
/boot/config/plugins/user.scripts/scripts/"appdata snap and replication"/script


Im thinking that the answer is in here, but I don't understand any of it and have been going at this for a couple hours now...
https://www.baeldung.com/linux/shell-call-script-from-another

Thank you for your patience/help.

Edited July 20 by RaidUnnewb

[foo_fighter]

Ah, sorry, I should have done a test case my self.

 

The scripts are stored as a text file under the directory and they are not set to executable. Also spaces in script names or directory name are generally a bad practice. Use _ or - if needed.

 

If you're comfortable with ssh and command line, you can copy or symlink them from a shell:

cd /boot/config/plugins/user.scripts/scripts/00001/ 

ln -s script script.sh (this makes a symbolic link so that script.sh is the same as script....If you edit script.sh, it will edit script since script.sh is just a pointer to script)

 

To make them excutable change the permissions (chmod +x script.sh)

 

I would try with a test case first, in each test script, only put a simple command

For example:

#!/bin/bash

echo "This is Script 1"

# Add a newline

echo ""

 

then the next one: 

#!/bin/bash

echo "This is Script 2"

# Add a newline

echo ""

 

so when you run all 5 you should see on the output:

This is Script 1

This is Script 2

 

...

This is Script 5

 

When that is working fine, symlink or copy the intended script.

 

If you want to wait between scripts, you can use the sleep command between each script, but generally unless you have background commands in the script, each one will start and finish before moving on to the next:

sleep 10m

to wait 10 minutes, 1h for 1 hour etc.

 

 

 

 

 

 

 

[itimpi]

For security reasons, the files on the flash drive cannot be made executable.

 

however in this case you can simply use

  sh pathtoscript

in the master script as then the file being run as a script does not need to be executable.

[RaidUnnewb]

I tried the sh pathtoscript first since that looked easiest lol.

 

#!/bin/bash
sh pathtoscript /boot/config/plugins/user.scripts/scripts/00001/
sh pathtoscript /boot/config/plugins/user.scripts/scripts/0 isos snap and replication/script.sh
sh pathtoscript /boot/config/plugins/user.scripts/scripts/"0 system snap and replication"
sh pathtoscript /boot/config/plugins/user.scripts/scripts/'0 VMs snap and replication'
sh pathtoscript /boot/config/plugins/user.scripts/scripts/"appdata snap and replication"/script

also did a quick test with 

sh pathtoscript /boot/config/plugins/user.scripts/scripts/00001/script.sh

same result..

 

I dont understand what to write in the main script for foo_fighter's comments. (And apparently forgot how quoting works on this forum).

Do I use the CLI to make a copy somewhere, put the command in the main script to make a copy daily then run that, something else?

 

 

 

Welp, I learned the hard way that I really do need to figure out this main script, because I was running one of the backups when my 'restart once a week' script ran. Ended up in an unclean shutdown so doing a 2-day parity check, when I just finished building parity yesterday (made some drive changes) lol. Now I have the 5 schedules running AFTER the weekly reboot.

Edited July 20 by RaidUnnewb

[itimpi]

The ‘pathtoscript’ was meant to represent the script full filename  (that starts with /boot) not to be included literally.   In other words just adding ‘sh’ (could also use ‘bash’) to start of each line.

[RaidUnnewb]

Tried 

#!/bin/bash
chmod +x /boot/config/plugins/user.scripts/scripts/00001/script
echo "Script 1 done"
echo ""
chmod +x /boot/config/plugins/user.scripts/scripts/'0 isos snap and replication'/script
echo "Script 2 done"
echo ""
chmod +x /boot/config/plugins/user.scripts/scripts/"0 system snap and replication"/script
echo "Script 3 done"
echo ""
/boot/config/plugins/user.scripts/scripts/'0 VMs snap and replication'
echo "Script 4 done"
echo ""
/boot/config/plugins/user.scripts/scripts/"appdata snap and replication"/script
echo "Script 5 done"
echo ""

 

got

So it looks like it mostly worked (made them all different so I could text syntax because I added spaces...)

But I checked the Iso snaphsot admin, and no new snapshots. Went and ran that script directly, and it did change the snapshot admin.
So none of the scripts actually ran from the main script.

 

 

 

Saw itimpi's comment that I got sh pathname wrong lol (I am new to this)

tried:

#!/bin/bash
sh  /boot/config/plugins/user.scripts/scripts/00001/script
echo "Script 1 done"
echo ""
sh  /boot/config/plugins/user.scripts/scripts/'0 isos snap and replication'/script
echo "Script 2 done"
echo ""
sh /boot/config/plugins/user.scripts/scripts/"0 system snap and replication"/script
echo "Script 3 done"
echo ""
sh /boot/config/plugins/user.scripts/scripts/'0 VMs snap and replication'
echo "Script 4 done"
echo ""
sh /boot/config/plugins/user.scripts/scripts/"appdata snap and replication"/script
echo "Script 5 done"
echo ""

 

And now its running sequentially which is GREAT. 

well, it ran script 1, script 2, script 3, failed script 4 due to syntax, and then ran script 5, which is what I expected!

Fixed the failed script, and boom! Good to go!

 

Thank you for both of your help! I have scoured this website search and google and couldnt find. Hope this is useful to others in the future.

[foo_fighter]

Just one small correction. I meant that the echo commands go inside of the script, like the last line, so that you would know when the script itself ran and finished. 

The way you have it now, it will always output "Script 4 done" even if it didn't actually run as you probably noticed. 

But that's a minor fix and you are making good progress and improving and learning which is half of the fun.