[UNRAID 6.7.1 easy way to backup and restore specific dockers]

On 12/4/2022 at 6:31 AM, Mustafa Aydin said:

At first. The Script is working like a charm...I'm recognize it's backup only the Container that are sitting in appdata. The Cache folders are not in the backup list. And its also not possible to backup there with direkt command. There are also showed in the backuplist.

 

./backupDockers.sh -l

 

Get only blank timestamps:

 

DATE: 2022-12-04-12-23-15-CET
DATE: 2022-12-04-12-23-15-CET

 

What can we do here? Correct the cache-dir not solve the problem.

 

cache="/mnt/cache"

 

Thanks in advance.

 

Edit: Its not backup all Docker Images that have some data in the cache directory.

 

I'm not sure what you are trying to do.

 

"./backupDockers.sh -l" only lists the dockers you have installed on the system, not the backups of the dockers.

 

You should never use /mnt/cache as your backup directory.  The backup will fail because it syncs the files to a folder called appdata in the backup directory.  appdata is usually located in /mnt/cache/appdata.  It would mean it is trying to overwrite itself.

[Not disclosed in the release notes]

8 hours ago, ljm42 said:

What version of Unraid are you coming from, and what version did you update to?

I updated from 6.11.0-rc3 to 6.11.0-rc5.

 

8 hours ago, ljm42 said:

Hmm... what makes you think glibc was removed? It is pretty critical to Linux and needs to be there. I highly recommend that you not change the version glibc that is installed.

 

You can see all the installed packages by opening a web terminal and running this command:

ls -al /var/log/packages/*

the ones that say 0 bytes are included in OS, the ones with with larger sizes are installed by plugins or by putting them in /boot/extra

I use a number of PERL modules.  After the install of 6.11.0-rc5 I tried updating the PERL modules and it was getting the following error:

 

     fatal error: sys/types.h: No such file or directory

 

I finally chased down the problem to glibc.  I know that it was working before.  While on 6.11.0-rc2 I had reloaded all the PERL modules from scratch and did not get this error.

 

The reason I had reloaded all the PERL modules from scratch was part of the process of figuring out which packages I needed to put in /boot/extra that were being provided by NerdPack and DevPack.  glibc was not one of the needed packages.

 

Since you told me where to find the installed packages, I removed glibc-2.36-x86_64-3.txz from /boot/extra and rebooted to see if glibc is there.

 

I see that glibc-2.36-x86_64-3_LT is installed.

 

I went ahead and reloaded my PERL modules from scratch and I get the same error:

 

     fatal error: sys/types.h: No such file or directory

 

So, I guess glibc-2.36-x86_64-3_LT is not loading sys/types.h correctly.

 

I put glibc-2.36-x86_64-3.txz back into /boot/extra, rebooted, and now everything is working fine again.
 

8 hours ago, ljm42 said:

Please provide an example of what used to work (in what version of Unraid?) and what no longer works (in what version of Unraid?).

I have been running the following script from the go file for years with no problems.  One of the things it does is appends to .bash_profile.  After logging into Unraid using SSH I tried to run an alias I set in .bash_profile called bin and it said command not found.  I typed alias and saw it wasn't set.  I checked .bash_profile and it had been overwritten.  I finally figured out that sometime during the boot process .bash_profile is now being overwritten.

 

#!/bin/bash

if [ -f /root/.bash_profile.orig ]
then
    cp /root/.bash_profile.orig /root/.bash_profile
else
    cp /root/.bash_profile /root/.bash_profile.orig
fi

(
cat <<"EOF"

# alias
alias bin="/usr/bin/ls -al /root/bin"

# Perl
PERLHOME="/root/perl5"; export PERLHOME;
PERL5LIB="$PERLHOME/lib/perl5"; export PERL5LIB;
PERL_MB_OPT="--install_base $PERLHOME"; export PERL_MB_OPT;
PERL_MM_OPT="INSTALL_BASE=$PERLHOME"; export PERL_MM_OPT;
PERL_LOCAL_LIB_ROOT="$PERLHOME"; export PERL_LOCAL_LIB_ROOT;

# Path
PATH=.:/root/bin:$PERLHOME/bin:$PATH

EOF
) >> /root/.bash_profile

ln -s /boot/config/perl5 /root/perl5

rsync -hrltgoD --delete --chown=root:root --chmod=ugo=rwx "/boot/config/bin/" /root/bin

modprobe i915
chmod -R 777 /dev/dri

 

I fixed it by adding a sleep 10 to the beginning of the script.

 

It would have been nice to know this change in the release notes.

[Not disclosed in the release notes]

Packages were removed such as glibc without any notice in the release notes. Only what has been updated is listed. If a package has been removed, it should also be listed.  There should be a place that defines all the packages that are included in the distribution. This is important information to have since NerdPack and DevPack are no longer supported and the user now has to maintain their own additional slackware packages.

 

There was a new change in writing to .bash_profile while booting however it was not mentioned in the release notes and those of us that append to .bash_profile from the go file get overwritten by the change.

 

@limetech

[how to find duplicate files]

See this post:

 

[Unraid OS version 6.10.3 available]

7 minutes ago, unRaide said:

Hi, just upgraded from 6.9.3 and everything seems to be working fine so far except for what seems like a permissions issue with Sonarr.

 

I had permission issues with my dockers after the upgrade and fixed the problems by running the following:

chown -cR nobody:users /mnt/user/appdata/*
chmod -cR ug+rw,ug+X,o-rwx /mnt/user/appdata/*

 

[UNRAID 6.7.1 easy way to backup and restore specific dockers]

1 hour ago, FreddyGSanford said:

For the life of me I cannot understand why this doesn't work for me. The file can no longer be downloaded, so on a Windows machine I opened Notepad++ copied the contents into a new file and saved as backupDockers.sh. I copied the file to X:\appdataBackup\containers\ this the path to the unRAID share where I have my weekly backups already going. I open terminal from the unRAID gui. I go to the path where I have the .sh file copied. I put this into the terminal window "backupDockers.sh -b -d /mnt/user/Backups/appdataBackup/containers/mariadb mariadb"

 

This is the response I get back from the terminal window "bash: ./backupDockers.sh: /bin/bash^M: bad interpreter: No such file or directory"

I just tried downloading the file from the link and the download still works.

[UNRAID 6.7.1 easy way to backup and restore specific dockers]

1 hour ago, jademonkee said:

Noob question: where do I store the script? I note that the UserScripts plugin doesn't work with interactive scripts, so I'm guessing I have to drop to terminal to execute this, but I have no idea what 'best practice' is for storing scripts. Should I just place it in the Backup directory? Or is it better to place it on the flash drive somewhere?

Thanks for your help.

 

You can run it from any folder except the flash drive.  You can then run it from within a script in the UserScripts plugin.

[UNRAID 6.7.1 easy way to backup and restore specific dockers]

On 10/4/2020 at 5:29 PM, rippedwarrior said:

Thanks for this script @JoeUnraidUser!

 

Quick question: Is there a reason a cache is used for this rather than running Gzip on the original files?  I'd prefer not to have a complete copy of my docker files in addition to the archive, but before I modify the script I want to make sure I'm not missing something that makes the cache necessary.  Thanks for any help you can provide!

 

EDIT: After using the script a bit further, it seems like the cache is to enable as minimal downtime to the docker as possible.  After the first cache is created, each subsequent "caching" only copies the changed/new files to the cache which means the docker is only offline for a short period of time.  Then the Gzip is run on the cached files while the docker is already back up and running.  If there's another reason I'm not seeing, please let me know.  Thanks again!

 

That's exactly why I used a cache.

[UNRAID 6.7.1 easy way to backup and restore specific dockers]

10 hours ago, 1unraid_user said:

Hi @JoeUnraidUser

I just tried your script (which is exactly what I am looking for), but the system gives me this error message

 

backupDockers.sh: line 52: syntax error near unexpected token `<'
backupDockers.sh: line 52: `    readarray -t all < <(printf '%s\n' "$(docker ps -a --format "{{.Names}}" | sort -fV)")'

 

I tried inserting as text and then downloaded the provided file. Buth without success.

As this post is ranked #1 in google for unraid docker backups, I think fixing this would probably be a big gain for the community.
(Maybe I am the problem).

 

I can't seem to make it get the same error.  What command line are you entering?

[How to install 7z?]

Sorry for my confusion.

 

To install p7zip:

• Install the NerdPack plugin.
• Go to SETTINGS and click on Nerd Pack.
• Turn on p7zip-16.02-x86_64-1sl.txz.
• Then click on APPLY at the bottom.
• You may have to reboot for it to take effect if the apply doesn't work.

[How to install 7z?]

Try typing in the full path: /usr/bin/7z

[How to install 7z?]

7z is already installed in /usr/bin/7z.  In a terminal type 7z and hit enter.

[[Plugin] CA User Scripts]

1 hour ago, je82 said:

can you elaborate on how to do this properly?

To set the umask to 0000 do the following on a line before your command:

umask 0

 

[[Plugin] CA User Scripts]

1 hour ago, je82 said:

I have a problem where whenever an rsync script is ran via user scripts,

When i test running this in the CLI it works fine, it sets both directories and files that are being backed up to the correct permissions for unraid, but whenever the same command runs via user scripts it for whatever reason sets the directory permissions to drwxrwx--- instead of drwxrwxrwx+, please advise?

Try adding the following to the beginning of your command so it runs in a login shell:

sudo -i -u root 

 

[[Plugin] CA User Scripts]

I had the same problem when running PERL scripts.  I had to hardcode library and config paths.

 

Try running the script in a login shell by doing the following:

su - root /mnt/user/test/test.sh <<<password

--or--

sudo -i -u root /mnt/user/test/test.sh

 

[[Plugin] CA User Scripts]

11 hours ago, l4t3b0 said:

I would like to ask for your help to solve a problem i have when executing a script. I have difficulties to have to get the home directory of the user.

You could use the following:

echo $(eval echo "~$(whoami)")

 

[[Plugin] CA User Scripts]

17 hours ago, Phoenix Down said:

You could write a small Perl script to do it. Maybe something like this:

 

#!/usr/bin/env perl

use strict;
use warnings;
use Carp;
use English;

# Read the file
my $file = "/mnt/user/text/mytestfile.txt";
my $fh;
open $fh, "<", $file
  or croak("\nERROR: Cannot open '$file' for reading: $OS_ERROR\n\n");
my @rawdata = <$fh>;
close $fh;

# Update each line
my @newdata;
foreach my $line (@rawdata)
{
  $line =~ s/#FF8C2F/#42ADFA/g;
  push @newdata, $line;
}

# Update the file with the updated data
open $fh, ">", $file
  or croak("\nERROR: Cannot open '$file' for writing: $OS_ERROR\n\n");
foreach my $line (@newdata)
{
  print $fh $line;
}
close $fh;

 

 

You can achieve the same thing from the command line with PERL:

perl -pi -e 's/#FF8C2F/#42ADFA/g' /mnt/user/text/mytestfile.txt

Perl one-liners

[[Plugin] CA User Scripts]

2 hours ago, Squid said:

Not 100% sure if MON, TUE etc works within unRaid.  Safest to use the numerals

Your right it doesn't work.  I tested it using "MON" and it failed.  It looks like Unraid is using "crond 4.5 dillon's cron daemon".

 

It's strange because the man page crontab(1) for "dillon's lightweight cron daemon" contains:

a symbolic range for the day of week and  month  in  year

# run at 11 am on the first and last Mon, Tue, Wed of each month
0 11 1,5 * mon-wed date

Edit:

I tested it again using "mon" and it worked.  It must only work with lowercase days of the week.

 

[[Plugin] CA User Scripts]

31 minutes ago, wgstarks said:

According to the man page you quoted the last field should be 0 to 7. Monday would be 1 rather than MON.

"or use names" means SUN-SAT or sun-sat.

 

Example crontab line from man page crontab(5):

5 4 * * sun     echo "run at 5 after 4 every sunday"

Edit:

Also from man page crontab(5):

Names can also be used for the 'month' and 'day of week' fields.  Use
the first three letters of the particular day or month (case does not
matter).  Ranges or lists of names are not allowed.

 

[[Plugin] CA User Scripts]

7 hours ago, itimpi said:

Where you have MON it should be a value in the range 1 to 12 for month of the year or * for all months.

The last field should be day of week.  Allowed values are 0-6, 7 for Sunday or SUN-SAT.

 

0 6 * * MON is 06:00 on Monday

 

From the crontab man page crontab(5) :

field          allowed values
-----          --------------
minute         0-59
hour           0-23
day of month   1-31
month          1-12 (or names, see below)
day of week    0-7 (0 or 7 is Sunday, or use names)

 

[[Plugin] CA User Scripts]

18 hours ago, rl2664 said:

Hello,

how can I start Mover after a rsync backup?

Greetings
Ronny

 

To run mover enter the following command:

/usr/local/sbin/mover

 

[Command to move all mkv files up a folder]

On 1/17/2020 at 1:57 AM, gnollo said:

It worked indeed! Thank you for that Joe. Now I need to identify which folder in my /mnt/user/test/movies contains subfolders. Need to get rid of them as emby does not deal well with them.

 

The following script will show the subfolders for each movie.

 

 

To remove the subfolders; uncomment the following line:

#rm -rf "$dir"

to

rm -rf "$dir"

 

movies_subdirs.sh

#!/bin/bash

movies="/mnt/user/test/movies"

process()
{
    local movieDir=$1

    readarray -t dirs < <(find "$movieDir" -mindepth 1 -maxdepth 1 -type d -print 2>/dev/null)

    if [ ${#dirs[@]} != 0 ]
    then
        printf "$movieDir\n"
        
        for dir in "${dirs[@]}"
        do
            printf "$dir\n"    

            #rm -rf "$dir"
                
        done
    fi
}

ls -1d "$movies"/*/ | sed 's/\/*$//' |\
while read movie
do
    if [ -d "$movie/DVD1" ]
    then
        ls -1d "$movie"/DVD*/ | sed 's/\/*$//' |\
        while read dvd
        do
            process "$dvd"
        done
    else
        process "$movie"
    fi
done

 

[Command to move all mkv files up a folder]

On 12/25/2019 at 1:44 PM, gnollo said:

Merry Xmas Joe, if I want to find out which movie folder within a directory does not have a .nfo file, do I just replace *.mkv with *.nfo in the script above? I don't have any dvd folders anymore as I have converted them all to mkvs.

Yes, it should work.

[Command to move all mkv files up a folder]

Check for movie.xml and mkv files in each movie folder.

check_movies.sh

#!/bin/bash

movies="/mnt/user/test/movies"

process()
{
    local movieDir=$1
    
    local mkv="$(find "$movieDir" -iname '*.mkv')"
    local xml="$(find "$movieDir" -iname 'movie.xml')"

    if [ -z "$mkv" ] || [ -z "$xml" ]
    then
        printf "$movieDir\n"
        [ -z "$mkv" ] && printf "\tmkv: file not found\n"
        [ -z "$xml" ] && printf "\tmovie.xml: file not found\n"
    fi
}

ls -1d "$movies"/*/ | sed 's/\/*$//' |\
while read movie
do
    if [ -d "$movie/DVD1" ]
    then
        ls -1d "$movie"/DVD*/ | sed 's/\/*$//' |\
        while read dvd
        do
            process "$dvd"
        done
    else
        process "$movie"
    fi
done

 

[Command to move all mkv files up a folder]

Do you want the script to move "movie.xml" to the parent directory like the mkv file?

 

I could modify the script so that at the time of processing it checks if an mkv file exists, if not print a message.

Or I could make a separate script to check if the files exist.