Jump to content

JoeUnraidUser

Members
 View Profile  See their activity

  • Posts

    208

  • Joined

 Content Type 

Posts posted by JoeUnraidUser

    Command to move all mkv files up a folder

    in General Support

    Posted

    Quote

    mv: cannot stat 'AUDIO_TS': No such file or directory

    mv: cannot stat 'VIDEO_TS': No such file or directory

    The mv will give an error if AUDIO_TS or VIDEO_TS do not exist.  I left the error in there so you could see if either directory wasn't present.  I can suppress the error if you want.

     

    Quote

    rev: stdin: Invalid or incomplete multibyte or wide character

    mv: cannot move 'VIDEO_TS' to '/mnt/user/unraid/dvdbackup/VIDEO_TS': Directory not empty

    I believe the reason the mv failed may be because of the rev error.

    I modified the script to use sed instead of rev.  It may fix the error.

     

    Command to move all mkv files up a folder

    in General Support

    Posted

    1 minute ago, gnollo said:

    run it on the same folders

     

    Script location: /tmp/user.scripts/tmpScripts/mkv dvd fixer/script
    Note that closing this window will abort the execution of this script
    ls: cannot access '/mnt/user/test/movies/*/': No such file or directory

    Are your movies in \\tower\test\movies all lowercase?

    Command to move all mkv files up a folder

    in General Support

    Posted

    4 minutes ago, gnollo said:

    Worked perfectly. One of the folders didn't have a VIDEO_TS folder, which is why it gave the error message. One last thing would be to also move across the AUDIO_TS folder? You are an absolute gem Joe this is going to save me a ton of time! And if anything goes wrong I can always fix it quickly as I have the original files in the backup folder! This is ace!

    I changed the script so it also moves AUDIO_TS to the backup.

    Command to move all mkv files up a folder

    in General Support

    Posted · Edited by JoeUnraidUser

    On 9/23/2019 at 6:57 PM, gnollo said:

    So if I create these folders

    \\tower\test

    \\tower\test\movies

    \\tower\test\backup

    drop some test dvds in the movies folder, I can run the script to test it?

    Yes.

     

    Note:

    1. On Linux, slash / is used for folders instead of the Windows backslash \

    2. On Unraid, the share \\tower\test is represented on Linux as /mnt/user/test

    3. Linux folder names are case sensitive.

     

    So your folder names in Linux should look like this: 

    movies="/mnt/user/test/movies"
backup="/mnt/user/test/backup"

     

    Command to move all mkv files up a folder

    in General Support

    Posted · Edited by JoeUnraidUser

    52 minutes ago, gnollo said:

    Ok, thank you again Joe, I am all set. How do I know that the script will not affect any other folder (containing another 3500 movies) than the one where all the DVDs are set? I moved them to a different location and left a few behind in the DVD folder, to perform a test run.

    It will only affect the folders under the folders you define with the variables "movies" and "backup"

    Command to move all mkv files up a folder

    in General Support

    Posted · Edited by JoeUnraidUser

    23 hours ago, gnollo said:

    I've never run a script on unraid before, how do I do that (I mostly still a windows guy)?

    1. Install the "CA User Scripts" app.
    2. Go to the settings tab and click on the "User Scripts" icon.
    3. Click on "ADD NEW SCRIPT".
    4. Give the script a name.
    5. Click on the name of the script.
    6. Select "Edit Script".
    7. Paste your script and make any changes.
    8. Hit "SAVE CHANGES".
    9. To the right of the name click "RUN SCRIPT".

     

    Edit:  I would test it first on a couple movies.  For example: 

    test
├── backup
└── movies
    ├── everything you wanted to ask about sex
    └── forrest gump

     

    Command to move all mkv files up a folder

    in General Support

    Posted · Edited by JoeUnraidUser

    On 9/22/2019 at 6:51 PM, gnollo said:

    Thank you Joe! I checked one more time and it is slightly more complex than what I stated.

    The program created an additional folder in each VIDEO_TS file, which adds underscores if the folder name has multiple words in it. Examples:

    \\tower\unraid\dvds\forrest gump\VIDEO_TS\FORREST_GUMP

    \\tower\unraid\dvds\everything you wanted to ask about sex\VIDEO_TS\EVERYTHING_YOU_WANTED_TO_ASK_ABOUT_SEX

    I modified the script to move "mkv" files from anywhere under the movie folder.

    Command to move all mkv files up a folder

    in General Support

    Posted · Edited by JoeUnraidUser

    Here is a script that should do what you need.  Set "movies" and "backup" to your folders.

    move_movies.sh 

    #!/bin/bash

movies="/mnt/user/test/movies"
backup="/mnt/user/test/backup"

process()
{
    local movieDir=$1
    local backupDir=$2
    
    pushd "$movieDir" >/dev/null
        find ./ -iname "*.mkv" -exec mv -f {} . \;
        mkdir -p "$backupDir"
        mv -f AUDIO_TS VIDEO_TS "$backupDir"
    popd >/dev/null
}

ls -1d "$movies"/*/ | sed 's/\/*$//' |\
while read movie
do
    name="$(sed 's/.*\///g' <<< $movie)"
    back="$backup/$name"

    printf "$name\n"

    if [ -d "$movie/DVD1" ]
    then
        ls -1d "$movie"/DVD*/ | sed 's/\/*$//' |\
        while read dvd
        do
            num="$(sed 's/.*\///g' <<< $dvd)"
            process "$dvd" "$back/$num"
        done
    else
        process "$movie" "$back"
    fi
done

     

    Additional Scripts For User.Scripts Plugin

    in Plugin Support

    Posted · Edited by JoeUnraidUser

    Script to convert text files from DOS to Unix format.

     

    dos2unix

      

    #!/bin/bash

# Convert text files from DOS to Unix format.

if [ $# -eq 0 ] || [ "$1" == "--help" ]
then
	printf "Usage: dos2unix <files>...\n"
	exit 0
fi

for file in "${@}"
do
	user=$(stat -c '%U' "$file")
	group=$(stat -c '%G' "$file")
	perms=$(stat -c "%a" "$file")

	tmp="$file.$(date +%N)"
	cat "$file" | fromdos > "$tmp"
	mv "$tmp" "$file"

	chown $user:$group "$file"
	chmod $perms "$file"
done

     

    Script to convert text files from Unix to DOS format.

     

     unix2dos 

    #!/bin/bash

# Convert text files from Unix to DOS format.

if [ $# -eq 0 ] || [ "$1" == "--help" ]
then
	printf "Usage: unix2dos <files>...\n"
	exit 0
fi

for file in "${@}"
do
	user=$(stat -c '%U' "$file")
	group=$(stat -c '%G' "$file")
	perms=$(stat -c "%a" "$file")

	tmp="$file.$(date +%N)"
	cat "$file" | todos > "$tmp"
	mv "$tmp" "$file"

	chown $user:$group "$file"
	chmod $perms "$file"
done

     

    • Upvote 1

    UNRAID 6.7.1 easy way to backup and restore specific dockers

    in General Support

    Posted · Edited by JoeUnraidUser

    Backup/Restore dockers.

     

    The backup directory contains cache files and Gzip files.  Running dockers are stopped and started during backup and restore.  Gzip compression is used so the owner and permissions are preserved in the backup files.  If you want to run with hard coded variables, then set the variables under Defaults.

     

    Usage: 

    Usage: backupDockers.sh: [-a] [-d <backup directory>] [<dockers and/or archive files>...]

  -b : backup mode
  -r : restore mode
  -l : list dockers
  -a : all dockers
  -c : crc comparison during rsync, default is check time and size
  -d : set backup directory
  -s : save backup during restore

    Examples:

     

    Backup dockers into a specific backup directory: 

    backupDockers.sh -b -d /mnt/user/backup/dockers binhex-plexpass transmission

    Restore docker latest file and a certain docker file from a specific backup directory: 

    backupDockers.sh -r -d /mnt/user/backup/dockers binhex-plexpass transmission.2019-07-02-12-30-38-EDT.tgz

    Backup all dockers into a specific backup directory: 

    backupDockers.sh -bad /mnt/user/backup/dockers

    Restore all dockers from a specific backup directory: 

    backupDockers.sh -rad /mnt/user/backup/dockers

    Source:

    If copy/paste doesn't work then download the file.  backupDockers.sh 

    #!/bin/bash

# Defaults
backup="/mnt/user/Backup/Dockers"
restore=false
all=false
checksum=false
dockers=()
files=()

usage()
{
    echo "Usage: backupDockers.sh: [-a] [-d <backup directory>] [<dockers and/or archive files>...]"
    echo
    echo "  -b : backup mode"
    echo "  -r : restore mode"
    echo "  -l : list dockers"
    echo "  -a : all dockers"
    echo "  -c : crc comparison during rsync, default is check time and size"
    echo "  -d : set backup directory"
    echo "  -s : save backup during restore"
    echo
    exit 1
}

while getopts 'brlacd:?h' opt
do
    case $opt
    in
        b) restore=false
           ;;
        r) restore=true
           ;;
        l) docker ps -a --format "{{.Names}}" | sort -fV
           exit 0
           ;;
        a) all=true
           ;;
        c) checksum=true
           ;;
        d) backup=${OPTARG%/}
           ;;
    h|?|*) usage
           ;;
    esac
done

shift $(($OPTIND - 1))

if [ "$all" == "true" ]
then
    readarray -t all < <(printf '%s\n' "$(docker ps -a --format "{{.Names}}" | sort -fV)")
else
    all=()
fi

readarray -t items < <(printf '%s\n' "$@" "${dockers[@]}" "${all[@]}" | awk '!x[$0]++')

[ "${items[0]}" == "" ] && usage

for item in "${items[@]}"
do
    if echo $item | grep -sqP ".+\.\d\d\d\d-\d\d-\d\d-\d\d-\d\d-\d\d-\w+\.tgz"
    then
        files+=("$item")
    else
        if [ ! -z $item ]
        then
            dockers+=("$item")
        fi
    fi
done

date=$(date +"%Y-%m-%d-%H-%M-%S-%Z")
echo "DATE: $date"

appdata="/mnt/user/appdata"
cache="$backup/appdata"

if [ "$restore" == "true" ]
then
    restores=()
    errors=()

    for docker in "${dockers[@]}"
    do
        file="$(ls -t $backup/$docker/*.tgz 2>/dev/null | head -1)"

        [ -e "$file" ] && restores+=("$file") || errors+=("$docker")
    done

    for file in "${files[@]}"
    do
        docker=$(echo $file | cut -d '.' -f 1)
        file="$backup/$docker/$file"

        [ -e "$file" ] && restores+=("$file") || errors+=("$file")
    done

    for error in "${errors[@]}"
    do
        archive=$(echo "$error" | rev | cut -d '/' -f 1 | rev)

        echo "ERROR: $archive: archive not found"
    done

    readarray -t restores < <(printf '%s\n' "${restores[@]}" | awk '!x[$0]++')

    for restore in "${restores[@]}"
    do
        archive=$(echo "$restore" | rev | cut -d '/' -f 1 | rev)
        docker=$(echo "$archive" | cut -d '.' -f 1)

        [ "$docker" == "" ] && continue

        echo "DOCKER: $docker"

        running=$(docker ps --format "{{.Names}}" -f name="^$docker$")

        if [ "$docker" == "$running" ]
        then
            echo "STOP: $docker"
            docker stop --time=30 "$docker" >/dev/null
        fi

        cd "$appdata"

        backup="$docker.$date"

        echo "MOVE: $docker -> $backup"
        mv -f "$docker" "$backup" 2> /dev/null

        if [ -d "$backup" ]
        then
            echo "RESTORE: $archive -> $appdata"
            #tar --same-owner --same-permissions -xzf "$restore"
            pv $restore | tar --same-owner --same-permissions -xzf -
            if [ ! -d "$appdata/$docker" ]
            then
                echo "ERROR: restore failed"
                mv -f "$backup" "$docker" 2> /dev/null

                if [ ! -d "$appdata/$docker" ]
                then
                    echo "ERROR: repair failed"
                fi
            fi

            if [ ! -d $docker ]
            then
                echo "ERROR: restore failed"
           fi
        else
            echo "ERROR: move failed"
        fi

        if [ "$docker" == "$running" ]
        then
            echo "START: $docker"
            docker start "$docker" >/dev/null
        fi
    done
else
    for docker in "${files[@]}" "${dockers[@]}"
    do
        if ! docker ps -a --format "{{.Names}}" | grep $docker -qs
        then
            echo "ERROR: $docker: docker not found"
        fi
    done

    mkdir -p "$backup" "$cache"
    chown nobody:users "$backup" "$cache"
    chmod ug+rw,ug+X,o-rwx "$backup" "$cache"

    for docker in "${dockers[@]}"
    do
        if [ -d "$appdata/$docker" ]
        then
            echo "DOCKER: $docker"

            running=$(docker ps --format "{{.Names}}" -f name="^$docker$")

            if [ "$docker" == "$running" ]
            then
                echo "STOP: $docker"
                docker stop --time=30 "$docker" >/dev/null
            fi

            echo "SYNC: $docker"
            [ "$checksum" == "true" ] && checksum=c || checksum=
            rsync -ha$checksum --delete "$appdata/$docker" "$cache"

            if [ "$docker" == "$running" ]
            then
                echo "START: $docker"
                docker start "$docker" >/dev/null
            fi

            mkdir -p "$backup/$docker"

            echo "GZIP: $docker.$date.tgz"
            tar cf - -C "$cache" "$docker" -P | pv -s $(du -sb "$cache/$docker" | cut -f 1) | gzip > "$backup/$docker/$docker.$date.tgz"
            chown -R nobody:users "$backup/$docker"
            chmod -R ug+rw,ug+X,o-rwx "$backup/$docker"
        fi
    done
fi

date=$(date +"%Y-%m-%d-%H-%M-%S-%Z")
echo "DATE: $date"

     

    • Like 5
    • Thanks 2

    [Plugin] CA User Scripts

    in Plugin Support

    Posted · Edited by JoeUnraidUser

    21 hours ago, ProphetSe7en said:

    I need a script to stopp all containers at a given time, and then restart them again after xx minutes. I want to run duplicati on my appdata once a week and for the backup to be done properly I need the dockers to be stopped during backup.

     

    Anyone have a script that I can use for this?

    This script will stop the running dockers, sleep for a specified duration, and start the previously running dockers.

    If you want to run your backup from this script, just replace "echo Pausing $pause..." and "sleep $pause" with your own code. 

    #!/bin/bash

pause=1m

running=$(docker ps --format "{{.Names}}" | sort -fV)

echo Stopping Dockers...
for docker in $running
{
	docker stop --time=30 $docker
}

echo Pausing $pause...
sleep $pause

echo Starting Dockers...
for docker in $running
{
	docker start $docker
}

    pauseDockers.sh

     

    Edit: Copy/Paste is a pain.

    Clean up command for folders under 10mb?

    in General Support

    Posted · Edited by JoeUnraidUser
    Changed wording.

    I know this is overkill, but it only took about 15 minutes.  Here are 3 different scripts for different situations:

     

    Remove directories within a source directory over a certain size: 

    #!/bin/bash

limit=20G
source="/mnt/user/Movies"

limit=$(echo $limit | cut -d 'G' -f 1)

for dir in "$source"/*/
do
	size=$(du -sBG "$dir" | cut -d 'G' -f 1)

	if (( $size > $limit ))
	then
		echo remove: $dir
		rm -rf "$dir"
	fi
done

    removeDirOverSize.sh

     

    Remove files within a source directory over a certain size: 

    #!/bin/bash

limit=20G
source="/mnt/user/Movies"

find "$source" -type f -size +$limit -delete -print

    removeFilesOverSize.sh

     

    Remove files within a source directory over a certain size with specific extensions: 

    #!/bin/bash

limit=20G
source="/mnt/user/Movies"

find "$source" -regextype posix-extended -iregex '.*\.(mpg|mkv)' -size +$limit -delete -print

    To change extensions you could replace mpg|mkv with avi or mpg|mkv|avi etc...

    removeFilesOverSizeByExtension.sh

     

    • Like 1

    Clean up command for folders under 10mb?

    in General Support

    Posted · Edited by JoeUnraidUser
    Renamed script.

    Here is a script that will remove directories within a source directory less than a certain size limit.  I used 100M, but you can change it to any size in MB.  The script can be easily changed to KB, GB, TB etc...  Don't forget to set your source directory. 

    #!/bin/bash

limit=100M
source="/mnt/user/Movies"

limit=$(echo $limit | cut -d 'M' -f 1)

for dir in "$source"/*/
do
	size=$(du -sBM "$dir" | cut -d 'M' -f 1)

	if (( $limit > $size ))
	then
		echo remove: $dir
		rm -rf "$dir"
	fi
done

    removeDirUnderSize.sh

    • Like 2
    • Thanks 1

    appdata permissions

    in General Support

    Posted

    4 hours ago, Squid said:

    Copying and pasting from the forum more often than not inserts hidden characters

    Sent via telekinesis
     

    When I copy and paste from now on, I will also attach a script with the code just to be safe.

    appdata permissions

    in General Support

    Posted · Edited by JoeUnraidUser

    try it again.  I repasted into the post for some reason there was a special character pasted in the code. 

    chown -cR nobody:users /mnt/user/appdata/letsencrypt /mnt/user/appdata/nextcloud
chmod -cR ug+rw,ug+X,o-rwx /mnt/user/appdata/letsencrypt /mnt/user/appdata/nextcloud

     

    • Thanks 1
×
×
  • Create New...