Log shows an error when running mover: "/usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 397: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")"

 

Any idea what this means and how to prevent it from occurring?

7 hours ago, helpful-tune3401 said:

Log shows an error when running mover: "/usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 397: ((: TOTALARRAYSIZE += : syntax error: operand expected (error token is "+= ")"

 

Any idea what this means and how to prevent it from occurring?

This code is here since a while. It will be replaced in next version (coming really soon)

 

Can you check if /tmp/Mover/Custom_Cache_Tuning_(date).list is populated?

 

You can also try to replace this:

((TOTALARRAYSIZE += $CUSTOMFILESIZE))

by that:

TOTALARRAYSIZE += $CUSTOMFILESIZE

In this file

/usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 397

 

Edited July 22, 20242 yr by Reynald

i removed the old and installed this version from the community apps (not manually with a link from the forum).

 

Jul 22 20:38:27 unraid emhttpd: shcmd (329893): /usr/local/sbin/mover start |& logger -t move &
Jul 22 20:38:27 unraid kernel: mdcmd (37): set md_write_method 1
Jul 22 20:38:27 unraid kernel: 
Jul 22 20:38:28 unraid move: Log Level: 1
Jul 22 20:38:28 unraid move: mover: started
Jul 22 20:38:29 unraid move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 260: /usr/local/sbin/move: No such file or directory
*repeated About 20 times*
Jul 22 20:38:30 unraid move: mover: finished

 

36 minutes ago, Grobalt said:

i removed the old and installed this version from the community apps (not manually with a link from the forum).
[..]
Jul 22 20:38:29 unraid move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 260: /usr/local/sbin/move: No such file or directory
*repeated About 20 times*
Jul 22 20:38:30 unraid move: mover: finished

Community Apps version is not compatible with unraid 7

 

19 hours ago, Reynald said:

Teasing: next version log example:

[...]
See for each share what "Cache mode" does.
See how smart are the 3 last lines

I've still some work to do on moving function (which send files to Unraid Mover) before releasing this new version

After the log, the UI with some new options

no changes with this one: 

https://github.com/R3yn4ld/ca.mover.tuning/blob/794d8b0fad8ea1c2db8096dbd753ecf75c8a6e22/plugins/ca.mover.tuning.plg

 

First test yes - then with test no:

 

Quote

Jul 22 21:52:16 unraid emhttpd: shcmd (331937): /usr/local/sbin/mover start |& logger -t move &
Jul 22 21:52:16 unraid kernel: mdcmd (51): set md_write_method 1
Jul 22 21:52:16 unraid kernel: 
Jul 22 21:52:16 unraid move: Log Level: 1
Jul 22 21:52:16 unraid move: mover: started
Jul 22 21:52:20 unraid move: mover: finished
Jul 22 21:52:20 unraid kernel: mdcmd (52): set md_write_method auto
Jul 22 21:52:20 unraid kernel: 
Jul 22 21:52:41 unraid emhttpd: shcmd (331938): /usr/local/sbin/mover start |& logger -t move &
Jul 22 21:52:41 unraid kernel: mdcmd (53): set md_write_method 1
Jul 22 21:52:41 unraid kernel: 
Jul 22 21:52:41 unraid move: Log Level: 1
Jul 22 21:52:41 unraid move: mover: started
Jul 22 21:52:42 unraid move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 260: /usr/local/sbin/move: No such file or directory
*repeated*
Jul 22 21:52:43 unraid move: mover: finished
Jul 22 21:52:43 unraid kernel: mdcmd (54): set md_write_method auto
Jul 22 21:52:43 unraid kernel: 

 

Edited July 22, 20242 yr by Grobalt

2 hours ago, Grobalt said:

no changes with this one: 

https://github.com/R3yn4ld/ca.mover.tuning/blob/794d8b0fad8ea1c2db8096dbd753ecf75c8a6e22/plugins/ca.mover.tuning.plg

 

First test yes - then with test no:

 

 

Very strange because creating symlinks is part of the installation of the latest plugin.

 

Please try this code in console and report:

if [[ -e /usr/local/bin/move ]]; then
  echo "Creating Soft link for 6.12 RC8 or newer since /usr/local/sbin/move is now located in /usr/local/bin/move"
  if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi
  ln -s /usr/local/bin/move /usr/local/sbin/move
elif [[ -e /usr/libexec/unraid/move ]]; then
  echo "Creating Soft link for 7.0 or newer since /usr/local/sbin/move is now located in /usr/libexec/unraid/move"
  if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi
  ln -s /usr/libexec/unraid/move /usr/local/sbin/move
fi

 

So I believe I've tried mover tuning before, but had to remove it due to bugs/issues at the time with my setup. As a note, I'm still on UnRAID version 6.12.10. I intend to stick to 6.12.xx until a couple minor releases into version 7.

 

So these are my pool devices:

 

And these are my share settings:

 

Is the plugin (either the Squid version or this more recent fork) able to handle the use case of me setting a % threshold on the 'download_drive' hard drive for triggering the mover? My drive named 'cache' holds my appdata, domains, and systems files - which is pretty consistently 200GB / 1TB. It's the `Download_drive` that is constantly having new media added to it - and that is the one I'd want to have watched for when it gets close to being filled (i.e. when the 'Download_drive' gets over 70%, start moving files off of it to the array). But I'm not sure if the plugin can be set up for that use case?

Edited July 23, 20242 yr by Magic815

4 hours ago, Magic815 said:

It's the `Download_drive` that is constantly having new media added to it - and that is the one I'd want to have watched for when it gets close to being filled (i.e. when the 'Download_drive' gets over 70%, start moving files off of it to the array). But I'm not sure if the plugin can be set up for that use case?

 

Actually 'per pool threshold' is not working, so with this plugin, you would set a 70% threshold and the files of each shares above 70% would be moved to array. That's not a problem as you cache drive is about 20%

On 7/23/2024 at 12:15 AM, Reynald said:

Very strange because creating symlinks is part of the installation of the latest plugin.

 

Please try this code in console and report:

if [[ -e /usr/local/bin/move ]]; then
  echo "Creating Soft link for 6.12 RC8 or newer since /usr/local/sbin/move is now located in /usr/local/bin/move"
  if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi
  ln -s /usr/local/bin/move /usr/local/sbin/move
elif [[ -e /usr/libexec/unraid/move ]]; then
  echo "Creating Soft link for 7.0 or newer since /usr/local/sbin/move is now located in /usr/libexec/unraid/move"
  if [[ -e /usr/local/sbin/move ]]; then rm /usr/local/sbin/move; fi
  ln -s /usr/libexec/unraid/move /usr/local/sbin/move
fi

 

./test.sh
Creating Soft link for 7.0 or newer since /usr/local/sbin/move is now located in /usr/libexec/unraid/move

Ok.

And is the age_mover running now?

fresh reinstalled and a run with test mode:

 

Jul 24 21:19:17 unraid plugin-manager: ca.mover.tuning.plg installed
Jul 24 21:20:03 unraid kernel: mdcmd (37): nocheck pause
Jul 24 21:20:03 unraid kernel: md: recovery thread: exit status: -4
Jul 24 21:20:06 unraid emhttpd: shcmd (1338): /usr/local/sbin/mover start |& logger -t move &
Jul 24 21:20:06 unraid move: Log Level: 1
Jul 24 21:20:06 unraid move: /tmp/Mover Directory does not exist. Creating it
Jul 24 21:20:06 unraid move: mover: started
Jul 24 21:20:10 unraid flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update
Jul 24 21:20:13 unraid move: mover: finished

 

on my cache disk are around 400GB to be moved .. dont know what the test mode should report - i expected filenames to be moved

 

i am not using "age" as criteria, i am using size above 14M to keep media library pictures on the flash drives

Edited July 24, 20242 yr by Grobalt

Is it possible to configure the mover to move files based on age and when the cache is over 60% full?

Currently, when I set it to move files based on age and Move All from Cache-yes shares pool percentage, it only works based on age.

2 hours ago, Grobalt said:

fresh reinstalled and a run with test mode:

 

on my cache disk are around 400GB to be moved .. dont know what the test mode should report - i expected filenames to be moved

 

i am not using "age" as criteria, i am using size above 14M to keep media library pictures on the flash drives

The report are in the file in /tmp/Mover. You can check the list of what would be moved with test mode: no

to keep media library pictures on ssd drives, set the share(s) to "Cache=only". In latest unraid version, do not set a secondary storage for it.

1 hour ago, Gobblerpl said:

Is it possible to configure the mover to move files based on age and when the cache is over 60% full?

Currently, when I set it to move files based on age and Move All from Cache-yes shares pool percentage, it only works based on age.

"Move All from Cache-yes" will move all based on the threhold for this option.
Do not select this option and just set "Only move at/to this threshold of used cache space:"  and "Move files off cache based on age?"

3 minutes ago, Reynald said:

The report are in the file in /tmp/Mover. You can check the list of what would be moved with test mode: no

to keep media library pictures on ssd drives, set the share(s) to "Cache=only". In latest unraid version, do not set a secondary storage for it.

that will not work as the movie scraper store the pictures in the same folder as the movie - otherwise it would be super easy to create an flash only share. the mover worked and moved all files above 14M size to array.

Ah OK, understood. Then the file size filter is effectivly your friend

On 7/24/2024 at 11:33 PM, Reynald said:

"Move All from Cache-yes" will move all based on the threhold for this option.
Do not select this option and just set "Only move at/to this threshold of used cache space:"  and "Move files off cache based on age?"

Doesn't work. 

Updated:

 

2024-07-29

• Automatic Array to Cache
• Complete rewrite of file listing functions (find, decide to move..) (R3yn4ld)
• Updated zfs functions getting usage of a pool (R3yn4ld, greetings to (Freender on forums) for pointing out and proposal)
• Added cache mode "prefer" smart moving in "Automatic age" mode (R3yn4ld)
• Added option to "repair" Cache:Only (moving everything on share to cache) and Cache:No (moving everything on share to array) shares (R3yn4ld)
• Added option to synchronize Cache:Yes and Cache:Prefer shares to array so data are parity protected (R3yn4ld)
• Turbo write mode forcing improvement to not wake spinners if not needed (R3yn4ld)
• UI improvements (R3yn4ld)

Full changelog: R3yn4ld/ca.mover.tuning at automatic-array-to-cache (github.com)

 

It is a huge huge update. I've tested for a couple of weeks. Hopefully you won't find many bugs, please report in any case.

 

Installation: ca.mover.tuning.plg

Please remove Community Application one before installing this one (at the moment via Plugins/Install plugin/insert above URL).

Please note that you'll have to reconfigure mover tuning.

Update: if already installed with above method, you should be able to update normally:

(above is the setting page of 2024-07-29 already  )

 

Advise: enable Test Mode as on picture above and verify the Mover_action_{date}T{time}list in the /tmp/ca.mover.tuning directory. This file contain the files that would be moved with Test Mode off.

If everything seems OK you can turn off Test mode.

Edited July 29, 20242 yr by Reynald

1) After a fresh installation (2024.07.29), I am not able to see the Mover settings in Settings > Scheduler.

 

I believe the issue is with the freshly generated /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg. The content inside is written as a single line. After I broke down the lines, it started working.
2) I also suggest setting the “Move Now button follows plug-in filters” option to ‘Yes’ by default.

Edited July 29, 20242 yr by Freender
added plugin version and suggestion

3)  In /boot/config/plugins/ca.mover.tuning/shareOverrideConfig/media.cfg

threshold is set:

 

shareThreshold="85"

 

However in https://github.com/R3yn4ld/ca.mover.tuning/blob/master/source/ca.mover.tuning/usr/local/emhttp/plugins/ca.mover.tuning/age_mover

 

threshold is misspelled (shareTreshold instead of shareThreshold)

 

    if [ -z $(cfg shareTreshold) ]; then
        SHAREPCTLIMIT="NA"
    else
        SHAREPCTLIMIT=$(cfg shareTreshold)
        mvlogger "Share threshold: $SHAREPCTLIMIT %"

 

As a result mover fails if a share has an override. 

Jul 29 00:58:46 NAS move: ---------- Settings override for media: ----------
Jul 29 00:58:46 NAS move: Default threshold: 
Jul 29 00:58:46 NAS move: Age: Automatic (smart caching)
Jul 29 00:58:46 NAS move: No Turbo Mode argument supplied
Jul 29 00:58:46 NAS move: Skip file list path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 00:58:46 NAS move: Skip folder list path: /mnt/cache/media/downloads/*
Jul 29 00:58:46 NAS move: No Repair argument provided
Jul 29 00:58:46 NAS move: No Sparness argument supplied
Jul 29 00:58:46 NAS move: No Synchronize Cache argument provided
Jul 29 00:58:46 NAS move: No Test Mode argument supplied
Jul 29 00:58:46 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 00:58:46 NAS move: Share Information: Name: media - Path: /mnt/cache/media - Cache mode: yes
Jul 29 00:58:46 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3362321969248 bytes (3 TiB)
Jul 29 00:58:46 NAS move: /usr/local/emhttp/plugins/ca.mover.tuning/age_mover: line 396: 3868846981120 *  /100: syntax error: operand expected (error token is "/100")

 

 

I am going to sleep, and I will check if this helps tomorrow.

6 hours ago, Freender said:

1) After a fresh installation (2024.07.29), I am not able to see the Mover settings in Settings > Scheduler.

 

I believe the issue is with the freshly generated /boot/config/plugins/ca.mover.tuning/ca.mover.tuning.cfg. The content inside is written as a single line. After I broke down the lines, it started working.
2) I also suggest setting the “Move Now button follows plug-in filters” option to ‘Yes’ by default.

1) corrected in 2024.07.29.b

2) good idea, implemented !

5 hours ago, Freender said:

3)  In /boot/config/plugins/ca.mover.tuning/shareOverrideConfig/media.cfg

threshold is set:

 

shareThreshold="85"

 

3) Corrected in 2024-07-29.b
I had to make some adaptation in getMoverSettings() function to not overwrite defaults when no set for share override.

Result:

13:03:47.499 ------------------------------------------------------------ Settings for media share -----------------------------------------------------------
13:03:47.505 Settings override:
13:03:47.509 ------------------
13:03:47.520 Age: Automatic (smart caching)
13:03:47.537 Share threshold: 60 %
13:03:47.552 ------------------------------------------------------------------------  -----------------------------------------------------------------------
13:03:47.562 Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
13:03:47.566 Share Information: Name: media - Path: /mnt/cache/media - Cache mode: yes
13:03:47.590 Pool Information: Name: cache - pool used: 89 % - size: 856872009728 bytes (798 GiB)
13:03:47.606 Default threshold: 95 % - media share threshold: 60 % => Threshold selected: 60 %,s size: 554372903731 bytes (516 GiB)
13:03:48.597 Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-07-29T130347.list for media
13:03:48.606 ----------------------------------------------------------------- Global settings ---------------------------------------------------------------

 

Edited July 29, 20242 yr by Reynald

New version: 2024-07-30.0000

 

Implement above fixes and automatically recreate config file if broken from 2024-07-29 version fresh install

Edited July 29, 20242 yr by Reynald

@Reynald thanks for your support.

1) Greatly appreciate these 2 settings:

• Show advanced filters
• Show advanced options:

This makes the plugin cleaner and focuses on the main functions.

2) To make debugging easier I simplified my config (disabled override shares). 

My current pool usage is 84% and threshold is 85%.

Expected Result: I don't expect any files to be moved from cache to array with this config

Actual Results: "/tmp/ca.mover.tuning/Filtered_files_2024-07-29T125104.list for media" file has created.

Size - 205K

 

-rw-r--r--  1 root root 205K Jul 29 12:51 Filtered_files_2024-07-29T125104.list

 

Full Log:

Jul 29 12:51:04 NAS move: ***************************************************************** FILTERING FILES ***************************************************************
Jul 29 12:51:04 NAS move: ----------------------------------------------------------- Settings for appdata share ----------------------------------------------------------
Jul 29 12:51:04 NAS move: Cache mode: Only. Nothing will be moved. Its usage is taken into account in the calculation of the threshold.
Jul 29 12:51:04 NAS move: Share Information: Name: appdata - Path: /mnt/cache/appdata - Cache mode: only
Jul 29 12:51:04 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671008256 bytes (3 TiB)
Jul 29 12:51:04 NAS move: cache/appdata usage: 277430949136 bytes (258 GiB)
Jul 29 12:51:04 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Skipfilelist string: find "/mnt/cache/appdata" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:04 NAS move: ------------------------------------------------------------ Settings for backup share ----------------------------------------------------------
Jul 29 12:51:04 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:04 NAS move: Share Information: Name: backup - Path: /mnt/cache/backup - Cache mode: yes
Jul 29 12:51:04 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671008256 bytes (3 TiB)
Jul 29 12:51:04 NAS move: Default threshold: 85 % - backup share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:04 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Skipfilelist string: find "/mnt/cache/backup" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:04 NAS move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-07-29T125104.list for backup
Jul 29 12:51:04 NAS move: ------------------------------------------------------------ Settings for docker share ----------------------------------------------------------
Jul 29 12:51:04 NAS move: Cache mode: Only. Nothing will be moved. Its usage is taken into account in the calculation of the threshold.
Jul 29 12:51:04 NAS move: Share Information: Name: docker - Path: /mnt/cache/docker - Cache mode: only
Jul 29 12:51:04 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671008256 bytes (3 TiB)
Jul 29 12:51:04 NAS move: cache/docker usage: 7979181760 bytes (7 GiB)
Jul 29 12:51:04 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Skipfilelist string: find "/mnt/cache/docker" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:04 NAS move: ------------------------------------------------------------- Settings for isos share -----------------------------------------------------------
Jul 29 12:51:04 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:04 NAS move: Share Information: Name: isos - Path: /mnt/cache/isos - Cache mode: yes
Jul 29 12:51:04 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671008256 bytes (3 TiB)
Jul 29 12:51:04 NAS move: Default threshold: 85 % - isos share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:04 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:04 NAS move: Skipfilelist string: find "/mnt/cache/isos" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:04 NAS move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-07-29T125104.list for isos
Jul 29 12:51:04 NAS move: ------------------------------------------------------------ Settings for media share -----------------------------------------------------------
Jul 29 12:51:04 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:04 NAS move: Share Information: Name: media - Path: /mnt/cache/media - Cache mode: yes
Jul 29 12:51:04 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671008256 bytes (3 TiB)
Jul 29 12:51:04 NAS move: Default threshold: 85 % - media share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:05 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:05 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:05 NAS move: Skipfilelist string: find "/mnt/cache/media" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: Updated Filtered filelist: /tmp/ca.mover.tuning/Filtered_files_2024-07-29T125104.list for media
Jul 29 12:51:08 NAS move: ----------------------------------------------------------- Settings for pictures share ---------------------------------------------------------
Jul 29 12:51:08 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:08 NAS move: Share Information: Name: pictures - Path: /mnt/cache/pictures - Cache mode: yes
Jul 29 12:51:08 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671668432 bytes (3 TiB)
Jul 29 12:51:08 NAS move: Default threshold: 85 % - pictures share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:08 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Skipfilelist string: find "/mnt/cache/pictures" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: ------------------------------------------------------------ Settings for steam share -----------------------------------------------------------
Jul 29 12:51:08 NAS move: Cache mode: Only. Nothing will be moved. Its usage is taken into account in the calculation of the threshold.
Jul 29 12:51:08 NAS move: Share Information: Name: steam - Path: /mnt/cache/steam - Cache mode: only
Jul 29 12:51:08 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671668432 bytes (3 TiB)
Jul 29 12:51:08 NAS move: cache/steam usage: 55265793616 bytes (51 GiB)
Jul 29 12:51:08 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Skipfilelist string: find "/mnt/cache/steam" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: ------------------------------------------------------------ Settings for system share ----------------------------------------------------------
Jul 29 12:51:08 NAS move: Cache mode: Only. Nothing will be moved. Its usage is taken into account in the calculation of the threshold.
Jul 29 12:51:08 NAS move: Share Information: Name: system - Path: /mnt/cache/system - Cache mode: only
Jul 29 12:51:08 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671668432 bytes (3 TiB)
Jul 29 12:51:08 NAS move: cache/system usage: 152768 bytes (149 KiB)
Jul 29 12:51:08 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Skipfilelist string: find "/mnt/cache/system" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: --------------------------------------------------------- Settings for time_machine share -------------------------------------------------------
Jul 29 12:51:08 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:08 NAS move: Share Information: Name: time_machine - Path: /mnt/cache/time_machine - Cache mode: yes
Jul 29 12:51:08 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671668432 bytes (3 TiB)
Jul 29 12:51:08 NAS move: Default threshold: 85 % - time_machine share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:08 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Skipfilelist string: find "/mnt/cache/time_machine" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: ------------------------------------------------------- Settings for time_machine_pc share ------------------------------------------------------
Jul 29 12:51:08 NAS move: Cache mode: Yes/Cache->Array. Will smart move old files from Cache to Array. Nothing will be moved from Array to Cache
Jul 29 12:51:08 NAS move: Share Information: Name: time_machine_pc - Path: /mnt/cache/time_machine_pc - Cache mode: yes
Jul 29 12:51:08 NAS move: Pool Information: Name: cache - pool used: 84 % - size: 3372671668432 bytes (3 TiB)
Jul 29 12:51:08 NAS move: Default threshold: 85 % - time_machine_pc share threshold: NA % => Threshold selected: 85 %,s size: 3288519933952 bytes (2 TiB)
Jul 29 12:51:08 NAS move: Adding Skip Folder List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Adding Skip File List. List Path: /mnt/user/backup/mover-ignore/mover_ignore.txt
Jul 29 12:51:08 NAS move: Skipfilelist string: find "/mnt/cache/time_machine_pc" -type f -depth -not -path '/mnt/cache/media/downloads/*' -printf '%T@|%s|%S|%n|%i|%p|\0' | awk -v RS='\0' -v FS='|' -v SPARSENESS=$SPARSENESS -v CACHEPOOLNAME=$CACHEPOOLNAME -v SHARENAME=$SHARENAME -v SHAREUSECACHE=$SHAREUSECACHE -v CACHESIZETHRESH=$CACHESIZETHRESH '{cmd="stat -c%Y \""$6"\""; cmd | getline result; close(cmd); sub(/\n$/, "", result); printf "%s|%s|%s|%d|%d|%d|%s|%d|%d|%s\n", CACHEPOOLNAME, SHARENAME, SHAREUSECACHE, result, CACHESIZETHRESH, $2, $3, $4, $5, $6}' | grep -vFx -f <(sed 's/\/*$//' '/mnt/user/backup/mover-ignore/mover_ignore.txt')
Jul 29 12:51:08 NAS move: ---------------------------------------------------------- Settings for zfs_backup share --------------------------------------------------------
Jul 29 12:51:08 NAS move: No shareCachePool entry found in config file, defaulting to cache
Jul 29 12:51:08 NAS move: Cache mode: No. Skipping
Jul 29 12:51:08 NAS move: ************************************************************ ANALYSING MOVING ACTIONS ***********************************************************
Jul 29 12:51:08 NAS move: Deciding the action (move/sync/keep) for each file. There are 1065 files, it can take a while...
Jul 29 12:51:10 NAS move: No new files will be moved/synced from cache to array
Jul 29 12:51:10 NAS move: No new files will be moved/synced from array to cache
Jul 29 12:51:10 NAS move: ****************************************************************** WE ARE DONE ! ****************************************************************