[SOLVED] MOVER issue with APPDATA and SYSTEM on cache:

[RobertP]

UnRAID 6.11.5

MOVER issue with APPDATA and SYSTEM on cache:
I’ve capitalized some names here just to make them stand out – but on the system they are in lower case as normal.

 

I’m having a problem with MOVER on one of my unRAID servers.  When there is nothing to move, it still runs for 10 minutes. (By that, I mean the only things on CACHE are APPDATA and SYSTEM folders, nothing else.)

 

I have both APPDATA and SYSTEM set to DISK 1 and CACHE PREFER, so it should not even be trying to move those items.

 

I turned on mover logging, and saw it was trying to move a LOT of Krusader files.

 

I tried setting appdata to CACHE YES and running it a few times – no fix, no change
I tried setting it to CACHE NO and running it a few times – runs instant, but no fix when I put it back to CACHE PREFER.  I re-ran mover several times under CACHE PERFERE no change, still takes 10 min and still tries to move Krusader.

 

I should note that Krusader is NOT active during all of these tests.  Also, Krusader is the only docker I currently have installed on this unRAID server (I did have another docker installed a while back for RGB lighting, but I de-installed that ages ago and no longer have RGB lighting running.)

 

I went to SETTINGS, DOCKER and set it to NO.

 

I then went through the above tests again:
MOVE with appdata set to CACHE PREFER – multiple runs.
MOVE with appdata set to CACHE YES – multiple runs.
MOVE with appdata set to CACHE NO – multiple runs.
MOVE with appdata set to CACHE PREFER – multiple runs.
(I have not attempted a MOVE with it set to CACHE ONLY.)

 

No changes, MOVER still runs long and tries to move Krusader files.

 

I re-enabled DOCKER, and de-installed Krusader.  I then ran through the same tests again.
I then disabled DOCKER and ran through the same tests again.

 

No success.  MOVE still runs long and tries to move Krusader files.

 

Right now, on the CACHE drive, there is a binhex-krusader folder even though that tool is no longer installed.

 

APPDATA is set to DISK 1 with cache set to PREFER. (same for SYSTEM share).
On disk 1, there is no APPDATA folder (and no SYSTEM share).
On cache, there is an APPDATA folder (and a SYSTEM folder.).  Inside APPDATA is a BINHEX-KRUSADER folder.   Inside that are more folders and files.

 

Under the SHARES tab, if I do a COMPUTE for APPDATA it shows there is data on CACHE and DISK 3.  As far as I recall, I never had it located on DISK 3, though it could be possible.  SYSTEM folder is also on CACHE and DISK 3.  (And, of course, the COMPUTE option shows both of the disk 3 shares are outside the list of designated disks.)  

 

I just now noticed the LOG file shows it is trying to move KRUSADER from DISK3 – did not see that before.  

 

In the log, most lines are white, some are yellow, some blue, some red – but it flies past so quick that I have not been able to catch why the color difference.  It LOOKS like they all say “
…file exists”, but it is flying by so fast I can not be sure.

 

The log file is so big and moves so fast, and once done I can only scroll back part way.  Is there a way to save the ENTIRE log file to a file so I can look through it more closely?  Or a way to pause the log screen display?  Since I now see there is a duplicate(?) SYSTEM and APPDATA folder on disk 3, I’d like to verify that it is only DISK 3 that is involved in the MOVE.

 

Disk 3 contents for these two shares:
Disk3/system/docker/docker.img  (4/29/2023 06:18)
Disk3/system/libvirt/libvirt.img (2/12/23 16:46)
Disk3/appdata/Binhex-krusader  folder
 Inside that is PERMS.TXT and SUPERVISORD.LOG and a HOME folder with LOTS.
 (at a QUICK GLANCE, all files/folders appear to be from 2022.)
On cache:
Cache/system/docker/docker.img (4/27/2023 2:21)
Cache/system/libvirt/libvirt.img (4/27/2023 2:22)
Cache/appdata/binhex-krusader  folder 
 Inside that is PERMS.TXT and SUPERVISORD.LOG and a HOME folder with LOTS.
(at a QUICK GLANCE, all files/folders appear to be from 2022.)

 

On disk 1 – neither SYSTEM nor APPDATA show up.

 

My thought is to delete all KRUSADER folders EVERYWHERE since it is no longer installed.  But I’m not sure what to do with the other folders/files on DISK 3 – delete them?  MOVE them to CACHE (that is, overwrite what’s in CACHE)? How do I tell which version is actually being used (the cache version or the disk 3 version).

 

I have CA BACKUP / RESTORE APPDATA plugin installed.  I’ve checked its settings:
Source:  /mnt/cache/appdata/
Destinations: /mnt/disk10/backupappdata, /mnt/disk10/backupflash/, /mnt/disk10/backuplibvert/
So there are no mis-named locations there.
(Why does BACKUP/RESTORE not have a place to put a SOURCE for SYSTEM?)

 

Below are the last few lines of the LOG file – and I now notice that MOVER is trying to move not only

/mnt/disk3/appdata/binhex-krusader . . . 
But also 
/mnt/disk3/system/ . . . 

 

I was able to grab a quick screen print of the front of the log file to see if there is anything showing up before the KRUSADER files –it starts with:
/mnt/disk3/appdata/binhex-krusader/home/.build/gtk/config/.gtkrc-2.0
So it would appear the only items are those disk3 folders.

 

. . . MANY more similar entries before this . . . 
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/gtk-3.0/assets/[email protected] File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/gtk-3.0/gtk-dark.css
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/gtk-3.0/gtk-dark.css File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/gtk-3.0/gtk.css
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/gtk-3.0/gtk.css File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/LICENSE
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/.themes/Ultimate-Maia-Blue-light/LICENSE File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/home/.gtkrc-2.0
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/.gtkrc-2.0 File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/home/novnc-16x16.png
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/home/novnc-16x16.png File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/perms.txt
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/perms.txt File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/appdata/binhex-krusader/supervisord.log
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/appdata/binhex-krusader/supervisord.log File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/system/docker/docker.img
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/system/docker/docker.img File exists
Apr 30 23:06:07 UR1-WOPR  move: file: /mnt/disk3/system/libvirt/libvirt.img
Apr 30 23:06:07 UR1-WOPR  move: move_object: /mnt/disk3/system/libvirt/libvirt.img File exists
Apr 30 23:06:07 UR1-WOPR root: mover: finished

 

Thanks in advance for any guidance/tips/suggestions.

Bob

[itimpi]

It might be a good idea to post a copy of your system’s diagnostics zip file so we can check how things are currently set up.

 

If you have the Dynamix File Manager plugin installed you would be able to delete the Krusader files directly.

 

if files exist in multiple locations then unfortunately you have to decide which copy to keep   In normal operation this scenario should not occur and files would only be at one location.

 

You also mention appdata being set to disk1 and Use Cache = Prefer.    This combination does not normally make sense as the Prefer setting means you want files for the share (on ANY array drive) to be moved to cache if space permits.   The disk1 setting would only be relevant for NEW files that did not fit on the cache.

[RobertP]

I’m a little confused about your comment about disk1/prefer.  We have to assign every share (including appdata and system) to a disk, there is not an option (that I see) to assign it to a disk of CACHE.  If I leave the disk selection of the appdata share blank, it defaults to ALL, which I don’t want.  If cache does fill up, I want to control and know where appdata ends up.  But since I have it set to PREFER, it is basically going to ignore that disk location anyway (right? – unless, heaven forbid, my cache drive fills up).

 

I’ve attached the diagnostic info.

 

Bottom line, you concur that I SHOULD be able to safely delete the KRUSADER files on disk 3 since KRUSADER is current no longer installed, correct?  Ditto for the APPDATA folder on disk 3 – it will be empty after above, so can safely delete it.

 

But for the items in SYSTEM folder, I’ll have to compare the CACHE version to the disk 3 version to try to figure out which is current (I’m guessing cache version is current, but . . . I’ll have to do some digging to be sure).

 

ur1-wopr-diagnostics-20230505-1158.zip

[itimpi]

1 minute ago, RobertP said:

I’m a little confused about your comment about disk1/prefer.  We have to assign every share (including appdata and system) to a disk, there is not an option (that I see) to assign it to a disk of CACHE.  If I leave the disk selection of the appdata share blank, it defaults to ALL, which I don’t want.  If cache does fill up, I want to control and know where appdata ends up.  But since I have it set to PREFER, it is basically going to ignore that disk location anyway (right? – unless, heaven forbid, my cache drive fills up).

If you want appdata to basically all be on the cache pool then most people do not bother to assign a specific disk (I.e. leave that set to ALL).   However if you want to control which disk files get sent to if they do not fit on the cache then it is OK to set a specific disk(s).   Once appdata is all on the cache you can (optionally) set Use Cache=Only to prevent any files subsequently going to the array. 

 

The 6.12 release (which is expected to go to ‘stable’ status any day now) is going to change some of the terminology around this to try and make it clearer (whether it does will be interesting to see) how files are initially placed and where they finally end up.  My suspicion is that it will be clearer for new users but some existing users might initially struggle to get the new presentation clear in their minds having gotten used to the existing terminology.  The actual functionality will not change at that point from what it is now but it will position things better for future releases. 

[RobertP]

Solved

Since I had de-installed Krusader, I was able to delete all left-over occurrences of Krusader folders on CACHE and DISK3.

The SYSTEM files on DISK3 – I renamed them (rather than deleting them – just in case). 

I then ran MOVER – it moved the DISK3 renamed SYSTEM files to CACHE and deleted them from DISK3.  I’ll let them sit on CACHE for a bit in case any issues come up, and then will probably delete them.

MOVER now runs instantly once again when there are no files to move, no more 10 minute lag and when I temporarily turned on MOVER LOGGING it was clean.  DISK3 no longer has any APPDATA nor SYSTEM files.  I re-installed Krusader, it is running fine.