Why is the mover leaving files behind?


FreeMan

Recommended Posts

In an attempt to clear my cache pool so I can replace a failing drive, I've set the use cache setting for all the previously "cache only" shares to be "yes", and I've run the mover. When it finished, it left 224MB of data on the cache pool.

 

After turning on mover logging, my log is full of messages like this:

Quote

Oct 23 18:09:50 NAS move: move: file /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/fileopen.svg
Oct 23 18:09:50 NAS move: move_object: /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/fileopen.svg No such file or directory
Oct 23 18:09:50 NAS root: Specified filename /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/folder_new.svg does not exist.
Oct 23 18:09:50 NAS move: move: file /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/folder_new.svg
Oct 23 18:09:50 NAS move: move_object: /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/folder_new.svg No such file or directory
Oct 23 18:09:50 NAS root: Specified filename /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-info.svg does not exist.
Oct 23 18:09:50 NAS move: move: file /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-info.svg
Oct 23 18:09:50 NAS move: move_object: /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-info.svg No such file or directory
Oct 23 18:09:50 NAS root: Specified filename /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-open.svg does not exist.
Oct 23 18:09:50 NAS move: move: file /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-open.svg
Oct 23 18:09:50 NAS move: move_object: /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-open.svg No such file or directory
Oct 23 18:09:50 NAS root: Specified filename /mnt/cache/appdata/binhex-preclear/home/.icons/BLACK-Ice-Numix-FLAT/24/actions/gtk-yes.svg does not exist.

 

The only thing that appears to be left are a few docker's config data in the appdata directory on my cache. UNRAID somehow is seeing them, but then, when it attempts to move them, it can't find them.

 

Why would this be? Could it be that these files are in the failing areas of the SSD that I'm trying to replace and may be lost forever? If that's the issue, would reinstalling these 4 dockers after I've replaced the cache drive and moved everything else back likely resolve the issue?

 

nas-diagnostics-20211023-1829.zip

 

 

 

Link to comment
One possibility that I know of is that Mover will refuse to 'move' a file to the array if the file already exists on the array. 
Weird error message logged.

I've used MC to try to move from cache to disk x where the mover put app data. So far, you are correct - all the files that remain also exist on the array.

Weird that it seemed to have moved them the first time it ran but didn't delete from cache.

I guess that I'll finish trying to move them and delete anything that already exists, just to be sure.

Sent from my moto g(7) using Tapatalk

Link to comment
3 minutes ago, FreeMan said:

I've used MC to try to move from cache to disk x where the mover put app data. So far, you are correct - all the files that remain also exist on the array.

 

What you now have to do is to try to figure what the circumstances were regarding each file in question.  I would assume that a modified file being rewritten to the array would force a write directly to the array which would overwrite the existing file.  Perhaps, and the following is all conjecture on my part, the file was open by two processes and when the write to the array failed, it wrote the file to the cache drive--  Checking the dates and times might provide some clues here. 

 

Note of disclosure:  I use a 'system' to prevent ransomware Malware from having direct write access to my array.  The system occasionally results in the above condition--- usually a bit of stupidity on my part.   You can read about it here if you are interested:

 

      https://forums.unraid.net/topic/58374-secure-writing-strategy-for-unraid-server-using-write-once-read-many-mode/#comment-572532

 

Link to comment
2 hours ago, Frank1940 said:

I would assume that a modified file being rewritten to the array would force a write directly to the array which would overwrite the existing file.

 

That seems like a reasonable assumption. However... These are all files from docker configs. I stopped all dockers and disabled the docker service prior to starting the mover.

 

If you look at the first post, there are svg files that are icons in use by Krusader that didn't get moved. Unless there's an update to Krusader itself, I doubt these get updated often. Actually, the file date on the cache drive versions are all 7 Sep 2020.

 

image.png.787e454c6b137926b8fe2cec284d6b83.png

As you can see in this screen shot of MC, I've got essentially the same file selected in both panes (cache on the left, disk2 on the right). the names are the same as are the file sizes. The dates are different because the one on the right was just written. What I don't understand is that the cache version starts with "!" while the array version starts with "@", yet MC tells me that the file already exists

image.png.c46ce9cdf69e66c13fb229aa7e77b229.png

 

At this point, it seems my only option is to continue to confirm by hand that they've all been copied, which I will do. Then continue with my SSD replacement process and hope that I don't run into this issue when I move everything back to the cache.

Link to comment

Even worse, as I'm moving through the Krusader detritus left over, I'm finding that some files do not exist in the destination directory. I guess it's a good thing I'm double checking.

 

I've got 3 other docker config directories to work through before I can shut everything down.

Link to comment

As I recall, it is recommended that the    appdata     share be restricted to the cache drive.  (Apparently, the system is more responsive if things are setup that way.)  I believe that if the share is restricted to the cache drive, Mover will actually move everything back to that drive.    (Turn on the  'Help' system to see what the choices are.  I think the choice you want is 'Prefer' or 'Only'.)   This will prevent the problem (at least, a portion of it) that you have run into. 

 

If you only have a single cache drive, the appdata share will not be protected.  There is a plugin which will back this share up to the array called CA Appdata Backup/Restore.

 

EDIT: you do have to be careful that you only map things to this folder that are actually used as 'settings' for the Dockers that you are using.  If a Docker creates actual data files, map those data files to the proper share where they are used.

Edited by Frank1940
Link to comment

My appdata share was set to Cache: Prefer (allowing overflow to hit the array should that ever become necessary - it wasn't at only about 60-70% full). I set it to Cache: Yes then ran the mover, which should have caused the mover to migrate everything to the array.

 

I've finished manually copying all the files that were left on /mnt/cache/appdata/* to /mnt/disk2/appdata/* (disk2 is where the mover decided to put everything).

 

Subsequently, I've shut down the server, replaced the failing SSD and now after powering up and setting the share back to Prefer, am in the process of running the mover to get everything back onto my SSDs.

 

Once the move is complete (and I'm 100% certain to check for any remaining appdata root-level directories on the array and clean them up if necessary), I'll restart the docker service & fire up all my dockers.

 

I've been using CA backup for my cache backup for a number of years now, even though the cache is on a 3-disk array. (I know 3 is a not ideal number, but it's what I've got, so I'm running with it).

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.