Better handling of unexpected mover conflicts


Recommended Posts

tl;dr: Improve handling of the uncommon situation that the mover can't move some files from cache to array because they unexpectedly already exist on the array.

 

In Unraid 6.8.3, it appears that if a file exists on the cache drive (e.g. /mnt/cache/foo/bar.txt) and also on the array (e.g. /mnt/user/foo/bar.txt), then mover simply leaves the file on the cache drive forever.

 

I recognize that this situation shouldn't naturally occur, but *shrug* things happen.

 

I can see four problems with this situation, whenever it somehow occurs:

  1. Endangered data (since the newer file is never propagated to the array)
  2. Wasted cache space
  3. User confusion (e.g. /mnt/cache/foo/bar.txt and /mnt/user/foo/bar.txt match, but /mnt/disk1/foo/bar.txt doesn't)
  4. Potential pointless spinning up of drives every time the mover runs, only to find it can't move these conflicting files (in my real-life situation, chasing of mysterious drive spin-ups is what led me to all this)

 

Here's my two suggestions to improve the situation:

  1. At the very least, some sort of user notification if some files are "stuck" in the cache due to conflicts.
    Plus, some mechanism for rectifying the situation would be a bonus.
  2. Optimally, Unraid would compare any such conflicting file to what exists on the array, then rectify the situation automatically if they match.
    This would be a potentially expensive operation when it occurs, but should be uncommon, and the results could arguably be worth the expense.
    Also, for a definition of files "matching", I suggest that if the file
    contents match, regardless of other metadata (timestamps, ownership, etc), then the metadata from the version of the file in the cache should end up on the array.

 

Apologies in advance if I'm wrong about how things work, but this appears to me to be the current situation with Unraid 6.8.3.

Edited by bland328
Clarification
Link to comment
20 minutes ago, bland328 said:

wrong about how things work

I think you've got it right. Where I often see this is helping users with their docker setup and trying to go the other way with cache-prefer data. Somehow they get system share on cache and array, probably by starting dockers with missing cache. This may result in duplicate docker.img for example.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.