Mover not moving

[Glonch]

Greetings Everyone,

 

I have searched all over the forums for a solution to my problem but none of the suggestions worked in my case.  

 

It appears my mover cannot move files to the array any more and it is continuously filling up.  All the directories are set to Cache=Yes.

 

In looking at the logs, I'm seeing "File exists" entries.  I have searched all the disks and the referenced file names do not exist on the array, only on the cache drive.

 

I have also read somewhere that there was bug introduced recently in unRaid where if a share name has a space in it, Mover will fail - I cannot locate that info again though.  Many of my shares and subfolders have spaces in them.

 

Attached is my diag file

 

Any help would be greatly appreciated. 

 

Edited October 10, 2021 by Glonch

[Squid]

The files already exist in /mnt/user0/download_cache.  (And there's a ton of them ->  How did this happen?)

 

Oct  2 15:16:39 WOPR move: move_object: /mnt/download_cache/Plex Media/Rich Plex/TV Shows/Chasing Classic Cars/Season 05/Chasing Classic Cars S05E25.mkv File exists

 

It's bit of a pain to fix this.  You'd have to use Krusader (with mappings of /mnt -> /UNRAID) and delete the afflicted files from /UNRAID/user0   DO NOT delete from /UNRAID/USER

[Glonch]

Thanks!

 

I little confused on the mapping....

 

I don't see UNRAID/user0 anywhere on my system

[Squid]

The path mapping for Krusader.  By default it's set to /mnt/user mapped to (IIRC) /UNRAID.  You need to change it to /mnt/ mapped to /UNRAID

 

Or if you're more comfortable working on the command line (or mc), then the folder you're looking for is /mnt/user0

 

I have been lobbying Limetech to make a minor change to mover that would make recovery from this situation far far easier.

[Glonch]

Ok... I can use MC and go to user0...

 

Two questions:

 

1. Am I just deleting the files noted in the logs as already existing or all in a particular directory within user0?

2. I reviewed all of my docker mappings and none of them have user0 - how they could have landed there?

 

Thanks for lobbying Limetech for a better solution and for the assistance.

[Squid]

There's the rub

 

/mnt/user contains all the files within the shares including the cache drives

/mnt/user0 contains all the files only on the array

/mnt/download_cache is only the files on the cache drive

 

Safest is to compare the /mnt/download_cache/Plex Media and /mnt/user0/Plex Media and delete the duplicates from /mnt/user0

 

I'm not really sure how you got into this situation though...

 

44 minutes ago, Glonch said:

Thanks for lobbying Limetech for a better solution and for the assistance.

My "solution" is to still copy the file(s) but to append ".incomplete" or ".mover" or the like so that at least it's easy to identify the applicable files and delete them accordingly.  It's not easy to wind up with duplicated files under normal circumstances though.

[Glonch]

Thanks so much... Off to search and destroy.   

[Glonch]

hmmm...

 

Removed about 10 duplicates and invoked the mover... still not moving.

 

None of the files in Download_cache\Plex Media exists in user0\Plex Media via Krusader or MC but the mover thinks so.

 

Attached the latest diag file.

 

Edited October 10, 2021 by Glonch

[Glonch]

Finally!  Figured it out (thanks to a friend of mine looking over my shoulder):

 

My download share settings:

 

My (original) Plex Media share settings:

 

 

So... you can't have one share (WORP_Downloads) on one cache pool (Download_Cache) and another share (Plex Media) that you will copy to on another cache pool (Cache).

 

Changed Plex Media share to use Download_Cache and Mover now moves without issue.

 

In my opinion, if I moved files (under 'user' via Krusader) from WORP_Download [Download_cache] to Plex Media [Cache] - unRaid should have moved the files to the cache drive named Cache.  Then, at 3am, Mover would have moved to the array.  This would be inefficient of course.

 

Having an error stating that 'file exists' doesn't point to having different cache settings between shares, thus was super hard to figure out.

 

So, I see three options:

1. This is bug with cache pools or
2. This setup should not be done, thus noted in the wiki (I haven't looked) or
3. This is normal behavior but the error should be updated to reflect the two cache pools are 'competing'

Opinions/Comments? 

 

PS - I had recently setup two cache pools.  Previously all cache data was saved to a cache named "Cache".  After changing the setup to have two cache pools, I never thought to change the Plex Media share cache pool setting until my friend mentioned it until today.

 

[itimpi]

11 hours ago, Glonch said:

In my opinion, if I moved files (under 'user' via Krusader) from WORP_Download [Download_cache] to Plex Media [Cache]

The problem was cause by the combination of Krusader and using a 'move' operation.  It would end up triggering a manifestation of the behaviour described here.  If you had done a copy + delete you would have gotten the result you expected.

[Glonch]

Understood with Krusader... 

 

But still doesn't explain Mover not moving the files unless they are coming from the same cache drive.

[trurl]

3 hours ago, Glonch said:

Understood with Krusader... 

 

But still doesn't explain Mover not moving the files unless they are coming from the same cache drive.

Mover only moves to/from the pool specified for the user share. Any files for the user share that are on a pool not specified for the user share are ignored by mover. This behavior is by design. Did you read the thread I linked in your report?

[rdagitz]

I cannot tell but am I in the same situation? I have a 500GB ssd that is just filling up and mover is not moving the files. When I click the "Move" button it goes gray but if I refresh the page it is as if nothing happened.  Attached is the syslog. I am new to Unraid so I am getting use to it. It looks to me like mover is trying to transfer some particular files and failing, so it just keeps looping on them. Does someone with a little more insight have some thoughts?

syslog.txt

[Squid]

Post your entire diagnostics zip file

[rdagitz]

Here you go 🙂

server-diagnostics-20211015-1646.zip

[hugenbdd]

On 10/2/2021 at 4:13 PM, Squid said:

I have been lobbying Limetech to make a minor change to mover that would make recovery from this situation far far easier.

@Squid

What's the minor change you are asking for?  (Just want to be prepaired for the plug-in if needed...)

 

[trurl]

10 hours ago, rdagitz said:

mover is not moving the files

Mover won't move duplicates (file already exists at destination), and nothing can move open files.

[rdagitz]

Can you please clarify? In the diagnostic files that I attached here, how are you seeing that duplicate files or open files are the problem? In addition, shouldn't mover just skip the duplicates and just go on to other files. I cannot see how I have 300+ GB of duplicate files.  Mover doesn't appear to be moving any files.  I want to be able to remedy the problem.

Edited October 22, 2021 by rdagitz

[Squid]

On 10/15/2021 at 5:04 PM, hugenbdd said:

@Squid

What's the minor change you are asking for?  (Just want to be prepaired for the plug-in if needed...)

 

For when mover runs, the file gets written as something like *.partial and then on successful move it deletes and renames accordingly.  Stops orphaned files being stranded on array / cache with no easy recourse beyond the command line to delete the duplicate.  Should be coming out in 6.10 rc2

[rdagitz]

So I have posted my server diagnostics but I am new to Unraid and unsure how to determine why more than 250GB of data continues to sit on my cache drive each day. Could someone help me know what is keeping the files on cache and not moving them to the array? I click the "Move" button but it seems to make little difference. I don't just want to "fix the problem" but want to understand the root of the issue so that I can get to the bottom of it. Thanks ahead of time.

[trurl]

post new diagnostics since those are over a week old now.

[Frank1940]

On 10/24/2021 at 10:22 AM, Squid said:

For when mover runs, the file gets written as something like *.partial and then on successful move it deletes and renames accordingly.  Stops orphaned files being stranded on array / cache with no easy recourse beyond the command line to delete the duplicate.  Should be coming out in 6.10 rc2

 

I hope this will be a selectable setting.  I check my cache drive a couple of times a week to make sure that everything got moved.  If something didn't get moved, I know (most likely) that there is a duplicate file.  I can then investigate and perform the remedy.  (The couple of times it has happened, it has been simple to run down the file chain and find the cause.)

[rdagitz]

Sorry it took me so long to get those diagnostics uploaded.  See attached.

server-diagnostics-20211106-1651.zip

[trurl]

Try it without the mover tuning plugin

[rdagitz]

Funny. I actually installed the mover tuning plugin attempting to solve the problem. I will uninstall it and see what happens.