Syslog filling up with mover errors

[GM_XIII]

Hi,

 

I have been having trouble to keep the server up for more than a couple of days as the log keeps filling up.

 

I stopped all the containers and run it as bare-bones as I could and it still fills ups. Looking at the logs, I noticed syslog growing in size:

 

It

 

It seems that this is because mover errors when trying to move things from the cache to the array. What could have caused it and what can I do about it?

 

 

 

Any help appreciated,

David

[JorgeB]

Please post the diagnostics, actual error is cutoff on that screenshot.

[GM_XIII]

Hi Jorge,

 

Thanks for coming back to me on this.

 

Having a bit of trouble with the diagnostic tool as it runs out of memory while generating it.

 

I am attaching the syslog on its own and will give it a reboot and run the diagnostics again. I suspect it has something to do with the mover as the log size stays reasonably constant while I watched it.

 

Thanks,

David

syslog.zip

[JorgeB]

File exists

 

Mover won't move any files that already exist on destination, you will need to delete one of the copies.

[GM_XIII]

Not sure if I fully follow. This probably comes down to a wrong setup of the shares post 6.12 update. My understanding is the appdata share set up to use cache would write there first and then the mover would transfer it to the array. My understanding that of course there would be some duplicates and it would handle it but not sure why is filling the log so aggressively.

[itimpi]

1 minute ago, GM_XIII said:

Not sure if I fully follow. This probably comes down to a wrong setup of the shares post 6.12 update. My understanding is the appdata share set up to use cache would write there first and then the mover would transfer it to the array. My understanding that of course there would be some duplicates and it would handle it but not sure why is filling the log so aggressively.

How any share uses a pool for caching depends on the share settings.   Since you could not get the diagnostics we cannot check what settings you have.

 

in normal operation you would not get duplicate files.    If you do end up with duplicates for any reason then it is a manual process to decide which copy to keep and delete the unwanted copy.

[GM_XIII]

Attaching the diagnostics file. I noticed some of the shares were transferring data from the array to the cache instead of the other way around.

vader-diagnostics-20231217-1541.zip

[itimpi]

According to your diagnostics all shares that have files on the ‘cache’ pool are set for mover to transfer files to the array.   Nothing should be attempting to transfer the other way with current settings.

 

The default settings for the ‘appdata’ and ‘system’ shares (which you must have changed) try and keep all their files on the cache pool as this gives far better performance of docker containers and avoids spinning up array drives unnecessarily.     Note, however, that files for these shares can only all be moved successfully if the docker service is disabled as it keeps files open and mover will not move open files.

[trurl]

3 minutes ago, GM_XIII said:

noticed some of the shares were transferring data from the array to the cache instead of the other way around.

Perhaps you made some change before taking those diagnostics, because none of your shares are currently configured to be moved to cache.

 

But, that isn't really what you want.

 

https://docs.unraid.net/unraid-os/manual/shares/user-shares/#default-shares

 

Ideally, appdata, domains, and system shares would have all their files on cache or other fast pool with no files for these shares on the array, and configured to stay that way. If these have files on the array, Docker/VM performance will be impacted by slower parity array, and array disks can't spin down since these files are always open.

 

You need to set these shares to be moved to cache.

 

But, nothing can move open files, so you also have to disable Docker and VM Manager in Settings before these files can be moved. VM Manager is already disabled, if you don't have any VMs, maybe you don't need any of the files in the domains share.

 

And, as already mentioned, mover won't replace files, so if some files already exists in both places you will have to decide which to keep and do some manual cleanup.

 

Dynamix File Manager plugin will let you work with files directly on the server.

[GM_XIII]

Thanks both. It is indeed correct that I changed it before running the diagnostics. Will make those changes and hopefully it will fix it.