I've always had intermittent issues with Mover not moving my files from the cache to the array using the (hourly) scheduler, but usually manually forcing a move seems to work. Lately, I've been having issues with Mover skipping files no matter if I force a move or not and I'm not sure why. I checked my share settings and Cache is set to Prefer. My cache drive is comp letely full and yet nothing wants to move to the array.

tower-diagnostics-20201207-1731.zip

18 minutes ago, kjac said:

Cache is set to Prefer.

That setting moves file FROM the array TO cache.

Looks like the only shares you have set to Prefer are the ones that should stay on cache.

 

Running mover hourly is unlikely to help if you are still writing to cache. Mover is intended for idle time. You should reconsider whether you really need to cache some of these other user shares.

I meant it is set to Yes, not Prefer, the wording confuses me for some reason. I actually did try setting it to Prefer and noticed it was pulling my files from the array to my cache drive, but oddly enough when I set it back to Yes it moved all the files to the array, including the ones it was skipping earlier.

3 minutes ago, trurl said:

Looks like the only shares you have set to Prefer are the ones that should stay on cache.

 

Running mover hourly is unlikely to help if you are still writing to cache. Mover is intended for idle time. You should reconsider whether you really need to cache some of these other user shares.

I had set it to move hourly because I was downloading a lot of stuff and my cache drive was filling up too fast, unless i'm missing something and that's not really how the cache drive works.

If this is the initial data load I recommend not caching anything (except appdata, domains, system which belong on cache, those should stay prefer).

 

Do you really expect to add hundreds of gigabytes per day to your server? As mentioned, mover should be scheduled for idle time and if you aren't currently having idle time don't use cache.

Mover can't move to the slower array as fast as cache can be written, simple as that. And if cache is being written at the same time as mover is running then it is just competing for those resources.

 

Mover is intended for idle time.

Just now, trurl said:

If this is the initial data load I recommend not caching anything (except appdata, domains, system which belong on cache, those should stay prefer).

 

Do you really expect to add hundreds of gigabytes per day to your server? As mentioned, mover should be scheduled for idle time and if you aren't currently having idle time don't use cache.

I've got over 100TB of drives to fill and an unlimited internet connection, so i'm adding a few TB of data per day (or as much as unraid transfer speeds will allow)

1 minute ago, kjac said:

I've got over 100TB of drives to fill and an unlimited internet connection, so i'm adding a few TB of data per day (or as much as unraid transfer speeds will allow)

Then don't cache any of it.

ok, i've set Cache to No on my Movies, TV, and Storage shares. I also set the Mover schedule to daily instead of hourly. My cache drive is still full of all those movies though and hitting the Move button doesn't move anything, I just get:

Dec 7 18:13:18 Tower emhttpd: shcmd (89621): /usr/local/sbin/mover |& logger & 

Dec 7 18:13:18 Tower root: mover: started

Dec 7 18:13:18 Tower root: mover: finished

 

Mover ignores cache-no and cache-only shares. To get them moved from cache to array they must be cache-yes.

 

Set them to cache-yes and run mover, but quit writing new data until mover has finished. Then set them to cache-no before writing new data again.

ok, looks like toggling those Cache settings triggered the mover to start working again, it's no longer skipping files. I'll set my storage shares to Cache: No when its finished. I assume that after I've loaded the bulk of my 100TBs of data to the array that I should turn Caching back on for my storage shares?

Damn I spoke too soon, it moved a couple movies and then went back to skipping 30 others. Any idea what might cause that? I never noticed that problem before and I don't think i'm doing anything different in the past month i've been using unraid.

 

EDIT: I manually enabled Mover again and it started moving the files again, not sure what causes it to do that though.

Edited December 8, 20205 yr by kjac

Mover can't move open files, so if these are torrent seeds for example or open for any other reason they can't be moved. Also, mover won't move duplicates so if the same filepath already exists on the array it won't be moved. And as already noted, mover ignores cache-no and cache-only shares.