6.5.3 - Use cache vs prefer vs only & how it all relates to best-practice

[digitalformula]

I have the Fix Common Problems plugin installed and it keeps warning about shares being set to cache only and files exist on the data disks.  Other shares are set to no cache but have files on the cache.

 

That's fine and I understand that ... but why?  If I have a new share set to cache only that *was* set to both, and files exist on cache and data disks, is the Mover supposed to clean things up?

 

I'd like to get rid of these warnings but I can see files do exist in both places.  For example the img file for one of my VMs.  Which file is the "active" one?  The "domains" share is set to not use cache at all.

[remotevisitor]

If you have files on both cache and data disks and have the share set to cache only, temporarily set it to cache prefer and manually run mover (which will move file from the data disks to cache) , then set to back to cache only.

 

if you have files on both cache and data disks and have the share set to cache no, temporally set it to cache yes and manually run mover (which will move files from cache to the data disks), then set it back to cache no.

 

update: realized I didn’t address the question of which files are the ‘active’ files .... the ones on the cache disk; these setting only affect the placement of new files, not the usage of existing files which uses cache first, data disks second.   Therefore for the ones set to cache no, you will probable just want to delete the ones on the data disk.

[digitalformula]

5 hours ago, remotevisitor said:

If you have files on both cache and data disks and have the share set to cache only, temporarily set it to cache prefer and manually run mover (which will move file from the data disks to cache) , then set to back to cache only.

 

if you have files on both cache and data disks and have the share set to cache no, temporally set it to cache yes and manually run mover (which will move files from cache to the data disks), then set it back to cache no.

 

update: realized I didn’t address the question of which files are the ‘active’ files .... the ones on the cache disk; these setting only affect the placement of new files, not the usage of existing files which uses cache first, data disks second.   Therefore for the ones set to cache no, you will probable just want to delete the ones on the data disk.

 

Thanks for the answer @remotevisitor but that's not working here.  I've done all of that and the files in both locations just stay where they are.  I can't find any log or sign that the mover has actually done anything.  The browser tab where I clicked run Mover is just sitting there hours later saying "Disabled - Mover is running."

 

Also, I've accidentally deleted a file from one location in the past and it got deleted from the other, too.  I'm not keen to try that again. 

[remotevisitor]

Ah ... my bad.   The advice I gave only ensures that files that do not already exist on the intended cache/data disk get moved.

 

i had forgotten that mover will not overwrite files that already exist in the destination.

 

do you have mover logging enabled, it should have at least shown it was skipping files?   Settings -> Scheduler -> Mover Settings -> Mover Logging -> Enabled.

 

 

[digitalformula]

12 hours ago, remotevisitor said:

Ah ... my bad.   The advice I gave only ensures that files that do not already exist on the intended cache/data disk get moved.

 

i had forgotten that mover will not overwrite files that already exist in the destination.

 

do you have mover logging enabled, it should have at least shown it was skipping files?   Settings -> Scheduler -> Mover Settings -> Mover Logging -> Enabled.

 

 

 

I have cache set to NO for appdata.  There are files on cache and files on disk1.

 

If I delete anything from the appdata directory on disk1, it is immediately deleted from the appdata directory on cache, too.

 

In fact, every change I made to appdata on disk1 is immediately reflected in appdata on cache.

[JonathanM]

8 hours ago, digitalformula said:

every change I made to appdata on disk1 is immediately reflected in appdata on cache.

The only way that would make sense is if you were confusing user shares and disk shares.

 

Can you post the exact paths you are looking at to come to that conclusion?

[digitalformula]

10 hours ago, jonathanm said:

The only way that would make sense is if you were confusing user shares and disk shares.

 

Can you post the exact paths you are looking at to come to that conclusion?

 

@jonathanm

 

Path 1: /mnt/user/appdata/<app>

Path 2: /mnt/cache/appdata/<app>

 

If I rename or touch <app> on Path 1, the change is immediately reflected in Path 2.

 

It's entirely possible (and likely) that I've confused user and disk shares, although I thought I had to be in /mnt/diskX/<share> for it to be a disk share.  If /mnt/cache is a disk share, that would explain this ... is it?

 

That said, if /mnt/cache is a disk share and not a user share, how do I go about making sure files are on the user shares and *not* on the cache (or the other way around).  The thing I'm missing and don't understand is how to check what the Fix Common Problems plugin is finding i.e. saying there's data on both.

 

[Squid]

/mnt/user is the aggregate  of all the top level folders (ie: shares) on every disk including the cache.  What your seeing is expected, normal, and by design.

18 minutes ago, digitalformula said:

Fix Common Problems plugin is finding i.e. saying there's data on both.

What FCP would be complaining about is that such and such share (presumably set to be cache-only) has files stored on /mnt/diskX also.

[digitalformula]

1 hour ago, Squid said:

/mnt/user is the aggregate  of all the top level folders (ie: shares) on every disk including the cache.  What your seeing is expected, normal, and by design.

What FCP would be complaining about is that such and such share (presumably set to be cache-only) has files stored on /mnt/diskX also.

 

If it is expected, normal and by design, why does FCP care at all?

 

In any case, I'm not overly concerned about the warning itself being shown.  I'm just trying to understand how I should configure things so that this warning *doesn't* happen (by design or not, since FCP does notice it).

 

[Squid]

11 minutes ago, digitalformula said:

If it is expected, normal and by design,

This is what's normal:

1 hour ago, digitalformula said:

Path 1: /mnt/user/appdata/<app>

Path 2: /mnt/cache/appdata/<app>

 

If I rename or touch <app> on Path 1, the change is immediately reflected in Path 2.

Because /mnt/user/appdata/file is the exact same file (assuming cache-only share) as /mnt/cache/appdata/file

 

13 minutes ago, digitalformula said:

why does FCP care at all?

If FCP is bitching, then you've got a file that's on say Disk1 when the share is set to be cache-only.  (as an aside, in this case, /mnt/user/appdata/file1 == /mnt/disk1/file1)

 

To fix the problem, you set the appdata share to be cache-prefer, then stop the docker service, and run mover.  Once its done, restart the docker service.  You can leave the share set to be prefer.  That way if the cache drive fills up for whatever reason, a write will not fail to it, but the file will go to the array until space is freed up and mover tosses it back onto the cache drive

 

[digitalformula]

58 minutes ago, Squid said:

This is what's normal:

Because /mnt/user/appdata/file is the exact same file (assuming cache-only share) as /mnt/cache/appdata/file

 

If FCP is bitching, then you've got a file that's on say Disk1 when the share is set to be cache-only.  (as an aside, in this case, /mnt/user/appdata/file1 == /mnt/disk1/file1)

 

To fix the problem, you set the appdata share to be cache-prefer, then stop the docker service, and run mover.  Once its done, restart the docker service.  You can leave the share set to be prefer.  That way if the cache drive fills up for whatever reason, a write will not fail to it, but the file will go to the array until space is freed up and mover tosses it back onto the cache drive

 

 

Ok cool.  So if I do all that, Mover logging should (I assume) show that some files are being moved, right?

 

I see this:

 

Oct 9 11:49:05 df-unRAID emhttpd: req (21): cmdStartMover=Move+now&csrf_token=****************
Oct 9 11:49:05 df-unRAID emhttpd: shcmd (7054): /usr/local/sbin/mover |& logger &
Oct 9 11:49:05 df-unRAID root: mover: started
Oct 9 11:49:05 df-unRAID root: mover: finished

 

Starting and finishing at the exact same second doesn't seem right since there are files that need to be moved.

[JonathanM]

9 minutes ago, digitalformula said:

there are files that need to be moved.

Which files, specifically?

[digitalformula]

14 minutes ago, jonathanm said:

Which files, specifically?

 

The files FCP is finding.  It doesn't say what they are (if it does, I just haven't found where).

 

That said, I'm starting to wonder if FCP isn't reporting incorrectly (sorry @Squid, I don't mean any offence by saying that).  I say that, though, because FCP is still saying 'appdata' is set to cache only when it isn't anymore.  It is also saying I don't have the SSD trim plugin installed, the plugin update check is disabled and the docker update check is disabled.  All those things are incorrect.

[bonienl]

5 hours ago, digitalformula said:

The files FCP is finding.  It doesn't say what they are

Go to Shares -> User Shares

Press "Compute All" will show where exactly each share exists