Newb Type Question on Placement of AppData

[gsd2012]

Ok this is going to seem like a newb question but here it goes. I have a 1Tb Cache Pool and 7 drives of 2TB storage. Now I have my Appdata on mtn/user/appdata which has been for years. I was doing some clean up on the drives and noticed that the Appdata was spread out over all the drives so I looked at the setting an it was set to use all drives, it that the correct way to do that? Same with Appdatabackup, system, trancoding are all in on the 7 storage drives. Would it not be better just to use the Cache Pool for those system type apps, dockers, and Plex transcoding? I was not 100% sure there since I set up all this years ago and kind of forgot about it until I had to replace a disk. I would think  the 7 drives would be just for media storage and data while the cache pool is for the system stuff or am I wrong there? What is the best and most optimal way to set these directories up? Sorry if that is a newb question just kept looking for an answer and got too many variations of configurations. Thanks ahead for any help

[trurl]

Better if appdata, domains, system shares are all on cache and set to stay on cache. With them on the array disks can't spin down since these files are always open. And performance will be better if they are on cache.

 

Setting those shares to cache-prefer will make mover try to move them to cache, but open files can't be moved, so you have to go to Settings and disable Docker and VM Manager so mover can move them.

[gsd2012]

45 minutes ago, trurl said:

Better if appdata, domains, system shares are all on cache and set to stay on cache. With them on the array disks can't spin down since these files are always open. And performance will be better if they are on cache.

 

Setting those shares to cache-prefer will make mover try to move them to cache, but open files can't be moved, so you have to go to Settings and disable Docker and VM Manager so mover can move them.

Ok that makes more since to me too. I must have goofed up setting it all up since I even had media files being placed on the Cache drive filling the pool up. I looked and forgot to exclude the cache drives. Ok so I have so major moving to do. Thanks

[gsd2012]

@trurl I also noticed that by default it splits all those system shares across all the drives so I turned that off so they stay only on the Cache drive. I would assume that all Transcoding would say on Cache too since it is moving information off and on for media? Mainly the 7 drives should be the media or data only since they are reading and writing all the time correct?

[gsd2012]

@trurl Ok here is another question. I got the AppData, Transcoding, System, and AppDatabackup moved to just the cache (pool) drives. Now on each of these shares it will not let you put them on just the Cache Drives in the Include disk section. Would you use all the disks for these shares? I would think it would let you just use the Cache Disk but they are not showing up as an Include or Exclude section. Now all the set to Cache Prefer accept AppDataBackup (set to "NO") but that was created by the backup plugin so not sure it should be changed to Prefer or not. I was also told way back that Media data share should be set to Use cache (for new files/directories): Yes but it did not seem correct at the time but since an expert told me it would be good for the system I did it.

[itimpi]

You are correct in that it is not possible to include only the cache disk for a share.    However if all the files for that share are on the cache and you set `use Cache=only then files will never get put on the array.    If you instead set Use Cache=Prefer then files will only be stored on the array if there is not room on the cache.

 

For your Media share then Use Cache=Yes is almost certainly the correct setting.

 

You might want to read the help built into the GUI for a good summary of the effect of the Use Cache setting.    There is also more detailed write up here in the online documentation accessible via the ‘Manual’ link at the bottom of the GUI.

[gsd2012]

7 minutes ago, itimpi said:

You are correct in that it is not possible to include only the cache disk for a share.    However if all the files for that share are on the cache and you set `use Cache=only then files will never get put on the array.    If you instead set Use Cache=Prefer then files will only be stored on the array if there is not room on the cache.

 

For your Media share then Use Cache=Yes is almost certainly the correct setting.

 

You might want to read the help built into the GUI for a good summary of the effect of the Use Cache setting.    There is also more detailed write up here in the online documentation accessible via the ‘Manual’ link at the bottom of the GUI.

@itimpi Ok I got it. If I set the AppData, Transcoding, and System to Cache=Only it makes it fixed to the Cache Pool Drives. Now for AppDataBackup it was set to Cache=No but would that be on the Cache too?

[gsd2012]

This is the way the docker is set so I am correct on that????

 

 

[itimpi]

15 minutes ago, gsd2012 said:

Now for AppDataBackup it was set to Cache=No but would that be on the Cache too?

 

You have to decide where you want it to end up.  Most people want it on the array in case they lose their cache for some reason.   This would probably best be set to No (as it is at the moment) to force the backup to go direct to the array.

[itimpi]

13 minutes ago, gsd2012 said:

This is the way the docker is set so I am correct on that????

 

 

This is quite typical.   By using /mnt/cache you are referencing the physical drive directly and bypassing the User Share system.  However since each top level folder on any drive is considered part of a User Share then files will also show up under the 'appdata' and 'system' shares.

 

You always have two ways of viewing files - directly by going to physical drive and a separate logical view at the User Share level that gives an amalgamated view across drives.  They are still the same files.

[gsd2012]

1 hour ago, itimpi said:

 

You have to decide where you want it to end up.  Most people want it on the array in case they lose their cache for some reason.   This would probably best be set to No (as it is at the moment) to force the backup to go direct to the array.

Ok that make sense on that one

[gsd2012]

1 hour ago, itimpi said:

This is quite typical.   By using /mnt/cache you are referencing the physical drive directly and bypassing the User Share system.  However since each top level folder on any drive is considered part of a User Share then files will also show up under the 'appdata' and 'system' shares.

 

You always have two ways of viewing files - directly by going to physical drive and a separate logical view at the User Share level that gives an amalgamated view across drives.  They are still the same files.

I was told a while back to put the appdata on the /mnt/user section but it seemed strange to put the working docker files on all the drives. I want to make sure I have the most optimal way to setup UnRAID to perform without any major issues that affect the system files or docker containers. The 7 storage drives are just for media or data only to spread out across all disks designated for media. Mainly the User Share is for the storage of videos, music, and media files for Plex.

[trurl]

cache is part of user shares, and if the appdata share was originally cache-only or cache-prefer, then all of its files would be on cache.

 

I'm pretty sure appdata, domains, system are cache-prefer by default. But I haven't tested that with a new install.

 

You can see how much of each drive each user share is using by clicking on Compute... for the share on the User Shares page, or by using the Compute All button.

[itimpi]

6 minutes ago, gsd2012 said:

I was told a while back to put the appdata on the /mnt/user section but it seemed strange to put the working docker files on all the drives

This is normally recommended because then at the application level you do not have to worry about what physical drive contains the files and they can be moved around without having to change application settings.    It does not mean that you WILL have files on all the drives - that is controlled by the Use Cache setting on the share.  
 

[gsd2012]

4 minutes ago, trurl said:

cache is part of user shares, and if the appdata share was originally cache-only or cache-prefer, then all of its files would be on cache.

 

I'm pretty sure appdata, domains, system are cache-prefer by default. But I haven't tested that with a new install.

 

You can see how much of each drive each user share is using by clicking on Compute... for the share on the User Shares page, or by using the Compute All button.

Yes you are correct it was Cache=Perfer by default but I want to force it to only go on the cache pool so switched it to "only". I want to separate the Users Share stuff from the System stuff.

[gsd2012]

5 minutes ago, itimpi said:

This is normally recommended because then at the application level you do not have to worry about what physical drive contains the files and they can be moved around without having to change application settings.    It does not mean that you WILL have files on all the drives - that is controlled by the Use Cache setting on the share.  
 

Ok that made it more clear so I will keep it as the recommended setting of /mtn/cache and set those system shares to cache=only to the cache pool. All other stuff will be User Shares to 7 main drives with it set to Cache=yes to use the pool to move files off and on the shares.  Thanks for all the help, much appreciated.

[trurl]

14 minutes ago, gsd2012 said:

Yes you are correct it was Cache=Perfer by default but I want to force it to only go on the cache pool so switched it to "only". I want to separate the Users Share stuff from the System stuff.

cache-no and cache-only shares are ignored by mover. cache-only will only write new files to cache, but if any are already on the array they won't be moved.

[gsd2012]

7 minutes ago, trurl said:

cache-no and cache-only shares are ignored by mover. cache-only will only write new files to cache, but if any are already on the array they won't be moved.

Ok so if I set AppData, and System shares to only then all the files will stay where they are at on the Cache Pool physical drive and not spread out across all the User Shared (7 Storage Drives). I am thinking that Transcoding needs to be set to "yes" or "prefer" so it can use all the User Shared Drives to transcode videos since it would move them in and out of cache not sure I should set it to "only". I am a bit ify on that part.

 

Would it make it easier to attach a diag file to show my settings for an expert set of eye to see if I have it correctly configured for optimal performance?

Edited April 13, 2021 by gsd2012
Left off line

[trurl]

2 minutes ago, gsd2012 said:

set AppData, and System shares to only then all the files will stay where they are at on the Cache Pool physical drive and not spread out across all the User Shared (7 Storage Drives)

Maybe I missed it, but did you actually ever get all those moved to cache?

 

As you say and as I noted, cache-only means the files will stay where they are, but as I also noted, it doesn't mean any array files will get moved to cache.

[gsd2012]

9 minutes ago, trurl said:

Maybe I missed it, but did you actually ever get all those moved to cache?

 

As you say and as I noted, cache-only means the files will stay where they are, but as I also noted, it doesn't mean any array files will get moved to cache.

I moved AppData, System, and Transcoding over to Cache drives with unBALANCED. They have all been moved over from the 7 storage drive to just on the cache or did I not follow that right?

[trurl]

It would have been possible to get those moved using the built-in mover by setting them to cache-prefer.

 

40 minutes ago, gsd2012 said:

They have all been moved over from the 7 storage drive to just on the cache or did I not follow that right?

 

1 hour ago, trurl said:

You can see how much of each drive each user share is using by clicking on Compute... for the share on the User Shares page, or by using the Compute All button.

 

[gsd2012]

12 minutes ago, trurl said:

It would have been possible to get those moved using the built-in mover by setting them to cache-prefer.

 

 

 

Ah didn't know that good to know for future. Thanks