Confused with Primary and Secondary Storage.

[sgibbers17]

I'm a little confused with the new primary and secondary storage and how the mover will work with these.

In unRAID it states this

Quote

Primary storage is where new files and folders are created. If Primary storage is below the minimum free space setting then new files and folders will be created in Secondary storage, if configured.

 

Secondary storage is where new files and directories are created if no room on Primary storage. When both Primary and Secondary storage are configured the 'mover' will transfer files between them.

Does this still work the same way it used to?

 

I want new files to be written to the cache drive then when the mover gets invoked the files get written to the array and stay there.

 

The way I read the new descriptions it sounds like you pick where you want the files stored primarily and moved back and forth between primary and secondary based on disk space.  This sounds like the old "Prefer Cache", where the files are stored on the cache drive unless the cache is full then moves to array, then when there is room on the cache drive again it moves back.

[itimpi]

28 minutes ago, sgibbers17 said:

I'm a little confused with the new primary and secondary storage and how the mover will work with these.

In unRAID it states this

Does this still work the same way it used to?

 

I want new files to be written to the cache drive then when the mover gets invoked the files get written to the array and stay there.

 

The way I read the new descriptions it sounds like you pick where you want the files stored primarily and moved back and forth between primary and secondary based on disk space.  This sounds like the old "Prefer Cache", where the files are stored on the cache drive unless the cache is full then moves to array, then when there is room on the cache drive again it moves back.

Thr mover direction maps to what used to be Use Cache = Yes and Prefer.    The behaviour therefore is unchanged, - just expressed differently.

[sgibbers17]

10 minutes ago, itimpi said:

Thr mover direction maps to what used to be Use Cache = Yes and Prefer.    The behaviour therefore is unchanged, - just expressed differently.

I have my mover scheduled to run at midnight every day will all files on the cache drive move from the primary storage to the secondary storage at that time?  What is stated on the shares page on my unRAID server makes me think it will not move till there is no room left on the cache drive.

[itimpi]

8 minutes ago, sgibbers17 said:

I have my mover scheduled to run at midnight every day will all files on the cache drive move from the primary storage to the secondary storage at that time?  What is stated on the shares page on my unRAID server makes me think it will not move till there is no room left on the cache drive.

It will move the files as long as mover direction is cache->array.   Not sure why you thought it would not

[sgibbers17]

Quote

Primary storage is where new files and folders are created. If Primary storage is below the minimum free space setting then new files and folders will be created in Secondary storage, if configured.

 

Secondary storage is where new files and directories are created if no room on Primary storage.

This is right from the shares page the way it is described is like what used to be called "cache prefered."

Edited October 8, 2023 by sgibbers17

[dboonthego]

6 hours ago, sgibbers17 said:

I want new files to be written to the cache drive then when the mover gets invoked the files get written to the array and stay there.

 

You do it like this:

• Primary Storage = Cache
• Secondary Storage = Array
• Mover Action=Cache --> Array

[sgibbers17]

So if I want my temp video folder to be cache prefered where if I am copying a lot to the cache drive to a share that is Cache only and it gets filled up and when the mover starts it will move files from the cache drive to the array to free up some space.  And once there is space back on the cache drive the mover will move the temp video files from the array to the cache drive.

 

Would I use these settings?

Primary = Cache

Secondary = Array

mover = Array --> Cache

[dboonthego]

On one hand you're saying you want to copy to the cache temp video folder and then have mover move those to array, but on the other hand you're saying you want to have the mover move those files back to cache?  The mover is one direction.  Also, there is no mover on a cache-only share, but I think I know what you meant.

 

1 hour ago, sgibbers17 said:

Would I use these settings?

Primary = Cache

Secondary = Array

mover = Array --> Cache

This is the old terminology of "prefer."  Use this if you primarily want the files to reside on the cache.  New files are written to the cache, and you want them to stay on the cache.  If the cache gets full, then new files go on the array.  The mover will move them back to cache when space is available.

 

1 hour ago, dboonthego said:

 

You do it like this:

• Primary Storage = Cache
• Secondary Storage = Array
• Mover Action=Cache --> Array

This is the old terminology of "Yes."  Use this if you primarily want files to reside on the array but want to take advantage of the cache disk for faster write speeds.  The mover will move files from the cache to the array.  This is what you said you wanted.

[sgibbers17]

Thanks,  I don't get on my server very often.  So when I got on yesterday half of my shares that should have moved to the array showed the yellow triangle for some of the files being stored on the cache drive.  I started looking at the share settings and seeing how they had changed and the new descriptions on that page made it sound that the mover was working differently than before.  It turns out that the mover was not deleting the empty folders off the cache drive causing unRAID to think I had files in the share stored on the cache drive.  I do have shares that I use in different ways I have shares that are cache only, prefer cache, use cache then write to array, and write directly to array.  Just wanted to make sure that the settings are correct.  I don't see why they needed to change the setting from the old way to the new way.  They could have used the old terminology and just added a drop down menu to select what cache pool to use.

Edited October 8, 2023 by sgibbers17

[dboonthego]

It was changed to make it easier to underatand for new users.

[primeval_god]

6 hours ago, sgibbers17 said:

They could have used the old terminology and just added a drop down menu to select what cache pool to use.

1 hour ago, dboonthego said:

It was changed to make it easier to underatand for new users.

More specifically it was meant to prepare for an eventual change where the "Array" will no longer be a separate required thing. Instead what is now the "Array" will be just another type of "pool" that can be created along side the existing BTRFS, ZFS, and XFS "cache pool" types. Ideally when implemented the mover will be able to move files between any two pools, essentially allowing any pool to cache data for any other pool. 

It also makes it clearer that pools are not just for caching data, that they can be used for other things where data lives only on the pool.

Edited October 8, 2023 by primeval_god

[heimdall-one]

Just to make sure, cause I have a missunderstanding I guess.

 

When I set a share to:

 

First Cache

Second Array

Mover Cache -> Array files are written from Cache to the Array and deleted afterwards, right?

 

Is it possible then to have for instance my appdata on the Cache and only a backup on the array? ^^"

[trurl]

29 minutes ago, heimdall-one said:

Mover Cache -> Array files are written from Cache to the Array and deleted afterwards, right?

RIght.

 

27 minutes ago, heimdall-one said:

Is it possible then to have for instance my appdata on the Cache and only a backup on the array?

If you backup your appdata to the array using some other method such as Appdata Backup plugin.

 

Primary, secondary, and mover has nothing to do with backups.

[dboonthego]

3 hours ago, heimdall-one said:

Is it possible then to have for instance my appdata on the Cache and only a backup on the array? ^^"

Most people set the appdata share to Primary-->Cache; Secondary-->None and as mentioned use the Apdata.Backup plugin for backups to the array.

[leonlin]

On 12/30/2023 at 10:44 AM, dboonthego said:

Most people set the appdata share to Primary-->Cache; Secondary-->None and as mentioned use the Apdata.Backup plugin for backups to the array.

what's the difference with secondary is array?
if i set secondary to array , and action: array>cache
the data in array is the older one, so it's pointless?! 
when i modified the file in cache the file in array is changed too

so when we need use array>cache instead of cache only?

Edited January 3 by leonlin

[trurl]

Secondary is where overflow goes, simple as that. If the pool has less than Minimum Free, new files will go to the array. They might be moved to cache when there is more than Minimum Free if that is the Mover action.

 

But nothing can move open files. And mover won't replace files.

 

All of this still works as it always has, it is just described differently in the webUI to help with the confusion often experienced by new users about the old Cache:Prefer setting.

[leonlin]

10 hours ago, trurl said:

Secondary is where overflow goes, simple as that. If the pool has less than Minimum Free, new files will go to the array. They might be moved to cache when there is more than Minimum Free if that is the Mover action.

 

But nothing can move open files. And mover won't replace files.

 

All of this still works as it always has, it is just described differently in the webUI to help with the confusion often experienced by new users about the old Cache:Prefer setting.

ok thanks