Cache and running out of space


Recommended Posts

After setting up my unRAID server and having everything working I started to migrate my data. While copying 150GB of data it stopped halfway saying it ran out of space.

 

I`ve got the following setup:

 

A single pool with:

1x 4TB HDD

2x 500GB SSD (will be replaced after migration)

 

A cache pool with:

1x 128GB NVMe (will be replaced after migration)

1x 500GB SSD

 

The share has the allocation method "High-water" and cache disk is set to yes.

 

The solution was to manually run the mover and the copy continued. But should it not be that when the cache gets filled up it starts writing data to the main pool?

Link to comment
4 minutes ago, Mentox said:

But should it not be that when the cache gets filled up it starts writing data to the main pool?

Yes, as long as the minimum free space is correctly set for that share and for the cache, but note that a pool with 2 different size devices reports the wrong free space.

 

Also, there's no point in using the cache pool for the initial data migration, you can and should enable turbo write.

  • Like 1
Link to comment

There are a couple of things you might have missed in your setup.

 

If you are using the default btrfs raid1 for the cache pool, then you only have 128GB total cache space, since it must mirror everything on 2 devices so you only get the capacity of the smallest.

 

https://lime-technology.com/forums/topic/46802-faq-for-unraid-v6/?tab=comments#comment-480420

 

Also, if you don't set a Minimum Free space for cache, then it will run out of space. unRAID has no way to know how large a file will become when it begins to write it. If a disk has more than Minimum Free, unRAID can choose it to begin writing the file. Then if the file gets too large you get out of space. If a disk has less than Minimum Free remaining unRAID will choose a different disk (in this case, on the array) to begin the write. So, you should set Minimum Free to larger than the largest file you expect to write. Minimum Free for cache is in Settings - Global Share Settings.

 

There is also a Minimum Free setting in each User Share which works similarly. If a disk has less than Minimum Free, unRAID will choose a different disk.

  • Like 1
Link to comment
41 minutes ago, johnnie.black said:

Yes, as long as the minimum free space is correctly set for that share and for the cache, but note that a pool with 2 different size devices reports the wrong free space.

 

Also, there's no point in using the cache pool for the initial data migration, you can and should enable turbo write.

The reason I enabled cache was to have it setup and tested in the way I will eventually run it.

 

Is Turbo write not for when you have more than 2 disks? I did read the post from RobJ about it.

 

40 minutes ago, trurl said:

There are a couple of things you might have missed in your setup.

 

If you are using the default btrfs raid1 for the cache pool, then you only have 128GB total cache space, since it must mirror everything on 2 devices so you only get the capacity of the smallest.

 

https://lime-technology.com/forums/topic/46802-faq-for-unraid-v6/?tab=comments#comment-480420

 

Also, if you don't set a Minimum Free space for cache, then it will run out of space. unRAID has no way to know how large a file will become when it begins to write it. If a disk has more than Minimum Free, unRAID can choose it to begin writing the file. Then if the file gets too large you get out of space. If a disk has less than Minimum Free remaining unRAID will choose a different disk (in this case, on the array) to begin the write. So, you should set Minimum Free to larger than the largest file you expect to write. Minimum Free for cache is in Settings - Global Share Settings.

 

There is also a Minimum Free setting in each User Share which works similarly. If a disk has less than Minimum Free, unRAID will choose a different disk.

Makes sense, I was under the impression that the cache was a bit more transparant but from what I understand now is that it is more like a separate volume. I have set the Minimum Free to 10GB and will test it again once the current job finished.

 

Both thanks a lot for the fast and detailed responses!

Link to comment

Yes the 4TB is parity, once I am happy with unRAID I will replace the SSDs with 4TB HDDs and then use the SSDs for cache.

 

Current:
1x 4TB HDD Parity

2x 500GB SSD Data

1x 128GB NVMe Cache

1x 500GB SSD Cache

 

Eventually:

1x 4TB HDD Parity

2x 4TB HDD Data

2x 500GB SSD Cache

Link to comment
34 minutes ago, Mentox said:

Interesting behaviour, I did configure the Minimum Free to 10GB and it still shows the same error but I can now click on retry and it continues. Had this message around 8 times.

 

I will need to read a bit more on how it exactly works.

 

Are you copying to a User Share or are you copying directly to cache? unRAID only decides where to put things for User Shares. If you are going directly to disk it doesn't decide anything. And it only overflows to the array when you are copying to a User Share and the User Share is set to Use cache Yes or Prefer.

 

https://lime-technology.com/forums/topic/46802-faq-for-unraid-v6/?page=2#comment-537383

 

 

Link to comment
  • 2 years later...
  • 3 months later...

In case anyone runs into the same issue I figured out my original problem. If you set the free space to 10GB, then copy a file of 50GB while there is, for example, 30GB left it will copy to your cache. Unraid does not check the file size before copying so make sure you set the free space to what you expect your biggest file will be.

Link to comment
1 hour ago, Mentox said:

Unraid does not check the file size before copying

In the general case, it has no way to know how large a file will become when it chooses a disk for it. Set Minimum Free for cache (or any other pool) to larger than the largest file you expect to write to it. Each user share also has a Minimum Free setting. If a disk has less than Minimum, Unraid will choose another disk. Also, note that Includes/Excludes or Split Level can still force Unraid to choose a disk that has less than Minimum.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.