Fix Common Problems is reporting one of my drives full. That's fine by me, but will it cause any problems?

[flyize]

As title suggests, I'm fine with the drive being full unless it presents some sort of problems. Logging maybe? No idea. 

 

What say you geniuses?

[itimpi]

7 minutes ago, flyize said:

As title suggests, I'm fine with the drive being full unless it presents some sort of problems. Logging maybe? No idea. 

 

What say you geniuses?

In principle it will not cause any problems,  but have you set Minimum Free Space settings for any shares that can use the drive to stop UnRaid trying to put onto it files that are too large for the remaining space (which will then cause errors).

[flyize]

8 minutes ago, itimpi said:

In principle it will not cause any problems,  but have you set Minimum Free Space settings for any shares that can use the drive to stop UnRaid trying to put onto it files that are too large for the remaining space (which will then cause errors).

Yes, I have minimum free space set, but there's like 100k left on the drive. Is there a scenario where it would try to put even a 200k file on the drive?

[trurl]

4 minutes ago, flyize said:

I have minimum free space set

What do you have it set to? Minimum Free only tells Unraid whether or not it can choose the disk, it doesn't control how much space Unraid tries to keep free on the disk.

5 minutes ago, flyize said:

Is there a scenario where it would try to put even a 200k file on the drive?

Split Level takes precedence over Allocation Method and Minimum Free, so if Split Level says files belong on the same disk it will try to put them there regardless. Also, if you have a file that already exists on the disk, and replace it with a larger file, that replacement will go to the disk the file is already on, so you could run out of space.

 

Probably the most important thing, though, is that filesystem repair needs some extra space to work, so if there is ever corruption on the disk you might not be able to repair.

 

3 minutes ago, flyize said:

100k left on the drive

I would never let any of my disks get that full.

[flyize]

Minimum free space is set to 50GB. I assume your suggestion of media getting replaced is how it got so full.

 

edit: If it matters, split level is on auto and allocation is high water.

 

edit2: No that can't be right. If Sonarr replaces something, its a different file name. How did this drive get so full?

Edited September 21, 2021 by flyize

[trurl]

6 minutes ago, flyize said:

Minimum free space is set to 50GB. I assume your suggestion of media getting replaced is how it got so full.

44 minutes ago, trurl said:

Minimum Free only tells Unraid whether or not it can choose the disk, it doesn't control how much space Unraid tries to keep free on the disk.

If Minimum Free is 50GB, and a disk has a little more free space than 50GB, Unraid can choose the disk when writing a file nearly 50GB in size., resulting in a disk with little space remaining. It can even choose the disk when writing a file 60GB in size, in which case the write will fail.

 

Unraid doesn't know how large a file will become when it chooses a disk for it. It can choose any disk with more than Minimum.

[flyize]

Wait, what? Write a 60GB file to a share where minimum free is only 50GB? These files are coming from a cache drive first, so it should know the size, right? I'm missing something here.

[trurl]

15 minutes ago, trurl said:

Unraid doesn't know how large a file will become when it chooses a disk for it. It can choose any disk with more than Minimum.

In the general case. I haven't actually tested with files that are already on cache.

 

Cache (and other pools) also has a Minimum Free setting which works in a similar manner as for user shares.

 

In the case of user shares, if an array disk has less than Minimum, Unraid will choose another array disk (if other factors allow it).

 

In the case of cache or other pools, if the pool has less than Minimum, Unraid will choose an array disk (overflow) if the user share is cache-prefer or cache-yes.

[itimpi]

6 minutes ago, flyize said:

Wait, what? Write a 60GB file to a share where minimum free is only 50GB? These files are coming from a cache drive first, so it should know the size, right? I'm missing something here.

The mover and the dtive selection logic are independent parts of the system that do not know about each other.  the drive selection logic knows nothing about the size of the file - just that it needs to be created (and at that point it is 0 bytes in size).    Thus the recommendation that Minimum Free Space I’d bigger than the largest file you expect to put in the share.

 

Maybe in a future release Limetech will manage to find a way to couple these parts of the system but at the moment they are not.

[trurl]

6 minutes ago, flyize said:

Write a 60GB file to a share where minimum free is only 50GB? These files are coming from a cache drive first

I can imagine situations where different filesystems in use might make edge cases fail.

[flyize]

1 minute ago, itimpi said:

The mover and the dtive selection logic are independent parts of the system that do not know about each other.  the drive selection logic knows nothing about the size of the file - just that it needs to be created (and at that point it is 0 bytes in size).    Thus the recommendation that Minimum Free Space I’d bigger than the largest file you expect to put in the share.

If the mover can't tell the drive selection process the size of the file, why does minimum file size on a share matter?

[itimpi]

Just now, flyize said:

If the mover can't tell the drive selection process the size of the file, why does minimum file size on a share matter?

Because once the free space falls below that value it is an indication to the drive selection logic to consider picking a different drive if possible.

[trurl]

1 minute ago, flyize said:

why does minimum file size on a share matter?

There is no setting for minimum file size. The setting is Minimum Free Space.

6 minutes ago, trurl said:

if an array disk has less than Minimum, Unraid will choose another array disk (if other factors allow it).

 

[flyize]

2 minutes ago, itimpi said:

Because once the free space falls below that value it is an indication to the drive selection logic to consider picking a different drive if possible.

Oh! I see now. Got it!

[Squid]

2 hours ago, flyize said:

100k left on the drive.

The specific test actually is a test for if the drive is read-only or completely full.  It writes a 4 byte file to the drive (which in reality takes up 4k on the drive). 

 

Is the drive formatted btrfs or xfs?

[flyize]

I have since freed up some space, but its XFS.

[JonathanM]

3 hours ago, flyize said:

I'm fine with the drive being full unless it presents some sort of problems

If you ever get file system corruption, free space is needed to fix it. I recommend keeping some arbitrary percent free, for multiple reasons.

 

The trend for file system overhead is also increasing, so if you have almost zero free and need to copy the contents of the drive to another drive of exactly the same size, it may not fit if the other drive is formatted differently.

 

Also, when sonarr updates metadata and additional files that go with shows, it can expand the space needed.

 

Bottom line, I'd recommend keeping 100GB or so free if you can manage it, for all the above reasons.

[flyize]

That would result in almost 1TB of slack. Ouch