High-water allocation method not working with share that has excluded disks

[greenythebeast]

This is the current setup of one of my shares:

 

Essentially, I want the share to be allocated with the high-water method over Disks 3 (2TB) and 4 (1TB). However, this is how my drives are filling up:

 

So my question is, why isn't any data being written to Disk 4? I have split level disabled so that shouldn't be the problem. Do I have something setup wrong? Please help!

[Joe L.]

What do you have under "Settings->Share Settings"

I'll bet they say disk1-3.  You really should just leave those include/exclude fields blank.

(I'll bet you do not have disk4 listed.) 

[greenythebeast]

What do you have under "Settings->Share Settings"

I'll bet they say disk1-3.  You really should just leave those include/exclude fields blank.

(I'll bet you do not have disk4 listed.)

 

All included, none excluded.

[dgaschk]

Show screen shots of the configuration screens.

[Joe L.]

The one configuration screen you have shown has a "space" in each of the disk names.  That is wrong.

 

Use "disk1"

Not "Disk 1"

 

Use all lower case for the disk names (although I'm not sure if that matters)

 

There is no need to populate BOTH the includes and excludes.  One is enough.

 

As I said, you've not listed "disk4"  Your include line is likely being ignored.  (Disk 4 != disk4 )

 

 

 

[greenythebeast]

The one configuration screen you have shown has a "space" in each of the disk names.  That is wrong.

 

Use "disk1"

Not "Disk 1"

 

Use all lower case for the disk names (although I'm not sure if that matters)

 

There is no need to populate BOTH the includes and excludes.  One is enough.

 

As I said, you've not listed "disk4"  Your include line is likely being ignored.  (Disk 4 != disk4 )

 

I use SimpleFeatures. It is my understanding that the capitalization and spacing is just a formatting decision on the developer's part and that's not how the actual config is written. What file does that config screen write to? I could check to see what the file looks like and go from there.

[garycase]

You don't need to (and shouldn't) specify BOTH incudes and excludes.    If you list includes, then all other disks are automatically excluded.    In addition, I'd do as JoeL noted ... list the disks with no spacing, as UnRAID requires.  I have no idea if Simple Features "translates" this correctly -- but why take a chance?

 

The disk allocation is indeed a bit strange.  Are ALL of your shares set to High Water?

 

List the allocation on each disk of this specific share -- just do a "Properties" of the high-level folder "Files" on each disk (be sure to check ALL disks for this file ... not just the "includes").

 

Finally, which disk is which?  I assume disk4 is the 1TB unit ... so it won't be written to until the high-water mark drops below 1TB ... meaning the 3TB unit needs to be more than 50% full.  If disk2 is your 3TB unit, then that hasn't happened yet.  But whether this what's happening or not depends on the actual interactions between the includes, excludes, and split levels -- and I'm not sure what impact your excludes are having, since you also have includes set ... and all of them aren't using the "no-space" format that UnRAID expects.

 

[greenythebeast]

The one configuration screen you have shown has a "space" in each of the disk names.  That is wrong.

 

Use "disk1"

Not "Disk 1"

 

Use all lower case for the disk names (although I'm not sure if that matters)

 

There is no need to populate BOTH the includes and excludes.  One is enough.

 

As I said, you've not listed "disk4"  Your include line is likely being ignored.  (Disk 4 != disk4 )

 

As I mentioned, SimpleFeatures doesn't allow you to type in the disk names. It uses a checkbox system:

[greenythebeast]

You don't need to (and shouldn't) specify BOTH incudes and excludes.    If you list includes, then all other disks are automatically excluded.    In addition, I'd do as JoeL noted ... list the disks with no spacing, as UnRAID requires.  I have no idea if Simple Features "translates" this correctly -- but why take a chance?

 

The disk allocation is indeed a bit strange.  Are ALL of your shares set to High Water?

 

List the allocation on each disk of this specific share -- just do a "Properties" of the high-level folder "Files" on each disk (be sure to check ALL disks for this file ... not just the "includes").

 

Finally, which disk is which?  I assume disk4 is the 1TB unit ... so it won't be written to until the high-water mark drops below 1TB ... meaning the 3TB unit needs to be more than 50% full.  If disk2 is your 3TB unit, then that hasn't happened yet.  But whether this what's happening or not depends on the actual interactions between the includes, excludes, and split levels -- and I'm not sure what impact your excludes are having, since you also have includes set ... and all of them aren't using the "no-space" format that UnRAID expects.

 

Disk 3 is a 2TB while Disk 4 is a 1TB, so the high water mark should have been crossed by now. All of my non-cache-only shares are set to High Water.

[garycase]

Disk 3 is a 2TB while Disk 4 is a 1TB, so the high water mark should have been crossed by now. All of my non-cache-only shares are set to High Water.

 

As I suspected, Disk 4 is the 1TB unit, which I presume means Disk 2 is a 3TB unit and is nowhere near the high-water mark.    I suspect that what is happening is the allocation logic is selecting Disk 2; but that selection is being over-ridden by the exclude logic, so it's then simply writing to the current folder name, and since it exists on Disk3, it just writes there.

 

Create an empty folder with the appropriate share name on Disk 4 and it may use that instead, since that disk has more space.    But I'm not sure, as I doubt the allocation logic is "re-checked" after it's done so the first time.    This may simply be a bug in that logic -- it would seem that the high-water logic should be applied only to those disks which are valid targets for the current write;  but is apparently being applied first; and then simply not used if the selected target disk is excluded from the current share.

 

[garycase]

... it also may be a bug in the way SimpleFeatures sets the includes/excludes.    I'd disable SimpleFeatures and see what the standard Web GUI shows for the Includes/Excludes settings.  As I noted earlier, you do NOT want to use both.

 

[greenythebeast]

... it also may be a bug in the way SimpleFeatures sets the includes/excludes.    I'd disable SimpleFeatures and see what the standard Web GUI shows for the Includes/Excludes settings.  As I noted earlier, you do NOT want to use both.

 

What file does the settings page write to? I could find it manually and see what is being set in the file.

[garycase]

What file does the settings page write to? I could find it manually and see what is being set in the file.

 

In the Config\Shares folder on the flash drive, there's a .CFG file for each of your shares.

 

It will have a shareInclude=<...>  line and a shareExclude=<...> line that will list all of the included and excluded disks.  You should NOT set both of these manually ... you set one; then UnRAID simply lists all of the other disks on the opposing list.  I don't know what happens if you set them manually, and the intersection of the two sets isn't exactly the complete set of disks.

 

But, for example, your Files.CFG should have these three lines in it (among others):

 

shareInclude=disk3,disk4

shareExclude=disk1,disk2

shareAllocator=highwater

 

[greenythebeast]

What file does the settings page write to? I could find it manually and see what is being set in the file.

 

In the Config\Shares folder on the flash drive, there's a .CFG file for each of your shares.

 

It will have a shareInclude=<...>  line and a shareExclude=<...> line that will list all of the included and excluded disks.  You should NOT set both of these manually ... you set one; then UnRAID simply lists all of the other disks on the opposing list.  I don't know what happens if you set them manually, and the intersection of the two sets isn't exactly the complete set of disks.

 

But, for example, your Files.CFG should have these three lines in it (among others):

 

shareInclude=disk3,disk4

shareExclude=disk1,disk2

shareAllocator=highwater

 

I removed the excludes from the settings page like you suggested. The config file now reads:

 

shareInclude="disk3,disk4"
shareExclude=""
shareUseCache="no"
shareAllocator="highwater"
shareSplitLevel="999"

 

So it seems the settings are being written correctly. Not sure what the deal is

[garycase]

Not sure if it matters, but on all my shares that don't have split levels set, it shows

 

shareSplitLevel=

 

(NOT a number, like your 999)

 

[greenythebeast]

Not sure if it matters, but on all my shares that don't have split levels set, it shows

 

shareSplitLevel=

 

(NOT a number, like your 999)

 

That doesn't fix the problem either. Not really sure what could be the problem at this point. Both drives have the root directory on them, but it just won't write to Disk 4 for whatever reason!

[garycase]

I suspect it's what I noted earlier -- the allocation method check is done before the include/exclude.    Seems like a bug !!

 

If that's the case, then it won't write to disk4 until either disk2 hits the 1.5TB point (50%); or disk3 doesn't have room.

 

... but before concluding that, you should at least upgrade to RC13 and see if that resolves it.

 

[greenythebeast]

I suspect it's what I noted earlier -- the allocation method check is done before the include/exclude.    Seems like a bug !!

 

If that's the case, then it won't write to disk4 until either disk2 hits the 1.5TB point (50%); or disk3 doesn't have room.

 

... but before concluding that, you should at least upgrade to RC13 and see if that resolves it.

 

I'd rather not upgrade to rc13 because of all the problems it's been having. I just remembered that disk2 is a 2TB disk. I'm going to try to get disk2 to 1TB to see if I can confirm your hypothesis.

[Freddie]

I think your allocation is working as expected.  You can think of it like a set of wells where a larger disk is like a deeper well.  And the only thing that matters is how much empty space there is at the top of the well.

 

For your setup I wouldn't expect the share to start filling disk4 until disk3 reaches 75%.

 

If disk3 were 50% full and disk 4 were empty, there would be 1T free on both disks. In this condition, new data will be written to disk3 until the next high water mark which is .5T free (or 75% used of 2T).  Then new data will be written to disk4 until it has less than .5T free (or 50% used of 1T).

 

[garycase]

Actually that analysis is probably right -- if so, high-water is working for the share as well, so my earlier thought r.e. the allocation decision preceding the include decision may not be right.

 

Consider your disk3 & disk4 share:  disk3 is 2TB, disk4 is 1TB.  So at high-water, the first thing that happens is disk3 will be written until there's 1TB left.  Then the high-water point for that share is set at 1/2 of the remaining 1TB .. i.e. 500GB.    On the next write, the system selects the lowest numbered disk that has 1TB remaining -- in this case disk3;  so it's now going to write to disk3 until the next high-water point (500GB);  at that point, it will choose the disk that's part of the share with the most space left, and use it until IT has 500GB left.    So ... when disk3 gets to 75% full, it will start writing to disk4, and will use it until it drops to 500GB free; then switch back to disk3 until 250GB; then disk4 until 250GB; etc.

 

[garycase]

... Note that it's fairly easy to confirm that.  Disk3 is already at 71%, so you only need to write 4% (80GB) to the Files share to "hit" the high-water mark.   

[BobPhoenix]

FYI and Highwater.

 

I have an array with 15 drives in it on 5.0rc10.  3tb parity, 12 2TB drives, 2 3TB data drives. 

 

I created a new share with the following values:

Name:			MediaCenter
Comments:	
Allocation method:	High-water
Min. free space:	90000000
Split level:		0
Included disk(s):	
Excluded disk(s):	disk1-9

Disks 10-14 were empty when share was created 3 2TB and 2 3TB.  I created a directory MediaCenter on each of the drives 10-14 so that unRAID would be sure to use them and not have problems writing to them - probably wasn't necessary but I usually write to disk shares until this experiment.

 

After writing a bunch of data here are the results:

Disk	Size	%Used	Free
disk10	2T	27%	1.47T
disk11	2T	26%	1.5T
disk12	2T	26%	1.49T
disk13	3T	61%	1.18T
disk14	3T	35%	1.96T

Currently writing to disk14.  It would have stopped on disk13 when it reached 1.5T free but I was copying directories and the whole directory took it from 49% to 61% used.  When that directory finished at 1.18T free it switched to disk14.

 

So I think that highwater will split when a drive hits the amount free space for 1/2 of the parity drive size - in my case 1.5T.  When disk14 hits 1.5T then I expect disk10 to be written to until it has 750GB free space same for rest, etc... Until I run out of data ~6TB from now.

[garycase]

So I think that highwater will split when a drive hits the amount free space for 1/2 of the parity drive size - in my case 1.5T. 

 

To be precise it's supposed to split when the drive his a free space <= 1/2 your largest data drive ... NOT your parity drive.    In MOST cases that's probably the same number; but with current drive sizes it's probably not unusual to have a 4TB parity drive and no data drives that large for a while after you add the new parity drive.

 

[BobPhoenix]

So I think that highwater will split when a drive hits the amount free space for 1/2 of the parity drive size - in my case 1.5T. 

 

To be precise it's supposed to split when the drive his a free space <= 1/2 your largest data drive ... NOT your parity drive.    In MOST cases that's probably the same number; but with current drive sizes it's probably not unusual to have a 4TB parity drive and no data drives that large for a while after you add the new parity drive.

Which doesn't match my results.  The copy started on my 2TB drives and when they had 1.5TB free approximately it moved to the next drive.  When it got to the 3TB drive it went beyond but only because the directory that was being copied was HUGE and it had 1.51TB free before the copy of that huge directory.  After that directory copy there was 1.18TB free.  Oh and this was a continuous copy operation from Windows box.  I selected all of the directories in Windows Explorer and pasted it into the unRAID user share and watched the ~2TB of data copy.  So for me it was based on the largest drive in the array when it figured half size and those drives with less still had the same space free.  So for 2TB drives it only used about 500GB before moving on to a new drive whereas the 3TB was >50% full.

[garycase]

So I think that highwater will split when a drive hits the amount free space for 1/2 of the parity drive size - in my case 1.5T. 

 

To be precise it's supposed to split when the drive his a free space <= 1/2 your largest data drive ... NOT your parity drive.    In MOST cases that's probably the same number; but with current drive sizes it's probably not unusual to have a 4TB parity drive and no data drives that large for a while after you add the new parity drive.

Which doesn't match my results.  The copy started on my 2TB drives and when they had 1.5TB free approximately it moved to the next drive.  When it got to the 3TB drive it went beyond but only because the directory that was being copied was HUGE and it had 1.51TB free before the copy of that huge directory.  After that directory copy there was 1.18TB free.  Oh and this was a continuous copy operation from Windows box.  I selected all of the directories in Windows Explorer and pasted it into the unRAID user share and watched the ~2TB of data copy.  So for me it was based on the largest drive in the array when it figured half size and those drives with less still had the same space free.  So for 2TB drives it only used about 500GB before moving on to a new drive whereas the 3TB was >50% full.

 

That's behaving fine.  The high-water marks are computed based on the largest data drive ... so in your case that's 1/2 of 3TB = 1.5TB.  This is NOT adjusted based on the size of other drives.  So, for example, once it starts writing to the 2TB drive, it will switch again after it's down to 1.5TB (1/2 of the 3TB size).    The next "high water" mark will be 750GB (half of the 1.5GB); etc.  So, for example, if you had a 750GB drive in the array, it would remain empty until all the larger drives were down to that size.