I have 3 User Shares all using Disk 1-8 of my Array (with Disk 9 Excluded) and a 4th Using only Disk 9

 

I have those 3 Shares all set to High Water Allocation and Split Level set to "2".  I have done multiple disk copies from various folders on various PC's and other NAS locations to fill up the drives to where they are now.  All copies were from XP or Vista directly to the shares, except for a couple times when I wrote directly to Disk 7 and 8 because they were getting ZERO writes even on big copies (Would Cycle back to Drive 1 after 5)

 

Is there a reason why my Disk #1, 750GB have WAY Less free space than the other drives (As a proportion of thier total space)?  I thought high water was supposed to fill to 50% on each drive and then cycle back up and Fill to 75%, etc, etc,etc.

 

The 2nd Screen shot has Disk 8 also in the included disc's

Improvement: now possible to specify a range of disks in Included disks and Excluded disks.  For example:

  "disk1-4, disk6, disk10-11"

 

The above comes from one of the 4.2 beta announcements.  You might try changing your list of included drives to disk1-8; can't guarantee any difference, just something to try.

 

Is there a reason why my Disk #1, 750GB have WAY Less free space than the other drives (As a proportion of thier total space)?

 

Just a guess. If you use SplitLevel=2 please look at the folders below that SplitLevel on disk1. E.g.

 

/mnt/disk1/Level1/Level2/Level3

 

If there's is one Level3 folder on your disk1 containing lots of data it indicates that the user share is not aware of data written to disk1 directly. I know, you said you didn't write to disk1 directly but it looks so.

 

I saw it once on my array too. I had to re-start the array and files were written to other disks then.

 

Regards

Harald

 

I'm having the same problem, and just started a similar thread. If one changes the allocation method, doe sit take effect immediately, or is a restart necessary?

I'm having the same problem, and just started a similar thread. If one changes the allocation method, doe sit take effect immediately, or is a restart necessary?

 

I'm not sure since I changed a few things on my unRAID box since seeing this error on mine, but it seems to be better now... sorta...

 

I replaced a failed drive, swaped a bunch of SATA Cables and changed all my user Shares to to include "disk1-8", instead of "disk1,disk2..." (except for the one share that is just "disk9"... so I have rebooted a few times and stopped and started my array a few more...

 

But all of my drives now appear to be filling to ~170-182GB Free... In this screenshot, Disk 7 is currently being written too, I expect down to that similar level... and then Disk 8... disk 2 has some writes because I'm moving and changing some of the files in the share (that happen to live physically on Disk 2)

 

 

As a preliminary experiment, I even created a folder on the empty disk with the same name as the share folders to see if this manual creation would be the equivalent of assigning that disk to the share. I transferred a couple of small files to it, too. I then rebooted the server and found that, yes indeed, the share was seeing the new files I moved to it on the disk that was previously not part of the share.

 

Then, I just went into the command line and manually moved a bunch of files within the same share, but between disks to even things up a little. I was getting annoyed seeing one disk with only 40 GB left filling up fast, while another had 720 GB free and wasn't being written to, even though it was supposedly part of the share.

 

Hopefully when I get home and restart the server, or maybe just take it offline and bring it back online, all the files I transferred will still be part of the same share as they were before the move.

 

I hope someone figures out what the problem with High Water allocation is.

I'm having the same problem, and just started a similar thread. If one changes the allocation method, doe sit take effect immediately, or is a restart necessary?

 

No restart necessary.

Here's the high-water allocation algorithm:

 

// first collect the size and free space of each data disk, and in the process
// determine which data disk is the largest
highwater = 0;
for (i=1; i<=num_data_disks; i++) {
  if (data disk 'i' is not active in the share)
    continue;

  size[i] = size of data disk number 'i'
  free[i] = amount of free space on disk number 'i'

  if (size[i] > highwater)
    highwater = size[i]
}

// now find the lowest data disk with free space still above highwater
while ((highwater /= 2) != 0) {
  for (i=1; i<=num_data_disks; i++) {
    if (free[i] > highwater)
      return i;  // ie, use disk 'i'
  }
}

return 0; // ie, no space available

 

So if you have one or two disks very much larger than the others, then you'll see space allocated on them until their free space more closely matches the others.

 

I might be an interesting to make a version like 'highwater', but instead of using absolute amount of free space, use percentage amount of free space.

 

Also, keep in mind if you have a directory deeper than 'split-level', then all files and subdirs created in that directory will stay on the disk that directory happens to be on.

Thanks for the answers limetech. In the thread I made, specific to my problem, I mention that all my disks are 750 GB and the split level is 1 for them all.

If it helps:

The fourth disk that I recently added (the disk that doesn't seem to be written to, or willingly become part of some of my shares) is on a Promise controller, while the other 4 disks (parity plaus disks 1-3) are on the motherboard.

Here's the high-water allocation algorithm:

 

A couple of questions on this example code, I do know it is not the actual, just a pseudo-code version of it, but:

1.  It's not clear that the set of disks being examined is the entire set of data disks, or the subset of disks involved with a particular user share.  You wouldn't want a large disk not a part of the user share making highwater decisions for the disks of the share.

2.  It returns 0 (no space) at the end, the same as returning 0 (first disk).  It probably never falls through to the end, but the calling function would be unable to determine whether it meant 'no space' or 'use the first disk'.  (Sorry, I know it's just pseudo-code, but my brain seems to have a built-in 'lint' tool.)

 

fitbrit:  I can't help wondering if a split level of 2 would work better for you, although I admit I don't know your file and directory layout.

 

A couple of questions on this example code, I do know it is not the actual, just a pseudo-code version of it, but:

1.  It's not clear that the set of disks being examined is the entire set of data disks, or the subset of disks involved with a particular user share.  You wouldn't want a large disk not a part of the user share making highwater decisions for the disks of the share.

2.  It returns 0 (no space) at the end, the same as returning 0 (first disk).  It probably never falls through to the end, but the calling function would be unable to determine whether it meant 'no space' or 'use the first disk'.  (Sorry, I know it's just pseudo-code, but my brain seems to have a built-in 'lint' tool.)

 

fitbrit:  I can't help wondering if a split level of 2 would work better for you, although I admit I don't know your file and directory layout.

 

 

Ok, I debugged the pseudo code 

 

1. Yes, it only considers the subset of disks 'included' in the share, as determined by current settings of 'included disks' and 'excluded disks' for the share.

2. Right, indexes start at 1 - corrected above.

 

Such is the risk of posting pseudo-code    The reason I did it is sometimes it's easier to see the algorithm than explain it!

Such is the risk of posting pseudo-code

 

Sorry, perfectionists can be such a nuisance!

 

Regarding uneven allocation being reported by SuperW2 and fitbrit - I would say this probably occurs because you are copying files into a directory deeper than 'split-level'.  In this case all such files will stay on the same disk as it's parent directory, regardless of how much free space remains; and eventually, if you keep copying to those deeper directories, it will report 'no more space' even though there might be lots of free space among the disks of the share - make sense?

 

Something to keep in mind with User Shares - it is NOT intended to be a general purpose file system which can handle efficiently any combination of large numbers of files and directories, without regards to the underlying disks.  You still need to be aware of the fact there are individual disk file systems, and that the user shares are really a "psuedo file system" kept in memory.

 

The intent is that unRAID is used primarily to store media and/or files which have some sort of organization to them.  (Complicating this is the desire to keep the contents of certain directories "together" on the same disk.)

 

My suggestion is that you divide up the physical disks in some logical manner.  For example, dedicate one disk say to Music, and have that disk be exclusive to the Music share.  If you outgrow that disk, either get a bigger disk, or allocate another dedicated disk.  Similar with other media types.

 

Having said this, however, we'll soon release ability to address more than 1GB of memory, so you could just put in say 4GB and never worry about it

Regarding uneven allocation being reported by SuperW2 and fitbrit - I would say this probably occurs because you are copying files into a directory deeper than 'split-level'.  In this case all such files will stay on the same disk as it's parent directory, regardless of how much free space remains; and eventually, if you keep copying to those deeper directories, it will report 'no more space' even though there might be lots of free space among the disks of the share - make sense?

 

Your supposition is true, and I get it now. I actually started writing a rebuttal, but then realised that I've changed the way I work/organise my media, and that's causing this problem... with split level, rather than allocation. I copy movies into a folder called Import residing within my Movies share. I then import these movies later into JRiver Media Center, and subsequently move them up a level so they're stored in the first level of the Movies share.

Before, I had a share called import that i used for all my media; now each form of media (photos, DVD-Rips, Music etc.) has its own import subfolder, which I copy into. So what I need is to go back to using the Import share, or get you guys to enable exceptions to split level on certain folders! 

 

Thanks a lot for the help, guys.