Balancing Disk Space Among Shares/Disks


Recommended Posts

  • 3 weeks later...

Yes, I have that installed, however, it only balances new data coming in, not existing data.  Ergo the issue.  If my disk is full, and I add a new disk, how to 'balance' them.  However (yet again), reading through the posts by @garycase, its better to let the drive fill up...especially for static content.  Better to have ALL of a movie on one drive, than split across multiple drives.  I think the perspective is that for the most part, fill-up (with an appropriate amount of free space) is a reasonable default.

 

Any opinions on best practices behind the disk fill process (most free, fill-up, etc)...?

Link to comment
52 minutes ago, jeffreywhunter said:

Yes, I have that installed, however, it only balances new data coming in, not existing data. 

Where did you get this idea? unBalance plugin can only balance existing data. Where new data gets put depends on your user share settings.

 

The default setting of High-Water is the default for a reason. It is a good compromise between balancing disk usage without making unRAID constantly switch between disks just because one has briefly got more free than another. Fill Up could be the choice if that is indeed what you want to do.

 

Make sure Minimum Free is set to larger than the largest file you will write. unRAID doesn't know how large a file will become when it starts writing it so if you don't set Minimum Free large enough it might choose a disk that doesn't have enough free for the whole file, and of course, it will never split a file.

 

Go to the setting page for a user share and turn on Help.

Link to comment
36 minutes ago, trurl said:

Where did you get this idea? unBalance plugin can only balance existing data. Where new data gets put depends on your user share settings.

 

The default setting of High-Water is the default for a reason. It is a good compromise between balancing disk usage without making unRAID constantly switch between disks just because one has briefly got more free than another. Fill Up could be the choice if that is indeed what you want to do.

 

Make sure Minimum Free is set to larger than the largest file you will write. unRAID doesn't know how large a file will become when it starts writing it so if you don't set Minimum Free large enough it might choose a disk that doesn't have enough free for the whole file, and of course, it will never split a file.

 

Go to the setting page for a user share and turn on Help.

Really?!?  Apologies for the errant statement.  I made that assumption because I had a drive that was nearly full, which never got 'balanced' after I installed the app.  I ended up moving the files manually...ergo the statement.  From what you say, unBalance actively balances existing data on the drives as well?  So I probably don't understand how it works fully, for instance, I have a drive that is again 'creeping' up as I add new movies.  Its now 80% full, with other drives from 21%-66% full in the same share.  Does unbalance not balance until drives reach a specific % full?  So maybe I'm missing something?  Screenshot of share settings...and stats.
 

Thanks for setting things straight.

2018-01-22_12-23.jpg

2018-01-22_12-24.jpg

Link to comment

Duh, that would explain a lot...:$  RTFM!  Its a utility.  When I installed it, "This plugin frees up space space from one of the disks in the array, by moving folders and files to other disks".

 

Not sure how I gathered it would do it automatically.  Perhaps my assumptions overwrote the truth...funny how that happens...  ;)

 

 

 

 

Edited by jeffreywhunter
Link to comment
On ‎1‎/‎22‎/‎2018 at 12:29 PM, jeffreywhunter said:

... I have a drive that is again 'creeping' up as I add new movies.  Its now 80% full, with other drives from 21%-66% full in the same share.

 

And you consider this an issue because ??     I can understand the psychology of wanting to "balance" your content, but really don't see any actual reason to do so.   Especially for effectively "static" content.    My main media server has dual parity plus 12 data disks (all 4TB Reds); with the first 8 being my main "DVDs" share, and the last 4 containing a few other shares that are modified more frequently.    The DVDs share uses "fill up" -- and the first 7 all report 100% utilization (actual free space varies from 88MB to 348MB).   The other shares are set to high water (the last disk was only recently added).

 

 

Media Disk Utilization.jpg

Link to comment
1 hour ago, garycase said:

 

And you consider this an issue because ??     I can understand the psychology of wanting to "balance" your content, but really don't see any actual reason to do so.   Especially for effectively "static" content.    My main media server has dual parity plus 12 data disks (all 4TB Reds); with the first 8 being my main "DVDs" share, and the last 4 containing a few other shares that are modified more frequently.    The DVDs share uses "fill up" -- and the first 7 all report 100% utilization (actual free space varies from 88MB to 348MB).   The other shares are set to high water (the last disk was only recently added).

 

 

Media Disk Utilization.jpg

Yeah, I'm getting it now.  Static content is just that Static.  Does not really matter in the end how much is on a specific drive...other than the performance hit of files stored on the outside vs the inside of the platter.  Not sure that really matters for 'movies and picture'...  I'm satisfied at this point...

 

And besides, the real issue was my ignorance in how UnBalance worked...or better, I didn't know it was a manual utility that one ran.  Lesson learned.  Now I just wish I could get my server to stay up.  Runs a few days, then crashes...but that's a different conversation!  ;)

Link to comment
1 hour ago, jeffreywhunter said:

Does not really matter in the end how much is on a specific drive...other than the performance hit of files stored on the outside vs the inside of the platter. 

 

Yep, that's the bottom line.   And the "performance hit" is really irrelevant => even on the innermost (slowest) cylinders modern drives can provide data far faster than you need for a streaming video (even 4K HD).    In fact, the higher density platters on high capacity drives can easily saturate a Gb Ethernet connection no matter where the data is on the drive.

 

Link to comment
  • 2 years later...
6 hours ago, SidebandSamurai said:

Wasn't it dangerous to have the drives fill up completely to 100%?

I wouldn't call it "dangerous", but it might not be in your best interest. Sometimes when a file system gets corrupted the file system utility program may need some free space to operate, and if you are ever in the position to copy the entire content of a single disk to another single disk, you can run into a situation where file system overhead on a newer or different format may make what fit on one disk not fit on another of the same size.

 

As an aside, why did you choose to reply to a two and a half year old thread to ask the question?

Link to comment
  • 3 weeks later...
On 6/18/2020 at 6:58 AM, jonathanm said:

I wouldn't call it "dangerous", but it might not be in your best interest. Sometimes when a file system gets corrupted the file system utility program may need some free space to operate, and if you are ever in the position to copy the entire content of a single disk to another single disk, you can run into a situation where file system overhead on a newer or different format may make what fit on one disk not fit on another of the same size.

 

As an aside, why did you choose to reply to a two and a half year old thread to ask the question?

I apologize but the thread really applies to me.  Instead of creating a new thread I thought I would keep the thread going so as to keep all the knowledge in one place.  Maybe not the best thing to do but the forums are a repository of knowledge, its better to have one thread that has the answer then have many small threads you need to paw through just to find the answer.  If I have done something wrong, please let me know.  Besides, I did not bother to look at the date. 😚

 

My system has 6 drives and a parity.  I have had this server running without any issue for 5 years.  Disk 1 through Disk 3 and Disk 6 are at 100%.  Disk 4 and Disk 5 are at 99%.  I am going to purchase larger drives shortly but right now budget does not allow for it.  Anyway when I start upgrading the drives, right now they are a mix of 2TB and 3TB drives, would it be a good idea to move some files on these older drives over to the newer drives.  Say keep moving files until the capacity is at least down to 90% for each drive?

Edited by SidebandSamurai
Link to comment
6 hours ago, SidebandSamurai said:

If I have done something wrong, please let me know.

Not wrong, just maybe not the best method to get the best answers. Regarding keeping all the answers together, the longer a thread ages, the more outdated the advice and knowledge is. For some things, it doesn't really matter much, for others, the information and answers may well be completely wrong as technology has moved on.

 

If you are upgrading an existing drive that's full, there is no need to move things around before the upgrade. Just keep the drive you pulled intact until the upgrade has completed successfully, that way if there is an issue you still have a copy of all the files. When the upgrade is done you will have plenty of free space on that drive for new files.

 

Don't move data around without a good reason. Every time you move data it involves some risk of corruption or accidental deletion. That risk is usually very small, but it wouldn't exist if you leave it be. Eventually all your smaller drives will be upgraded.

 

As long as nothing is written or deleted on the full drives, the chances of file system corruption that would need the extra space for recovery is non-existent.

 

Link to comment
  • 9 months later...

I have been reading these posting in regards to balancing drives and the argument of splitting or not splitting files. I have tried to get an experts answer on this on best practice for add media across many disks. I have had my UnRAID setup and configured for over 6 years but really did nothing new to it. I just put more drives in and let it do its thing. Now I am upgrade hardware and wanted to find out is the best (safest for data) is when configuring the shares. Below is the setup I have had forever since I started with UnRAID, never changed it. I am just looking for a second pair of expert eyes if I am doing all this right or am I way wrong. At the beginning I was confused on the settings so I did what I thought was right...

 

Untitled.thumb.png.7b30630265eba7e2e6174e7452be3c1c.png

Edited by gsd2012
Add more
Link to comment

Looks fine to me.

 

Is that Included list all of your disks? Or do you actually intend to exclude some? Seems like a long list of included disks that might be better as Include All or perhaps a shorter Exclude list, but nothing really wrong.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.