Hi,

 

I've got the issue recently that transfers were timed out 90% before actually they started, and in the other 10% they somehow got copied after minutes of struggling (I've seen the HDD led active for minutes for a 100KB file).

 

Was looking at the forum and found the reason, namely that disks shouldn't be more filled than the largest file on the same disk, because of the journaling file system.

 

I can live with that, but I think that's a bug. If this is the nature of the file system, then I would say unRAID shouldn't report this space as free and definitely shouldn't try to write on it. Especially new or unexperienced users who doesn't even aware of the forum can struggle a lot until (if ever) they find a solution.

 

Don't you guys agree?

Hi,

 

I've got the issue recently that transfers were timed out 90% before actually they started, and in the other 10% they somehow got copied after minutes of struggling (I've seen the HDD led active for minutes for a 100KB file).

 

Was looking at the forum and found the reason, namely that disks shouldn't be more filled than the largest file on the same disk, because of the journaling file system.

 

I can live with that, but I think that's a bug. If this is the nature of the file system, then I would say unRAID shouldn't report this space as free and definitely shouldn't try to write on it. Especially new or unexperienced users who doesn't even aware of the forum can struggle a lot until (if ever) they find a solution.

 

Don't you guys agree?

 

I've never done an extensive analysis on what happens when the disks become near full.  I know that I can fill disks to within a couple GB's and not see any issues, but then I'm writing mostly large files to them and rarely deleting.

 

In your case, it would be interesting to run a 'reiserfsck' operation on that disk to see if there's any corruption.

 

I've never heard of the the "reserve 10% for journaling file system" rule, or seen it mentioned in regards to ReiserFS.  Source?

Hi Tom,

 

source was Joe L. in this case

http://lime-technology.com/forum/index.php?topic=9158.msg87493#msg87493

 

it's surely not a corruption issue and was reproducable on multiple disk.

It is actually only happened to me because I've taken out a few disks so temporarly I copied over their contents to other disks, so the utilization went up very high on all disks. I could copy to all disks where there was enough space and only had this issue with disks whose utilization were high. All disks were holding large files, it just happened by accident that I actually wanted to copy over a small pdf file.

 

When I replaced the disks and redistributed the content again, the issue gone.

I typically leave 6GB to 10GB free on all of my disks.  I don't think I have run into the problem you are describing but like I said I have always left free space on my drives.

is there some "golden tip" about this as to which amount free space we should set our disks ?

i have them set for the moment on 50 GB

but on a 10 disk array this is 500 GB of unused space which is rather big

 

guess setting them on 10 GB for 1 TB drives and 20 GB on 2 TB drive would be acceptable .... but is that not too little ?

 

 

is there some "golden tip" about this as to which amount free space we should set our disks ?

i have them set for the moment on 50 GB

but on a 10 disk array this is 500 GB of unused space which is rather big

 

guess setting them on 10 GB for 1 TB drives and 20 GB on 2 TB drive would be acceptable .... but is that not too little ?

 

 

 

I figure leaving 10GB on each disk is about right.

 

NOTE: the above number is COMPLETELY different from the Min free space setting for user shares.  I generally right directly to a disk share which bypasses the min free space related stuff.  I use User shares mainly for read-only access.

I understand what you are saying, but I am pretty sure there is an issue around that.

Not something dramatic, and like I said I can live with that since I am also able work around it, but this is not so robust this way I think.

 

When I experienced this iirc I had about 3-4GB free space on the drives where I tried to copy the 100KB file with the mentioned issues, but probably the largest files on all those disks was about 20GB or more.

I aim to fill up all my drives. I don't really care about space for journalling because if my server crashes during a write, I'll just delete the partially written files and re-copy them from the source again. I'm not creating anything new or unique directly on the server.

 

I've got plenty of drives on my server with a few MB free, not GB.

 

If there's any reason to have free space other than for journal replays, let me know, otherwise, I'll continue filling them up.

 

P.S. I only write to disk shares, not user shares, so I ignore any "fill up" rules on the share.

 

I aim to fill up all my drives. I don't really care about space for journalling because if my server crashes during a write, I'll just delete the partially written files and re-copy them from the source again. I'm not creating anything new or unique directly on the server.

 

I've got plenty of drives on my server with a few MB free, not GB.

 

If there's any reason to have free space other than for journal replays, let me know, otherwise, I'll continue filling them up.

 

P.S. I only write to disk shares, not user shares, so I ignore any "fill up" rules on the share.

 

 

I don't have a hard and fast reason to leave some free space but my gut and general knowledge of computers tells me it is not a good idea to fill up a drive like you have.

As far as I know journaling is the only reason to leave free space on a drive (and you should leave the same amount of free space as your largest file on that drive).  If you don't care about journaling, there's no reason to leave free space.  I've also filled drives to the brim with no problems.

 

prostuff1, you may be thinking of boot drives.  Boot drives should never be filled to the brim as the OS will then have no caching space.  However, for data drives like unRAID uses it doesn't matter.  Just don't fill your flash drive all the way

prostuff1, you may be thinking of boot drives.  Boot drives should never be filled to the brim as the OS will then have no caching space.  However, for data drives like unRAID uses it doesn't matter.  Just don't fill your flash drive all the way

 

Right, I agree. I always like to leave quite a few GB free on any OS drive. Data drives? They must be filled to the brim otherwise they are just "drive is half empty". 

Like neilt0, I have a number of drives with just a few MB free. Before all this talk about leaving space = largest file I did this, and the drives are now pure R/O. Been considering moving some files off these disks, but would likely lose 500g or so following advice to leave space >= largest file on every disk. So still undercided whether to do or not.

Like neilt0, I have a number of drives with just a few MB free. Before all this talk about leaving space = largest file I did this, and the drives are now pure R/O. Been considering moving some files off these disks, but would likely lose 500g or so following advice to leave space >= largest file on every disk. So still undercided whether to do or not.

 

I don't believe you have to leave that much free space on a drive.  That would likely be 30Gb to 40GB for a BluRay DVD.  The reason that number is thrown out there is for User Shares.

 

If I set a Min Free Space of 40GB and have 41GB left I should be able to write a 30GB file to the drive and end up with 10ish GB of space left on the drive. From there writing to the drive via a User Share should be impossible as there is now less space then is specified in the Min Free Space setting.

I aim to fill up all my drives. I don't really care about space for journalling because if my server crashes during a write, I'll just delete the partially written files and re-copy them from the source again. I'm not creating anything new or unique directly on the server.

 

It seems you also experienced these "crash"-es what I am talking about. It's not really a crash though, well a kind of transfer crash.

 

...but what about you guys with filled up disks? You never experienced that?

I think that user shares have a problem with low free space drives. It's hard to quantify and it's not all that serious. If you want to top off a disk using disk shares I don't think this problem will occur.

It is indeed occur. I've tested this with disk shares.

 

Anyhow, I really don't want to pump on this problem. If you all think it's a marginal issue, then I am fine with that.

I haven't experienced this problem but a number of people have reported this problem. If we gather information about it into a chart then we can identify the problem and add a solution or a fix to the roadmap. Focusing on disk shares rather than user shares simplifies the issue. This may be a problem with RieserFS and then it is on the roadmap a long way off. It may not be a problem with RieserFS and then we can expect a resolution sooner.

I think that user shares have a problem with low free space drives. It's hard to quantify and it's not all that serious. If you want to top off a disk using disk shares I don't think this problem will occur.

 

Right -- as i mentioned before I don't write to user shares, I only write to disk shares, and I check how much is free before I start writing to the disk. I tend to write hundreds of GB at a time.

 

I aim to fill up all my drives. I don't really care about space for journalling because if my server crashes during a write, I'll just delete the partially written files and re-copy them from the source again. I'm not creating anything new or unique directly on the server.

 

It seems you also experienced these "crash"-es what I am talking about. It's not really a crash though, well a kind of transfer crash.

 

...but what about you guys with filled up disks? You never experienced that?

 

Never had a "transfer crash" as I don't write to user shares. I have had the server crash if there's a loose power or SATA cable. Sometimes it overheats.

I've recently rebuilt my large server (pics to follow) to avoid these problems.

I think that user shares have a problem with low free space drives. It's hard to quantify and it's not all that serious. If you want to top off a disk using disk shares I don't think this problem will occur.

 

I agree, however, it's still annoying.  I had ~15 gigs free on each of the 15 data drives (12 750gbs and 3 2tbs) in my server and it would no longer write to user shares.  I was attempting to copy over the last 11gbs (not all in one folder), but it kept failing.  So I just did it through disk shares and dumped it all on one drive so now one of them has 6 gigs free.  I already added another 4 hd's so I can't replicate the problem presently, but it had me scratching my head for a minute.  I could understand if the directory was 15 gigs, but it wasn't so....

 

Nate

Did you try reducing the minimum free user share setting?

could one of you guys with almost full disks (with large, >10GB files) try to write a 100KB file directly to the disk share?

Did you try reducing the minimum free user share setting?

 

All of mine have always been set to 0.

could one of you guys with almost full disks (with large, >10GB files) try to write a 100KB file directly to the disk share?

 

Why? It works. I only write to disk shares. I can see issues writing to user shares, which is why I don't do that.