Which filesystem should I choose?

[thany]

I would thing this question has been asked before, but digging throught topics in the search for "filesystem" only yields topics with concerns about bugs and misconfigurations.

 

So here's the big question: what filesystem do I choose?

And I don't really want "it depends" as an answer. Seriously. That's not helping anyone. JSYK.

 

Let's face it: unRAID is used to host virtual machines. So yes, big files. And yes, lots of random access.

Perhaps it depends on whether or not to use softraid. Well, what if I'm using softraid? And what if I'm not?

 

There has to be a "best choice".

On VMware ESX, it's VMFS - take it or leave it.

On Windows, it's NTFS - take it or leave it.

On OSX, it's HPFS - take it or leave it. (why am I mentioning this one, this is not even a hypervisor )

 

Having choices is a good thing, but having no way to know which choice to make, is a bad thing.

I was hoping for some solid documentation on this, but for a commercial product, I'm seeing suspiciously little in the way of "how to choose a filesystem" or similar. So, please add this. Or explain here.

 

A filesystem is, after all, chosen only once. There's no way - in practice - to just switch over like that (snaps fingers), and be done with it. So the first choice needs to be the right choice. So, which is the right choice?

[ashman70]

XFS

[trurl]

Please pay attention to the subforum you are posting in. You posted this in Defect Reports. I have moved it.

[John_M]

Choices are quite restricted, actually. XFS, ReiserFS or BTRFS for array disks. Most people choose XFS because it's reliable and well supported. ReiserFS was the only choice with previous unRAID versions but it is no longer being developed. However, it has very good recovery tools that have stood the test of time so people with legacy unRAID setups often keep their existing disks in that format. BTRFS recovery tools are not as mature. I've tried BTRFS on a couple of array disks but I now use XFS exclusively.

 

For single cache disks the choice is between XFS and BTRFS. For multiple disk cache pools, the only choice is BTRFS, so that's what I use.

 

What is this "softraid" you mention? It doesn't apply to unRAID - the md driver has been replaced with unRAID's custom parity driver.

 

I recommend reading a little about unRAID so that you're clear about what it is and what it does. You could start here.

 

HPFS was OS/2's file system of choice, BTW. macOS currently uses HFS+ (with optional journalling), though APFS is in development and may be an option one day.

 

[gubbgnutten]

I would thing this question has been asked before, but digging throught topics in the search for "filesystem" only yields topics with concerns about bugs and misconfigurations.

 

So here's the big question: what filesystem do I choose?

 

Currently? XFS. Unless it is a cache disk that will participate in a cache pool (now or in the future), then BTRFS.

 

And I don't really want "it depends" as an answer. Seriously. That's not helping anyone. JSYK.

 

Let's face it: unRAID is used to host virtual machines. So yes, big files. And yes, lots of random access.

Perhaps it depends on whether or not to use softraid. Well, what if I'm using softraid? And what if I'm not?

 

Let's face it: you might use unRAID to host virtual machines, but that does not mean everyone else does. My use case is NAS, Plex and mostly sequential access. Not quite sure what you mean by using softraid in this context.

 

I can agree that "it depends" is not very helpful, but "it depends" followed by a question about intended use should not be frowned upon if the answer is different for different cases.

 

There has to be a "best choice".

 

Why?

 

On VMware ESX, it's VMFS - take it or leave it.

On Windows, it's NTFS - take it or leave it.

On OSX, it's HPFS - take it or leave it. (why am I mentioning this one, this is not even a hypervisor )

 

You seem to confuse "best choice" with "no choice". Would you be happier if unRAID only provided one sub-par filesystem rather than multiple decent ones?

 

Not sure either why you mentioned OSX, especially considering HPFS was the OS/2 file system. HFS+ is a different beast.

[RobJ]

There are a few comments here.  It hasn't been updated in a year, but the only update I can think of is that the XFS repair tool was improved.  It doesn't say hardly anything more than was stated above though.

[johnny121b]

For single cache disks the choice is between XFS and BTRFS.

 

Wait- so you're saying my current cache disk (v5.06, so ReiserFS) can't follow me into a version 6 upgrade?  That's a bit of bad news.  My cache drive has a ridiculous number of PLEX folders & files nested so deeply I can't even copy them w/o leaving my comfy Windows environment.  I was really hoping to avoid that complication.

 

Presupposing ReiserFS, while not being developed, is already stable and reliable, what advantage(s) does XFS offer?

[JonathanM]

For single cache disks the choice is between XFS and BTRFS.

 

Wait- so you're saying my current cache disk (v5.06, so ReiserFS) can't follow me into a version 6 upgrade?  That's a bit of bad news.  My cache drive has a ridiculous number of PLEX folders & files nested so deeply I can't even copy them w/o leaving my comfy Windows environment.  I was really hoping to avoid that complication.

 

Presupposing ReiserFS, while not being developed, is already stable and reliable, what advantage(s) does XFS offer?

Assuming you follow the upgrade instructions, I think you will be able to keep your current cache format. However... I and many others experienced a problem with mostly full ReiserFS disks on V6+ that manifests as a complete system lockup under heavy load. You may be lucky and everything works fine, but be prepared to migrate all your disks to XFS if you start to experience issues. That is the downside of staying with a dying file system, you have to be prepared for people to address issues by telling you to change file systems, which involves copying ALL your data from disk to disk as you successively format each empty disk in turn. Long painful process that hopefully will only happen once.

[itimpi]

For single cache disks the choice is between XFS and BTRFS.

 

Wait- so you're saying my current cache disk (v5.06, so ReiserFS) can't follow me into a version 6 upgrade?  That's a bit of bad news.  My cache drive has a ridiculous number of PLEX folders & files nested so deeply I can't even copy them w/o leaving my comfy Windows environment.  I was really hoping to avoid that complication.

 

Presupposing ReiserFS, while not being developed, is already stable and reliable, what advantage(s) does XFS offer?

you can continue to use ReiserFS as long as you only want a single cache drive.    For new users XFS or BTRFS are recommended as being higher performance (particularly as the disk gets near full).

[John_M]

For single cache disks the choice is between XFS and BTRFS.

 

Wait- so you're saying my current cache disk (v5.06, so ReiserFS) can't follow me into a version 6 upgrade?

 

I was answering the OP who is clearly intending to build a new server. I don't think unRAID 6.2 gives the choice of ReiserFS when creating a cache disk (though it is there as an option for data disks). However, I'm not able to test this right now and I use a cache pool anyway. I don't know what the situation is with legacy systems so no, I'm not saying that at all.

 

[JorgeB]

I was answering the OP who is clearly intending to build a new server. I don't think unRAID 6.2 gives the choice of ReiserFS when creating a cache disk (though it is there as an option for data disks). However, I'm not able to test this right now and I use a cache pool anyway. I don't know what the situation is with legacy systems so no, I'm not saying that at all.

 

You still can use RFS, but not recommended for new servers, especially for SSDs as it doesn't support TRIM.

[John_M]

Thanks for the clarification johnnie.black. I guess that answers this question too:

 

Presupposing ReiserFS, while not being developed, is already stable and reliable, what advantage(s) does XFS offer?

[garycase]

Thanks for the clarification johnnie.black. I guess that answers this question too:

 

Presupposing ReiserFS, while not being developed, is already stable and reliable, what advantage(s) does XFS offer?

 

If the cache disk is an SSD, then yes, that's an important feature.

 

In addition to that, RFS also has the known issue of slow writes when a disk gets nearly full (over 90% or so).  But if the disk isn't full; and isn't likely to get that way, there's no compelling reason to switch.    As already noted, the RFS recovery tools are superb -- much better than those for XFS -- so if you ever need them that's a consideration.

 

Personally, I would use XFS or BTRFS for the cache; and XFS for new array drives.  But on an older system where your array drives are all RFS, there's no real reason to convert them -- especially if (as is often the case) they're essentially static content (i.e. media).    But I'd probably convert a cache disk from RFS to XFS or BTRFS => yes, that could be a pain, as you'd need to Stop all activity (Dockers, VM's, etc.); not use the array for anything else;  copy the entire cache contents to another location;  change the file system; and then copy everything back.    But since the cache is a very active disk (lots of writes), it's probably worth the bother for that specific drive (unlike the array drives).  However -- if the drive isn't close to full; you may want to simply wait until you upgrade to v6 and see how it performs before making a decision r.e. the file system.

 

 

 

[thany]

@John_M @gubbgnutten

Softraid is software RAID. As opposed to hardware RAID. I thought it was a common term by now. For instace, the €10000 server at work uses hardware RAID, while a NAS uses software RAID.

Not to be confused with firmware RAID

 

Would you be happier if unRAID only provided one sub-par filesystem rather than multiple decent ones?

If the only choice is a good one.

 

Not sure either why you mentioned OSX, especially considering HPFS was the OS/2 file system. HFS+ is a different beast.

I'm not an OSX user, so I probably meant HFS+.

[trurl]

@John_M @gubbgnutten

Softraid is software RAID. As opposed to hardware RAID. I thought it was a common term by now. For instace, the €10000 server at work uses hardware RAID, while a NAS uses software RAID.

Not to be confused with firmware RAID

Their point was that unRAID is not RAID.

[garycase]

Although UnRAID does not use a traditional RAID level, it is for all intents and purposes a software RAID implementation.  In fact, there's no option to use a hardware RAID controller for the UnRAID parity calculations -- a lot of folks who run UnRAID use hardware RAID controllers to provide additional SATA ports -- but NOT to actually implement a RAID configuration.

 

[thany]

@John_M @gubbgnutten

Softraid is software RAID. As opposed to hardware RAID. I thought it was a common term by now. For instace, the €10000 server at work uses hardware RAID, while a NAS uses software RAID.

Not to be confused with firmware RAID

Their point was that unRAID is not RAID.

It's still Redundand, and it's still an Array, and they are still Independend Disks I've connected

 

It's not RAID5 or RAID6 or whatever, it's certainly an implementation of a custom RAID topology.

[John_M]

My point was this:

 

What is this "softraid" you mention? It doesn't apply to unRAID - the md driver has been replaced with unRAID's custom parity driver.

 

I assumed that by "softraid" the OP meant md-based software RAID, which isn't an option with the unRAID OS, for the reason I gave.

 

[GreenDolphin]

I'll hitch a ride on this thread...

I'm in the process of ordering parts for my first UnRAID build, so also need to make the same decision as the OP.

For the cache drive(s):

I'll be starting out with one (SSD) drive, but adding a second in a pool later on, so obvious choice is BTRFS.

 

Personally, I would use XFS or BTRFS for the cache; and XFS for new array drives. 

I'm curious why XFS for the array drives.

I recall reading that BTRFS's "native" RAID support has issues (particularly RAID5/RAID6), but UnRAID doesn't use that.

Are there stability issues (and if so, which) with the BTRFS version UnRAID uses?

Or maybe there aren't as good disk recovery/analysis tools?

 

The BTRFS feature I find particularly attractive is bitrot detection/correction (eventually if it's not working yet).

 

I expect my array to hold ~15TB to start, and expand continually. I'd rather avoid the hassle of a large-copy project later on just to switch filesystems,

and was hoping BTRFS would be reliable enough to start with.

[tdallen]

Are there stability issues (and if so, which) with the BTRFS version...?

Do a full forum search on "btrfs corrupt".  My conclusion is that it is not yet as stable/mature as XFS.  Fine for a cache drive where everything is backed up/recoverable and holds promise for array drives, imo.  But I wouldn't use it for the array yet.

[ashman70]

Yet you read an article like this and it makes me wonder if BTRFS is not ready for prime time.

 

http://www.virtualtothecore.com/en/2016-btrfs-really-next-filesystem/

[JorgeB]

I really like btrfs's checksum feature, so I recently converted my main server to btrfs, if all goes well I'm thinking of doing the same to my other servers.

 

From what I read and I my own experience btrfs seems mostly stable for single disk usage and the recovery tools are not so bad once you know how to use them, especially restore.

 

I believe btrfs achilles heel is when there's an intermittent problem with one of the disks in a redundant pool, e.g., a raid1 pool where on the disks is having connection issues, btrfs doesn't know to kick it and keeps trying to write to that disk and if it goes on long enough it may end up corrupting the whole pool.

 

Still, I would only recommend using it for completely stable servers with a UPS.