I just completed a copy of a 3T RFS drive to a 3T XFS drive, and the free space went up by 1G. The drive was almost full (75G remaining in RFS, and went up to 76G in XFS). The drive contained files in the 10G to 40G range, and NOT a bunch of small files.

Note:  I don't really know much about RFS or XFS.

 

Could this be an effect of copying the entire drive at once?  Maybe the journal is smaller because of that?  Try doing the copy from XFS to a formatted RFS and measure.

Did you measure the available space before the copy? Both filesystems have several parameters to adjust "overhead". Since you know how many of what size files you need to put on the filesystem, you can use xfs_info and optimize your filesystem layout.

 

What you are probably seeing is the effect of XFS's dynamic inode allocation vs RFS metadata structure (does not use inodes). With that file size distribution, most filesystems would over allocate metadata space with default settings.

Did you measure the available space before the copy? Both filesystems have several parameters to adjust "overhead". Since you know how many of what size files you need to put on the filesystem, you can use xfs_info and optimize your filesystem layout.

 

What you are probably seeing is the effect of XFS's dynamic inode allocation vs RFS metadata structure (does not use inodes). With that file size distribution, most filesystems would over allocate metadata space with default settings.

 

I was copying with "cp -rpv ...". Seems unlikely there would be any insights XFS would glean to somehow optimize it's disk usage or layout.

 

Makes sense that XFS is saving space on metadata, which might lead to different results if copying a bunch of smaller files.

I believe XFS will always use less space than RFS. I'll try to dig up the links to where this was discussed.

Did you measure the available space before the copy? Both filesystems have several parameters to adjust "overhead". Since you know how many of what size files you need to put on the filesystem, you can use xfs_info and optimize your filesystem layout.

 

What you are probably seeing is the effect of XFS's dynamic inode allocation vs RFS metadata structure (does not use inodes). With that file size distribution, most filesystems would over allocate metadata space with default settings.

 

I was copying with "cp -rpv ...". Seems unlikely there would be any insights XFS would glean to somehow optimize it's disk usage or layout.

 

Makes sense that XFS is saving space on metadata, which might lead to different results if copying a bunch of smaller files.

 

XFS does several things to reduce diskspace usage. But this seems like a case of metadata allocation.

 

By not allocating inodes for files which do not yet exist, XFS does indeed optimize the disk usage. By using xfs_info, you might optimize the disk layout. This is a bit overkill in most cases, but your file size distribution is outside the range considered typical. You listed 10GB as the smallest file. If I swag it, you have about 111-186 files on that drive, not sure how many directories, but you probably have inodes to spare (or in excess), but XFS by default allocates the minimum number of inodes (you can not reduce, only increase).

 

If the drive is full and future writes are not going to happen, you can reduce the log allocation. Also you might reduce the number of AG since parallelism is a non issue. But at 1GB/2793GB, the savings are not huge.

 

Pretty good guess. I have 112 items on the drive, 107 files + 5 directories.

 

Here is the xfs_info output.

 

meta-data=/dev/md3              isize=256    agcount=4, agsize=183141659 blks

        =                      sectsz=512  attr=2

data    =                      bsize=4096  blocks=732566633, imaxpct=5

        =                      sunit=0      swidth=0 blks

naming  =version 2              bsize=4096  ascii-ci=0

log      =internal              bsize=4096  blocks=357698, version=2

        =                      sectsz=512  sunit=0 blks, lazy-count=1

realtime =none                  extsz=4096  blocks=0, rtextents=0

 

Can you provide some insight into what that is telling me?

Just copied another disk. Also 3T. About 2600 files.

 

Disk nearly full (15G free). This time the new (XFS) disk used MORE space than the RFS disk - it only had 8G free after the copy - meaning XFS consumed 7G more.

 

Here is the xfs_info output. Seems identical to the othe one.

 

meta-data=/dev/md6              isize=256    agcount=4, agsize=183141659 blks

        =                      sectsz=512  attr=2

data    =                      bsize=4096  blocks=732566633, imaxpct=5

        =                      sunit=0      swidth=0 blks

naming  =version 2              bsize=4096  ascii-ci=0

log      =internal              bsize=4096  blocks=357698, version=2

        =                      sectsz=512  sunit=0 blks, lazy-count=1

realtime =none                  extsz=4096  blocks=0, rtextents=0

Yes, the two are the same since the filesystem was created with the defaults.

 

Even with 2600 files the typical size is large, 1+GB average size.

 

The most interesting thing about XFS is the Allocation Groups (AG). They are meant to allow simultaneous writes to the file system. Each AG is a subvolume with it's own free block list. AG are limited to 1TB size, so 4 on the 3TB drive is near the minimum. With your filesystem on a single drive, these AGs wont help much.

 

isize is size for inodes. The 256 is the minimum and XFS will allocate more if needed. But if you knew you were going to be small size/high count you could pre-allocate more.

 

On a striped storage system, the sunit and swidth would be tune to match, Again, single drive, change will not help.

 

If the filesystem is going to be read only, the log will not be used. So, you could reduce the size of the log, but again such a change is not worth it, 1GB/2793GB.

 

No idea why XFS is using more than ReiserFS. 7GB is a sizable jump for 2600 files, but possible if half the files are under 512 bytes.

Is it worth the effort to migrate drives from RFS to XFS?  How are the recovery options?  I'm setting new disks to XFS but migrating all of my old disks would be very time consuming.  What are you reasons for migrating if you don't mind me asking?

Is it worth the effort to migrate drives from RFS to XFS?  How are the recovery options?  I'm setting new disks to XFS but migrating all of my old disks would be very time consuming.  What are you reasons for migrating if you don't mind me asking?

 

 

Maintenance on ReiserFS over the past couple years has lead to some potential bugs which could corrupt data. Fortunately they were caught in betas and resolve by patches.

ReiserFS is not getting the attention it used to and from what I understand, it's not part of the mainline kernel code any longer.

Agree that using XFS for new drives make sense; but I see no reason to migrate current RFS drives to XFS ... not only is Reiser VERY stable; but if you do get some corruption, reiserfsck is VERY robust and has often recovered data for folks on this forum that I never expected they'd be able to restore.

 

I have no intention of changing my current arrays; but will certainly use XFS for the next server I build.

 

 

As Weebo points out, RFS is quickly going the way of the dodo bird. It has done a yeoman's job in the past, but with the recent data corruption issue and the fact it is falling into disfavor with the Linux community, I think now is a good time to move off of it completely. It is not an emergency, but I am doing it a disk at a time and will complete it in January. (The original author is also in jail for murdering his wife so I have no sentimental attachment to RFS.)

 

Another reason - I have had a persistent problem with Samba when an RFS disk starts to fill up. It seems worse with larger disks. An attempted copy of a file from Windows will time out, and it could take multiple tries to finally get the copy to start. I have found XFS to be much better.

 

One last point is the extremely high usage of XFS in production systems. I like to be on the same file system as the big boys. The recent corruption bug took a while to find due, in part, to the fact that its usage is waning. And I am sure this bug is hastening the departure.

 

We have had good luck (I might say even say phenomenal luck) with RFS and its recoverability (i.e., using reiserfsck). We have had no experience with XFS with unRAID. I have to believe that with the extensive use of XFS out there, that is has to be reasonably good. But only time will tell.

Funny you mention samba performance. I can't say it has anything to do with anything, or anything to do with samba, but I can say since I just converted all over to XFS, my performance copying to/from the array and my win7 desktop over samba is muuuuuch smoother and faster. I used to see a lot of start/stop/start behavior in the transfer stream. Now it is rock steady and saturating my network.  It isn't scientific but there you have it. A nice juicy anecdote for you

Thanks all,  sorry to derail the thread somewhat,  had a chat about this on IRC too,  think I will probably migrate... slowly.

If you have a drive with lots of files or a drive that is written to a lot, and those drives get anywhere near full, it's worth migrating.

 

I migrated two out of my six 4TB drives to XFS because they slowed writes down so much, the drives were essentially useless.

Deleting files doesn't fix it.

 

Writes are now much quicker, and deletes are also very fast. 

...  think I will probably migrate... slowly.

 

That was my point as well => my RFS drives are rock-solid, so I see no real reason to switch; but I won't use RFS for any NEW drives I add to the array ... so I'll simply migrate as I add drives, rather than change any current drives.  I'll probably build a new UnRAID server sometime next year, so clearly that unit will use XFS [and will probably have all 8TB drives  ]

 

Hi guys! I am building a new array and want to use XFS only. I still have 1 hdd in my new setup formatted as RFS since this array was built for testing kvm. This hdd is going to be formatted as XFS soon. When done I would like to migrate all data from my old array (v5 + RFS) to the new one. Pls, how would you suggest would be the best way to do it (based on your experience)?

My subjective experience is VM running in an ssd formatted as xfs runs smoother and show no single error (btrfs dooes show errors).

Another reason - I have had a persistent problem with Samba when an RFS disk starts to fill up. It seems worse with larger disks. An attempted copy of a file from Windows will time out, and it could take multiple tries to finally get the copy to start. I have found XFS to be much better.

 

 

Someone posted a change to the windows registry to allow a longer timeout over the network.

 

However, this still doesn't change the fact that RieserFS support is waning.

I've built a new server and plan to migrate to XFS myself.

Interesting that you're seeing time-outs.  I've never had a time-out from Windows, and several of my disks are down to 10-15MB of free space (clearly I don't write to these anymore, but the writes that filled them to this point worked fine).

 

I certainly agree that for any new server with v6 it's a good idea to use XFS.  And while it's certainly true that support for Reiser is waning, it's equally true that on systems where it's working perfectly (e.g. v5) there's no reason to be concerned.  It's rock-solid, and given that v5 is essentially static (barring any major latent but I doubt there will be any future changes), there's no reason to be concerned about having RFS disks.

 

With v6 it's a different story -- as v6 evolves, it's going to use newer core components; and there MAY be some Reiser issues that get introduced in those (as we saw a few Beta's ago).    For a v6 server, I'd use XFS for the disks for sure.

 

The "limbo spot" is for upgrades from 5 to 6 (or v4.7 to 6 for those still on that) => in those cases it's still not, IMHO, necessary to migrate FULL disks from RFS to XFS, since they're likely never going to be modified (and it's VERY unlikely there will be any READ issues).  But for disks that are still actively being used for writes, the best approach is probably to install new XFS disks and copy the data from the old RFS disks to those; and then reformat them as XFS ... or just relegate them to backup use with whatever file system is appropriate (perhaps NTFS if they're being written via Windows).

 

Personally, my ultimate "migration" to XFS will simply be via a new server, and I'll copy everything from the old servers to the new one, without bothering the old disks.

 

... I would like to migrate all data from my old array (v5 + RFS) to the new one. Pls, how would you suggest would be the best way to do it (based on your experience)?

 

Just copy the data across the network from the old server to the new one.    After the copy's done, you can either keep the old server as your backup; or just reformat the disks in whatever file system you want.

 

... I would like to migrate all data from my old array (v5 + RFS) to the new one. Pls, how would you suggest would be the best way to do it (based on your experience)?

 

Just copy the data across the network from the old server to the new one.    After the copy's done, you can either keep the old server as your backup; or just reformat the disks in whatever file system you want.

 

 

use teracopy in verify mode or create md5sums of the whole disk and verify it before reformatting the original disks.

Frankly I prefer rsync, yet in any case, do something that verifies the source and destination before erasing the source.

Interesting that you're seeing time-outs.  I've never had a time-out from Windows, and several of my disks are down to 10-15MB of free space (clearly I don't write to these anymore, but the writes that filled them to this point worked fine).

 

Seems related to pre-allocating the sectors for a large copy. Worse on very large files (40G+ on larger disks (3T and 4T) that are getting full. It does not happen except on Samba - copying within the server works fine. I have this on two very different server configurations.

 

I certainly agree that for any new server with v6 it's a good idea to use XFS.  And while it's certainly true that support for Reiser is waning, it's equally true that on systems where it's working perfectly (e.g. v5) there's no reason to be concerned.  It's rock-solid, and given that v5 is essentially static (barring any major latent but I doubt there will be any future changes), there's no reason to be concerned about having RFS disks.

 

Version 5 does not have an XFS option and is static. RFS has been solid in that version. (I might be nervous if there is ever a maintenance release with a new RFS FS version. Version 6 does have alternatives and RFS has proven itself NOT rock-solid on version 6. I would recommend those on version 6 to get their disks converted over, new server or not.

Interesting that you're seeing time-outs.  I've never had a time-out from Windows, and several of my disks are down to 10-15MB of free space (clearly I don't write to these anymore, but the writes that filled them to this point worked fine).

 

Seems related to pre-allocating the sectors for a large copy. Worse on very large files (40G+ on larger disks (3T and 4T) that are getting full. It does not happen except on Samba - copying within the server works fine. I have this on two very different server configurations.

 

I've seen this issue in days gone by. Especially for the slower drives as ReiserFS walks through the data looking for free space.

Once the file was allocated it seemed that ReiserFS was now happy with all the dentry/inode data in ram.

My shortcut solution would be to login to the server and make a dummy file in the destination directory which would cause loading of the dentry/inode tables.  I've not had to do this in ages. Larger drives, faster drives, larger ram caches on the drives, more ram in the OS or narrower arrays for me. I haven't put my finger on it, but it hasn't happened in a long time.

 

 

I certainly agree that for any new server with v6 it's a good idea to use XFS.  And while it's certainly true that support for Reiser is waning, it's equally true that on systems where it's working perfectly (e.g. v5) there's no reason to be concerned.  It's rock-solid, and given that v5 is essentially static (barring any major latent but I doubt there will be any future changes), there's no reason to be concerned about having RFS disks.

 

Version 5 does not have an XFS option and is static. RFS has been solid in that version. (I might be nervous if there is ever a maintenance release with a new RFS FS version. Version 6 does have alternatives and RFS has proven itself NOT rock-solid on version 6. I would recommend those on version 6 to get their disks converted over, new server or not.

 

I'm with bjp999 on this one.

Version 5 does not have an XFS option and is static. RFS has been solid in that version. (I might be nervous if there is ever a maintenance release with a new RFS FS version. Version 6 does have alternatives and RFS has proven itself NOT rock-solid on version 6. I would recommend those on version 6 to get their disks converted over, new server or not.

 

Agree ... as I noted the static nature of v5 means it's simply not an issue.    Note that my comment about "... no reason to be concerned about having RFS disks" was specific to v5:

 

...  on systems where it's working perfectly (e.g. v5) there's no reason to be concerned.  It's rock-solid, and given that v5 is essentially static ... there's no reason to be concerned about having RFS disks.

 

I agree that if v5 has further releases all bets are off if the RFS components are changed.  But I very much doubt that will be the case -- it's pretty clear that LimeTech is completely focused on v6.

 

I do think, however, that any "read only disks" (e.g. full disks in RFS that aren't going to have further write activity .. not an unusual occurrence in large arrays with a lot of media -- I have nearly 20 disks in that situation) don't need to be converted, even if moved to v6.    I'm not likely to move them to v6 anyway, since if I build a v6 server it'll simply be a new server with 6 or 8TB drives, so I'd just copy all the data to it instead of moving the old, lower-capacity drives ... but some folks may not want to buy new drives or build a new server when they migrate to v6.