TSM

Was just about to say I've been wanting to go from RFS to XFS. Perfect application. Wish I'd have had this when I did my conversion. It would have saved me quite a bit of time emptying that first 6TB drive. Trying to find space to put everything it had on it.

Personally I wouldn't because I'd be afraid the drive would fail during the parity check. And if my understanding of the process is correct, depending on how the drive fails you could be in a worse state than you are right now. And I guess it would depend on whether the check was correcting or non-correcting. But others here might have a more educated and intelligible reason for doing it or not doing it. And I could be wrong...

Where I used to work, we had a lot of Cat 5 cabling in the walls. They'd upgrade the switch in an area to one that was gigabit capable, and the next day we'd get a lot of calls from people whose nic in their pc's had auto-negotiated with the new switch gigabit speed and their network performance was terrible. Hard set both ends to 100 Full and they were good to go.

NIC supports 1000 but only connected at 100. Check cables and ports at both ends. And make sure all cables are Cat 6 or Cat 5e and not Cat 5.

MB/s is a megabyte per seconds. Mb/s is a megabit per second. If the nic is connecting at 100 mb/s and transfers are going at 8 MB/s on a powerline connection, I'd think that was pretty OK. As to why it slowed down when cache was added I couldn't say. My only thought on that is possibly change the sata cable, make sure it's rated for Sata 3 and not 1 or 2. Could be a bad cable. Same thing with the sata port. Could be a bad drive. When I was converting my drives to xfs, and was in the process of moving files around. I noticed that one drive seemed to be a lot slower than the other drives during this process. I connected it to a different available sata port (thankfully I had a port available on a different card) and it worked normally after that.

Thanks Boof, Helps a lot. I guess at this point Crashplan is the defacto unraid standard application for a highly configurable backup solution. I'm thinking I know the answer to this question, since you are currently using Crashplan, but which do you like better? Which is easier to administer on a daily basis? Which is easier to fine tune the behavior of? To me it looks like you might be able to run BackupPC natively on unraid, instead of a docker like crashplan. Is that how you did it? Did you have instances with either where you actually had to restore a backed up file? What was the experience like? I also like the fact with BackupPC that it doesn't require a client running on the machine being backed up. Not an issue for me, but may be for some. I read on crashplan's site that if an identical file exists on 2 computers, being backed up by crashplan, that crashplan will have 2 separate copies of the file. With BackupPC they say they will only have 1 copy of that file, and links to all the other locations. Might save some people some disk space.

I've been contemplating setting up a backup server recently. I've looked at some of the various options mentioned on the board like crashplan and rsync. Googling rsync showed me a few other options that expand upon rsync's capabilities. One that's particularly interesting to me is BackupPC. There are some older posts on the forum about BackupPC, but nothing recently. I'm curious if anyone out there is quietly using it successfully without posting on the forums about it. I'd also like to hear from some of the folks on here who might have more expertise than me as far as what it's drawbacks might be in comparison to Crashplan for example. If it can be made to work with unraid(I'm sure a smarter person than I could get it to work easily), it looks like it might have a smaller memory footprint than crashplan, and it looks to also implement some pretty impressive compression and deduplication features. http://backuppc.sourceforge.net/index.html

My unraid box's motherboard has been in continual service since February of 2008. Time to get a new one. I've actually been coveting this board for a couple of months now. But I've been hesitating on pulling the trigger because there is a review on Newegg from a guy who said he couldn't get it to boot from usb. http://www.newegg.com/Product/Product.aspx?Item=N82E16813132230 Also in the last paragraph of the tweaktown review they mention memory compatibility issues. "The P9A-I/C2550/SAS/4L motherboard is also fussy about memory used. None of our normal DDR3 server memory would work with this motherboard. The PA-I only has support for UDIMM with ECC memory, non-ECC UDIMM's are allowed, but not recommended. Be sure to check the Memory/Device support section for supported memory on this platform." I was only able to find Asus's suggested memory at one or 2 online retailers I've never used before. And nowhere could I find memory sticks that would allow you to max out the ram on this board.

More recent thread. http://lime-technology.com/forum/index.php?topic=34432.0 In this thread we even started talking about the fact that some of us would be willing to pay for the dual parity feature if it was something that would require a significant amount of Limetech's time and effort.

I think that's a great idea. No idea if it's something they can implement or not. But this gets my vote too as a requested feature. I'm thinking protection could also be achieved during this process by adding dual parity. But there are a lot of other threads discussing that, and it's a feature that I don't even think is on Limetech's radar right now. It's a shame really. This would accomplish what you are asking for, plus it would have other positive advantages as well.

This is a pretty good thread. All the talk about backup servers has got me thinking. What if instead of syncing the servers, you wanted to do a true incremental backup from one server to the other? I'm of course thinking about those times when the drives are fine, but you've accidentally deleted a file or changed a file and need an old version of it back, if you don't realize that before a sync is scheduled you'd be screwed. With what I'm thinking, maybe you'd be able to go into a console of some sort and restore an older version of it. I know software like this exists, but the question is, is there a version that would work on an unraid server? Maybe as a docker?

The price right now is $208.99, which is still pretty nice especially if you are a prime customer. I have 2 of these drives. One is going on 9 or 10 months old, and the other was given to me as a Christmas present. They aren't flashy, but they've been stable good drives so far. YMMV

I agree completely, and I don't think they should drop the items that are in the current development pipeline to work on this. But adding it to "next in line" in the pipeline wouldn't be too bad. I'd pay up to $100 or so. That's probably more than what other people would want to pay, but it's a feature I really want.

I am, and I have purchased licenses for spare USB keys to help fund the current development stream even though I don't need them. I suppose the proof is in the pudding. People who want to fund the effort will need to make it known. I have purchased licenses for spare usb keys that I have not redeemed in the many years I have been running unraid to fund development efforts. I also contribute significant time on the forums to help the community which contributes to their product. There are many like Weebotech and myself who have already contributed significantly to fund the project in various ways, such as all the moderators, all the addon developers wether it be plugins or dockers, and all the addon service providers who support unraid. I think you may have misunderstood the intent of my comments. I want dual parity, and am willing to pay a reasonable amount of money to get it. That's all I was trying to say.

The response to this request in the past has always been Limetech saying it would be a lot of work, and "helpful" folks on the forum arguing it's not needed. I don't know how much work it would be on Limetech's part to implement this, but to me at least it would seem like a better use of their time to develop, than many other features that have been added since this was first requested. I'd even be willing to purchase an upgraded license version to get this feature. A Pro-Advanced or a Super-Pro license maybe. And I haven't given Limetech any money since 2008, but yet I've enjoyed the benefits of their work through several upgrade cycles for the past 7 years. I'm wondering how many other people would be willing to pay for an upgrade like this as well. A scenario like that might make it more worthwhile for them to spend their time on development, by adding a potential revenue stream from people who already purchased a pro license long ago. And the people who don't think it's needed, well they don't have to buy it.

+1

Dumb question here guys, but I'm assuming you could pass multiple cards right? Like in a situation where you have multiple cards connecting your unraid drives. Or would it work better if all unraid drives were connected to the same card?

As you add/remove files to/from directories, those directories get fragmented, plus the metadata starts to be placed around the filesystem (Wherever free space may be). The larger and deeper the directory tree, the more prone to this problem on reads. As the filesystem becomes full, the filesystem driver has to search through the trees to allocate directory space and/or metadata space for stat information. I've noticed that rsyncing data from one drive to another causes all the directories to be built first. This pre-allocation of directory space and putting them at the outer tracks provides a speed boost. I.E. The directory can possibly be read sequentially without random head movement while searching for other blocks/meta data. I've noticed this type of benefit with rsyncing and moving filesystems with ext3 and reiserfs. I'm not that familiar with XFS at this point in time. From what I remember, historically it was better at file allocation and removal. Now File System fragmentation like you describe, that makes sense. You say that rsync pre-allocates the directory space. Would MC do the same thing? I think somebody said that MC is really just a gui front end for the underlying Linux commands that would do the same thing.

+1 + Infinity All this talking back and forth about what drives are most important. All my drives are important. I don't want any of them to die. I had a drive die about 6 months ago and I did not enjoy the experience. Dual parity would have made it much more pleasant. Double Drive failure? No. But until I was able to replace the drive that died I felt extremely on edge and agitated. And no I don't have a good back up plan, and I know that parity is not the same as a good backup, but I still desperately want dual parity.

+1 + Infinity All this talking back and forth about what drives are most important. All my drives are important. I don't want any of them to die. I had a drive die about 6 months ago and I did not enjoy the experience. Dual parity would have made it much more pleasant.

What you say makes sense logically, but I can't see how my drives could have been fragmented badly enough for that to have single handedly caused the problems I saw. All things being equal, I'm sure I had some fragmentation, but for the most part my files are written once, and then read multiple times. I have maybe 20 smallish files that ever get changed, and almost nothing ever got moved around once written. Plus, I think there are other posts where people have said that reiserfs disks that are very full can cause performance issues with unraid. I'm not saying you're wrong, because you may know a lot more about the topic than I do, I'm just saying that I'm not convinced.

Saturday morning I finished migrating all of my drives to XFS, and I must say the difference in system performance is striking. It feels like I gave the server a hardware upgrade. My server has 14 data drives with a complex directory structure, with many sub-folders having contents that reside on multiple different drives. I also have a lot of drives that are very close to being full, which I know can cause reiserfs to have issues sometimes. When opening folders with a lot of files in them that are strewn across multiple drives, there is still a delay but not big enough to cause me any anguish. I used to have some folders that would cause Windows Explorer to time out once maybe even twice before I could finally view their contents. Those folders took maybe 30 seconds to open after the upgrade. And folders that took 10 to 20 seconds to open before, now seem to open almost instantly or with a brief hesitation. Before doing the migration I was seriously considering upgrading my server's core hardware, but now I'm not so sure. I might just leave it alone for another few years.

I've been migrating all my drives to XFS, slow going, but thus far no problems that weren't caused by me and explainable. Until yesterday. But this one was interesting. I was moving all the files off of an almost full 4TB reiserFS drive onto a newly formatted XFS drive, also 4TB, using MC inside a screen session. And when I say the drive was almost full, the reiserFS drive had maybe a few megabytes free. But everything I've read in the forums about migrations says that after formatting with XFS a drive will at least have the same amount of space on it for files, and I think some people have claimed it might even have more. It seems that the very last file on the drive to be moved was a bluray iso image of about 30gigs in size. And it got stuck moving about 3/4 of the way through. Via Windows explorer I moved 10gigs of files off of the destination drive onto the hard drive of my desktop pc, told MC to try again and it worked. Then through windows explorer I moved the same files I just moved off, back on to the destination drive, and it worked. Unraid gui shows a few megs free on the drive, unfortunately I didn't take a screen shot or write down how much free space was on the ReiserFS drive before the move, but I think the space is pretty similar. Anyone have any idea what might have happened here? Not a big deal because all files involved have a safe and secure home now, it was just weird.

Part of me wants to believe that this is the only thing you post on forums. Also the technology is impressive. I know he also posts about beer because I replied to one earlier. That is a reasonable alternative. I want to believe. I supposed I deserve some ridicule for that. Perhaps I should expound on my thought process, to reduce the ire of the more puritanical among us. I read a great article a few years ago that talked in quite a bit of detail about how the rise of online pornography contributed to the development and refinement of many internet technologies. Things like E-Commerce, streaming video and compression. The writer of the article gave examples of how this or that specific feature or iteration of an existing technology was used for the first time in a porn related thing. My guess is that this new Hololens technology will be no different. I'm at work right now, so I obviously can't do much web searching related to this, but when I get home tonight I'll try to find the article.

porn