SSD Posted February 13, 2008 Share Posted February 13, 2008 My TIP is to NOT assign a parity drive when you are initially loading your array. Of course you'll lose the ability to recover from a failure - but during the loading process you must have your files stored somewhere else in order to copy them to the unRAID - so your data is already protected. Write performance is a lot better without the parity, and you'll save some serious time completing the load. It also saves a little wear and tear on the parity drive. My plan is to get the array loaded, add the parity drive, let it build parity, and finally to take out a drive and let the system rebuild it. Once I know all that is working, I will delete the copies on my desktop machines and run with parity on a go forward basis. - Brian Quote Link to comment
RobJ Posted February 15, 2008 Share Posted February 15, 2008 Sounds like a very good and efficient plan (I especially like the remove and rebuild testing part), *IF* you can afford to buy a whole new set of drives to move the data to. Many of us wish we could have done it that way, but had to start with 2 new drives, and move the data over one drive at a time, and then add that drive to the array, to have space for the next drive. Quote Link to comment
hawihoney Posted February 16, 2008 Share Posted February 16, 2008 My plan is to get the array loaded, add the parity drive, let it build parity Hmm, I'm currently loading with the parity-drive doing millions of read/write cycles. Is there a way to remove the parity drive, complete filling the data drives, and then re-add the parity drive with a final recreate of the parity information? What would be the neccessary steps? Thanks Harald Quote Link to comment
RobJ Posted February 16, 2008 Share Posted February 16, 2008 Very easy, just un-assign it. When you are ready with all other data disks filled the way you want them, then assign the parity drive and click the Start button, and parity will be built automatically. Quote Link to comment
hawihoney Posted February 16, 2008 Share Posted February 16, 2008 Very easy, just un-assign it. When you are ready with all other data disks filled the way you want them, then assign the parity drive and click the Start button, and parity will be built automatically. When re-assigning the parity-drive it makes no problem that it is filled to some extend? Thanks Harald Quote Link to comment
RobJ Posted February 16, 2008 Share Posted February 16, 2008 The parity drive is not a normal drive, it has no file system, and therefore has no concept of being full or empty. Every bit position on it has a parity bit calculated from the corresponding bit positions on all other drives. A parity sync/build/rebuild always overwrites every single bit on the drive. Quote Link to comment
hawihoney Posted February 16, 2008 Share Posted February 16, 2008 The parity drive is not a normal drive, it has no file system, and therefore has no concept of being full or empty. Every bit position on it has a parity bit calculated from the corresponding bit positions on all other drives. A parity sync/build/rebuild always overwrites every single bit on the drive. Thanks. Just an hour ago I un-assigned the parity drive and went on filling my new unRAID server. The performance didn't get that boost I expected ... Greetings Harald Quote Link to comment
SSD Posted February 17, 2008 Author Share Posted February 17, 2008 Hmmm ... I am surprised. My performance is much faster without the parity drive. You might want to make sure that your drives are properly jumpered. Are you sure that you actually removed the parity disk - that you didn't just stop the parity process? I don't think that stopping the parity process stops real-time updates to the parity drive. Quote Link to comment
hawihoney Posted February 17, 2008 Share Posted February 17, 2008 Hmmm ... I am surprised. My performance is much faster without the parity drive. You might want to make sure that your drives are properly jumpered. Yes, I did un-assign the parity drive. It's marked red and "not installed". I'm filling SATA-2 drives they don't need jumpers (usually). I'm writing to the user-shares - not to the disks directly. I'm not really familiar with Linux so I only can imagine what's happening. There's a shfs process eating all ressources (CPU _and_ memory). Perhaps this is the reason. In an article I found in the web was a factor 4 increase in writing without the parity drive on unRAID. I only can dream of such a performance boost. It's ok for me. I calculated 10 days to fill the new beast with all the data. I'm still right on track. Regards Harald Quote Link to comment
Billped Posted February 17, 2008 Share Posted February 17, 2008 Not sure what network speed you are running, but if it is 100Mb, that's the problem. That speed gets saturated with the parity, so taking out the parity would offer no improvement. If you think you are running GigE, you should validate that. Bill Quote Link to comment
Joe L. Posted February 17, 2008 Share Posted February 17, 2008 shfs is the shared file system. If you can initially decide where to copy your files you can speed things up a bit by copying directly to disk1,2,3,... etc instead of the user shares. Writing directly to the disk shares is generally twice as fast as writing through the user-share interface. Quote Link to comment
hawihoney Posted February 18, 2008 Share Posted February 18, 2008 Thanks for all your answers. Writing directly to the disk shares is generally twice as fast as writing through the user-share interface. Yep. That's what I thought. Seems that one of my Teras dropped back from 1000MBit/s to 100MBit/s. This is the real bottleneck. Harald Quote Link to comment
SSD Posted February 21, 2008 Author Share Posted February 21, 2008 Sounds like a very good and efficient plan (I especially like the remove and rebuild testing part), *IF* you can afford to buy a whole new set of drives to move the data to. Many of us wish we could have done it that way, but had to start with 2 new drives, and move the data over one drive at a time, and then add that drive to the array, to have space for the next drive. Thought I'd post back and report success in completing my load process, including having unRAID rebuild a single drive. I wonder if anyone has done the "swap disable" drive replacement. I don't have a 1T drive to try it with. I used every gig on every computer in my home network to free the space on most of the hard disks I wanted to put in the array. Everyone could do this for the initial load. Even if you can't do everything, you can at least do the frst step with parity turned off and speed up at least that part. Then you can turn parity on and complete the incremental drive additions to the array. Oh, and be careful when parity is disabled! I made a terrible mistake that could have been a disaster. Either have good backups or have parity enabled. (Note: you should have good backups of important data anyway!) Quote Link to comment
cmhardwick Posted February 22, 2008 Share Posted February 22, 2008 hawihoney, Seagate Sata drives still have jumpers for 150 and 300, which come jumpered for 150 by default for some reason. Just an FYI Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.