October 8, 201015 yr I am doing a fairly major array update - swapping out some 750G and swapping in some new 2T, reducing overall the number of drives in my array while increasing the capacity and leaving room to grow. I have already copied all of the data from the 750Gs to larger array disks, The new 2T disks are precleared. So the puzzle is this - what is the quickest way to accomplish. Obviously to do it quickly you need to minimize lengthy operations like parity builds, parity checks, and drive rebuilds. And one more wrinkle - the server has got to maintain the ability to recover from a drive failure throughout the process, until parity protection is in place on the array. Hero members - please keep the answer to yourself until we give some of the others in the forum a chance to respond.
October 8, 201015 yr Is this some kind of test that you already know the answer to ... or are you really looking for the best way to do this? I'm VERY new to unRAID, so I'm sure I don't have the 'proper' solution. However, why not just install the new drives and then rebuild parity from scratch. You say you need parity protection, but why? If you just copied the data from a bunch of smaller disks, don't you still have a copy of that data on the smaller disks to protect you from a possible failure? EDIT: Ok, I see you want to avoid parity builds. I'm not heroic enough to know if this is even possible. I don't see how you could do this without running at least one parity build. But then again, I just started playing with unRAID.
October 8, 201015 yr Okay, I'll bite. -Write all zeros to any drive you wish to remove from the array -Shutdown -Remove the drives that were zeroed from the machine -Add any precleared drives to the machine -Boot -Unassign removed drives from the devices page -Assign added drives in the devices page -Invoke "trust my parity" procedure -Run parity check just for fun Most of this is laid out very nicely in this post (you may recognize the author): http://lime-technology.com/forum/index.php?topic=2591.msg20919#msg20919 Why do I feel like I'm being set up.
October 8, 201015 yr Author Is this some kind of test that you already know the answer to ... or are you really looking for the best way to do this? I'm VERY new to unRAID, so I'm sure I don't have the 'proper' solution. However, why not just install the new drives and then rebuild parity from scratch. You say you need parity protection, but why? If you just copied the data from a bunch of smaller disks, don't you still have a copy of that data on the smaller disks to protect you from a possible failure? EDIT: Ok, I see you want to avoid parity builds. I'm not heroic enough to know if this is even possible. I don't see how you could do this without running at least one parity build. But then again, I just started playing with unRAID. Your method would definitely work. And is the way that is documented. The problem is that your array is exposed to a drive failure during the new parity build. That risk is what I am trying to minimize.
October 8, 201015 yr Author Okay, I'll bite. -Write all zeros to any drive you wish to remove from the array -Shutdown -Remove the drives that were zeroed from the machine -Add any new precleared drives to the array -Add any precleared drives to the machine -Boot -Unassign removed drives from the devices page -Assign added drives in the devices page -Invoke "trust my parity" procedure -Run parity check just for fun Most of this is laid out very nicely in this post (you may recognize the author): http://lime-technology.com/forum/index.php?topic=2591.msg20919#msg20919 Why do I feel like I'm being set up. Outstanding! This is a very good method and accomplishes the objectives. Not the one I chose, but in some ways this method is better. Once other methods is discovered, we can debate the pros and cons of each. Not a setup at all. Just trying to get people thinking a little outside of the box to best utilize unRAID to protect their data. Other ideas?
October 8, 201015 yr Okay here is another idea. -Stop array -Remove the drives that are no longer needed -Remove the parity drive -Put in a new drive for parity -Put in new precleared drives -Start array and calculate parity Any failures along the way can be recovered by replacing the old parity drive and the old 750s. If you wish to use your old parity drive, it will require a swap and a promise not to write to the array while parity is being calculated.
October 8, 201015 yr Author Okay here is another idea. -Stop array -Remove the drives that are no longer needed -Remove the parity drive -Put in a new drive for parity -Put in new precleared drives -Start array and calculate parity Any failures along the way can be recovered by replacing the old parity drive and the old 750s. Can you elaborate on steps to recover from a disk failure during the parity build? If you wish to use your old parity drive, it will require a swap and a promise not to write to the array while parity is being calculated. I DO want my current parity disk to be in the array in the end, either as the parity disk or as a data disk. Can you elaborate on how you are suggesting this would happen?
October 8, 201015 yr bjp999, to use your method and maintain the parity protection, you have to copy the "config" folder from the usb drive before the "initconfig" procedure, then you can add your current parity drive to the array "NOT FORMATTING" it yet, and during all process of new parity reconstruction you must keep your array as read-only killing all process like torrent/usenet and setting all your shares as read-only. When your parity construction is over and verified, you can format your old parity drive and make your array writable again. I suggest keep all drives unmounted since the "initconfig" until everything worked out successfully. If you have any failure during the parity calculation, you can shutdown your server and revert your "config" files to those backed up, and boot up the server with the old array configuration and parity.
October 8, 201015 yr Can you elaborate on steps to recover from a disk failure during the parity build? Okay, I see my mistake. -Stop array -Backup the array config -Remove the drives that are no longer needed -Remove the parity drive -Put in a new drive for parity -Put in new precleared drives -Start array and calculate parity Now a drive fails. -Stop array -Remove new parity drive -Install old parity drive -Remove new precleared drives -Install old 750 drives -Restore config backup -Start -Rebuild failed drive I DO want my current parity disk to be in the array in the end, either as the parity disk or as a data disk. Can you elaborate on how you are suggesting this would happen? Old parity --> New data disk -Preclear old parity -Add to array Old parity --> New parity -Stop array -Backup the array config -Remove the parity drive (that we just built) -Put in a parity drive (original parity disk) -Rebuild parity Should a disk fail during this we can do what we did before -Stop array -Remove new parity drive -Install old parity drive -Restore config backup -Start -Rebuild failed drive
October 8, 201015 yr Fun thread! However, I feel that we need more information. Is the current parity drive already a 2 TB (and thus staying in place)? Or is the 'old' array all 750GB drives, all of which will be replaced with 2 TB drives? It is unclear to me whether or not we are upgrading the parity drive or just the data drives (or both).
October 8, 201015 yr Okay, I'll bite. -Write all zeros to any drive you wish to remove from the array -Shutdown -Remove the drives that were zeroed from the machine -Add any precleared drives to the machine -Boot -Unassign removed drives from the devices page -Assign added drives in the devices page -Invoke "trust my parity" procedure -Run parity check just for fun Most of this is laid out very nicely in this post (you may recognize the author): http://lime-technology.com/forum/index.php?topic=2591.msg20919#msg20919 Why do I feel like I'm being set up. Would this be a valid route? bjp stated that he was replacing smaller drives with fewer larger drives. If that's the case, then there must have been some data consolidation (i.e. data from two different drives written onto a single new drive). Maybe there was no data consolidation, but if there was, I don't think 'trust my parity' would work.
October 8, 201015 yr Fun thread! However, I feel that we need more information. Is the current parity drive already a 2 TB (and thus staying in place)? Or is the 'old' array all 750GB drives, all of which will be replaced with 2 TB drives? It is unclear to me whether or not we are upgrading the parity drive or just the data drives (or both). I DO want my current parity disk to be in the array in the end, either as the parity disk or as a data disk. Can you elaborate on how you are suggesting this would happen? I'm assuming the current parity drive is smaller than 2TB. If the parity drive has 2TB, I see no need to do any of these steps, just "zero" your current drives, replace them and do the trust my parity trick.
October 8, 201015 yr If you are removing two identically sized 750Gig drives and have already copied their contents elsewhere, you can try this trick. Let's assume they are currently assigned as disk1 and disk2 in the array. You can clone disk1 as disk2 by using dd if=/dev/md1 of=/dev/md2 bs=4k Then, once cloned, you can Stop the array, un-assign both, and then type initconfig /root/mdcmd set invalidslot 99 When you re-start, the parity will still be good (you removed an even number of bits since the two drives were clones of each other) This keeps parity AND satisfies the initial criteria of minimal time ("quickest way to accomplish") since you only copied 750gig from the one drive to the other in the cloning process. You can do this with any "pair" of drives you are removing, as long as they are the same size. Joe L.
October 8, 201015 yr OK, Joe, just want to make sure I understand your method. I understand that cloning the drives will allow you to remove both without breaking parity (same as zeroing both drives, just faster). However, what does the 'invalid slot' command do? As I understand it, it forces unRAID to 'red ball' a drive, allowing it to be reconstructed from parity. So, in essence, are you making unRAID think that instead of two 750 GB drives, the red balled drive is actually a 1.5 TB drive? Then you replace it with a 2 TB and unRAID reconstructs the contents of the original 750 GB drive onto the 2 TB? I have a feeling that is wrong. Also, doesn't 'initconfig' throw out any good parity data?
October 8, 201015 yr OK, here's my tongue-in-cheek solution: You emphasized speed, but placed no cost limits on the puzzle. Therefore, I propose that you build a second server or expand the first server so that it is large enough to hold all the 750 GB drives and the 2 TB drives at the same time. Format all the 2 TB drives manually (either with mkreiserfs or by using the cache drive slot). Mount all the 2 TB drives except for one outside the array with SNAP or similar. Copy over all the data. Now remove all the 750 GB drives and run 'initconfig'. Next, assign a 2 TB drive as parity and all the others as data drives. Finally, run a parity check. OK, that's kind of a joke solution. It would work, but it isn't cost effective. However, even if you didn't have unlimited funds, you could still make this method work as long as you had at least one free slot in the 750 GB server. You would just have to format and copy data to each 2 TB drive one at a time. The key is to mount them outside the array so that there's no slowdown from parity.
October 8, 201015 yr initconfig /root/mdcmd set invalidslot 99 is trust my parity http://www.lime-technology.com/wiki/index.php?title=Make_unRAID_Trust_the_Parity_Drive,_Avoid_Rebuilding_Parity_Unnecessarily
October 8, 201015 yr OK, Joe, just want to make sure I understand your method. I understand that cloning the drives will allow you to remove both without breaking parity (same as zeroing both drives, just faster). However, what does the 'invalid slot' command do? As I understand it, it forces unRAID to 'red ball' a drive, allowing it to be reconstructed from parity. So, in essence, are you making unRAID think that instead of two 750 GB drives, the red balled drive is actually a 1.5 TB drive? Then you replace it with a 2 TB and unRAID reconstructs the contents of the original 750 GB drive onto the 2 TB? I have a feeling that is wrong. Also, doesn't 'initconfig' throw out any good parity data? The ONLY way to reduce the number of disks in an array is to establish a new disk configuration. You must issue an "initconfig" command (or press "restore" on the older releases of unRAID... they do the exact same thing) That command immediately sets a new disk configuration based on the currently assigned drives and invalidates the drive in slot 0. (the parity drive) so it will be forced to be completely re-calculated when you next start the array. The "/root/mdcmd set invalidslot 99" command then overrides that and sets all disks as valid and disk 99 (a non-existent disk) as invalid. It allows you to start the array with the current (new) configuration with the parity as valid. An added benefit is you really only need to copy 750gig of data as you clone the one drive to the other. The technique will only work when removing a pair of identically sized drives, but it does minimize the time needed as long as their contents is already been moved. It does keep parity protection throughout the process. Joe L.
October 8, 201015 yr Great, thanks for the explanation. So initconfig just invalidates the drive slot, it doesn't actually touch the parity data? I didn't know that. Makes sense, I guess.
October 8, 201015 yr Great, thanks for the explanation. So initconfig just invalidates the drive slot, it doesn't actually touch the parity data? I didn't know that. Makes sense, I guess. Initconfig ALSO re-names the config/super.dat file to config/super.old (it does not just invalidate parity) This forces the "md" driver to save a new configuration when you next start it. The "md" driver will write a new config/super.dat file to replace the missing one, exactly as if you were first configuring the array when you initially set up your server. Joe L.
October 8, 201015 yr Author A lot of good ideas! I never thought about the cloning disk concept. Proabably would have been quicker than the method I chose. But the reason I don't like the zeroing or cloning is that when I remove a disk from my array I like to keep it as a backup for a period of time. If you are wiping out the data you lose that. Guess it depends on whether you want to immediately repurpose the drive. The method that I used is what gfjardim originally suggested. The key point is that you can take the old parity disk, leave it in the array, and use a new disk for parity. unRAID can compute parity with unformatted disks in the array (and the parity will appear as an unformatted disk) and will not update the disk in the process. With this setup, you can recover from a disk failure while rebuilding parity. Although unlikely that you’d have a disk failure after running a parity check shortly beforehand, it is still possible and the impact could be huge. Low risk, high impact. Worth taking precautions? The only major thing missing from all of the responses was to start out by running a parity check. This is needed to ensure that all of the drives are working and that parity is correct to begin. The following are the steps laid out for anyone referencing: 1 – Perform parity check – do not proceed unless parity check is good with no sync errors 2 – Stop the array and make a copy of the config folder on the flash drive (copy should be when array is stopped!). This can be done over the net using the “flash” share. 3 – If you are automatically running addins that result in updates to disks in the array (e.g., torrents), disable those addins so they don’t run. Suggest rebooting to verify that all the addins are disabled. 4 – Make note of the disk slots that each disk is assigned to. For example, take a screenshot of the unRAID GUI and print it. 5 – Power down the server, remove the old disks, add the new disks. Note that new disks do NOT have to be precleared. For example, you could add some disk that was previously in the array that you want to reinsert. Or you could add a disk being transferred from one array to another. (using the zeroing or cloning method, all new disks would need to be precleared) 6 – Power up the server. Watch the BIOS screens to make sure the disks are being recognized. 7 – Array should not start. (If it does you must have forgotten to remove your disks). Go to the devices tab and assign one of the new large drives to parity. Then assign the remainder of the drives, including old parity, to the disk slots. This is an opportunity to do penalty free, wholesale rearranging. Some have suggested that parity check speeds are improved by not having consecutive disks on the same controller. So you might like to have parity on the MB, disk1 on controller1, disk2 on controller2, disk3 on MB, etc. etc. YMMV 8 – Go back to main tab – drive list should look a mess if you’ve done lots of rearranging. Press the restore button if on an older version of unRAID, or go to a telenet session and run the initconfig command. 9 – (Refresh the screen) All drives should now have blue icons. TRIPLE CHECK all of your disk assignments. Make sure you’re not missing any disks. Do not press the format button (I don’t think it is presented until you start the array – but if it is do not press it). 10 – Start the array. Parity build will begin. DO NOT PRESS THE FORMAT BUTTON. Remember this is a parity build not a parity check so speeds will be slower. Do not do any updates to the array. NONE. Playing movies probably wouldn’t hurt unless the array is updating last access times. I would not use the array at all while the parity build occurs. Maybe Joe L. will chime in on safety of doing read only operations during this process. 11 – In the event that you have a cabling or other problem while building parity, you can stop the parity check, bring down the array, power down, fix the problem, and start again. 12 – If there is a true disk failure, the recovery gets a little involved. I have laid out the steps below. 13 – After parity builds, run a parity check immediately. Make sure it runs to completion and you have 0 sync errors. 14 – At this point the new array configuration is parity protected. 15 - Once satisfied that you will not ever need to go back to the old array configuration, press the format button to format the old parity disk and any new disks added to the array. Shouldn’t take more than 5 minutes. But note that the disks can stay in the array unformatted for an extended period if you choose. 16 – Fix your user share configurations, reestablish any addins that run on boot, test everything out. 17 – DONE Recovery steps is a disk fails during the parity build: 1 – Make sure the disk really failed. Cabling problems (data or power) make up a large percentage of perceived drive failures. Intermittent cabling issues are hard to detect and easy to misinterpret as a drive failure. 2 – Power down the server, and put all of the 750G drives back in the server, removing the failed disk and the newly added drives, EXCEPT the one you plan to use to replace the failed disk. 3 – Take the flash disk to another computer and restore the backup copy of the config folder (taken in step 2 of the main procedure). Put the flash disk back in the unRAID server. 4 – Reboot, watch the BIOS screens to ensure all drives are recognized (except the one that failed obviously). 5 – On the tower GUI, go to the devices page and assign your disks to the correct disk slots based on your screenshot taken in step 4 of the main procedure 6 – Assign the new disk to the slot containing the failed disk (sometimes this happens automatically) 7 – Back on the main tab, all disks should be green except the new one in the old slot should be blue. 8 – There should be some unRAID prompts that it is prepared to rebuild the disk. There is a checkbox under the start button you will have to check and then the start button will be enabled. 9 – Press the start button – the disk should rebuild. 10 – When it is completed, you should run a parity check 11 – Done – ready to try original procedure again
October 8, 201015 yr 7 – Array should not start. (If it does you must have forgotten to remove your disks). Go to the devices tab and assign one of the new large drives to parity. Then assign the remainder of the drives, including old parity, to the disk slots. This is an opportunity to do penalty free, wholesale rearranging. Some have suggested that parity check speeds are improved by not having consecutive disks on the same controller. So you might like to have parity on the MB, disk1 on controller1, disk2 on controller2, disk3 on MB, etc. etc. YMMV I believe that this 'round robin' style of cabling only mattered when most of us were using many drives on the PCI bus. However, now that most of us use fast PCIe expansion cards, I don't think that it matters any more. 10 – Start the array. Parity build will begin. DO NOT PRESS THE FORMAT BUTTON. Remember this is a parity build not a parity check so speeds will be slower. Do not do any updates to the array. NONE. Playing movies probably wouldn’t hurt unless the array is updating last access times. I would not use the array at all while the parity build occurs. Maybe Joe L. will chime in on safety of doing read only operations during this process. As I understand it, it doesn't matter at all if you read or write to the array during a parity build/check/rebuild. Say you started writing a file during a parity build/check/rebuild. The build/check/rebuild will simply pause while the new parity info is calculated for the file being written. Once that is complete, the parity build/check/rebuild will continue. Reading shouldn't make the build/check/rebuild even pause unless perhaps you are reading from the slowest drive in your array. So the worst that will happen is that you slow down the build/check/rebuild by a few minutes or a few hours, but it shouldn't actually hurt anything. PLEASE CORRECT ME IF THIS IS WRONG. Also, I would imagine that if you use a cache drive you could read and write to that drive all you want during a build/check/rebuild, since it is outside the array. It might be prudent to disable the mover script during that time (or set the it to run months in the future, or something like that).
October 8, 201015 yr Author Writing to the array would interfere with being able to rebuild a failed disk. Since the new parity but not the old parity would be updated. Reading shouldn't hurt anything, except that depending on how the disk is mounted, the disk can update last access date/time fields. Even these trivial updates would affect the ability to rebuild a failed disk.
Archived
This topic is now archived and is closed to further replies.