June 21, 201115 yr Summary of my issues with 5.0b7: 1 - Sync errors after drive rebuild (occurs in 4.7 as well) 2 - Adding 3T precleared disk to array not working 3 - "Trust my parity procedure" (initiconfig / mdcmd set invalidslot 99) not working I think number 3 got broken somewhere after 5.04 the moment we changed the way the disk is recognized by the OS .... i remember using it one time before that and it worked .... but after that it doesn't work any more... i tried 2 times in 5.06a....
June 21, 201115 yr using a new flash drive w.o. upgrade - I get just unassigned. if I assign the disks to the proper slots, I am able to start the array. txt now reads: "Start will record all disk information, bring the array on-line, and start Parity-Sync (if parity is present). The array is immediately available, but is unprotected until Parity-Sync completes." i should be "safe" then right?
June 21, 201115 yr I hot added my new 3Tb drive, started a preclear with the beta version of preclear 1.12beta that should accomidate drives larger than 2.2TB. after 2 cycles 66 hours. the drive successfully precleared. The array was running this whole time and all drives spun down. I decided I would reboot the box before adding the new preclear 3TB drive into the array. Once again, facing same problem, I clicked "Stop" array stopped. I clicked "Reboot" server is hung. Tom I have not done a single thing since I ran the reiserfsck checks on all disks as you requested (granted a 66 hr preclear). What do I do to troubleshoot. My goal was it was supposed to reboot and I was going to add the 3TB drive into the array and see if it accepts the drive as cleared already. If you notice the 80GB WD 10k cache drive even though is on a SAS controller seems to be being sent a hpdarm command not the native linux command, dont know if that has something to do with it. Console is showing its stuck at "mdcmd (48): spinup 20" it been like this for an hour already. Update, had to warm reboot the unRAID server. I re-ran all the steps Tom original posted for me here (reiserfsck, guys please remember the step Tom gave me are for a configuration with NO parity drive, dont do it unless thats the case for you). Once again the 80GB WD 10k drive replayed journal, wish I knew what that meant, anyone? Attaching "reiserfsck_log2.txt" to this post (its the last drive in the log). So this is what I decided to do. I shutdown the system successfully after running these steps. I removed this drive as I though it would be a good kick around cache drive, and since I was going to eventually replace it with another drive before going production with my system, its no big deal in my case to remove it !from the equation. I will keep an eye out if another drive (it will be brand new) that i place as a cache drive does the same thing, or in the meantime without one if the next last drive starts to have the same behavior on reboot/shutdowns. Now I booted the system back up, all was well, array was ready to be started. I assigned my 3TB precleared drive to the array it took!: Jun 20 22:34:33 PNTower emhttp: ckmbr: open: No such file or directory Jun 20 22:34:33 PNTower emhttp: writing GPT on disk 10 (sdj) with partition 1 of fset 64 Jun 20 22:34:33 PNTower emhttp: shcmd (86): sgdisk -o -a 64 -n 1:64:0 /dev/sdj 2 >&1 |logger Jun 20 22:34:36 PNTower logger: Creating new GPT entries. Jun 20 22:34:36 PNTower logger: The operation has completed successfully. Jun 20 22:34:36 PNTower kernel: sdj: sdj1 Jun 20 22:34:36 PNTower emhttp: re-reading (sdj) partition table Jun 20 22:34:36 PNTower kernel: sdj: sdj1 Jun 20 22:34:37 PNTower emhttp: Start array... I started the array, it stated just the 3TB drive was unformatted (very good, as expected), I checked off to format, clicked format, 35 seconds later she was done and ready to go. Great job on this Tom!!! and big thanks to Loe L. for the 1.12beta preclear_disk script as well!!! I am attaching a tail log ("add 3TB drive to array.txt") to this post showing from start to finish of me assigning the drive into the array and formating the drive. I will keep testing, crossing my fingers that taking that drive out of the equation helps, as I did not have problems before this version with it and was a cache drive from the beginning. reiserfsck_log2.txt add_3TB_drive_to_array.txt
June 21, 201115 yr ... Do you have a parity disk assigned? Meantioned many times over in this forum here, nope no parity drive. U have a 7k I have a 5k, dont think it would matter, u may have interrupted your preclear thougth, all my drives are on SAS controllers (precleared on one, moved to another before added to array to test both see 3TB's) Do u need me to try anything?
June 21, 201115 yr ... Do you have a parity disk assigned? Meantioned many times over in this forum here, nope no parity drive. U have a 7k I have a 5k, dont think it would matter, u may have interrupted your preclear thougth, all my drives are on SAS controllers (precleared on one, moved to another before added to array to test both see 3TB's) Do u need me to try anything? Ok - just confirming. At first I thought you were able to add a preclered disk to the arry without experiencing the bug I found that resulted in unRAID wanting to clear the disk. But without parity, you weren't testing that. The preclear process is useful for burning in a drive, but you could have used even the old version and it would have served that purpose. unRAID would have still added the GPT and partitioned the disk for you as you added it to the array after the prclear ended, much as it did. The reason for the new preclear is to put the new preclear signature in place, which is supposed to allow a precleared disk to be added to a parity protected array without a lengthy unRAID clearing process. But that does not work in this version.
June 21, 201115 yr I am not sure how u came to that conclusion, as if u don't preclear a drive (which preps the what u call signature) and try to insert it into an array with or without a parity disk in the array it is a lengthy process. My drive was precleared with Joe L. new beta Script, and the mbr was prepped with the flag that is needed for unRaid to detect as if it had cleared it and this removed the length process of unRaid doing it all over again. Then on a new drive it will always format it, 35 seconds for my 3tb drive and I was done. If I am wrong, I appologize , but I don't see this being the case. Joe L. And Tom can confirm.
June 21, 201115 yr Author An update... Summary of my issues with 5.0b7: 1 - Sync errors after drive rebuild (occurs in 4.7 as well) Pretty sure this is an extreme corner-case driver bug (that is to say, it has to be just the exact right case I/O requests, sync requests, and timing for it to happen). I've been working on this for several days now and will soon have it fixed. This may require an update to 4.7 as well (to 4.7.1). More details on this later... 2 - Adding 3T precleared disk to array not working Haven't looked at this yet, but should be easy to figure out. 3 - "Trust my parity procedure" (initiconfig / mdcmd set invalidslot 99) not working Yes this "feature" got broken some betas back, now fixed in next beta. [You might have noticed that when array is Stopped, every time you refresh the webGui the driver is unloaded/loaded... this is to detect hot-swap drives, and what caused this feature to fail (driver being reloaded cleared effect of the mdcmd command).] webGui seems to hang when I upgrade a smaller disk to a really larger disk. This will be fixed in next beta. What's happening here is that the 'mount' command sequence includes a 'resize' function to expand the size of the reiserfs file system on the upgraded disk. The webGui is not actually "hanging" in the sense that something has broken - it's just sitting there waiting for the mount to finish, which for a couple reasons, is taking a really long time. Finally, a new feature: A feature added in next beta is something I call "Cache-only Shares". Mainly this is a change to the 'mover' script. The new 'mover' will first examine all the top-level directories on the cache disk, and for each directory that also exists on the array, it will move the files from the cache disk to the array as usual - but - if a directory on the cache disk is not also present on the array, then the directory contents are not moved. This lets you create some normally named directories on the cache disk which will not get moved (instead of just skipping directories that start with a '.' character). What I use this for is to create a movie metadata share on the cache disk so that only the cache disk is spinning and no pauses in navigating a movie wall via PCH. Other uses would be for other kinds of "temporary" shares where you want fast access and don't care if the underlying device fails.
June 21, 201115 yr what's the easiest way to delete/recreate my shares? since my 4.7 to 5b7 was hosed, i started a fresh install of the beta, and ran a parity sync, and ran the permissions script. I tried deleting the files in /config/shares, which didn't work. I even tried creating a new share, then using that as a "template" to create a .cfg file for the old shares. Basically, I'm getting a weird permissions error. Doesn't seem to happen on new shares, just two of my older shares. help, appreciated. so far my move to the beta has not been a good one.
June 21, 201115 yr Finally, a new feature: A feature added in next beta is something I call "Cache-only Shares". Mainly this is a change to the 'mover' script. The new 'mover' will first examine all the top-level directories on the cache disk, and for each directory that also exists on the array, it will move the files from the cache disk to the array as usual - but - if a directory on the cache disk is not also present on the array, then the directory contents are not moved. This lets you create some normally named directories on the cache disk which will not get moved (instead of just skipping directories that start with a '.' character). What I use this for is to create a movie metadata share on the cache disk so that only the cache disk is spinning and no pauses in navigating a movie wall via PCH. Other uses would be for other kinds of "temporary" shares where you want fast access and don't care if the underlying device fails. That's cool. So would we be able to create User Shares from the cache disk, or must we mount the cache disk as a disk share? You use the symbolic links to your Media on the Cache disk, for PCH/YAMJ, correct? I've only recently been playing with YAMJ living on unRAID, so currently only the YAMJ application is on the cache drive, and my Media/Jukebox lives under my "Media" user share.
June 21, 201115 yr Author Finally, a new feature: A feature added in next beta is something I call "Cache-only Shares". Mainly this is a change to the 'mover' script. The new 'mover' will first examine all the top-level directories on the cache disk, and for each directory that also exists on the array, it will move the files from the cache disk to the array as usual - but - if a directory on the cache disk is not also present on the array, then the directory contents are not moved. This lets you create some normally named directories on the cache disk which will not get moved (instead of just skipping directories that start with a '.' character). What I use this for is to create a movie metadata share on the cache disk so that only the cache disk is spinning and no pauses in navigating a movie wall via PCH. Other uses would be for other kinds of "temporary" shares where you want fast access and don't care if the underlying device fails. That's cool. So would we be able to create User Shares from the cache disk, or must we mount the cache disk as a disk share? You use the symbolic links to your Media on the Cache disk, for PCH/YAMJ, correct? I've only recently been playing with YAMJ living on unRAID, so currently only the YAMJ application is on the cache drive, and my Media/Jukebox lives under my "Media" user share. When you create a share there is an option to make it "cache only", so the top-level directory gets created on the cache disk instead of on one of the array disks. Later you can remove the 'cache-only' option if you want which will cause a directory of that share name to get created on the array. As for YAMJ, what I do is this. I create a cache-only directory called "YAMJ". Then from the unRaid command line I create a symlink to the Movies share inside the YAMJ share: cd /mnt/cache/YAMJ ln -s /mnt/user/Movies (Need the symlink because of PCH limitation of only 1 share mounted at a time.) Next, I have YAMJ and associated mediaInfo application, and custom skin, installed on a PC - not the Server, and I use "YAMJ GUI Config" to set up the batch files. Here is My_YAMJ.cmd: @Echo OFF C: CD "\Program Files (x86)\YAMJ\YAMJ moviejukebox" CALL moviejukebox My_Library.xml -o \\TOWER\YAMJ pause exit and here is My_Library.xml: <!-- Library file generated by Omertron's GUI Config --> <!-- YAMJ GUI Config v0.51 --> <!-- http://mediaplayersite.com/YAMJ_GUI_Config --> <libraries> <library> <path>\\\\TOWER\YAMJ\Movies</path> <playerpath>file:///opt/sybhttpd/localhost.drives/NETWORK_SHARE/TOWER:YAMJ/Movies/</playerpath> <exclude name="sample,tmp/,temp/,RECYCLER/,RECYCLE.BIN/"/> <description>unRAID Server</description> <prebuf></prebuf> <scrapeLibrary>true</scrapeLibrary> </library> </libraries> When My_YAMJ.cmd is invoked on the PC, it scans \\tower\yamj\movies and creates a directory called Jukebox in the YAMJ share (which contains all the metadata) and "index.htm" in the YAMJ share. When you navigate to the YAMJ share on the PCH, the PCH sees the "index.htm" file, and off we go... (of course you have previously defined the YAMJ on the PCH). In the end, if you navigate to the YAMJ share you see this: Jukebox <- a directory Movies <- appears as a directory index.htm
June 21, 201115 yr Right, I understand what you've done. I assume you have it process on your PC because it's faster than your unRAID server? I just wanted to isolate my laptop from doing these tasks, plus it takes much longer to copy over the Jukebox to unRAID from my laptop, than from the cache drive to the Media drive -- but I may try keeping the YAMJ application and Jukebox output on the cache drive, and using the symbolic link to my Media folders.
June 21, 201115 yr I am not sure how u came to that conclusion, as if u don't preclear a drive (which preps the what u call signature) and try to insert it into an array with or without a parity disk in the array it is a lengthy process. My drive was precleared with Joe L. new beta Script, and the mbr was prepped with the flag that is needed for unRaid to detect as if it had cleared it and this removed the length process of unRaid doing it all over again. Then on a new drive it will always format it, 35 seconds for my 3tb drive and I was done. If I am wrong, I appologize , but I don't see this being the case. Joe L. And Tom can confirm. If you look HERE, you'll see that the official documentation explains that the clearing is only to maintain integrity of the parity protected array. If there is no parity, there is no need to clear it, and unRAID is smart enough to NOT clear a disk added to an unprotected array. So you could have precleared or not precleared and results would have been the same. I should add that the preclear script has another use. It gives a disk a workout. This is useful in finding bad disks before you start to use them to store your data. So doing a preclear is a good idea even if you don't need the disk cleared.
June 21, 201115 yr A feature added in next beta is something I call "Cache-only Shares". Excellent idea - many thanks!
June 22, 201115 yr I am not sure how u came to that conclusion, as if u don't preclear a drive (which preps the what u call signature) and try to insert it into an array with or without a parity disk in the array it is a lengthy process. My drive was precleared with Joe L. new beta Script, and the mbr was prepped with the flag that is needed for unRaid to detect as if it had cleared it and this removed the length process of unRaid doing it all over again. Then on a new drive it will always format it, 35 seconds for my 3tb drive and I was done. If I am wrong, I appologize , but I don't see this being the case. Joe L. And Tom can confirm. If you look HERE, you'll see that the official documentation explains that the clearing is only to maintain integrity of the parity protected array. If there is no parity, there is no need to clear it, and unRAID is smart enough to NOT clear a disk added to an unprotected array. So you could have precleared or not precleared and results would have been the same. I should add that the preclear script has another use. It gives a disk a workout. This is useful in finding bad disks before you start to use them to store your data. So doing a preclear is a good idea even if you don't need the disk cleared. Sorry bjp999 I dont see were it states with won't "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" then it will not do this if there is no parity drive? So lets say you are correct, my data drives all have data on them and tomorrow I add a parity drive, will it want to run a clearing phase on all the disks "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" and wipe out all my data on the data disks due to this? or will it just build parity from all the data i have on each drive and NOT run a clearing phase. I think if anyone of us was neighbors and say their HW died (no spare available) but needed to desperately access a bit of data on one drive, they could run over and I could be able to add that drive into my array and fetch it for them right? without it clearing that drive no? I know I dont have more experience than you with this product and learning, thats why I am asking these questions, but I remember when I first started testing (before I knew about preclear_disk) when I added a disk with no parity unRAID cleared it and it took a long time, and thats when I ready about Joe L. script and I thought OH thats another benefit to be able to prep a disk without having to bring down the array and already have the mbr prepped so unRAID would not do it again. What I saw and I added the tail log for all to review, is unRAID saw the prepped mbr (so it didnt need to clear it), detected the 0XFFFF blah blah blah (greater than 2.2TB) so it created the GPT and it was ready to be formatted... thats my understanding of it. I hope Tom and Joe L. can chime in (not to prove anyone wrong or anything like that) but to for us to have the proper understanding. As I stated previously if my understanding is wrong I appologize, I am not by any means challenging you. Cheers
June 22, 201115 yr I think if anyone of us was neighbors and say their HW died (no spare available) but needed to desperately access a bit of data on one drive, they could run over and I could be able to add that drive into my array and fetch it for them right? without it clearing that drive no? Yes. But this has nothing to do with clearing. The drive has been prepared by unRAID and unRAID will recognize the drive as such and its contents will be readable. No clearing is required if there is not parity drive in place. When a parity drive is added parity will be generated based on all the bits on the drives. If a parity drive is in place then a drive must be cleared/zeroed in order to maintain correct parity.
June 22, 201115 yr .... or will it just build parity from all the data i have on each drive and NOT run a clearing phase. Yes. I think if anyone of us was neighbors and say their HW died (no spare available) but needed to desperately access a bit of data on one drive, they could run over and I could be able to add that drive into my array and fetch it for them right? without it clearing that drive no? Why would you want to add the drive to your array? All that is required is to mount the drive (partition?), outside of the array, using the ReiserFS option, then you can access all of the data on that drive (partition?).
June 22, 201115 yr I am not sure how u came to that conclusion, as if u don't preclear a drive (which preps the what u call signature) and try to insert it into an array with or without a parity disk in the array it is a lengthy process. My drive was precleared with Joe L. new beta Script, and the mbr was prepped with the flag that is needed for unRaid to detect as if it had cleared it and this removed the length process of unRaid doing it all over again. Then on a new drive it will always format it, 35 seconds for my 3tb drive and I was done. If I am wrong, I appologize , but I don't see this being the case. Joe L. And Tom can confirm. If you look HERE, you'll see that the official documentation explains that the clearing is only to maintain integrity of the parity protected array. If there is no parity, there is no need to clear it, and unRAID is smart enough to NOT clear a disk added to an unprotected array. So you could have precleared or not precleared and results would have been the same. I should add that the preclear script has another use. It gives a disk a workout. This is useful in finding bad disks before you start to use them to store your data. So doing a preclear is a good idea even if you don't need the disk cleared. Sorry bjp999 I dont see were it states with won't "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" then it will not do this if there is no parity drive? So lets say you are correct, my data drives all have data on them and tomorrow I add a parity drive, will it want to run a clearing phase on all the disks No, it is also smart enough to only clear the drives if there is existing parity protection. Adding a drive to an array with no assigned parity drive will never "clear" the added drive. Conversely, adding a data drive to a parity protected array will always clear the drive unless it has a special pre-cleared signature. "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" and wipe out all my data on the data disks due to this? or will it just build parity from all the data i have on each drive and NOT run a clearing phase.Again, if there is existing parity it will wipe the data from the drive by writing all zeros to it. The only way to not clear an added drive is to set a new disk configuration. As soon as you perform that operation parity is immediately invalidated. Then, since there is no valid parity disk the data disk being added is not cleared. I think if anyone of us was neighbors and say their HW died (no spare available) but needed to desperately access a bit of data on one drive, they could run over and I could be able to add that drive into my array and fetch it for them right? without it clearing that drive no? If you add the drive to your array as a new data drive in a previously unused logical "slot" in the array, it would be zeroed if you already had parity defined. The only way to add it to the "protected" array is to invalidate parity. On most unRAID versions it is the initconfig command. (On the 5.0beta7 version there is also a button on the user-interface that invokes "initconfig") As mentioned, it is very east to mount the drive outside of the protected array and get to its contents. Your neighbor's data is safe unless YOU mis-manage it. I know I dont have more experience than you with this product and learning, thats why I am asking these questions, but I remember when I first started testing (before I knew about preclear_disk) when I added a disk with no parity unRAID cleared it and it took a long time, and thats when I ready about Joe L. script and I thought OH thats another benefit to be able to prep a disk without having to bring down the array and already have the mbr prepped so unRAID would not do it again. What I saw and I added the tail log for all to review, is unRAID saw the prepped mbr (so it didnt need to clear it), detected the 0XFFFF blah blah blah (greater than 2.2TB) so it created the GPT and it was ready to be formatted... thats my understanding of it. I hope Tom and Joe L. can chime in (not to prove anyone wrong or anything like that) but to for us to have the proper understanding. As I stated previously if my understanding is wrong I appologize, I am not by any means challenging you. Cheers The pre-clear recognition is broken for > 2.2TB drives. You are using beta software... what do you think "beta" means? Since you do not have parity disk assigned, it makes no difference at all to you. The pre-clear gives you some assurance the disk is working properly... but was un-needed if you just wanted to eliminate the down-time. The down-time only occurs when adding a non-pre-cleared drive to a parity protected array.
June 22, 201115 yr Sorry bjp999 I dont see were it states with won't "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" then it will not do this if there is no parity drive? The point I was making is it does not HAVE TO clear the disk. And it doesn't. I did this this past weekend with 5.0b7 and there was no clearing. So lets say you are correct, my data drives all have data on them and tomorrow I add a parity drive, will it want to run a clearing phase on all the disks "The clearing phase is necessary to preserve the fault tolerance characteristic of the array" and wipe out all my data on the data disks due to this? or will it just build parity from all the data i have on each drive and NOT run a clearing phase. No, it will build parity based on your data drives. The array will them be parity protected. And if you were to add another data disk to the array, the disk would have to be cleared first (etiher precleared or unRAID will clear it). I think if anyone of us was neighbors and say their HW died (no spare available) but needed to desperately access a bit of data on one drive, they could run over and I could be able to add that drive into my array and fetch it for them right? without it clearing that drive no? You could add it as your cache disk and access it. There are also addons to let you mount it outside the array. But if you went to add it to a parity protected array, unRAID would clear it with binary zeros and its contents would be lost. If you wanted to add a disk to your array that already had data on it, what you would have to do is reinitialize the array, re-add all of the disks, add the new disk, and assign the parity disk. When you start the new array configuration, unRAID would then rebuild parity based on current data on the drives. Then you would again have a parity protected array. I am not sure if unRAID would reinitialize a disk if you tried to add it to a non-parity protected array. I have wondered, but never tried it. I expect it would be reinitialized, but that's a guess. I know I dont have more experience than you with this product and learning, thats why I am asking these questions, but I remember when I first started testing (before I knew about preclear_disk) when I added a disk with no parity unRAID cleared it and it took a long time, and thats when I ready about Joe L. script and I thought OH thats another benefit to be able to prep a disk without having to bring down the array and already have the mbr prepped so unRAID would not do it again. Define a long time. You talking about 6 or 7 hours? I would need more details to understand what happened. What I saw and I added the tail log for all to review, is unRAID saw the prepped mbr (so it didnt need to clear it), detected the 0XFFFF blah blah blah (greater than 2.2TB) so it created the GPT and it was ready to be formatted... thats my understanding of it. I hope Tom and Joe L. can chime in (not to prove anyone wrong or anything like that) but to for us to have the proper understanding. As I stated previously if my understanding is wrong I appologize, I am not by any means challenging you. Cheers There is a difference between the reinitializing a disk (i.e,. creating MBR or GPT) and zeroing every last byte. unRAID WILL reinitialize a new or foreign disk when it is added to the array whether the array is protected or not. Joe L. or Tom are free to chime in. I see Joe L. already responded. I think we said basically the same thing.
June 22, 201115 yr The point I was making is it does not HAVE TO clear the disk. And it doesn't. I did this this past weekend with 5.0b7 and there was no clearing. I hope you can understand why when I saw both you and I were the only one to try (so far) adding a 3TB drive after preclearing with 1.12beta yours failed mine did not, I was thinking like “how do you know mine is not a valid test”, because I was under the impression that unRAID treats additions of hard disks the same whether there was a parity drive assigned or not, as I never ran into any documentation stating it IS different. Which clearly I do understand now thanks to these last posts here. I understand now that when you guys saw that I clearly stated I do NOT have a parity drive assign it automatically flag you to know it did not matter what the end of the preclear did to my drive unRAID will not zero out the disk because there was no parity drive assigned. No, it will build parity based on your data drives. The array will them be parity protected. And if you were to add another data disk to the array, the disk would have to be cleared first (etiher precleared or unRAID will clear it). This is where I knew it would NOT zero them out (but threw it out there to make sure I am on the right page on this) but what is throwing me off I guess because I feel since no zeroing was forced (due to no parity drive originally) just a format, why are the drives trusted once a parity drive is assigned, it just builds parity based on hard drives it does not know if they have EVER have been zero’ed out. AND I remembered a statement Tom made earlier in this announcement “ BTW where the code base is moving is to have parity generated over the entire physical address space of data disks, not just partition 1. This will let me be able to reconstruct an entire data disk contents, and thus let you plug in disks partitioned and formatted in "foreign" systems. For example, I want to be able to take a large GPT partitioned hard drive from a Windows 7 system with NTFS file system and incorporate into the array as-is. Then, on disk-by-disk basis user can specify which partition of each disk to "export", default being partition 1 (but often, esp. with GPT partitioned drives, partition 1 is not the main data storage partition). Then if that disk dies, unRaid can reproduce the entire disk, including partitions not being used.” So in my mind NO hard disk would ever be zero’ed out right??? In order to accomplish this. You could add it as your cache disk and access it. There are also addons to let you mount it outside the array. But if you went to add it to a parity protected array, unRAID would clear it with binary zeros and its contents would be lost. If you wanted to add a disk to your array that already had data on it, what you would have to do is reinitialize the array, re-add all of the disks, add the new disk, and assign the parity disk. When you start the new array configuration, unRAID would then rebuild parity based on current data on the drives. Then you would again have a parity protected array. I am not sure if unRAID would reinitialize a disk if you tried to add it to a non-parity protected array. I have wondered, but never tried it. I expect it would be reinitialized, but that's a guess. I never new one could just mount a drive and access that data (see I don’t know as much as most of u assumed I did, or should have LOL), I wont waste anymore precious space in this post, I will search the forums, learn about these options and practice (test) doing this. As you stated reinitialize would be an option but you would lose your parity and that’s not a good thing as I would be vulnerable until it completed a new parity build, so the options given here are the way to go and I will research them. Define a long time. You talking about 6 or 7 hours? I would need more details to understand what happened. No it was more like 1-2 hours but scratch that altogether as it seems so long ago for me and I did not know anything at all about unRAID back then, it could have been just a format or something else as I didn’t even know about zeroing/clearing… There is a difference between the reinitializing a disk (i.e,. creating MBR or GPT) and zeroing every last byte. unRAID WILL reinitialize a new or foreign disk when it is added to the array whether the array is protected or not. Clear on this. I really appreciate when Joe L. chimes in because he has a way to phrase things and uses keywords that just auto click for me. I did not want to reply to both posts but appreciate both. Thank you all very much. Sorry for my miss understandings and taking the time out to clarify these key points for my understanding. Very much look forward to Tom’s next beta with all the fix’s mentioned in his replies on this announcement. I am sure he appreciates all the head’s up on things missed/dropped/bugs etc. that you guys quickly posted for him to pick up on quickly for the benefit of all and the product itself. Last thing I can share at the moment is so far after removing that old WD drive that keep getting that journal issues, it has been rebooting fine for me in between tests. The only thing I have noticed and confirmed for the third time is again I introduced the new 3TB drive it would not auto spin down. This was happening to all the disks as bjp999 stated in the most detail the disk file is not writing a “-1” until you actually go to the disk settings and just click apply. I am not running any disk groups, for testing purposes I select “15 mins” and clicked apply, this should have been set to all current disks and any future disks that are introduced. But it is not working.
June 22, 201115 yr ... Sounds like you have a much better understanding of unRAID now than when you started with beta7! Understanding all of the rules is very important when something goes wrong and you are trying to recover. ...why are the drives trusted once a parity drive is assigned, it just builds parity based on hard drives it does not know if they have EVER have been zero’ed out. AND I remembered a statement Tom made earlier in this announcement “ BTW where the code base is moving is to have parity generated over the entire physical address space of data disks, not just partition 1. This will let me be able to reconstruct an entire data disk contents, and thus let you plug in disks partitioned and formatted in "foreign" systems. For example, I want to be able to take a large GPT partitioned hard drive from a Windows 7 system with NTFS file system and incorporate into the array as-is. Then, on disk-by-disk basis user can specify which partition of each disk to "export", default being partition 1 (but often, esp. with GPT partitioned drives, partition 1 is not the main data storage partition). Then if that disk dies, unRaid can reproduce the entire disk, including partitions not being used.” So in my mind NO hard disk would ever be zero’ed out right??? In order to accomplish this. Each bit on the parity disk is set to either 1 or 0, so that the sum of all of the corresponding bits on all of the array disks is even. If you add a disk of all zeros to an array, you do not disturb parity, because an even number + 0 is still an even number. That is why the disk is zeroed before adding to a parity protected array - it makes it quick and easy. But if the disk was full of random data, you can rebuild parity and then the corresponding bits all add up to even numbers. So there is nothing special about the zeroing of the disk, other than the fact such a disk can be added to an existing array without rebuilding parity. When unRAID rebuilds a disk, it sums the corresponding bits on each remaining drive (all but the one being rebuilt). If the sum is even, the rebuilt disk needs a zero bit there. If the sum is odd, the rebuilt disk needs a one bit there. In this way the entire disk is reconstructed to be, bit for bit, what the old one was. We'll have to learn more about Tom's idea to parity protect the entire disk and not just the partition. But I don't think this will have any impact on the need to zero a disk to quickly add it to a protected array.
Archived
This topic is now archived and is closed to further replies.