bblue

Your Norco looks like it has a lot of commonality with my Chenbro RM41416b 4U case. How many active drives are you running in there now? I ask because I'm wondering how effective your cooling solution actually is. I used 5 80mm Noctual fans across that center divider because there was already hardware and plug-in shells for each fan. The fans it came with were much higher RPM and quite noisy. With 13 drives (1.5T and 2T) installed, 11 in the canisters and two (cache and parity) in the top positions, I'm finding that during a parity calc (any event that keeps all drives running for extended periods of time) the temperature of some of the drives gets into the low 50's C. Normally they run in the high 30's. It seems that three 120mm fans may marginally produce more air flow than five 80mm, depeding on their ratings So I'm wondering how many active drives you are running and how the drive temperatures maintain under an all-running scenario? --Bill

I meant to, sorry. Yes, very interesting. It was distressing to learn how much storage you *actually* had... :-) What was your basis for calculating the cache size? --Bill

Joe, Many thanks! Works like a champ. --Bill

I'm not finding that ssmtp package on the Slackware site anywhere from build 11 to 13. The only reference is to a libcddb that has a simple smtp backend. Any idea where to locate it? That version of mailx is there, however. --Bill

Very clever! That's what I first thought, but the 'signature' bytes threw me away from that notion for some reason. So what makes it show as uncleared after an unRAID format is that the filesystem type (83) is now defined, which breaks the tests. That seems like a lot of redundancy since bashmail can already talk to a local smtp server, it just doesn't have the right front end, plus relies on unraid-notify's conf file for default smtp info. Seems like it wouldn't be too difficult to make a new front end for bashmail and add a .conf file and reader just for it for a quick and simple delivery backend. I don't have a netcat, what's that about? Yeah... I guess that would be easier (less time involved) and more universal overall. I was hesitant to be loading packages that are far more than they need to be to get the job done. But then there's the time issue (mine)... --Bill

Thanks for all the info, Joe! When you say that unRAID "completely recreates it" (the MBR) are you talking about just when you put in an uncleared disk, or could that also happen during the formatting phase (pre or post)? I notice in your script you don't ever clear it, but just write to the appropriate places to size the filesystem, partition #1 and pointers. Is it just unnecessary as other bytes don't matter, or what? It would be ok if unRAID clears all of 1-62 sectors with any flag bytes in it, because I think I'd only want that extra flag there if preclear_disk actually did the work. Bytes 510 and 511 of MBR are preclear_disk.sh's flag, not really a function of MBR bytes, correct? That's the only place in your code I can see a signature write, even though it's labeled as signing the MBR. Are the 0x55AA values arbitrary or significant in some way (other than being mirrored alternating bit-sets)? On a different subject, which mail binary is supposed to be used with this script? It looks like it's the old standard /bin/mail by the arguments, but I don't find anything like that on the distribution site. Bashmail is a totally different animal although I found some apparently incorrect references on the Forum that if you had unraid_notify running you could use its back-end for mail. Oh man. It sounds like that would have been both exciting and incredibly frustrating at the same time! That was back when, during the early transition from tubes? Sounds like you were quite 'inside' at the hardware level. What insight that would give you! And who even remembers CP/M for that matter. It was kind of cool in its own way, though. At least compared to what else was available at the time. The whole OS could relocate anywhere in memory to suit the hardware or other requirements. I had (among other things) one of those 'portable' Kaypro 10 CP/M computers, maybe around 1982-1983. Very cool at the time with that huge 10M drive... --Bill

Actually, no. But cumulatively on one boot, yes. I pre-cleared two 1.5T, then a 500G. For some reason it didn't show problems until the end of the 500G drive. I have 4G RAM installed (2.9G available to Linux), but I failed to notice how much was freed each time. Actually, yes... Your 4Gig of RAM (2.9Gig available for programs/buffers) is most certainly less than even the smallest of your disk drives (500gig) Uh, yes, you're right. I had some kind of a 'moment' back there... --Bill

That's really good information! Writing a historical pre-clear byte there is problematic, though. Let's say you had to quickly install a replacement drive (new, never written to) and didn't have time to pre-clear it. So it's installed and being restored by the system via the parity drive. Now, since the pre-clear byte from the original (now failed) disk is part of the restore, you now have an indicator that says this new drive was pre-cleared, when in fact it wasn't. You'd need to manually clear that byte if it was to be accurate. You said earlier "If you accidentally made a drive look like it was pre-cleared, and it had anything other than zeros on the remainder of the disk, you would completely break all the parity calculations and be unable to restore from a disk failure unless you did a full parity check and fixed all the parity ... prior to the failure...". We'd have to go with the presumption that only preclear_disk.sh would set that byte appropriately. But I'm curious, here you're talking about anything that existed within the ReiserFS space? The first 62 sectors before the FS starts are cleared when sector 0 is rewritten? So presumably nothing written anywhere in there would count in parity, or would be preserved when sector 0 was updated, correct? It doesn't seem that the formatting (creating the ReiserFS) takes enough time to clear the entire remaining partition, it just sets up the directory and FS structures. So any drive that has been in use in another system and is not pre-cleared could have blocks with garbage in them in the filesystem space. But they wouldn't be 'seen' because there would be no pointers to it, but that same garbage *would* be seen by parity calcs. If that's true, then if a drive had been previously used in another system and NOT pre-cleared, you absolutely could not depend on its restorability and there would be no indication of a problem. ? Or is the parity calc aware of allocated FS blocks and only takes the data within them for the calcs? Oh yes... I go back a bit more than that... Do you remember 8" floppy disks? (or loading programs via punched paper tape on a teletype machine?) Talk about slow... No, I missed all the fun with punched tape and all that, but I was running a bulletin board (BBS) that I wrote in 1981 that used a 5.25 floppy boot drive (about 90KB) and two dual Persci 8" data drives with something like 220KB per disk. What capacity! Shortly after that I bought my first hard drive, a big external cabinet with a 10MB drive in it. That and the adaptor cost just shy of $2k. I still have a bunch of those 8" disks around here someplace. --Bill

Actually, no. But cumulatively on one boot, yes. I pre-cleared two 1.5T, then a 500G. For some reason it didn't show problems until the end of the 500G drive. I have 4G RAM installed (2.9G available to Linux), but I failed to notice how much was freed each time. Gotcha. Ok, thanks. I'll try that. --Bill

Another issue, if I may. Preclear_disk.sh seems to suck up a lot of memory -- sometimes. Specifically, dd is the culprit. When free memory (shown by top) gets down to around 400M, access from the outside world via ethernet (the unRAID manager and unMenu) is all but dead. I don't know if you could still serve video or audio at that point. On the console everything continues to be peppy, despite the slowly increasing load average. As soon as preclear is done, free memory shoots back to normal and remote services return. The behavior is the same with 4GB RAM (2.98G usable) as with 2GB RAM except that it just takes longer to happen. But sometimes it doesn't happen at all, everything works normally through the entire pre-clear. It seems to have something to do with how long the machine has been up, and/or how many previous pre-clears have been done. I'm running 4.4.2. Any thoughts on this behavior? --Bill

Until the array is started, yes. There's no indication of what is new and cleared. But as soon as you start the array, the drives that were pre-cleared and newly installed show up as needing formatting. At least that how it was on the last two I just did with the 9.6 version of preclear_disk.sh. Why would it matter if there was a parity drive or not? Don't think I understand this. If the drive had something extra written to say, sector 2, and that was consistent throughout the life of the drive in that array, the parity check would always be consistent, wouldn't it? Why would you not be able to restore? Oh yes. I can't tell you the number of times I've had to enter the old drive parms into a BIOS manually, in the (not so) 'good old days'. But we were in heaven then with the new 386-25 or 33Mhz motherboard and our 50-100MB full size drives... Screamers, you know. --Bill

Hmmm. It seems obvious that I must have done something wrong in the sequence, but I'm quite sure I didn't format any drives. In fact I recall being surprised that the supposedly pre-cleared ones didn't show up needing to be formatted, since with all zeros just written there can't be a filesystem structure there. I'm ready to believe I could have gotten confused over the two 1.5T drives, but the 750G is an oddball size in this array and I know I did not format it - ever. Is there any mode in unRAID that upon seeing a pre-cleared drive will put a filesystem on without requesting you to format it? Puzzling. I wonder, is there a byte in that first sector (or somewhere) where a pre-cleared flag could be placed and remain after the filesystem is added? It seems like an indication that a given drive, now in operation, was ever pre-cleared. It might be useful to know later, if you popped in a replacement drive in a hurry without time for the lengthy pre-clear process, so you could pre-clear it later when files could be moved around and there was more time available. Thanks for the geometry lesson! I knew things had changed considerably in the newer and larger drives but never had really paid much attention to it as long as they just worked! --Bill

No, no fdisk. I put them into their current device order, assigned them to their RAID role (disk1, disk3 and disk4 respectively) and started unRAID without a parity drive. I did not format any drive or do any writing. Shouldn't they have come up as unformatted in the Main menu? None of them did. Is the act of formatting in unRAID the only way the fs would get created (setting offset 83 to 83)? I just finished pre-clearing another drive and this one does show pre-cleared with the -t option, and fdisk -l shows an ID of 0, and System=empty so as you said it would appear that preclear_disk.sh is not the culprit. fdisk does indicate the error about partition 1 not ending on a cylinder boundary, though. Is that normal? --Bill

Thanks Joe, here it is: Disk /dev/sde: 1500.3 GB, 1500301910016 bytes 1 heads, 63 sectors/track, 46512336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sde1 2 46512336 1465138552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 7af1 aea8 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 Disk /dev/sdg: 1500.3 GB, 1500301910016 bytes 1 heads, 63 sectors/track, 46512336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdg1 2 46512336 1465138552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 7af1 aea8 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 Disk /dev/sdb: 750.1 GB, 750156374016 bytes 1 heads, 63 sectors/track, 23256336 cylinders Units = cylinders of 63 * 512 = 32256 bytes Disk identifier: 0x00000000 Device Boot Start End Blocks Id System /dev/sdb1 2 23256336 732574552+ 83 Linux Partition 1 does not end on cylinder boundary. 0000000 0000 0000 0000 0000 0000 0000 0000 0000 * 0000448 0000 0083 0000 003f 0000 66b1 5754 0000 0000464 0000 0000 0000 0000 0000 0000 0000 0000 * 0000496 0000 0000 0000 0000 0000 0000 0000 aa55 0000512 --Bill

I've just finished pre-clearing four drives; two seagate 1.5T's that were new, a Seagate 750G a few months old but hardly used and an older WD 500G drive. Each and every one went through all ten steps, especially #7 where the pre-cleared flag is set on the drive, and all passed with no errors. Yes, when I review each of them with preclear_disk.sh -t, in every case it says the drive has NOT been pre-cleared! This is with 0.9.3 of the script, and each were done separately (no concurrents). There is also the statement that Partition 1 does not end on a cylinder boundary. Any idea what might be going on? Each of the processes ended with the diff display. I do have the missing library for smartctl installed. --Bill