February 11, 201115 yr Hi, I am systematically replacing (long story) all the drives in my array. I had 5 x 1GB WD Caviar Black (WD1001FALS) in the array ie 1 Parity + 4 data disks. I am replacing them with 1GB WD Caviar Black (WD1002FAEX). I have replaced one of the failing data drives and it finished in a reasonable time frame. I decided to do the parity drive next. Here is where Im a confused as it is taking a loooong time to synch up. As I look at it now its done about 56% after about 20hrs and stating it needs another 1100 or so minutes to finish. Its average is sitting at around 5MB/sec (when it started it hovered around 12MB/sec). The config is - Supermicro X8SIL-F - 4 GB RAM - Intel Xeon X3440 - 3 x Supermicro SASLP-MV8 PCIe cards - Norco 4220 case - WD Cav Green 500GB as cache drive (running off the MB) - All but the cache drive are running off the 3 MV8 PCIe cards - unRaid Pro - All the drives in the array look like they are connected at Sata2 speeds. 1. Is this about right from a speed POV? 2. Anything I can check, or do I have to let it run its course for now? Thanks
February 11, 201115 yr Author Thanks for the reply Joe. Are you saying yes to point one ie its processing at a normalish speed? On a separate note there is nothing in the syslog from what I can see other than nntp, mover, etc. Here is snippet of the last few hours of Syslog (sSMTP is tripping as I havnt debugged it yet, and although the mover is activating it has nothing to move). Feb 11 09:47:06 odyssey sSMTP[20416]: 501 Syntactically invalid EHLO argument(s) () Feb 11 10:47:08 odyssey sSMTP[20857]: 501 Syntactically invalid EHLO argument(s) () Feb 11 11:47:05 odyssey sSMTP[21176]: 501 Syntactically invalid EHLO argument(s) () Feb 11 12:00:01 odyssey logger: mover started Feb 11 12:00:01 odyssey logger: . Feb 11 12:00:05 odyssey logger: mover finished Feb 11 12:47:04 odyssey sSMTP[21382]: 501 Syntactically invalid EHLO argument(s) () Feb 11 13:10:49 odyssey ntpd[2404]: no servers reachable Feb 11 13:19:22 odyssey ntpd[2404]: synchronized to A.B.C.D, stratum 11 Feb 11 13:27:49 odyssey ntpd[2404]: time reset -5.951098 s Feb 11 13:28:27 odyssey ntpd[2404]: synchronized to A.B.C.D, stratum 11 Feb 11 13:47:05 odyssey sSMTP[21695]: 501 Syntactically invalid EHLO argument(s) () Feb 11 14:47:09 odyssey sSMTP[21886]: 501 Syntactically invalid EHLO argument(s) () Feb 11 15:47:05 odyssey sSMTP[22059]: 501 Syntactically invalid EHLO argument(s) () Feb 11 16:00:01 odyssey logger: mover started Feb 11 16:00:01 odyssey logger: . Feb 11 16:00:08 odyssey logger: mover finished Feb 11 16:47:05 odyssey sSMTP[22247]: 501 Syntactically invalid EHLO argument(s) () Feb 11 17:47:06 odyssey sSMTP[23682]: 501 Syntactically invalid EHLO argument(s) ()
February 11, 201115 yr I do not get the point of your build. You have Norco 4220 - hence 20 disks max and this is best achievable by 16 HDs on external controllers (2 x 8 ports) and then 4 HDs to the motherboard SATA ports. You are running the cache drive from the motherboard so I assume you have the required "reverse" breakout cable. If you think that if you use 6GBps HDs they will somehow be limited by the onboard SATA II Intel ports you are shouting yourself in the foot as these drivers are way more mature than the Supermicro's own. I will give an example - move one of the WD1002FAEX HDs to the motherboard ports and then copy 1 GB file to it. Copy the same file to one of the others WD1002FAEX attached to the SM cards and just observe the numbers of reads and writes. And then compare. So I do not see the point of a third SM controller in you configuration. It definitely wont help you to break the "speed record" with your system even if you only use fast 7200 rpm SATA III drives. Also you do need to post the complete syslog as there is definitely something fishy with your system - perhaps the SM cards do not like the new WD SATA III drives. In theory a system with small numbers of 7200 rpm 1TB drives should complete the parity build or check in around 4-5 hours.
February 11, 201115 yr Author Hi, My answers in blue You have Norco 4220 - hence 20 disks max and this is best achievable by 16 HDs on external controllers (2 x 8 ports) and then 4 HDs to the motherboard SATA ports. You are running the cache drive from the motherboard so I assume you have the required "reverse" breakout cable. No I decided to go with the three cards instead to drive the hotswap area (20 drives). I have the cache drive and dvd rom mounted in the case above the hotswap cage so no need for reverse breakout cable. If you think that if you use 6GBps HDs they will somehow be limited by the onboard SATA II Intel ports you are shouting yourself in the foot as these drivers are way more mature than the Supermicro's own. I will give an example - move one of the WD1002FAEX HDs to the motherboard ports and then copy 1 GB file to it. Copy the same file to one of the others WD1002FAEX attached to the SM cards and just observe the numbers of reads and writes. And then compare. The drives are limited to SATA II by a jumper on the drive (pins 5 and 6). I understand what your saying re performance on the MB vs in the hotswap area on the PCIe cards, but having said that reviewing what hardware unRaid is compatible with these cards were listed prominently (http://www.lime-technology.com/wiki/index.php?title=Hardware_Compatibility). Note Im not saying it would be faster than the MB SATA, just effective. The drives and cards were cheap (and good warrantee on the drives) and it was simply a choice I made to go for them. So I do not see the point of a third SM controller in you configuration. It definitely wont help you to break the "speed record" with your system even if you only use fast 7200 rpm SATA III drives. The question of speed is relative. Where I started with this topic was that the parity drive rebuild was taking a long time (at the time 20 hrs and now almost two days and counting, at this rate should finish a 2 days and 4 odd hours). Previously when I replaced one of the data drives it completed in 4 hrs or so hours. Im asking about something thats not right and not about something thats taking 2.6 seconds for 0-100 instead of 2.5 seconds. Also you do need to post the complete syslog as there is definitely something fishy with your system - perhaps the SM cards do not like the new WD SATA III drives. In theory a system with small numbers of 7200 rpm 1TB drives should complete the parity build or check in around 4-5 hours. Yep as soon as the thing is done I will restart and dump the syslog. Re the card and drive not liking each other, its possible. 4-5 hours, thats the number I was after to confirm that there was something wrong when I saw the rebuild rate initially. Watching it take two days gave it away as well. Thanks for the post.
February 12, 201115 yr Author All, Just dawned on me. The WD1002FAEX drives are specified as not being Advanced Format Drives. When I precleared I used the -A option (DOH) to force the format for AFD. Does anyone who knows the ins and outs of AFD know if pre clearing (with -A) a drive that isnt AFD causes it to perform badly eg I know that getting an AFD drive and not treating it like an AFD unit (or not jumpering it) makes it perform terribly. thanks
February 12, 201115 yr Author All, Thought Id share my learnings. The sloooowww parity synch with the replacement drive completed (after 2.5 days.....started with a throughput of 12MB/s and ended with 4MB/s). I corrected two things : - The WD1002FAEX drive is not an AF format drive and I incorrectly used the -A option for preclear. So I took another free precleared drive and performed a preclear -C 63 on it and swapped the new parity drive out with it. - The parity drive is on a SASLP-MV8 card by itself but while I was prepping the new replacement drives there was a WD1002FAEX drive on the same card. So why is this important? Well it looks like there at least two variants of the WD1002FAEX drives a 6G drive (WD1002FAEX-00Z3A0) and a 3G(?) drive (WD1002FAEX-00Y9A0). If you jumper pins 5 and 6 on a 00Z3A0 drive it sets the physical interface to 3G while if you do the same to the 00Y9A0 drive it sets it to 1.5G (confirmed it with WD). I had a 1.5G set drive on the same card as the parity drive (spun down but still there). I dont know which of the above (or combination) caused the issue but the new resynch is running at 104MB/s. Maybe its the advanced formating I used on a non AF disk or maybe its the SASLP-MV8 card downgrading performance to all interfaces to match the lowest drive interface speed hence affecting the parity drive...... Note there was nothing in the syslog about any of the drives being connected at a degraded speed or in any info of the hd tools I used to check the drive setting. In anycase I hope this helps someone else.
February 12, 201115 yr All, Just dawned on me. The WD1002FAEX drives are specified as not being Advanced Format Drives. When I precleared I used the -A option (DOH) to force the format for AFD. Does anyone who knows the ins and outs of AFD know if pre clearing (with -A) a drive that isnt AFD causes it to perform badly eg I know that getting an AFD drive and not treating it like an AFD unit (or not jumpering it) makes it perform terribly. thanks It would not have made any difference at all. A drive that uses 512 byte sectors internally is automatically aligned when its partition is aligned at ANY 512 byte multiple. You could start the partition on ANY sector and the disk would would be perfectly happy and work as best as it could. A disk that uses 4k sectors internally works at its best performance when the partition is accessed at 4096 byte boundaries. It would work when best if the partition starts at sector 1,4,8,16,20,24,28,32,36,40,44,48,52,56,60,64,68, etc... That is why partition 64 works better than 63. for it. To read sector 63 it would need to read the first part from the first 4k segment starting on logical sector 60, then the remainder from the 4k-physical sector starting on logical sector 64.. You'll need to look elsewhere for the cause of your poor performance. Any older disk can be accessed at ANY sector boundary. The "-A" option had no impact on performance. Joe L.
February 12, 201115 yr Does anyone who knows the ins and outs of AFD know if pre clearing (with -A) a drive that isnt AFD causes it to perform badly No. In fact, it's best to just use the -A option all the time (and do not jumper WD drives) and then you won't ever have an alignment issue by mistakenly not using the -A option on an advanced format drive. The -A option on those drives did not cause your issue. You likely had a bad drive or connector but maybe it was the SATA1 vs SATA2 drives conflicting. Peter
Archived
This topic is now archived and is closed to further replies.