Anyone using two ADD ON cards? Supermicro?

[opentoe]

Building my new rig this weekend and wanted to know if anyone is using TWO SuperMicro add on SAS2 6GB cards? I've noticed all my hard drives have a 6GB connection and 4 SATA ports on the mainboard have only a 3GB connection. So I was thinking of just installing TWO SAS2 cards and using the on-board SATA ports for other uses. I do have three PCI-16 slots, so I have the space. I don't see why it wouldn't work.

 

The mainboard has a total of 8 SATA connections. 4 are 6GB and 4 are 3GB. If I use my one Supermicro add on card (total of 8 6GB ports) and the remaining 6GB on the mainboard that is a total of 12  ports and my system has 14 drives. 13 for the array and one unassigned. I really wish I just had an additional add on card right now. You know how you want to just do it right for the first time?

 

Anyway, I do have a left over Supermicro 3GB older card I may use temporarily and try until another 6GB card comes in. Arggh.

[Kode]

As I understand it there isn't much benefit to having harddrives on SATA3 vs SATA2 as harddrives don't get anywhere near 300MB/s, if all your drives were SSDs that might be a different matter.

[JorgeB]

There's no problem using 2 Supermicro SAS2 cards, or one SAS and one SAS2.

 

By the way, in my experience latest Unraid V6 with SAS2 is much slower during parity check / disk rebuild than with older SAS, works ok during parity sync.

 

You can see my results here:

http://lime-technology.com/forum/index.php?topic=39125.msg386462#msg386462

 

Would love to know your results.

[Kode]

That's very interesting, I currently have my parity and cache drive running from the onboard SATA2 ports and all the other drives on a SASLP, I recently bought a SAS2LP with the intention of moving the cache (which is a 512GB SSD) and harddrives to that instead, might just stick the cache drive on it for now if that is the case.

[opentoe]

I don't get it, how can the slower card be faster then the faster card? Funny question, right?

 

Anyway, right now I have one SAS installed and one SAS2 installed. Amazingly on the very first boot everything came up as normal. All the drives were set, parity was good and it's humming along now.

 

I'm wondering how I can test the speeds of each individual SAS controller. I'd love to test it out and see if there really is a difference. I mean the SAS2 card is actually readily available for the same price as the orignal SAS/older card. I left the side case open to make sure all the fans were running good.

[Kode]

if you only have 14 drives you could put 6 on the motherboard and 8 on the SASLP, do a parity check and record the finish time, then move the drives to the SAS2LP card and run the check again, that should give you a rough indication of how they compare.

[bkastner]

I have 2 SAS2LP controllers which manage all my drives. Works fine. I have a Norco 4224 with backplanes to have each SAS2LP controlling 2 backplanes and I don't use my on-board SATA ports for anything.

 

I've never noticed any performance issue either.

 

My last parity check was reasonable:

? Duration: 17 hours, 31 minutes, 29 seconds. Average speed: 95.1 MB/sec

 

I have not re-run the tunables script in quite a while, so I may be able to get a bit better if I played with it, but I am okay with the current speed.

[JorgeB]

That is a very good parity check speed, I have a couple of SAS2LP that I tried in 4 different board/cpu combinations and cannot get nowhere near those speeds.

 

Do you mind posting your complete hardware and confirming you're on v6 final.

 

Thanks.

 

EDIT: Never mind, I was on Tapatalk and didn't see your signature.

[JorgeB]

My last parity check was reasonable:

? Duration: 17 hours, 31 minutes, 29 seconds. Average speed: 95.1 MB/sec

 

I see you have a lot of different size disks so after the 3tb point your speed should improve a lot and so will the average speed, can you tell me the starting speed of a parity check, while reading all disks?

 

Thanks.

 

[opentoe]

Ok, I ran a parity on my 4TB parity drive connected to my SAS2 card. Here were the stats.

 

Last checked on Sat 27 Jun 2015 11:29:37 AM EDT (today), finding 5 errors.

? Duration: 16 hours, 15 minutes, 47 seconds. Average speed: 68.3 MB/sec

 

I do remember last time, it took over 21 hours to complete. But I'll run it again when it is connected to the older card and see if there is really a difference.

 

 

 

[Kode]

Are you not concerned it found 5 errors?

[bkastner]

Ok, I ran a parity on my 4TB parity drive connected to my SAS2 card. Here were the stats.

 

Last checked on Sat 27 Jun 2015 11:29:37 AM EDT (today), finding 5 errors.

? Duration: 16 hours, 15 minutes, 47 seconds. Average speed: 68.3 MB/sec

 

I do remember last time, it took over 21 hours to complete. But I'll run it again when it is connected to the older card and see if there is really a difference.

 

If you haven't done this already, I would run the tunables script on your environment to get optimal settings prior to testing.

 

http://lime-technology.com/forum/index.php?topic=29009.0

 

 

[opentoe]

Are you not concerned it found 5 errors?

 

I've checked all the SMART reports on all the drives. They are all good. I rebooted already once I updated to 6.0.1, so I'm not sure where to even look for what errors it found. Does the parity check throw it's own log somewhere?

 

[opentoe]

Ok, I ran a parity on my 4TB parity drive connected to my SAS2 card. Here were the stats.

 

Last checked on Sat 27 Jun 2015 11:29:37 AM EDT (today), finding 5 errors.

? Duration: 16 hours, 15 minutes, 47 seconds. Average speed: 68.3 MB/sec

 

I do remember last time, it took over 21 hours to complete. But I'll run it again when it is connected to the older card and see if there is really a difference.

 

If you haven't done this already, I would run the tunables script on your environment to get optimal settings prior to testing.

 

http://lime-technology.com/forum/index.php?topic=29009.0

 

Wow, didn't even knew this existed. Will check it out. I always left those settings alone since I never knew what they were.

Thanks, I'll try it.

 

 

[bkastner]

Ok, I ran a parity on my 4TB parity drive connected to my SAS2 card. Here were the stats.

 

Last checked on Sat 27 Jun 2015 11:29:37 AM EDT (today), finding 5 errors.

? Duration: 16 hours, 15 minutes, 47 seconds. Average speed: 68.3 MB/sec

 

I do remember last time, it took over 21 hours to complete. But I'll run it again when it is connected to the older card and see if there is really a difference.

 

If you haven't done this already, I would run the tunables script on your environment to get optimal settings prior to testing.

 

http://lime-technology.com/forum/index.php?topic=29009.0

 

Wow, didn't even knew this existed. Will check it out. I always left those settings alone since I never knew what they were.

Thanks, I'll try it.

 

I know it can make a big difference (i.e. 20-30MB/sec) depending on how far off you are by default.

 

Just a warning - it can take several hours to run it's course so it may be something you want to kick off before you go to bed one night.

[opentoe]

Ok, I ran a parity on my 4TB parity drive connected to my SAS2 card. Here were the stats.

 

Last checked on Sat 27 Jun 2015 11:29:37 AM EDT (today), finding 5 errors.

? Duration: 16 hours, 15 minutes, 47 seconds. Average speed: 68.3 MB/sec

 

I do remember last time, it took over 21 hours to complete. But I'll run it again when it is connected to the older card and see if there is really a difference.

 

If you haven't done this already, I would run the tunables script on your environment to get optimal settings prior to testing.

 

http://lime-technology.com/forum/index.php?topic=29009.0

 

Wow, didn't even knew this existed. Will check it out. I always left those settings alone since I never knew what they were.

Thanks, I'll try it.

 

I know it can make a big difference (i.e. 20-30MB/sec) depending on how far off you are by default.

 

Just a warning - it can take several hours to run it's course so it may be something you want to kick off before you go to bed one night.

 

Awesome, I'll do that! Looking forward to the results.

 

[opentoe]

Ran the tunable script but posted a question in that related thread. It has a setting that it recommend I change but is no longer in V6. So I would be only able to change two of the settings. I have poll_attributes in my settings but no sync_write. Anyway, the detailed post is in that tunable thread.