Guys, browsed quickly trough the thread but couldn't find the final build list?

 

I would like to upgrade my 20-drive unraid server (based on the X7SBE) to X7SPA-HF board because of the low power consumtion and sleep mode ability. Thanks!

Guys, browsed quickly trough the thread but couldn't find the final build list?

 

I would like to upgrade my 20-drive unraid server (based on the X7SBE) to X7SPA-HF board because of the low power consumtion and sleep mode ability. Thanks!

 

Hi starcat,

 

I hadn't published the final build yet.  I was waiting to finish tuning and testing, working for max power efficiency and making careful measurements.

 

But since you're asking, here's the core details:

 

Motherboard:  Foxconn H61S

CPU:  Intel Celeron G1610

Memory: G.SKILL ECO Series 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1600 (PC3 12800)

SAS Controller:  HighPoint 2760A

Power Supply: KINGWIN LZP-650 650W ATX

Chassis:  X-Case RM-424

 

The Motherboard, CPU, Memory and Power Supply are all excellent choices for a 24 drive, power efficient server, though the MB forces you to add a single slot 24-drive controller card (and choices are limited).  You might be able to cobble together something a smidge more efficient, but it will be more expensive.

 

The 2760A is not exactly efficient, but at the same time isn't exactly horrible either, at least for what it does.

 

Motherboards with more PCIe x16 slots would give you more SAS/SATA controller options, but you also end up burning more power on the MB to support those slots, and every additional controller card adds wattage too, so ultimately you might be back in the same neighborhood as I am with the H61S+2760A.

 

The X-Case RM-424 does consume about 16W (10W fans, ~6W backplanes, but I need to measure again).  I never considered this in my build.  The RM-424 is superior to the Norco 4020 of my previous build, and worth every penny, so no regrets even though it added 16W to my build.  I plan to measure my 4220 for comparison.

 

I looked as well but couldn't find it either.  It would be great if the original post was updated with spec and energy usage

 

 

Sent from my Nexus 4

 

 

So I've been digging into the performance issues, ...

 

When I was first examining the output of lspci -vv

Hey, good for you! That's more like excavating (than digging).

 

A favor please: could you post with an attachment of your lspci -vv output. Thanks. (Apologies to all others, but PM was too restricting [no attachments; msg size too limited])

 

--UhClem

 

A favor please: could you post with an attachment of your lspci -vv output. Thanks. (Apologies to all others, but PM was too restricting [no attachments; msg size too limited])

 

For you, UhClem, absolutely!  Your dskt.sh script was a great help!

 

Attached.

lspci-vv.txt

Thanks guys. However, I am looking for the same build based on the X7SPA-HF board. I have tested the X7SPA-HF thoroughly in my other 6-bay enclosure and all the advanced scripting stuff like disk temperature based fan speed control, sleep after inactivity of certain time, inactivity of clients, wake up, etc. work excellent with this board. Any thoughts about using the 2760 controller with this board?

Thanks guys. However, I am looking for the same build based on the X7SPA-HF board. I have tested the X7SPA-HF thoroughly in my other 6-bay enclosure and all the advanced scripting stuff like disk temperature based fan speed control, sleep after inactivity of certain time, inactivity of clients, wake up, etc. work excellent with this board. Any thoughts about using the 2760 controller with this board?

 

I would not recommend it, total waste of money. 

 

Here's why, from the X7SPA-HF specs: 1 (x4) PCI-E (in x16 slot)

 

The 2760A has 16 lanes of PCIe 2.0, for a total of 8GB/s of bandwidth.  This is a high-performance controller card, with a price to match.

 

The X7SPA-HF only has 4 lanes of PCIe 1.0, for a total of 1GB/s of bandwidth.  Instead of giving each drive (assuming 24 total) 333 MB/s of bandwidth (good), the X7SPA-HF would be choking them to death with only 41MB/s of bandwidth.

 

You see reports of some people having 48 hour parity checks... and wonder why... this is why.

 

You can buy the H61S + G1610 for $100, and it will smoke that expensive X7SPA-HF all day... all night.. until it begs for mercy.

 

If you really, really want to use the X7SPA-HF, buy a more appropriate controller card that uses a PCIe x4 connection.

 

I would not recommend it, total waste of money. 

 

Well ...  (I LOVE my X7SPA-HF-D525 build !!)

 

... I don't disagree that it's not a good choice with a 2760A because of the x4 interface on the sole add-in card.    HOWEVER ... just as a nit ... it IS a PCIe v2.0 slot, so the bandwidth would be 2GB (500MB/lane) -- not the 1GB you noted.      So it would work fine with a 2760a as long as you didn't install more than a dozen drives on the card (thus supporting a total of 18 drives with the motherboard ports).    But a SuperMicro 8-port card like the SASLP-MV8 or SAS2LP-MV8 would be a better choice ... 8 ports at a FAR lower cost than the 2760a 

... Basically, if you bought a 2760a and wanted to preserve full bandwidth to your drives, you'd be paying the cost difference between the 2760a and a SASLP-MV8 just to gain 4 more useable ports !!

 

Note, by the way, that this is the board Limetech is using in their new server (along with one of the SuperMicro cards I just noted).

 

You can buy the H61S + G1610 for $100, and it will smoke that expensive X7SPA-HF all day... all night.. until it begs for mercy.

 

Ahem !!  My X7SPA does a parity check in 8:07 with six 3TB WD Reds ... and I suspect would do it in ~ the same time with 14 of them if I added an 8-port controller.    That kinda "smokes" your H61S + G1610 times    [Although I readily admit that your times are extended a good bit due to the mix of drive sizes you have in your array.

 

AND, I expect that 8:07 to be notably less with a bit of "tweaking" of the md_sync_window parameter.    I just started that a few hours ago ... changed it to 768 ... and it's looking like it's going to shave at least half an hour off that time (I'll know in a couple hours).

 

AND ... my system draws 20W on idle  8)  [it would of course be more if I added 8 more drives and an add-in controller]

Are you sure it is PCIe 2.0 capable? It looks like 1.0 from the web page and manual. 

 

If it is PCIe 2.0, then the 2740 would be better than the 2760A in that it supports 16 drives and is $150 less.  You could use up the 6 MB SATA ports first, then fill up the remainder from the 2740.

 

If it is only PCIe 1.0, then I wouldn't want to go beyond 14 drives total, 6 from the motherboard and 8 from an add-in card like the SASLP-MV8.

 

Still, I don't believe the X7SPA-HF is a good choice for a 20 (like you are talking about) or 24 drive server.  garycase get's excellent use in a super energy efficient 6 drive server, which I think is this boards strong point, and I think it would work well up to 14 drives (like the Lime-Tech server), but beyond that you are really throttling each drives bandwidth because of the limited PCIe connection.

 

 

You can buy the H61S + G1610 for $100, and it will smoke that expensive X7SPA-HF all day... all night.. until it begs for mercy.

 

Ahem !!  My X7SPA does a parity check in 8:07 with six 3TB WD Reds ... and I suspect would do it in ~ the same time with 14 of them if I added an 8-port controller.    That kinda "smokes" your H61S + G1610 times    [Although I readily admit that your times are extended a good bit due to the mix of drive sizes you have in your array.

 

AND, I expect that 8:07 to be notably less with a bit of "tweaking" of the md_sync_window parameter.    I just started that a few hours ago ... changed it to 768 ... and it's looking like it's going to shave at least half an hour off that time (I'll know in a couple hours).

 

AND ... my system draws 20W on idle  8)  [it would of course be more if I added 8 more drives and an add-in controller]

 

Like I just said, your board makes an excellent 6-drive energy efficient server, far superior to the H61S (since it only has 2 SATA ports!  ).

 

But scale it up to 16 drives and I think it will start to choke.  24 and it just might have a heart attack.

 

Your board is awesome.... just not 20/24 awesome. 

My X7SPA does a parity check in 8:07 with six 3TB WD Reds ...

 

By the way, I'm slowly removing all my Samsung drives, and will soon have only WD Red 3TB drives.  Then we can compare apples to apples, my friend.  These Samsungs are molasses slow...

But scale it up to 16 drives and I think it will start to choke.  24 and it just might have a heart attack.

 

Your board it awesome.... just not 20/24 awesome. 

 

I definitely agree it's not "20/24 awesome"  ==> but I think it would work every bit as good as it does now with up to 14 drives [the 6 it supports natively plus one fully-loaded 8-port card].    Obviously the idle consumption would jump up [probably to ~ 35-40 watts with the card plus 8 more idle WD Reds] ... but I think it would still perform just fine.    Go above that 14 drive threshold, however (by using a 2760a or an extender), and I agree the performance would begin to suffer.

 

Pauven, sorry if I missed this (and you answered this before), but to help Tom and others if tweaking these setting only worked for you as of RC13 (slowness issue)? or you never tried them previously in say 12a. IF you have not tried this in a prior release, can you? This would help to know if tweaking these settings only helped with the latest kernel (in RC13) only or helps with any of the prior release (I use any prior release as a loose term).

 

 

Pauven, sorry if I missed this (and you answered this before), but to help Tom and others if tweaking these setting only worked for you as of RC13 (slowness issue)? or you never tried them previously in say 12a. IF you have not tried this in a prior release, can you? This would help to know if tweaking these settings only helped with the latest kernel (in RC13) only or helps with any of the prior release (I use any prior release as a loose term).

 

Clearly he's used these settings in 12a, since the results were posted BEFORE RC13 was released      ... and if you read the details of what they do, it's virtually certain that ANY release will benefit.    Very simple to see what the effect is for any given system:  simply try it !!

 

My X7SPA does a parity check in 8:07 with six 3TB WD Reds ...

 

By the way, I'm slowly removing all my Samsung drives, and will soon have only WD Red 3TB drives.  Then we can compare apples to apples, my friend.  These Samsungs are molasses slow...

 

My parity check completed in 7:33 (34 minutes better) with my first "tweak" to the md_sync_window ... a simple doubling of the setting from 384 to 768.    I'm going to try a few more settings ... perhaps in conjunction with also adjusting the other two parameters => but not it will be a few days, as I'm not going to be home much the next two days.

 

I'd expect your parity check to drop to the same general time once you're all WD Reds.

 

Firstly, I clearly stated "sorry if I missed this", I did not follow the thread from the beginning, only from the point Pauven post in the RC thread he had some good news, so the assumptions was as of RC13.

 

I see an assumption in the RC13 thread that this has been solved now with the new kernel version with RC13, unless I misunderstood something; hence the question to Pauven, if the slow write can be resolved on any prior release by tweaking these settings then the fix is not due to the latest kernel in RC13 and does not need to be set in stone to keep that kernel version. Thats all and nothing more.

 

I do not suffer a slow issue (I could definitely say the last few RC's have not been the fastest for me but not as slow as some are clearly reporting).

The "tunables" were added about two years ago, so they've ben around for a LOT of releases 

 

This thread gives a good overview of what they do ... and how they interact:

http://lime-technology.com/forum/index.php?topic=15224.msg142767#msg142767

 

 

The "tunables" were added about two years ago, so they've ben around for a LOT of releases 

 

This thread gives a good overview of what they do ... and how they interact:

http://lime-technology.com/forum/index.php?topic=15224.msg142767#msg142767

Agreed, they have been around for as long as I can remember using 5, original posts questioning those setting were replied to DONT MESS with them. Then some experiments later, no one got anywhere. Now this slowness issue hit some MB and Cards Hard and its been tapped into again. Again my question is strictly if it helped those affecedt in prior release not just RC13, if so at least Tom then know's its not a pick or choose situation, that is if he decides to downgrade the kernel would he be stranding the people with affected MB and/or controllers, thats it. They would have an avenue to still use these tunables, and I am not saying this is or not acceptable for those people affect either. Just where its at, facts are facts.

My parity check completed in 7:33...

 

You keep moving the bar... let me catch up first! 

Pauven, sorry if I missed this (and you answered this before), but to help Tom and others if tweaking these setting only worked for you as of RC13 (slowness issue)? or you never tried them previously in say 12a. IF you have not tried this in a prior release, can you? This would help to know if tweaking these settings only helped with the latest kernel (in RC13) only or helps with any of the prior release (I use any prior release as a loose term).

 

Hi madburg, I've only tried the md_ settings on 5RC12a.  I originally raised the issue in the Lime-Tech AVS-10/4 server thread, not the RC13 thread.  I think someone else reported their SAS2LP md_sync_window results in the RC13 thread.

 

I've only had the 2760A a very short while, and I've only used two unRAID versions with it:  5RC6 (which I had been running for a while on my old server) and 5RC12a (which was the latest available when I got the 2760A).

 

Under 5RC6, I had horrible parity check speeds of 4 MB/s with the 2760A!  My first trouble-shooting step was to go to 5RC12a, which improved speeds tenfold, to 40 MB/s (still horrible), and with some clever drive re-arranging I got it into the 60MB/s range (decent, but not great).  That was when Tom directed me to try the md_sync_window, which resolved the parity speed issue on 5RC12a.

 

From what I've read, these parity speed issues have been affecting SAS2LP owners since 5B11, and I would hazard a guess that the md_sync_window would probably help all versions since then, but that's just a guess until someone tries it.  As I illustrated above with 5RC6, some versions were notably worse than others, so I don't imagine md_sync_window is a cure all; if a particular release is really bad, there's not much that's gonna help it.

 

From what I've been reading about RC13, I'm planning to skip it.  While I admire Tom's efforts to resolve the outstanding issues and drive towards a 5.0 Final, I believe the changes he made in RC13 would be more appropriately classified as Beta.  I don't put beta's on my production server.

 

As far as I am aware, I am the first and only unRAID user with a 2760A (though I think that will soon change now that I've blazed the trail).  There are many other users with the SAS2LP using the same Marvel controller chip, and many of them are having the the same parity check performance issue.  My recommendation would be for them to begin testing md_sync_window with whatever versions they are running (RC10 sure seems popular) - that will be the quickest way to determine that this setting truly addresses the SAS2LP performance issues and that it works across multiple version.

 

Myself, I'm staying with RC12a for now.  It seems to be working well for me.  Sorry I couldn't be more help, but since this isn't a test server, I have to be particular about what versions and what tests I run on my production server.

 

I'll continue to evaluate the md_ settings on RC12a and report my findings - I do expect that setting  the values too high will cause stability issues.

 

-Paul

I definitely agree it's not "20/24 awesome"  ==> but I think it would work every bit as good as it does now with up to 14 drives [the 6 it supports natively plus one fully-loaded 8-port card].    Obviously the idle consumption would jump up [probably to ~ 35-40 watts with the card plus 8 more idle WD Reds] ... but I think it would still perform just fine.    Go above that 14 drive threshold, however (by using a 2760a or an extender), and I agree the performance would begin to suffer.

 

Sooo... is it evil now that I want you to build a 24 drive server using your MB?    I want to know how good or bad it would be.  I can't help but imagine 4-day parity checks, but maybe it wouldn't be so bad... 

 

I once built a race car from a Volkswagen Golf MkI....  Sometimes from humble beginnings comes greatness.

Sooo... is it evil now that I want you to build a 24 drive server using your MB?    I want to know how good or bad it would be.  I can't help but imagine 4-day parity checks, but maybe it wouldn't be so bad... 

 

Pure evil ... devil incarnate !!!

 

The math is relatively simple -- the x4 bandwidth limit would start to be pretty bad as you exceeded 12 drives on the controller.  To get to 24, that would require 18 drives on the controller, so they'd be limited to 111MB/s /drive.    That would also, of course, mean that even the motherboard-attached drives would be restricted to that, since the parity check can't run any faster than the slowest drive.  Parity checks start off just over 140MB/s, and stay well above the 110MB threshold for at least the first 60% or so.  Once they get below 110MB/s the timing should be the same as what I'm seeing now.    So ... about a 20% reduction in speed = about 25% longer for the first 2/3rds of the check (until it's far enough "in" the drive to be under 110MB).  I don't know exactly what point it hits that, but I'd guess about 6 hrs in ... so probably about 1.5 extra hours.    So instead of 7:33 the checks should take just over 9 hours.  Assuming that the actual useful bandwidth is a bit less than the theoretical 2GB/s (likely), that would likely be a bit longer.    So ... I'd estimate ~ 9:30  for a parity check on a 24 drive system with 18 drives attached to a 2760a and the other 6 on the motherboard, assuming all drives were 3TB WD Reds (since that's what all my timings are based on).    If all 24 were attached to the 2760a it would be appreciably worse, since then the per/drive bandwidth would be dropped to about 83MB/s.  This is probably about as low as the check ever goes (I didn't actually note the final rate) ... but if so, it's easy to compute that time:  my current check (at 7:33) is averaging just over 110MB/s, so at 83MB/s it would take 1.33 times as long ... or just over 10 hours.

 

... Not really "bad" for a 69TB system    ==> but I'd expect that your system will do this in something very close to 7:30 once you have all WD Reds (unless, of course, you mix sizes).

 

P.S.  If anyone wants to know the ACTUAL timings for that, just send me a 2760a and 18 3TB WD Reds and I'll be glad to oblige  8)

 

 

 

I once built a race car from a Volkswagen Golf MkI....  Sometimes from humble beginnings comes greatness.

 

... I had a friend in the early 80's who had added a turbocharger to his Golf (US MkI) and could easily beat my Porsche on takeoff 

... one other factor that could potentially make the checks a bit longer with 24 drives ==> I'm not sure if the md_sync_window tunings would still allow the same rate with that many drives.    Your experience tends to indicate they would ... but without actually doing it on my system it's difficult to really know.    But in any event, I doubt the impact would be more than perhaps an extra 30 minutes or so.

 

Pauven, sorry if I missed this (and you answered this before), but to help Tom and others if tweaking these setting only worked for you as of RC13 (slowness issue)? or you never tried them previously in say 12a. IF you have not tried this in a prior release, can you? This would help to know if tweaking these settings only helped with the latest kernel (in RC13) only or helps with any of the prior release (I use any prior release as a loose term).

 

Hi madburg, I've only tried the md_ settings on 5RC12a.  I originally raised the issue in the Lime-Tech AVS-10/4 server thread, not the RC13 thread.  I think someone else reported their SAS2LP md_sync_window results in the RC13 thread.

 

I've only had the 2760A a very short while, and I've only used two unRAID versions with it:  5RC6 (which I had been running for a while on my old server) and 5RC12a (which was the latest available when I got the 2760A).

 

Under 5RC6, I had horrible parity check speeds of 4 MB/s with the 2760A!  My first trouble-shooting step was to go to 5RC12a, which improved speeds tenfold, to 40 MB/s (still horrible), and with some clever drive re-arranging I got it into the 60MB/s range (decent, but not great).  That was when Tom directed me to try the md_sync_window, which resolved the parity speed issue on 5RC12a.

 

From what I've read, these parity speed issues have been affecting SAS2LP owners since 5B11, and I would hazard a guess that the md_sync_window would probably help all versions since then, but that's just a guess until someone tries it.  As I illustrated above with 5RC6, some versions were notably worse than others, so I don't imagine md_sync_window is a cure all; if a particular release is really bad, there's not much that's gonna help it.

 

From what I've been reading about RC13, I'm planning to skip it.  While I admire Tom's efforts to resolve the outstanding issues and drive towards a 5.0 Final, I believe the changes he made in RC13 would be more appropriately classified as Beta.  I don't put beta's on my production server.

 

As far as I am aware, I am the first and only unRAID user with a 2760A (though I think that will soon change now that I've blazed the trail).  There are many other users with the SAS2LP using the same Marvel controller chip, and many of them are having the the same parity check performance issue.  My recommendation would be for them to begin testing md_sync_window with whatever versions they are running (RC10 sure seems popular) - that will be the quickest way to determine that this setting truly addresses the SAS2LP performance issues and that it works across multiple version.

 

Myself, I'm staying with RC12a for now.  It seems to be working well for me.  Sorry I couldn't be more help, but since this isn't a test server, I have to be particular about what versions and what tests I run on my production server.

 

I'll continue to evaluate the md_ settings on RC12a and report my findings - I do expect that setting  the values too high will cause stability issues.

 

-Paul

Completely understand and appreciate all your points. I think this will all be very helpfully to Tom and others. I feel the same way about RC13, it's a pass for me as well.

Thanks again for sharing with everyone.