Hey guys,

 

I've got a modest setup that ive been running for a few years and have always been plagued with the array dropping off after a few weeks, nothing in the logs so i suspect it's always been a PSU issue as nothing else is coming up as an issue.

 

I run the server with:

22 x Hitachi 2TB 7200rpm drives (2 for parity)  they peak at around 11w (0.9amps according to this link) (https://content.it4profit.com/itshop/itemcard_cs.jsp?ITEM=91012054645819599&THEME=asbis&LANG=en)

3 x Dell H200 HBA's, (14w max each)

1 x 10gb PCIe nic (14w max)

2 x Samsung EVO 500GB SSD's (<5w)

MOBO: Supermicro X9SCM-F (<20w)

CPU: E3-1240v2 (69w TDP)

24 bay server Midplane (<20w)

PSU: Corsair CX750M (Single rail)

 

=466w max draw. About 37 amps of 12v i believe? (if i used 24 disks rather than 22 = 439w)

 

So as mentioned i'm in the market for a replacement PSU. Based on the hardware above what would you recommend if anyone has a similar setup could you let me know what PSU's you've had good luck with as a starting point? Any advice would be great. Not looking for the latest and greatest just something solid and reliable that will allow me to populate my other 2 drive bays at a later date without worry for what it matters my midplane uses 6 x MOLEX connectors to power the drives)

 

Do i even need too replace my CX750M? Is there an easy way to tell? I've recently removed it and it was pretty dusty so just in the process of cleaning it out...

 

Any chance that using unraid as a persistant syslog server would help to capture anything psu related? What should i be looking for in the logs to indicate it? (https://wiki.unraid.net/Manual/Troubleshooting#Persistent_Logs_.28Syslog_server.29)

 

Thinking of the following PSU models, anyone got any experience in a similar rig as they have icue link for power monitoring and are single rail switchable.

RM750i
RM850i
HX750i
HX850i

Edited November 3, 20214 yr by thestraycat
adding

5 hours ago, thestraycat said:

22 x Hitachi 2TB 7200rpm drives (2 for parity)  they peak at around 11w (0.9amps according to this link) (https://content.it4profit.com/itshop/itemcard_cs.jsp?ITEM=91012054645819599&THEME=asbis&LANG=en)

That link claims that max. power consumption is 11W, but this is only for normal operation.

You need to find the number, required for spinup (in Amps), as this is what the spindle motor would draw, when trying to spin-up the weight of the platters (running on 12V)

See here: https://www.hdsentinel.com/storageinfo_details.php?lang=en&model=HUA722020ALA330

This claims a draw of 3.3 Amps !!!! (almost 40W!!)

 

So, with a 750W PSU and 20+ HDDs (and if you trust that document I linked above) you better set this Rig to do staggered spinup!!

 

6 hours ago, thestraycat said:

RM750i
RM850i
HX750i
HX850i

I suggest you should focus on the 5v rating.

 

By quick check, CX750M or RM__ HX__i, 5v rating is 25A ( combine is 130w or 150w )

 

 

 

In my opinion, the major problem of CX750M should be 5v regulate with 12v together, btw you should avoid use single PSU for 24 disks. ( If you can, pls visual check does CX750M any capacitor exploded )

 

https://linustechtips.com/topic/1122694-why-group-regulated-units-shouldnt-be-boughtsold-in-2019-and-on/

 

Edited November 3, 20214 yr by Vr2Io

6 hours ago, thestraycat said:

Any chance that using unraid as a persistant syslog server would help to capture anything psu related? What should i be looking for in the logs to indicate it? (https://wiki.unraid.net/Manual/Troubleshooting#Persistent_Logs_.28Syslog_server.29)

You could try pbertera/syslogserver, I run it in docker with standalone IP ( need enable host network in docker setting ) and redirect syslog file to UD disk path.

 

Edited November 3, 20214 yr by Vr2Io

4 hours ago, Ford Prefect said:

That link claims that max. power consumption is 11W, but this is only for normal operation.

You need to find the number, required for spinup (in Amps), as this is what the spindle motor would draw, when trying to spin-up the weight of the platters (running on 12V)

See here: https://www.hdsentinel.com/storageinfo_details.php?lang=en&model=HUA722020ALA330

This claims a draw of 3.3 Amps !!!! (almost 40W!!)

 

So, with a 750W PSU and 20+ HDDs (and if you trust that document I linked above) you better set this Rig to do staggered spinup!!

 

Hmmm i've run it for many years on the CX750M - I wonder how i've got away with it for so long during partiy checks? I'd assue it has something to do with the use of 5v and 12v ampage being utilized by the spin up process... but it dosnt make sense that this should be pulling 800w on just the disks alone  + 3 x HBAs, 1 x 10gb nic, 2 x SSD, MOBO + CPU during a parity check... That should be enough to push my cx750m's peak surely...

Edited November 3, 20214 yr by thestraycat

3 hours ago, Vr2Io said:

I suggest you should focus on the 5v rating.

 

By quick check, CX750M or RM__ HX__i, 5v rating is 25A ( combine is 130w or 150w )

 

 

 

In my opinion, the major problem of CX750M should be 5v regulate with 12v together, btw you should avoid use single PSU for 24 disks. ( If you can, pls visual check does CX750M any capacitor exploded )

 

https://linustechtips.com/topic/1122694-why-group-regulated-units-shouldnt-be-boughtsold-in-2019-and-on/

 

 

Thanks. Yeah no bulging capacitors but i'll take a pic and upload it later. My case is the 550mm  XCase RM424S. So dosnt have room for 2 x PSU's so i'll have to stick to a single large PSU... I've been thinking of upping the size up to the EVGA Supernova 1000 or 1300 models... This gives me around 83amp (1000w)-> 108amps (1300w) on the 12v rail and around 25 on the 5v... Do you think this should hold up well for my system? I think going forward rather than expand further i'll simply start the upgrading process of moving these power hungry 2tb disks over to larger and newer lower spin up draw disks as a battle plan... I do however need to get the server back up prior to migrating the disks over to more energy efficient models...

 

 

1 hour ago, thestraycat said:

Hmmm i've run it for many years on the CX750M - I wonder how i've got away with it for so long during partiy checks?

This high power draw only occurs during cold boot, hence the staggered spinup topic.

I gather, when the system is already booted, re-spin-up draw will be slightly less, as 5V/3.3V draw is not part of that initial amount but only the motor firing up the platters.

Wenn spun up, the draw is only 11W per disk during the parity check run.

It all comes down for how long the PSU must sustain that amount of high/peak current, actually a spread over some milliseconds while spinning up the drives could make a difference here.

 

1 hour ago, thestraycat said:

I'd assue it has something to do with the use of 5v and 12v ampage being utilized by the spin up process... but it dosnt make sense that this should be pulling 800w on just the disks alone  + 3 x HBAs, 1 x 10gb nic, 2 x SSD, MOBO + CPU during a parity check... That should be enough to push my cx750m's peak surely...

You could check these with a good energy meter....some utilities/energy retailers even offer to lend these out to customers for some days/weeks. A good model will capture min/max/average values auf Voltage, Current and Power for the timeframe measured, not just Energy consumption (sum).

 

Edit: FWIW, i have a 900W PSU in my Norco-like case, equipped with 14x3TB (Seagate, HGST & WD) disks..during cold boot, the system peaks at 360W and runs during parity check with 174W (AMD Opteron 2350HE/4C, 8GB ECC, 120GB SSD, 9211-8i HBA+ RES2SV240 Expander).

 

Edit2: For comparison of the effect we are talking about here...I have a larger heatpump, that nominally will consume 2kW when in operation (delivering 10kW of heat then) and this equals to approx 9A (at 230V - across 3 phases, so 3A each))

During the cold start of the inverter motor, fuses are "slow, 20A", it draws approx 40A (luckily across 3 phases, so 40/3 = 14A per phase...hence the 20A fuse).

So the factor is 3A vs 14A here.

Edited November 3, 20214 yr by Ford Prefect

My setup also similar power usage, 28 disks in two case, 2 PSU , 1 10GNIC, 2 HBA , 1 Expander, 1 low power GPU, during parity sync, it use ~422w ( 522w-100w floor). I like Corsair more then EVGA.

 

3 hours ago, Ford Prefect said:

This high power draw only occurs during cold boot, hence the staggered spinup topic.

I gather, when the system is already booted, re-spin-up draw will be slightly less, as 5V/3.3V draw is not part of that initial amount but only the motor firing up the platters.

Wenn spun up, the draw is only 11W per disk during the parity check run.

It all comes down for how long the PSU must sustain that amount of high/peak current, actually a spread over some milliseconds while spinning up the drives could make a difference here.

 

You could check these with a good energy meter....some utilities/energy retailers even offer to lend these out to customers for some days/weeks. A good model will capture min/max/average values auf Voltage, Current and Power for the timeframe measured, not just Energy consumption (sum).

 

Edit: FWIW, i have a 900W PSU in my Norco-like case, equipped with 14x3TB (Seagate, HGST & WD) disks..during cold boot, the system peaks at 360W and runs during parity check with 174W (AMD Opteron 2350HE/4C, 8GB ECC, 120GB SSD, 9211-8i HBA+ RES2SV240 Expander).

 

Edit2: For comparison of the effect we are talking about here...I have a larger heatpump, that nominally will consume 2kW when in operation (delivering 10kW of heat then) and this equals to approx 9A (at 230V - across 3 phases, so 3A each))

During the cold start of the inverter motor, fuses are "slow, 20A", it draws approx 40A (luckily across 3 phases, so 40/3 = 14A per phase...hence the 20A fuse).

So the factor is 3A vs 14A here.

 

 

@Vr2Io

@Ford Prefect

 

 

 

Yeah i get that. Thanks for confirming. An issue im trying to wrap my head round is my midplane used 6 x molex to power 24 disks. From the PSU most have 2 seperate lines of 3 or 4 molex cables at most. In escence in a case like mine, with drives like mine (3.3amp cold start) isnt that potentially:

 

12 disks x 3.3amp = 39.6amp (480w) for the first molex cable 

12 disks x 3.3amp = 39.6amp (480w) for the 2nd molex cable 

 

 

 

And if molex connectors are rated at 54w each and theres 3 of them per wire the max draw over all 3 combined shouldnt exceed 162w! I understand this is a burst of current and they can potentially take it on the short term... but still.... 

 

It seems realistic that the configuration for any of these sort of 4u cases with 6 molex and 24 drives is impossible due to exceeding the maximum

ampeage a 4 pin molex can take (54w)

 

In escence:

*If 16awg cable is only rated at 15amps 

*and a single molex is rated at 11amps 

*but the drives cold start at 13.2amps on the midplame

 

then it seems physically impossible for me to power my midplane! Unless the molex can withstand the extra  2.2 amps of draw for a short time?

 

 

I drew this crude pic to demonstrate what i mean:

 

 

*A Powersupply that has over 85amps on the 12v rail for headroom of onboard and pcie cards (1000w+)

*A powersupply that is single rail

*A powersupply that has the potential to run 6 seperate cables with a molex connector on each of them? Although this dosnt explain how the molex connector on each of the 6 cables can take 150w / 12.2amps of current for the time it takes to spin that bank of drives up! however lots of people run them, and even the most energy efficient drives probably take around 2amps to spin up...

 

 

The best fit i can find is something like this EVGA SuperNOVA 1300w G2 what do you think? Obviously, new case, new psu new disks would be amazing, but it's just not an option for me right now.  What PSU model are you using @Vr2Io??

 

 

 

 

Edited November 3, 20214 yr by thestraycat
added

2 hours ago, thestraycat said:

Yeah i get that. Thanks for confirming. An issue im trying to wrap my head round is my midplane used 6 x molex to power 24 disks. From the PSU most have 2 seperate lines of 3 or 4 molex cables at most. In escence in a case like mine, with drives like mine (3.3amp cold start) isnt that potentially:

Yes, most PSU won't have 6 independence wire for peripheral.

 

2 hours ago, thestraycat said:

The best fit i can find is something like this EVGA SuperNOVA 1300w G2 what do you think? Obviously, new case, new psu new disks would be amazing, but it's just not an option for me right now.  What PSU model are you using @Vr2Io??

 

** Never think only use 2 wire set to power up 24 disks **

 

My 28 disks config are 16 ( 4 molex ) + 12 ( 3 molex ), both is 2U PSU with 4 peripheral wire and each have 2 molex, so no problem on wiring. The model is Enhance ENH-2180, it is not a high quality PSU, but I haven't much choice.

 

EVGA SuperNOVA 1300w G2 have 6 peripheral socket, you need DIY to make SATA 1-4 wire be molex, cut it and insert to molex DIY plug.

 

 

Corsair HX-850, 1000, 1200 also support 6 peripheral socket and 1200 have 5v 30A and one more SATA cable, but you also need cut those SATA wire be molex.

 

Edited November 3, 20214 yr by Vr2Io

5 minutes ago, Vr2Io said:

Yes, most PSU won't have 6 independence wire for peripheral. My 28 disks config are 16 ( 4 molex ) + 12 ( 3 molex ), both is 2U PSU with 4 peripheral wire and each have 2 molex, so no problem on wiring.

 

>>Wouldnt this mean on your 16 drives though that you have 1 molex connector per 4 drives? Depending on the cold starting draw of your disks that would put you in the same camp as me:

 

16awg cable = 15amp

Molex connector = 11amp

4 x 3.3amp disks on a molex connector = 13.2amps

 

Obviously if your disks wernt as power hungry as mine you'd be ok?

 

5 minutes ago, Vr2Io said:

 

 

Quote

 

EVGA SuperNOVA 1300w G2 have 6 peripheral socket, you need DIY to make SATA 1-4 wire be molex, cut it and insert to molex DIY plug.

 

>> Rather than modding the cables couldnt i just order up additional pci-e 6 pin > Molex cables from EVGA? They give you 1 or 2 with the PSU anyway and it has 6 x 6pin peripheral ports on the PSU... wouldnt that work?

 

 

Quote

 

Corsair HX-850, 1000 also support 6 peripheral socket and 1200 is 7 and 5v 30A, but you also need cut those SATA wire be molex.

 

>> Woulnd't this be the same as what i mentioned above. Rather than DIY cables, just order spares from the company of the ones you need? (In my case i'd order 4 extra cables with Molex connectors on them?

 

7 minutes ago, thestraycat said:

Wouldnt this mean on your 16 drives though that you have 1 molex connector per 4 drives?

 

Yes, exactly. 3U and 2U rack case also in this way.

 

7 minutes ago, thestraycat said:

Obviously if your disks wernt as power hungry as mine you'd be ok?

Sure, because I use two PSU, loading are 16 disk and 12 disk. Always 1 wire set for 4 disks. Not one PSU for 28 disk.

 

Edited November 3, 20214 yr by Vr2Io

9 minutes ago, thestraycat said:

4 x 3.3amp disks on a molex connector = 13.2amps

AFAIR the backplanes on my Norco 4224 "clone" have either 2x Molex or 1x Molex + 1 SATA ... I' gather that, with both deployed, the current will be split between the two connected lines and hence send 13,2/2amps through each.

@Vr2Io -Are your disks individually powered? 1 disk = 1 molex connector or are you using a mid plane? It's the midplane that is the issue.

 

@Ford Prefect - Ok. Mine has the same, but the 2nd row of molex connectors on each row of 4 disks are mainly for redundant power supplies, I'm sure you can power them from the same PSU but you would still run out of single wires coming from your PSU without using molex connectors from a previous row to power the next row, which in turn would bring the ampeage up over the 15amp rating for a single wire back to the PSU. Your way would only work for the 1st or 2nd row....after than it's over the ampage of the cable... So as only the 2nd picture "!" is the most efficient. You still need 6 seperate cables back to the PSU to complete the midplane. And i'm assuming your not using 6 to power it?

 

@Vr2Io

@Ford Prefect

 

The only way any of this makes sense is if the maximum ampeage of any single cable coming from the PSU with molex or sata on it can withstand 15amps of current for a short period of time.. Still sketchy at best though. 

 

This is why the only way i can think of doing this is to get a PSU of around 1200w that can take my 79amps of potential 24 disks startup current and mobo/cpu/hba loads and distribute the power to 6 seperate peripheral cables (with a few molex on each cable)  from the PSU to keep the wire loads down to 12.2amps a wire. Only the larger wattage PSU seems to have 6 x peripheral modular connector slots.

Edited November 3, 20214 yr by thestraycat

6 minutes ago, thestraycat said:

@Vr2Io

@Ford Prefect

 

The only way any of this makes sense is if the maximum ampeage of any single cable coming from the PSU with molex or sata on it can withstand 15amps of current for a short period of time.. Still sketchy at best though. 

Another possibility is, that the data sheet I linked previously is simply incorrect.

AFAIR one of my 3TB HDDs, at the time I did the research before purchase will draw about 25W, not 40W as in your case.

 

11 minutes ago, thestraycat said:

@Ford Prefect - Ok. Mine has the same, but the 2nd row of molex connectors on each row of 4 disks are mainly for redundant power supplies, I'm sure you can power them from the same PSU but you would still run out of single wires coming from your PSU without using molex connectors from a previous row to power the next row, which in turn would bring the ampeage up over the 15amp rating for a single wire back to the PSU.

I actually cannot remember how the wiring inside my rig is actually done...running since years as well

But is there not an option to get another, additional set of aux cables for a modular PSU?

21 minutes ago, thestraycat said:

@Vr2Io -Are your disks individually powered? 1 disk = 1 molex connector or are you using a mid plane? It's the midplane that is the issue

Of course backplane, 1 need 4 molex and 1 need 3 molex, all molex are individual wire, so total 7 wire.

Edited November 3, 20214 yr by Vr2Io

thats what i was thinking above. Possibly. Problem is most PSU's <850w only have 4 slots for sticking in peripheral cables ... hence my move to a 1000-1200w psu to get 6 slots for peripheral cables. Much better that they pull 25w! But if your spanning those molex's too thinly you may end up in the same position as me as your PSU degrades really depends on how many of those banks of 4 disks each is powering i suppose.

1 minute ago, Vr2Io said:

Of course backplane, 1 need 4 molex and 1 need 3 molex.

The devils in the detail. I suppose in your case it would all depend on how your distributing the power  between the 2 psu's. If one PSU is responsible on 1 wire to power 4 disks. Your in the same position as me if your disks pull 3.3amps at cold. It would be good to see exactly how everythings wired to be sure. But either way it wont help my conundrum if you've got 2 PSU's. lol. 

Quote

Of course backplane, 1 need 4 molex and 1 need 3 molex, all molex are individual wire, so total 7 wire.

Sounds like you might be fine then. Not an option for me though  Stuck with ATX form factor PSU's

3 minutes ago, thestraycat said:

But either way it wont help my conundrum if you've got 2 PSU's. lol. 

I mention in first, 2 case 2 psu.

 

BTW, above suggested psu support 6 peripheral which suitable for your case, just not all in molex.

Quote

I mention in first, 2 case 2 psu.

Yup.

 

Quote

BTW, above suggested psu support 6 peripheral which suitable for your case, just not all in molex.

Why not all in Molex? I need them all in molex... I understand the PSU wont come with that many but will 4 more wires of molex (including the 2 it comes with) to go to the 1200w psu... its what the midplane takes. Are you just warning me that the PSU's dont come with 6 x Molex modular cables?

6 minutes ago, thestraycat said:

Are you just warning me that the PSU's dont come with 6 x Molex modular cables?

Yes and solution is DIY those SATA cable be Molex, got 6 molex cable is the best.

Edited November 3, 20214 yr by Vr2Io

Yup. I agree. So out of the PSU's i listed above what would you pick? I'm leaning towards the EVGA Supernova 1200w or 1300w

For me, I will got Corsair HX850

 

. Previous post already show 28 disk ( 2 rack ) consum 4xxW, why need a 1300w psu

 

. And I trust Corsair more then EVGA

 

. HX850 cheaper then EVGA 1300W

 

. Best power factor when psu in 50% load

 

. Why not Rmi or HXi, because those support 5 peripheral only

Edited November 3, 20214 yr by Vr2Io