Nothing but problems for weeks now. Spent over $800 on new controllers, disks, and cables and no better than I was before


timekiller

Recommended Posts

I have a very large array - 24 drives, all 10,12, or 14 TB. A couple of weeks again I started having issues with the parity drives getting disabled. I was told to ditch my Marvel based controllers, which I did. I bought two LSI 9201-16i cards and installed them. I still had issues. I was told my firmware is very old, so I updated to the latest. I thought maybe the drives were faulty, so I bought new 14TB drives for parity. I've swapped my cables I don't even know how many times at this point. I moved the server out of my network closet to a cooler room.

 

Every time I made a change I had to start the parity build over. With this many drives it takes several days to complete. Most of the time, it doesn't complete. It did complete once, and a few hours after completion a data drive was disabled, taking all it's data offline.

 

Yesterday I moved the server and updated the controller firmware. Now I have read errors on 3 drives and unraid has disabled one of the parity drives.

 

I'm pulling my hair out with frustration. I've swapped out all the hardware short of building a new server from scratch. I need to nail down the actual issue and get my server back.

 

Please help!

 

diag attached

storage-diagnostics-20211121-1156.zip

Link to comment
11 minutes ago, timekiller said:

No. The drive was "disabled" and all of it's contents were removed from the array. Not emulated.

If you have parity, then a disabled disk is emulated from the parity calculation be reading parity plus all other disk. If the emulated disk was unmountable then its contents wouldn't be available but filesystem repair probably would have made it mountable again.

Link to comment
5 minutes ago, trurl said:

If you have parity, then a disabled disk is emulated from the parity calculation be reading parity plus all other disk. If the emulated disk was unmountable then its contents wouldn't be available but filesystem repair probably would have made it mountable again.

I understand how it's supposed to work. Both parity drives were online. The disabled drive was no emulated. Don't know what else to tell you on that. Doesn't really matter since this was like 3 weeks ago and not relevant to my current issues.

Link to comment
7 minutes ago, timekiller said:

I understand how it's supposed to work. Both parity drives were online. The disabled drive was no emulated. Don't know what else to tell you on that. Doesn't really matter since this was like 3 weeks ago and not relevant to my current issues.

This thread where the emulated drive was unmountable?

 

Link to comment
2 hours ago, trurl said:

This thread where the emulated drive was unmountable?

 

Yup. My mistake. The data drive was marked unmountable, not disabled. The 2 parity drives were still online and the data on the unmountable drive was missing fromt he array. That was 3 weeks ago and I've been dealing with so many issues since then I forgot the specific error for the drive. This doesn't answer my current problem though. New drives, new cables, new controller cards, and I still can't get a successful parity rebuild.

Link to comment
2 hours ago, Michael_P said:

 

Any splitters?

 

2 hours ago, Michael_P said:

 

Any splitters?

a couple, just due to the runs I have. Not as many as you might be thinking. It's a 1,000 watt PSU so there are plenty of ports for peripherals. I really on't think the PSU or the way I have the power wired is the issue. 

Link to comment
7 hours ago, timekiller said:

I really on't think the PSU or the way I have the power wired is the issue.

 

Splitting power to too many drives can cause power to sag, and cause the issues you describe when all drives are under load. The wattage of the PSU is plenty sufficient, but power handing per connector may not be (4 drives per connector is pushing it, for example). Eliminate the splitters and see if it solves your problem.

Link to comment
3 hours ago, Michael_P said:

 

Splitting power to too many drives can cause power to sag, and cause the issues you describe when all drives are under load. The wattage of the PSU is plenty sufficient, but power handing per connector may not be (4 drives per connector is pushing it, for example). Eliminate the splitters and see if it solves your problem.

of the 24 drives, there is a grand total of 2 splitters (not daisy chained, and not on the same line). Again, I don't believe this is causing the issue.

Link to comment
5 minutes ago, timekiller said:

I don't believe this is causing the issue.

Understandable, but everything you have posted about the issue points to a power delivery problem. Until that's ruled out by part substitution, it's going to be an issue on the table.

 

17 hours ago, timekiller said:

EVGA SuperNOVA 1000 G+, 80 Plus Gold.

New doesn't mean it's good. Plenty of partially defective parts get shipped every day.

 

Also, those modular cable assemblies can have connector issues, so it wouldn't hurt to pull out and reconnect every power cable connection, especially the ones at the PSU itself.

 

When swapping a PSU, make sure ALL cables are swapped, since modular cables often aren't compatible between different PSU's, sometimes even of the same brand, and also the cables themselves can be the problem.

Link to comment

Posting an update in case it's helpful to others. I believe I have this solved. I realized that the drive issues I was having were all on drives in this drive cage I bought to try and squeeze an extra drive into the server. I removed the cage, rearranging drives to get everything mounted securely and started yet another parity rebuild. This one took about 37 hours and completed successfully last night. The array has been online and stable since then. Hopefully That was the source of all my issues and I can move on.

 

Thank you everyone who gave advice, even if I didn't necessarily take all of it.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.