buster84

Update 10/11/23 (found the real reason disks were being disabled) Shortly after my last post i got a power supply on sale and threw it on. I also remembered that you said that this whole issue could be my power supply, so I thought maybe you were right.. So i did more testing and moved all 8 14TB exos drives back onto the LSI. I then copied over all my data and its been working 100% normally for almost 3 weeks now. No disabled disks, write errors or anything with 2 party drives. This is my advice for anyone getting disabled disks in unraid. Change your power supply first, it’s probably on its way out. I spent so much time troubleshooting when the entire time it was actually the power supply. I hope this post saves others from losing their data and saving time on troubleshooting.

Just updating the post. Its been almost a week now and this is what i noticed. When i had 4 14tb drives connected to the lsi disk 3 got disabled again shortly after starting my data copy. I then moved disk 3 to the motherboard sata connector and rebuilt the drive, its been running for 5 days now without any issues at all. So either my card caps rejects 56gb of capacity and only accepts 42gb in capacity or my card is defective or it got damaged from not having the proper cooling. At this point im happy with where the setup is and i think ive trouble shooted this to the max. I still have 1 more free sata slot on my motherboard if needed. If i run into any more issues, im immediately buying the 9300-16i. Hopefully this post will help anyone else in the future as i tried looking up the lsi 9207-8i with high capacity drives and came up short.

Thanks everyone for their reply's. Their was a hardare issue with my lsi, but i think ive fixed it by throwing a fan on it. If i get errors again im buying a sas 9300. As for zfs, i decided to just do xfs again for now and tossed some hard drives in my old readynas and im using that to backup all my important data. MrGrey thanks for explaining parity and backup. I was using the term backup in regards to parity. I didnt need a true backup, I just wanted a way to recover the data if a hdd or 2 failed and it looks like parity works for me that way on zfs or xfs. Im a little confused with this. I actually tested it, zfs 6 drives with no pool and parity built fine. Or were you refering to the snapshots not spanning multiple drives?

I think i figured out the issue. So far everything has been working fine since i increased my internal case fan speeds. I also threw a 40mm fan on the lsi heat sync yesterday and its doing much better. Ive been able to copy data back over without an errors popping up like before. I'm starting to think the issue was that it was overheating since it was made for a server not a desktop. Thank you for all your help, if it happens again ill update the thread and buy the 9300-8i or a 9300-16i.

Ok thank you, thats a relief. Time to go shopping. edit: In my search for a 9300 i came accross a thread talking about the sas controllers overheating. I took a thermal gun and noticed that its at 180-190f. I read max is 55c of 131f so its definitly overheating at the heatsync. I decided to buy a small fan for the card and see if it makes a difference. It'll arrive tomorrow. I know these cards have issues on their own, but im wondering if maybe its caused by overheating. I cant return this card so i figure ill give this a try first and report back. edit2: Interesting. I turned all my fans on high and now its reading around 100-130f. Still hot, but the parity check isnt showing an errors this time and the speed is at 250mbps vs 150mbps before when i started parity.

I'll be on the lookout for one. Thank you. quick question, i contacted seagate because my read error rate was 14million and they said i can swap it out for a refurbished on since its under warranty, should i swap it out or is the hard drive interpreting the erros from the LSI and not real errors?

Well i didn’t get far in the parity rebuild and disk 3 became disabled. I then swapped the cords with another disk to see if it followed the cord and it didn’t. I then noticed more disk read errors so i didn’t do anything after that except run the smart extended test. took 20ish hours and passed. I realize now that in my old setup that worked without any issues had 6 14tb drives on the sas motherboard controller and only 3tb drives on the lsi, so I’m thinking your right that this controller is failing to work with the 14tb drives. I’m going to move at least 6 of them to my motherboard then leave the last 2 on the lsi and see if maybe it can handle just 2 drives for now. I’m going to have to order another one and do some research to make sure i get a sas3 version. Do you have any recommendations for any specific cards to look for since i thought i bought a sas3 3.0card already (lsi2308)? Im also posting my logs encase you see something else. nas-smart-20230917-0651 (drive3).zip nas-diagnostics-20230917-1004 (9:17).zip

Ok so to test i pulled 2 drives off the lsi card and removed one of the cords. I only have 4 drives on the lsi now. I then tried to do xfs_repair and realized their formated zfs LOL. I now see that doing it this way may not be the best. I either do all xfs, or do a xraidz2. Is there a repair method for zfs like xfs? If not I'll just reformat everything and do the pool again. edit: went ahead and did xfs, leting the parities rebuild now.

Ok, wasnt sure, but i copied a little data over to test and disk 3 got disabled. Was this the LSI? or another error? I thought at first maybe the power supply, but now im really suspecting the LSI card and might just buy another one as i cant even restore my server with it continuing to happen. Another thought is maybe the lsi to sasa cable is bad. nas-diagnostics-20230915-1439.zip

Ok. using terminal didnt work. I force restarted it. I then tookout the "LSI" card and checked the thermal paste. Im not sure if it was completly dried out or a specific type that was used, but it was rock solid, had to use a plastic card to scrape it off the heat synce and the chip die along with soaking it in alcohol. Eventually i cleaned it up and threw on new thermal paste. Hopefully this fixes my issues, if not the next step will be a new LSI card that is sas3 (wish i realized i was buying a sas2, need to look out for that next time). After this error should i re-run parity or since it say's that its valid does that mean it wrote corrections to the drive if it needed them already?

Ok, after a few days i became indeciive. I went back to xfs and ran the duel parity check. Then deleted that, made every drive zfs but without pools, then redid parity. Finished but this time one of the disks had IO error. Im going to run a full smart extended test on this once i figure out how to restart the server, so far i havnt been able to shut the system down or stop the array, its stuck at stopping. In out previous conversation i updated the firmware on the lsi sas and now with this new error im wondering what else it could be? Is this truly hardware that i need find out that is failing or maybe a bug with zfs? Also without forcfully turning it off with the power button, is there anyway to force a shut down in terminal? Its stuck right now. I posted the diagnostics. nas-diagnostics-20230915-1024.zip

long story short. I put on a second parity and somehow it messed everything up and i kept getting drives disabled, one after another even though they were fine and passed the smart tests. This caused me to lose some data because I didn’t want to go through the lost and found and the thousands of folders it was put in. I did pull out all my photos and videos, but the rest was so unorganized I deleted it. Since then, I backed up my server and now I have 8 14tb exos drives waiting on a file system. I was going to do Raidz2 but then after I formatted it I realized I couldn’t use my cache drive anymore, That brings me to this forum to ask for your opinions and ask a few questions. I like the idea of snapshots to keep all my data organized and backed up, i like that raidz2 lets 2 drives fail just like dual parity would. What I dislike about parity is that it destroys the folder layout and file system organization if I lose parity and have to rebuild it. 1) if I go back to XFS is there a plugin that snapshots the files and folder systems that way if I lose my parity or drives get disabled again I can recover the data exactly how it was originally? Or a sway to snapshot with xfs? 2) If i go ZFS and do a pool for raidz2? is there a way to use a use cache drive? 3) would there be any benefit to using zfs on all individual drives, with parity? Can you use snapshots with it like this? Would I be able to create a share that spans across multiple zfs drives not combined in a pool?

Perfect thank you!

Ok that makes sense. so im left with option 1 and 3. Witch one would have faster read/write speeds? Option 1 with raid0 and duel parity or option 3 using raidz2?

That fixed it! thank you. Since im here, Which setup would be best for access to 2 future drive failures. 1) 6x 14tb with 2 14tb parity drives 2) 7x 14tb in Raidz1 with 1 parity drive and one zfs drive failure? 3) 8x 14tb in Raidz2