Another unmountable disk problem

[NoJokeBeatz]

I have been running into this unmoutable disk problem one disk at a time. However this last one just stopped mounting last night. I was able to mount it a couple times after the -L check, but it is not mounting anymore. It is being emulated but it says it is disabled. Here is my last Check -L:

Phase 1 - find and verify superblock... Phase 2 - using internal log - zero log... - scan filesystem freespace and inode maps... - found root inode chunk Phase 3 - for each AG... - scan and clear agi unlinked lists... - process known inodes and perform inode discovery... - agno = 0 - agno = 1 - agno = 2 - agno = 3 - agno = 4 - agno = 5 - agno = 6 - agno = 7 - process newly discovered inodes... Phase 4 - check for duplicate blocks... - setting up duplicate extent list... - check for inodes claiming duplicate blocks... - agno = 0 - agno = 1 - agno = 4 - agno = 6 - agno = 3 - agno = 7 - agno = 5 - agno = 2 Phase 5 - rebuild AG headers and trees... - reset superblock... Phase 6 - check inode connectivity... - resetting contents of realtime bitmap and summary inodes - traversing filesystem ... - traversal finished ... - moving disconnected inodes to lost+found ... Phase 7 - verify and correct link counts... Maximum metadata LSN (10:22) is ahead of log (1:2). Format log to cycle 13. done

 

Attaching my diagnostics as well. What is going wrong and what are next steps?

wakanda-diagnostics-20220112-2044.zip

[trurl]

36 minutes ago, NoJokeBeatz said:

It is being emulated but it says it is disabled

Emulated/Disabled are the same. Mountable/Unmountable is something else.

 

Looks like emulated/disabled disk8 was mounting according to syslog, but it would be better if you had posted diagnostics with the array started so I could see for sure and how much data it might have.

 

Connection problems with multiple disks.

 

Marvell controllers are NOT recommended.

06:00.0 SATA controller [0106]: Marvell Technology Group Ltd. 88SE9215 PCIe 2.0 x1 4-port SATA 6 Gb/s Controller [1b4b:9215] (rev 11)

 

[NoJokeBeatz]

Thank you so much for your response. What controller would be better?

wakanda-diagnostics-20220113-2113.zip

[ChatNoir]

3 hours ago, NoJokeBeatz said:

What controller would be better?

 

[NoJokeBeatz]

Thank you both. Anything turn up on the new diagnostic?

[trurl]

SMART for disk8 looks OK, but emulated disk8 isn't mounting, probably can't be accurately emulated due to those connection problems on multiple disks.

 

Do you have enough ports without that Marvell?

 

If you don't need hardware passthru for any VMs you might try disabling IOMMU (AMD-Vi) in BIOS to see if Marvell will work better.

[NoJokeBeatz]

I am going to replace the Marvell. Let me see if that helps anything.

[NoJokeBeatz]

I replaced the controller and it is running. However the re-build is taking increasi9ngly longer. Up to 222 days now.

wakanda-diagnostics-20220115-1905.zip

[itimpi]

Y

1 hour ago, NoJokeBeatz said:

I replaced the controller and it is running. However the re-build is taking increasi9ngly longer. Up to 222 days now.

wakanda-diagnostics-20220115-1905.zip 170.68 kB · 0 downloads

You are getting continual resets on disk4 which suggests a cabling issue with the drive.

[NoJokeBeatz]

Replaced the cables and still drives start crashing the entire server. I think the bad equipment isn't on the array anymore. Do you see anymore problems? Thanks a lot.

wakanda-diagnostics-20220117-2352.zip

[trurl]

10 hours ago, NoJokeBeatz said:

crashing the entire server

setup syslog server

[NoJokeBeatz]

Alright, syslog set up locally and not on cache drive.

wakanda-diagnostics-20220118-1924.zip

[trurl]

Connection problems with parity and disk2

[NoJokeBeatz]

After a week it seems removing nearly 5 drives stabilized things. All that data gone but there haven’t been any real problems so far. That you all.