## System info

- Unraid version: 7.0.1 (per web UI)

- Array: 7 data/parity devices, ~25TB total, XFS on each disk

- Cache pool: separate single SSD (not implicated in this issue, confirmed healthy)

## Timeline of events

1. Ran a du -sh /mnt/user/*/ command directly at the shell to check share sizes. This is a large array with many TB of data.

2. Some time later (exact duration unclear — I didn't watch it start-to-finish), the Unraid web UI became completely unreachable ("connection refused"), as did SSH ("connection refused", later "no route to host").

3. Docker containers running on the box (confirmed via direct container port access, bypassing the web UI) continued to respond normally throughout — e.g. a Syncthing container and another app container were both still reachable on their own ports.

4. No IPMI/BMC available on this hardware, no monitor connected at the time. Waited approximately 10+ minutes total with no recovery of SSH/web UI.

5. Pressed the power button briefly (intending a graceful ACPI shutdown) — system powered off almost immediately, which did not look like a normal graceful shutdown sequence (no extended drive activity burst, no gradual fan spin-down).

6. Powered back on. After it failed to come back on the network within a few more minutes, did a second forced power-off (held button ~5-10 sec) and power-on cycle after failing to get any video signal on a monitor connected mid-boot (probably too late to be detected).

7. Eventually the web UI came back up. Array had auto-started and begun a parity check due to the unclean shutdown(s).

8. Noticed 3 disks (2 array disks + the cache SSD) showing SMART "error" indicators on the Dashboard tab (though Main tab showed all green / DISK_OK). Pulled SMART data on all three — all showed 0 reallocated sectors, 0 pending sectors, only small UDMA_CRC_Error_Count values (1-3) on the two array disks. Interpreted this as likely power-cycle artifacts rather than genuine media failure, since the parity check was still running and no disk was disabled/red.

9. Attempted to access a Syncthing-synced folder that lives on this array (not the cache pool) and got Input/output error on every single file in the directory listing — including on a ls -la of the folder root itself, which showed files with completely unreadable permissions/ownership -????????? ? ? ? ? ?).

10. Checked dmesg and found repeated XFS metadata I/O errors (see below) on two array devices, md1p1 and md2p1.

11. Parity check eventually completed: "Parity is valid... Finding 975,823,662 errors" (i.e., it ran in default/correcting mode and rewrote parity to match current disk contents — I understand this does NOT mean the underlying data/filesystem is actually correct, just that parity now matches whatever is currently on the data disks, right or wrong).

12. Checked the Dashboard again later and now see elevated error counts on 4 disks (not just the original 2), with those 4 disks showing an unreadable temperature *) — only Parity + 2 other disks show clean temps and zero errors.

## Relevant dmesg output

```

[81288.467022] XFS (md1p1): metadata I/O error in "xfs_da_read_buf+0x9f/0x104" at daddr 0x15d535bc0 len 8 error 5

[81288.473872] XFS (md1p1): metadata I/O error in "xfs_da_read_buf+0x9f/0x104" at daddr 0x246917a40 len 8 error 5

[81339.298418] XFS (md2p1): metadata I/O error in "xfs_da_read_buf+0x9f/0x104" at daddr 0x140362100 len 8 error 5

[81339.298562] XFS (md2p1): metadata I/O error in "xfs_da_read_buf+0x9f/0x104" at daddr 0x17a7c1b90 len 8 error 5

... (many more of the same pattern, repeating on both md1p1 and md2p1)

```

All instances are error 5 (EIO) from xfs_da_read_buf, recurring continuously (log shows "XFS: metadata IO error: 706 callbacks suppressed" style throttling messages between bursts).

## Current disk status (from Dashboard tab)

| Disk | SMART | Errors | Temp |

|---|---|---|---|

| Parity | healthy | 0 | 99°F |

| Disk 1 | error | 976,483,134 | * |

| Disk 2 | error | 488,041,243 | * |

| Disk 3 | error | 976,482,966 | * |

| Disk 4 | error | 243,718,249 | * |

| Disk 5 | healthy | 0 | 91°F |

| Disk 6 | healthy | 0 | 90°F |

(Note: Disk 1 and Disk 2 were the ones that showed actual XFS corruption in dmesg above. Disks 3 and 4 developed high error counts more recently and I have not yet gotten fresh SMART/dmesg data specifically on those two — will pull and add that if helpful.)

## What I've done so far (nothing destructive)

- Have NOT run xfs_repair or any other repair/write command on any of the affected disks.

- Stopped all Docker containers that were reading/writing to the affected array paths.

- Paused the relevant Syncthing sync folder (both on this box and its remote peer) to stop any further I/O against the affected paths.

- Failed over the affected services (Paperless-ngx, a budgeting app) to a second physical Unraid server I run, restoring from the most recent good exports/backups I had — so there's no time pressure to rush a repair, I just want to do it correctly.

## Questions

1. Given the pattern above (unclean shutdown → XFS metadata I/O errors on specific disks → parity check ran in correcting mode afterward), what's the correct order of operations from here? My understanding is I should NOT have let the parity check run before addressing the filesystem corruption, since it may have "corrected" parity to match already-corrupted data — is that right, and does it matter at this point since it's already completed?

2. Is xfs_repair (via Unraid's own maintenance mode / xfs_repair GUI tool) the appropriate next step for Disk 1 and Disk 2? Should I be concerned that Disk 3 and Disk 4 are now also showing high error counts, or is that possibly a knock-on effect from the parity check itself (since it was reading everything)?

3. Given all four affected disks currently show temp * (unreadable) on the Dashboard — is that itself meaningful, or just a reporting quirk?

Appreciate any guidance — I'd rather move carefully and correctly than fast, given what's on this array.

server-diagnostics-20260816-1743.zip

There are errors with multiple disks, I also see you are using a controller with SATA port multipliers. Before trying to repair the filesystem, you need ot resolve those errors, and ideally replace that controller https://forums.unraid.net/topic/102010-recommended-controllers-for-unraid/

Thank you for pointing that out — I hadn't realized the controller itself could be the root cause rather than just the disks/filesystem.

I've ordered a replacement — Internal 5 Port Non-Raid SATA III 6GB/S Pci-E X4 Controller Card for Desktop PC Support SSD and HDD with Low Profile Bracket. JMB585 Chipset SI-PEX40139 and should arrive late this evening.

Plan is to swap the controller first, reconnect the affected drives, and confirm Unraid recognizes everything cleanly before attempting any filesystem repair on the disks with the XFS metadata errors. Will report back once the card's in and I have fresh diagnostics.

Appreciate the guidance — will update this thread once the new controller is in.

I would recommend first posting new diags with the new controller (after array start)

Wanted to give an update, I replaced the card but I encountered a verified broken sata port on the card. The good thing is when I initially started the server, it registered the other drives previously not registered except the broken port drive. I have a replacement card ordered and should be here Wednesday. To be continued....

Controller came in a day earlier. I just installed a new GLOTRENDS 6-port card built on the ASM1166 chipset (also native, non-multiplier), which fit the same PCIEX4 slot as a straight swap.

Just finished installing it and starting the array:

• All 6 array members (Parity + Disk1–Disk6) are recognized cleanly by the new controller, all showing green/healthy

• Disk1 and Disk2 (the two that had the original XFS corruption) are both present and assigned correctly — no "missing" or unassigned devices this time

• Array started normally with no parity rebuild triggered, confirming Unraid recognized all drives by their existing identity rather than treating anything as new/replaced

• Disk3 and Disk4 (the ones that may have had secondary/knock-on corruption) are also present and healthy

Fresh diagnostics are attached, generated after array start per your instructions.

Given everything's now on a clean, native controller with no multiplier in the chain, what's the recommended next step for addressing the XFS corruption on Disk1 and Disk2 (and confirming whether Disk3/Disk4 need attention too)? Want to make sure I approach the repair correctly if required at all.

Thanks again for the guidance so far — the controller swap was clearly the right call.

server-diagnostics-20260818-1325.zip