Jump to content
Sign in to follow this  
Devy

unmountable file system after a normal reboot and installing a GPU

7 posts in this topic Last Reply

Recommended Posts

i was running Unraid with just one HDD and one Parity Disk, today after a reboot for installing a new GPU

after the reboot everything was gone, so i checked the main tab.

My Disk one, a WDC one showed "unmountable file system". now i just got pretty much the parity disk and i could install another one, but would it recover the files from the parity drive if i install another disk? because  i think after the array started it tried to do a parity check from the unmountable drive

im really scared now, of the most important things i got backups but it would still hurt me and cost me weeks to rebuild it exactly like it has been

 

the problem in the past with the cache drive was kinda meh but that was my fault for not having an ups.

but i'm really confused with this suddenly unmountable drive

(i also can't moutn it over the unassigned devices plugin)

 

 

here is a log:

Quote

Dec 3 23:02:30 unrawr kernel: CPU: 0 PID: 9500 Comm: mount Tainted: P O 5.3.12-Unraid #1
Dec 3 23:02:30 unrawr kernel: Hardware name: System manufacturer System Product Name/PRIME X470-PRO, BIOS 5406 11/13/2019
Dec 3 23:02:30 unrawr kernel: Call Trace:
Dec 3 23:02:30 unrawr kernel: dump_stack+0x67/0x83
Dec 3 23:02:30 unrawr kernel: xfs_trans_cancel+0x55/0xcd [xfs]
Dec 3 23:02:30 unrawr kernel: xfs_efi_recover+0x195/0x1db [xfs]
Dec 3 23:02:30 unrawr kernel: xlog_recover_process_efi+0x2d/0x3f [xfs]
Dec 3 23:02:30 unrawr kernel: xlog_recover_process_intents+0xd1/0x1b2 [xfs]
Dec 3 23:02:30 unrawr kernel: xlog_recover_finish+0x14/0x85 [xfs]
Dec 3 23:02:30 unrawr kernel: xfs_log_mount_finish+0x5a/0xc3 [xfs]
Dec 3 23:02:30 unrawr kernel: xfs_mountfs+0x50c/0x700 [xfs]
Dec 3 23:02:30 unrawr kernel: ? xfs_mru_cache_create+0x11c/0x13c [xfs]
Dec 3 23:02:30 unrawr kernel: xfs_fs_fill_super+0x498/0x56d [xfs]
Dec 3 23:02:30 unrawr kernel: mount_bdev+0x134/0x183
Dec 3 23:02:30 unrawr kernel: ? xfs_test_remount_options+0x54/0x54 [xfs]
Dec 3 23:02:30 unrawr kernel: legacy_get_tree+0x22/0x3b
Dec 3 23:02:30 unrawr kernel: vfs_get_tree+0x1d/0xc1
Dec 3 23:02:30 unrawr kernel: do_mount+0x6b6/0x76f
Dec 3 23:02:30 unrawr kernel: ? memdup_user+0x3a/0x57
Dec 3 23:02:30 unrawr kernel: ksys_mount+0x71/0x99
Dec 3 23:02:30 unrawr kernel: __x64_sys_mount+0x1c/0x1f
Dec 3 23:02:30 unrawr kernel: do_syscall_64+0x57/0xfd
Dec 3 23:02:30 unrawr kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Dec 3 23:02:30 unrawr kernel: RIP: 0033:0x14da2583225a
Dec 3 23:02:30 unrawr kernel: Code: 48 8b 0d 39 7c 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 06 7c 0c 00 f7 d8 64 89 01 48
Dec 3 23:02:30 unrawr kernel: RSP: 002b:00007ffc89170018 EFLAGS: 00000206 ORIG_RAX: 00000000000000a5
Dec 3 23:02:30 unrawr kernel: RAX: ffffffffffffffda RBX: 000014da259b9f64 RCX: 000014da2583225a
Dec 3 23:02:30 unrawr kernel: RDX: 000000000040d500 RSI: 000000000040d580 RDI: 000000000040d560
Dec 3 23:02:30 unrawr kernel: RBP: 000000000040d2f0 R08: 0000000000000000 R09: 000000000040d535
Dec 3 23:02:30 unrawr kernel: R10: 0000000000000c00 R11: 0000000000000206 R12: 0000000000000000
Dec 3 23:02:30 unrawr kernel: R13: 000000000040d560 R14: 000000000040d500 R15: 000000000040d2f0
Dec 3 23:02:30 unrawr kernel: XFS (sde1): xfs_do_force_shutdown(0x8) called from line 1049 of file fs/xfs/xfs_trans.c. Return address = 000000000336dac2
Dec 3 23:02:30 unrawr kernel: XFS (sde1): Corruption of in-memory data detected. Shutting down filesystem
Dec 3 23:02:30 unrawr kernel: XFS (sde1): Please unmount the filesystem and rectify the problem(s)
Dec 3 23:02:30 unrawr kernel: XFS (sde1): Failed to recover intents
Dec 3 23:02:30 unrawr kernel: XFS (sde1): log mount finish failed
Dec 3 23:02:31 unrawr unassigned.devices: Mount of '/dev/sde1' failed. Error message: mount: /mnt/disks/WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4ARL: mount(2) system call failed: Structure needs cleaning.
Dec 3 23:02:31 unrawr unassigned.devices: Partition 'WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4ARL' could not be mounted...

 

Edited by Devy

Share this post


Link to post

Update

 

 i inserted a new disk, different SATA cable/port and after a rebuild , it showed the new disk as emulated and ran for 7 hours in maintenance mode just reading parity and writing to the other disk

 

 

 now after that it shows again as unmountable file system

Share this post


Link to post
1 minute ago, Devy said:

Update

 

 i inserted a new disk, different SATA cable/port and after a rebuild , it showed the new disk as emulated and ran for 7 hours in maintenance mode just reading parity and writing to the other disk

 

 

 now after that it shows again as unmountable file system

If a disk that is being emulated shows as ‘unmountable’ before a rebuild it will show the same after completing the rebuild as the rebuild process is simply making the physical disk correspond to the emulated one.    If you have not previously successfully formatted the drive this will still be required.   Note, however, that a format wipes any existing data on a drive so should not be used if a drive that was previously fine suddenly goes unmountable (in such a case a file system repair is the way forward).

 

the diagnostics that you posted showed you were getting continual resets on the drive which is probably why the format never completed!   Such resets are typically cabling related (although other causes are possible), so hopefully the change you made has rectified the issue for this drive.

 

you should probably post new diagnostics now you have made a change so any feedback can be based on the current setup..

Share this post


Link to post

Hello. just to make sure my previous Steps:

 

- Reboot and noticed that the file system is suddenly unmountable

- Stopped the Array

- Started in Maintenance Mode
- Checked with this -n command the xfs_repair but it gave a lot of errors

 

- Got another Disk that i plugge din on another sata cable on another Sata Port ( got the old one still plugged in )

- in Maintenance mode i did a data_rebuild that went for 7 hours that lasted the whole night, in the morning it was done but still unmountable

 

Disk Log from the new Disk:

Quote

ErrorWarningSystemArray


Dec 3 22:45:01 unrawr kernel: ata3: SATA max UDMA/133 abar m131072@0xfcd80000 port 0xfcd80200 irq 53
Dec 3 22:45:01 unrawr kernel: ata3: SATA link down (SStatus 0 SControl 300)
Dec 3 23:35:28 unrawr kernel: ata3: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Dec 3 23:35:28 unrawr kernel: ata3.00: ATA-9: WDC WD30EFRX-68EUZN0, WD-WCC4N5ZA4578, 82.00A82, max UDMA/133
Dec 3 23:35:28 unrawr kernel: ata3.00: 5860533168 sectors, multi 0: LBA48 NCQ (depth 32), AA
Dec 3 23:35:28 unrawr kernel: ata3.00: configured for UDMA/133
Dec 3 23:35:28 unrawr kernel: sd 3:0:0:0: [sdf] 5860533168 512-byte logical blocks: (3.00 TB/2.73 TiB)
Dec 3 23:35:28 unrawr kernel: sd 3:0:0:0: [sdf] 4096-byte physical blocks
Dec 3 23:35:28 unrawr kernel: sd 3:0:0:0: [sdf] Write Protect is off
Dec 3 23:35:28 unrawr kernel: sd 3:0:0:0: [sdf] Mode Sense: 00 3a 00 00
Dec 3 23:35:28 unrawr kernel: sd 3:0:0:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 3 23:35:29 unrawr kernel: sdf: sdf1
Dec 3 23:35:29 unrawr kernel: sd 3:0:0:0: [sdf] Attached SCSI disk
Dec 3 23:36:50 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:37:11 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:37:11 unrawr kernel: mdcmd (3): import 2 sdf 64 2930266532 0 WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578
Dec 3 23:37:11 unrawr kernel: md: import disk2: (sdf) WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 size: 2930266532
Dec 3 23:37:26 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:37:26 unrawr kernel: mdcmd (3): import 2 sdf 64 2930266532 0 WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578
Dec 3 23:37:26 unrawr kernel: md: import disk2: (sdf) WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 size: 2930266532
Dec 3 23:37:32 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:37:38 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:37:38 unrawr kernel: mdcmd (2): import 1 sdf 64 2930266532 0 WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578
Dec 3 23:37:38 unrawr kernel: md: import disk1: (sdf) WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 size: 2930266532
Dec 3 23:38:06 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:38:06 unrawr kernel: mdcmd (2): import 1 sdf 64 2930266532 0 WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578
Dec 3 23:38:06 unrawr kernel: md: import disk1: (sdf) WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 size: 2930266532
Dec 3 23:38:43 unrawr emhttpd: WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 (sdf) 512 5860533168
Dec 3 23:38:43 unrawr kernel: mdcmd (2): import 1 sdf 64 2930266532 0 WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578
Dec 3 23:38:43 unrawr kernel: md: import disk1: (sdf) WDC_WD30EFRX-68EUZN0_WD-WCC4N5ZA4578 size: 2930266532

 

now i'm really unsure on what to do now,

 

the Disk Log from the Parity

 

Quote

ErrorWarningSystemArray


Dec 3 22:45:01 unrawr kernel: ata1: SATA max UDMA/133 abar m131072@0xfcd80000 port 0xfcd80100 irq 53
Dec 3 22:45:01 unrawr kernel: ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Dec 3 22:45:01 unrawr kernel: ata1.00: ATA-9: WDC WD40EFRX-68WT0N0, WD-WCC4E3YCCVZY, 82.00A82, max UDMA/133
Dec 3 22:45:01 unrawr kernel: ata1.00: 7814037168 sectors, multi 16: LBA48 NCQ (depth 32), AA
Dec 3 22:45:01 unrawr kernel: ata1.00: configured for UDMA/133
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] 7814037168 512-byte logical blocks: (4.00 TB/3.64 TiB)
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] 4096-byte physical blocks
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] Write Protect is off
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Dec 3 22:45:01 unrawr kernel: sdb: sdb1
Dec 3 22:45:01 unrawr kernel: sd 1:0:0:0: [sdb] Attached SCSI disk
Dec 3 22:45:09 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 22:45:09 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 22:45:09 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 22:45:46 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 22:45:46 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 22:45:46 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 22:51:15 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 22:51:15 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 22:51:15 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:08:13 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:08:13 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:08:13 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:35:23 unrawr kernel: ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
Dec 3 23:35:23 unrawr kernel: ata1.00: configured for UDMA/133
Dec 3 23:36:50 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:36:50 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:36:50 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:37:11 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:37:11 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:37:11 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:37:26 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:37:26 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:37:26 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:37:32 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:37:32 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:37:32 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:37:38 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:37:38 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:37:38 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:38:06 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:38:06 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:38:06 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532
Dec 3 23:38:43 unrawr emhttpd: WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY (sdb) 512 7814037168
Dec 3 23:38:43 unrawr kernel: mdcmd (1): import 0 sdb 64 3907018532 0 WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY
Dec 3 23:38:43 unrawr kernel: md: import disk0: (sdb) WDC_WD40EFRX-68WT0N0_WD-WCC4E3YCCVZY size: 3907018532

 

 

The unassigned Devices that i got all work without any issue. i still want to try and not lose all of the data and i'm a bit clueless on what steps to take next.

This is the Current state, the one in Unassigned Devices is the old One that gave ma all the Troubles and the one above sdd is the one that i uses to replace it

 

Bildschirmfoto 2019-12-04 um 09.22.35.png

Edited by Devy

Share this post


Link to post

I also got the log of the repair with -n and it doesnt look "healthy" that was the newer drive that i placed into the system AFTER the data-rebuild.

 

warning: there is a lot of text.

 

the one of the drive before looked similiar, guess there isnt much hope which is pretty sad :(

 

 

xfsrepair.log

Share this post


Link to post

okay i tried that and in fact only 100GB are missing in Total, i did this on the new Disk and it looks pretty good so far, the iso folder is missing and one docker, there can't be much else that is missing

i mean its a bit annoying to reconfigure the proxy manager and dump some gpu bios but thats nothing compared to if everything was lost

 

will anothe rprogram get any more success out of this? like this ufs explorer i'm reading in some posts about?

 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

Sign in to follow this