September 3, 20241 yr Hi all, My system does not start up properly anymore after a reboot after I found that a scrub on disk2 did not want to finish. The array hangs on mounting disk2. I also cannot download the logs, because it is trying to get the pool status which is not working. Is threre a CLI command that I can use? Can you help me find the issue?
September 3, 20241 yr Community Expert Please post the diagnostics, or if that doesn't work, at least the syslog: cp /var/log /boot/syslog.txt
September 3, 20241 yr Author Hi JorgeB, I have the logs from a in Maintenance started array, because I cannot start the array. unraid-diagnostics-20240903-1039.zip Edited September 3, 20241 yr by TallMan206
September 3, 20241 yr Author Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sdd Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sde Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sdb Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sdf Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sdc Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/nvme1n1 Sep 3 11:47:56 UnRAID emhttpd: read SMART /dev/sda Sep 3 11:47:56 UnRAID SysDrivers: SysDrivers Build Starting Sep 3 11:48:06 UnRAID Parity Check Tuning: Unclean shutdown detected Sep 3 11:48:06 UnRAID Parity Check Tuning: Send notification: Unclean shutdown detected: See **Troubleshooting** section of the Unraid OS Manual in the online documetaion to get guidance on resolving this (type=alert link=) Sep 3 11:48:06 UnRAID kernel: mdcmd (31): set md_num_stripes 1280 Sep 3 11:48:06 UnRAID kernel: mdcmd (32): set md_queue_limit 80 Sep 3 11:48:06 UnRAID kernel: mdcmd (33): set md_sync_limit 5 Sep 3 11:48:06 UnRAID kernel: mdcmd (34): set md_write_method 1 Sep 3 11:48:06 UnRAID kernel: mdcmd (35): start DISABLE_DISK Sep 3 11:48:06 UnRAID kernel: unraid: allocating 20870K for 1280 stripes (4 disks) Sep 3 11:48:06 UnRAID kernel: md1p1: running, size: 19531825100 blocks Sep 3 11:48:06 UnRAID kernel: md2p1: running, size: 15625879500 blocks Sep 3 11:48:07 UnRAID emhttpd: shcmd (58): udevadm settle Sep 3 11:48:07 UnRAID emhttpd: Opening encrypted volumes... Sep 3 11:48:07 UnRAID emhttpd: shcmd (59): touch /boot/config/forcesync Sep 3 11:48:08 UnRAID emhttpd: Mounting disks... Sep 3 11:48:08 UnRAID emhttpd: mounting /mnt/disk1 Sep 3 11:48:08 UnRAID emhttpd: shcmd (60): mkdir -p /mnt/disk1 Sep 3 11:48:08 UnRAID emhttpd: /usr/sbin/zpool import -f -d /dev/md1p1 2>&1 Sep 3 11:48:11 UnRAID emhttpd: pool: disk1 Sep 3 11:48:11 UnRAID emhttpd: id: 4076012607772953184 Sep 3 11:48:11 UnRAID emhttpd: shcmd (61): /usr/sbin/zpool import -f -N -o autoexpand=on -d /dev/md1p1 4076012607772953184 disk1 Sep 3 11:48:16 UnRAID emhttpd: shcmd (62): /usr/sbin/zpool online -e disk1 /dev/md1p1 Sep 3 11:48:17 UnRAID emhttpd: /usr/sbin/zpool status -PL disk1 2>&1 Sep 3 11:48:17 UnRAID emhttpd: pool: disk1 Sep 3 11:48:17 UnRAID emhttpd: state: ONLINE Sep 3 11:48:17 UnRAID emhttpd: scan: scrub repaired 0B in 12:11:54 with 0 errors on Mon Sep 2 00:12:08 2024 Sep 3 11:48:17 UnRAID emhttpd: config: Sep 3 11:48:17 UnRAID emhttpd: #011NAME STATE READ WRITE CKSUM Sep 3 11:48:17 UnRAID emhttpd: #011disk1 ONLINE 0 0 0 Sep 3 11:48:17 UnRAID emhttpd: #011 /dev/md1p1 ONLINE 0 0 0 Sep 3 11:48:17 UnRAID emhttpd: errors: No known data errors Sep 3 11:48:17 UnRAID emhttpd: shcmd (63): /usr/sbin/zfs set mountpoint=/mnt/disk1 disk1 Sep 3 11:48:17 UnRAID emhttpd: shcmd (64): /usr/sbin/zfs set atime=off disk1 Sep 3 11:48:17 UnRAID emhttpd: shcmd (65): /usr/sbin/zfs mount disk1 Sep 3 11:48:17 UnRAID emhttpd: shcmd (66): /usr/sbin/zpool set autotrim=off disk1 Sep 3 11:48:17 UnRAID emhttpd: shcmd (67): /usr/sbin/zfs set compression=on disk1 Sep 3 11:48:18 UnRAID emhttpd: mounting /mnt/disk2 Sep 3 11:48:18 UnRAID emhttpd: shcmd (68): mkdir -p /mnt/disk2 Sep 3 11:48:18 UnRAID emhttpd: /usr/sbin/zpool import -f -d /dev/md2p1 2>&1 Sep 3 11:48:21 UnRAID emhttpd: pool: disk2 Sep 3 11:48:21 UnRAID emhttpd: id: 570875432498759471 Sep 3 11:48:21 UnRAID emhttpd: shcmd (69): /usr/sbin/zpool import -f -N -o autoexpand=on -d /dev/md2p1 570875432498759471 disk2 Sep 3 11:48:25 UnRAID kernel: VERIFY3(0 == dmu_buf_hold_array(os, object, offset, size, FALSE, FTAG, &numbufs, &dbp)) failed (0 == 5) Sep 3 11:48:25 UnRAID kernel: PANIC at dmu.c:1123:dmu_write() Sep 3 11:48:25 UnRAID kernel: Showing stack for process 17758 Sep 3 11:48:25 UnRAID kernel: CPU: 26 PID: 17758 Comm: txg_sync Tainted: P O 6.1.106-Unraid #1 Sep 3 11:48:25 UnRAID kernel: Hardware name: Dell Inc. PowerEdge R720/0W7JN5, BIOS 2.9.0 12/06/2019 Sep 3 11:48:25 UnRAID kernel: Call Trace: Sep 3 11:48:25 UnRAID kernel: <TASK> Sep 3 11:48:25 UnRAID kernel: dump_stack_lvl+0x44/0x5c Sep 3 11:48:25 UnRAID kernel: spl_panic+0xd0/0xe8 [spl] Sep 3 11:48:25 UnRAID kernel: ? dmu_buf_hold_array_by_dnode+0x279/0x41c [zfs] Sep 3 11:48:25 UnRAID kernel: ? preempt_latency_start+0x2b/0x46 Sep 3 11:48:25 UnRAID kernel: ? _raw_spin_lock+0x13/0x1c Sep 3 11:48:25 UnRAID kernel: ? _raw_spin_unlock+0x14/0x29 Sep 3 11:48:25 UnRAID kernel: ? dnode_rele_and_unlock+0x67/0xb6 [zfs] Sep 3 11:48:25 UnRAID kernel: ? dmu_buf_hold_array.constprop.0+0x7b/0xa2 [zfs] Sep 3 11:48:25 UnRAID kernel: ? dmu_buf_hold_array_by_dnode+0x24d/0x41c [zfs] Sep 3 11:48:25 UnRAID kernel: dmu_write+0x77/0xf0 [zfs] Sep 3 11:48:25 UnRAID kernel: ? _raw_spin_lock+0x13/0x1c Sep 3 11:48:25 UnRAID kernel: space_map_write+0x19c/0x605 [zfs] Sep 3 11:48:25 UnRAID kernel: ? dmu_object_size_from_db+0x22/0x6d [zfs] Sep 3 11:48:25 UnRAID kernel: ? space_map_open_impl+0x48/0x76 [zfs] Sep 3 11:48:25 UnRAID kernel: metaslab_flush+0x1ba/0x326 [zfs] Sep 3 11:48:25 UnRAID kernel: ? spa_generate_syncing_log_sm+0x1f4/0x21a [zfs] Sep 3 11:48:25 UnRAID kernel: spa_flush_metaslabs+0x2bf/0x334 [zfs] Sep 3 11:48:25 UnRAID kernel: spa_sync+0x7c9/0xd12 [zfs] Sep 3 11:48:25 UnRAID kernel: txg_sync_thread+0x1f3/0x375 [zfs] Sep 3 11:48:25 UnRAID kernel: ? __kmem_cache_free+0x6a/0xf5 Sep 3 11:48:25 UnRAID kernel: ? txg_fini+0x1c2/0x1c2 [zfs] Sep 3 11:48:25 UnRAID kernel: ? __thread_exit+0x13/0x13 [spl] Sep 3 11:48:25 UnRAID kernel: thread_generic_wrapper+0x5a/0x65 [spl] Sep 3 11:48:25 UnRAID kernel: kthread+0xe7/0xef Sep 3 11:48:25 UnRAID kernel: ? kthread_complete_and_exit+0x1b/0x1b Sep 3 11:48:25 UnRAID kernel: ret_from_fork+0x22/0x30 Sep 3 11:48:25 UnRAID kernel: </TASK> Sep 3 11:48:26 UnRAID SysDrivers: SysDrivers Build Complete Sep 3 11:48:38 UnRAID SysDrivers: SysDrivers Build Complete Sep 3 11:51:04 UnRAID nginx: 2024/09/03 11:51:04 [error] 9887#9887: *40 upstream timed out (110: Connection timed out) while reading upstream, client: 192.168.100.160, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket:/update.htm", host: "192.168.100.30", referrer: "http://192.168.100.30/Main" Sep 3 11:51:04 UnRAID nginx: 2024/09/03 11:51:04 [alert] 9887#9887: *148 open socket #13 left in connection 7 Sep 3 11:51:04 UnRAID nginx: 2024/09/03 11:51:04 [alert] 9887#9887: *120 open socket #4 left in connection 10 Sep 3 11:51:04 UnRAID nginx: 2024/09/03 11:51:04 [alert] 9887#9887: *103 open socket #28 left in connection 12 Sep 3 11:51:04 UnRAID nginx: 2024/09/03 11:51:04 [alert] 9887#9887: aborting Edited September 3, 20241 yr by TallMan206
September 3, 20241 yr Community Expert Solution See if you can mount that disk read-only before starting the array: zpool import -o readonly=on disk2 If it mounts the data should be under /mnt/disk2, you can then backup the contents and re-format
September 3, 20241 yr Author Hi JorgeB, That works. I have 2 data disks in the system, and disk1 also did not want to start anymore. Now with disk2 in Read Only I can move the data from disk2 to disk1. Perfect! What do you suggest after copying? pre-clear disk2?
September 3, 20241 yr Community Expert Disk itself looks healthy, looks like it was only a filesystem issue, so just reformatting it should be enough.
September 6, 20241 yr Author Hi, I pre cleared the disk and no errors. I added it to the array again and everything is fine. Thanks for everything
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.