Jump to content

Forced shutdown / unmountable cache drive

MvL

By MvL
in General Support

Recommended Posts

MvL Enthusiast

MvL

Posted
  • Author
Posted

Interesting.. I found this in the syslog:

 

Quote

Apr 25 08:20:06 RackServer kernel: BTRFS critical (device sdb1): corrupt leaf: root=2 block=159318016 slot=36, unexpected item end, have 2430846058 expect 14375
### [PREVIOUS LINE REPEATED 1 TIMES] ###
Apr 25 08:20:06 RackServer kernel: BTRFS: error (device sdb1) in btrfs_replay_log:2277: errno=-5 IO failure (Failed to recover log tree)
Apr 25 08:20:06 RackServer kernel: BTRFS error (device sdb1): pending csums is 57344
Apr 25 08:20:06 RackServer kernel: WARNING: CPU: 18 PID: 7098 at fs/btrfs/extent-tree.c:107 btrfs_put_block_group+0x1a/0x44
Apr 25 08:20:06 RackServer kernel: Modules linked in: xfs dm_crypt algif_skcipher af_alg dm_mod dax nfsd lockd grace sunrpc md_mod bonding igb i2c_algo_bit sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd ipmi_ssif cryptd glue_helper mpt3sas intel_cstate intel_uncore ahci intel_rapl_perf libahci i2c_i801 i2c_core raid_class scsi_transport_sas wmi ipmi_si acpi_power_meter pcc_cpufreq button acpi_pad [last unloaded: i2c_algo_bit]
Apr 25 08:20:06 RackServer kernel: CPU: 18 PID: 7098 Comm: mount Not tainted 4.19.33-Unraid #1
Apr 25 08:20:06 RackServer kernel: Hardware name: Supermicro Super Server/X10SRi-F, BIOS 3.1 06/06/2018
Apr 25 08:20:06 RackServer kernel: RIP: 0010:btrfs_put_block_group+0x1a/0x44
Apr 25 08:20:06 RackServer kernel: Code: ff 48 83 c4 50 5b 5d 41 5c 41 5d 41 5e 41 5f c3 53 f0 ff 8f 08 01 00 00 75 38 48 83 7f 48 00 74 02 0f 0b 48 83 7f 50 00 74 02 <0f> 0b 48 8b 87 e8 01 00 00 48 85 c0 74 02 0f 0b 48 89 fb 48 8b bf
Apr 25 08:20:06 RackServer kernel: RSP: 0018:ffffc900071cbb00 EFLAGS: 00010206
Apr 25 08:20:06 RackServer kernel: RAX: 0000000000000000 RBX: ffff888ff0022000 RCX: 0000000000000048
Apr 25 08:20:06 RackServer kernel: RDX: 0000000000000048 RSI: 0000000000004000 RDI: ffff888fed5dbc00
Apr 25 08:20:06 RackServer kernel: RBP: ffff888fed5dbce0 R08: 0000000000000001 R09: ffffffff81243d00
Apr 25 08:20:06 RackServer kernel: R10: ffffea003fb422c0 R11: ffff889033878001 R12: ffff888ff0022090
Apr 25 08:20:06 RackServer kernel: R13: ffff888fed5dbc00 R14: dead000000000100 R15: ffff888ff00220a0
Apr 25 08:20:06 RackServer kernel: FS:  000014adb5be5780(0000) GS:ffff88903f680000(0000) knlGS:0000000000000000
Apr 25 08:20:06 RackServer kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 25 08:20:06 RackServer kernel: CR2: 000014adb5c7d200 CR3: 0000000fedcf0003 CR4: 00000000001606e0
Apr 25 08:20:06 RackServer kernel: Call Trace:
Apr 25 08:20:06 RackServer kernel: btrfs_free_block_groups+0x17a/0x2c8
Apr 25 08:20:06 RackServer kernel: open_ctree+0x1bb2/0x1c61
Apr 25 08:20:06 RackServer kernel: btrfs_mount_root+0x382/0x4f5
Apr 25 08:20:06 RackServer kernel: ? cpumask_next+0x16/0x17
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc+0x366/0x471
Apr 25 08:20:06 RackServer kernel: mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: vfs_kern_mount+0x62/0xfa
Apr 25 08:20:06 RackServer kernel: btrfs_mount+0x163/0x7f0
Apr 25 08:20:06 RackServer kernel: ? pcpu_block_update_hint_alloc+0x9c/0x1a0
Apr 25 08:20:06 RackServer kernel: ? pcpu_chunk_relocate+0x8/0x5a
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc_area+0xc7/0xd7
Apr 25 08:20:06 RackServer kernel: ? pcpu_next_unpop+0x31/0x3c
Apr 25 08:20:06 RackServer kernel: ? cpumask_next+0x16/0x17
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc+0x366/0x471
Apr 25 08:20:06 RackServer kernel: ? mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: vfs_kern_mount+0x62/0xfa
Apr 25 08:20:06 RackServer kernel: do_mount+0x7b3/0xa2f
Apr 25 08:20:06 RackServer kernel: ? __kmalloc_track_caller+0x65/0x122
Apr 25 08:20:06 RackServer kernel: ? _copy_from_user+0x2f/0x4d
Apr 25 08:20:06 RackServer kernel: ? memdup_user+0x39/0x55
Apr 25 08:20:06 RackServer kernel: ksys_mount+0x6d/0x92
Apr 25 08:20:06 RackServer kernel: __x64_sys_mount+0x1c/0x1f
Apr 25 08:20:06 RackServer kernel: do_syscall_64+0x57/0xe6
Apr 25 08:20:06 RackServer kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 25 08:20:06 RackServer kernel: RIP: 0033:0x14adb5d21fca
Apr 25 08:20:06 RackServer kernel: Code: 48 8b 0d c9 7e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 96 7e 0c 00 f7 d8 64 89 01 48
Apr 25 08:20:06 RackServer kernel: RSP: 002b:00007ffcb1aeb568 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
Apr 25 08:20:06 RackServer kernel: RAX: ffffffffffffffda RBX: 000000000040d2b0 RCX: 000014adb5d21fca
Apr 25 08:20:06 RackServer kernel: RDX: 000000000040d4c0 RSI: 000000000040d540 RDI: 000000000040d520
Apr 25 08:20:06 RackServer kernel: RBP: 000014adb5eaef10 R08: 0000000000000000 R09: 0000000000000000
Apr 25 08:20:06 RackServer kernel: R10: 0000000000000c00 R11: 0000000000000202 R12: 0000000000000000
Apr 25 08:20:06 RackServer kernel: R13: 0000000000000c00 R14: 000000000040d520 R15: 000000000040d4c0
Apr 25 08:20:06 RackServer kernel: ---[ end trace 62ba4894ff66ba3e ]---
Apr 25 08:20:06 RackServer kernel: WARNING: CPU: 18 PID: 7098 at fs/btrfs/extent-tree.c:9747 btrfs_free_block_groups+0x233/0x2c8
Apr 25 08:20:06 RackServer kernel: Modules linked in: xfs dm_crypt algif_skcipher af_alg dm_mod dax nfsd lockd grace sunrpc md_mod bonding igb i2c_algo_bit sb_edac x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel pcbc aesni_intel aes_x86_64 crypto_simd ipmi_ssif cryptd glue_helper mpt3sas intel_cstate intel_uncore ahci intel_rapl_perf libahci i2c_i801 i2c_core raid_class scsi_transport_sas wmi ipmi_si acpi_power_meter pcc_cpufreq button acpi_pad [last unloaded: i2c_algo_bit]
Apr 25 08:20:06 RackServer kernel: CPU: 18 PID: 7098 Comm: mount Tainted: G        W         4.19.33-Unraid #1
Apr 25 08:20:06 RackServer kernel: Hardware name: Supermicro Super Server/X10SRi-F, BIOS 3.1 06/06/2018
Apr 25 08:20:06 RackServer kernel: RIP: 0010:btrfs_free_block_groups+0x233/0x2c8
Apr 25 08:20:06 RackServer kernel: Code: 93 38 0e 00 00 48 39 c2 0f 84 a4 00 00 00 48 8b ab 38 0e 00 00 48 83 7d 90 00 75 0e 48 83 7d 98 00 75 07 48 83 7d a0 00 74 15 <0f> 0b 31 c9 31 d2 48 89 df 48 8d b5 78 ff ff ff e8 4f 97 ff ff 48
Apr 25 08:20:06 RackServer kernel: RSP: 0018:ffffc900071cbb10 EFLAGS: 00010206
Apr 25 08:20:06 RackServer kernel: RAX: ffff888fed5c8488 RBX: ffff888ff0022000 RCX: 0000000000000000
Apr 25 08:20:06 RackServer kernel: RDX: ffff888ff0022e38 RSI: 0000000000000001 RDI: ffff888fed5c8200
Apr 25 08:20:06 RackServer kernel: RBP: ffff888fed5c8488 R08: 0000000000000000 R09: 0000000000000000
Apr 25 08:20:06 RackServer kernel: R10: 0000000000000fc0 R11: ffff889033878080 R12: ffff888ff0022090
Apr 25 08:20:06 RackServer kernel: R13: ffff889034575400 R14: dead000000000100 R15: ffff888ff00220a0
Apr 25 08:20:06 RackServer kernel: FS:  000014adb5be5780(0000) GS:ffff88903f680000(0000) knlGS:0000000000000000
Apr 25 08:20:06 RackServer kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Apr 25 08:20:06 RackServer kernel: CR2: 000014adb5c7d200 CR3: 0000000fedcf0003 CR4: 00000000001606e0
Apr 25 08:20:06 RackServer kernel: Call Trace:
Apr 25 08:20:06 RackServer kernel: open_ctree+0x1bb2/0x1c61
Apr 25 08:20:06 RackServer kernel: btrfs_mount_root+0x382/0x4f5
Apr 25 08:20:06 RackServer kernel: ? cpumask_next+0x16/0x17
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc+0x366/0x471
Apr 25 08:20:06 RackServer kernel: mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: vfs_kern_mount+0x62/0xfa
Apr 25 08:20:06 RackServer kernel: btrfs_mount+0x163/0x7f0
Apr 25 08:20:06 RackServer kernel: ? pcpu_block_update_hint_alloc+0x9c/0x1a0
Apr 25 08:20:06 RackServer kernel: ? pcpu_chunk_relocate+0x8/0x5a
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc_area+0xc7/0xd7
Apr 25 08:20:06 RackServer kernel: ? pcpu_next_unpop+0x31/0x3c
Apr 25 08:20:06 RackServer kernel: ? cpumask_next+0x16/0x17
Apr 25 08:20:06 RackServer kernel: ? pcpu_alloc+0x366/0x471
Apr 25 08:20:06 RackServer kernel: ? mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: mount_fs+0x10/0x6b
Apr 25 08:20:06 RackServer kernel: vfs_kern_mount+0x62/0xfa
Apr 25 08:20:06 RackServer kernel: do_mount+0x7b3/0xa2f
Apr 25 08:20:06 RackServer kernel: ? __kmalloc_track_caller+0x65/0x122
Apr 25 08:20:06 RackServer kernel: ? _copy_from_user+0x2f/0x4d
Apr 25 08:20:06 RackServer kernel: ? memdup_user+0x39/0x55
Apr 25 08:20:06 RackServer kernel: ksys_mount+0x6d/0x92
Apr 25 08:20:06 RackServer kernel: __x64_sys_mount+0x1c/0x1f
Apr 25 08:20:06 RackServer kernel: do_syscall_64+0x57/0xe6
Apr 25 08:20:06 RackServer kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9
Apr 25 08:20:06 RackServer kernel: RIP: 0033:0x14adb5d21fca
Apr 25 08:20:06 RackServer kernel: Code: 48 8b 0d c9 7e 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 96 7e 0c 00 f7 d8 64 89 01 48
Apr 25 08:20:06 RackServer kernel: RSP: 002b:00007ffcb1aeb568 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5
Apr 25 08:20:06 RackServer kernel: RAX: ffffffffffffffda RBX: 000000000040d2b0 RCX: 000014adb5d21fca
Apr 25 08:20:06 RackServer kernel: RDX: 000000000040d4c0 RSI: 000000000040d540 RDI: 000000000040d520
Apr 25 08:20:06 RackServer kernel: RBP: 000014adb5eaef10 R08: 0000000000000000 R09: 0000000000000000
Apr 25 08:20:06 RackServer kernel: R10: 0000000000000c00 R11: 0000000000000202 R12: 0000000000000000
Apr 25 08:20:06 RackServer kernel: R13: 0000000000000c00 R14: 000000000040d520 R15: 000000000040d4c0
Apr 25 08:20:06 RackServer kernel: ---[ end trace 62ba4894ff66ba3f ]---
Apr 25 08:20:06 RackServer kernel: BTRFS info (device sdb1): space_info 1 has 32111308800 free, is not full
Apr 25 08:20:06 RackServer kernel: BTRFS info (device sdb1): space_info total=126709923840, used=94598602752, pinned=0, reserved=12288, may_use=0, readonly=0
Apr 25 08:20:06 RackServer kernel: BTRFS error (device sdb1): open_ctree failed
Apr 25 08:20:06 RackServer root: mount: /mnt/cache: can't read superblock on /dev/sdb1.
Apr 25 08:20:06 RackServer emhttpd: shcmd (222): exit status: 32
Apr 25 08:20:06 RackServer emhttpd: /mnt/cache mount error: No file system
Apr 25 08:20:06 RackServer emhttpd: shcmd (223): umount /mnt/cache
Apr 25 08:20:06 RackServer root: umount: /mnt/cache: not mounted.
Apr 25 08:20:06 RackServer emhttpd: shcmd (223): exit status: 32
Apr 25 08:20:06 RackServer emhttpd: shcmd (224): rmdir /mnt/cache
Apr 25 08:20:06 RackServer emhttpd: shcmd (225): sync
Apr 25 08:20:06 RackServer emhttpd: shcmd (226): mkdir /mnt/user0
Apr 25 08:20:06 RackServer emhttpd: shcmd (227): /usr/local/sbin/shfs /mnt/user0 -disks 510 -o noatime,big_writes,allow_other   |& logger
Apr 25 08:20:06 RackServer shfs: stderr redirected to syslog
Apr 25 08:20:06 RackServer emhttpd: shcmd (228): mkdir /mnt/user
Apr 25 08:20:06 RackServer emhttpd: shcmd (229): /usr/local/sbin/shfs /mnt/user -disks 511 2048000000 -o noatime,big_writes,allow_other  -o remember=330  |& logger
Apr 25 08:20:06 RackServer shfs: stderr redirected to syslog
Apr 25 08:20:06 RackServer emhttpd: shcmd (231): /usr/local/sbin/update_cron
Apr 25 08:20:06 RackServer root: Creating custom tabs
Apr 25 08:20:06 RackServer emhttpd: Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/customtab/scripts/createTab.php on line 33
Apr 25 08:20:06 RackServer cache_dirs: Arguments=-l off
Apr 25 08:20:06 RackServer cache_dirs: Max Scan Secs=10, Min Scan Secs=1
Apr 25 08:20:06 RackServer cache_dirs: Scan Type=adaptive
Apr 25 08:20:06 RackServer cache_dirs: Min Scan Depth=4
Apr 25 08:20:06 RackServer cache_dirs: Max Scan Depth=none
Apr 25 08:20:06 RackServer cache_dirs: Use Command='find -noleaf'
Apr 25 08:20:06 RackServer cache_dirs: ---------- Caching Directories ---------------
Apr 25 08:20:06 RackServer cache_dirs: h..e
Apr 25 08:20:06 RackServer cache_dirs: m..)
### [PREVIOUS LINE REPEATED 6 TIMES] ###

 

MvL Enthusiast

MvL

Posted
  • Author
Posted

So cache drive is recovered thanks to the post that I linked above. 

 

I'm only not sure why my server was hanging.. Something to investigate further. If you have any idea then please leave a comment behind.

Archived

This topic is now archived and is closed to further replies.

Go to topic listing
×
×
  • Create New...