- Minor
In relation to this thread:
and this one:
and my comment here:
So while what I noted above seemed to solve that problem. Yesterday I had an issue where my Ubuntu VM seemed to become partially unresponsive while doing various things over NFS (sab downloading).
Several hours later, I tried to figure out what was going on as several things on the VM were unhappy. I ended up rebooting it. Then tried to load a Unraid NFS share via autofs. Whenever I did this, it'd just lock up and not mount or anything. No error logs on either side. I tried restarting NFS and RPC on unraid, that didn't help.
I shut all my VMs down then tried to stop the array to try a reboot. The array hung trying to unmount /mnt/user
I ran lsof and found that the 'shfs' process was holding a couple of files open on /mnt/cache and another share.
I did a 'kill <pid of shfs>' and then the unmount proceeded to happen correctly.
I rebooted the server, and now everything is okay again. Except the files that were being manipulated at the time over NFS are either zero file size or didn't get recreated.
Previously on 6.12.10 my server has been rock solid for weeks. I upgraded to 6.12.14 2 days ago and the up time was about 2 days.
This is the first time i've ever seen an error like this.
It seems like sab running inside the ubuntu vm, with all of its data directories mounted over NFS had a sad related to NFS....
This is from the syslog of the vm (I cannot find anything else related in unraid logs or anywhere else:
Dec 07 19:46:39 <vm hostname> kernel: INFO: task sab********:4126 blocked for more than 122 seconds. Dec 07 19:46:39 <vm hostname> kernel: Not tainted 6.8.0-45-generic #45~22.04.1-Ubuntu Dec 07 19:46:39 <vm hostname> kernel: "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. Dec 07 19:46:39 <vm hostname> kernel: task:sab******** state:D stack:0 pid:4126 tgid:4094 ppid:1 flags:0x00000002 Dec 07 19:46:39 <vm hostname> kernel: Call Trace: Dec 07 19:46:39 <vm hostname> kernel: <TASK> Dec 07 19:46:39 <vm hostname> kernel: __schedule+0x27c/0x6a0 Dec 07 19:46:39 <vm hostname> kernel: ? nfs_pageio_complete+0xee/0x140 [nfs] Dec 07 19:46:39 <vm hostname> kernel: schedule+0x33/0x110 Dec 07 19:46:39 <vm hostname> kernel: io_schedule+0x46/0x80 Dec 07 19:46:39 <vm hostname> kernel: folio_wait_bit_common+0x136/0x330 Dec 07 19:46:39 <vm hostname> kernel: ? __pfx_wake_page_function+0x10/0x10 Dec 07 19:46:39 <vm hostname> kernel: folio_wait_bit+0x18/0x30 Dec 07 19:46:39 <vm hostname> kernel: folio_wait_writeback+0x2b/0xa0 Dec 07 19:46:39 <vm hostname> kernel: __filemap_fdatawait_range+0x93/0x110 Dec 07 19:46:39 <vm hostname> kernel: filemap_write_and_wait_range+0x94/0xc0 Dec 07 19:46:39 <vm hostname> kernel: nfs_wb_all+0x27/0x130 [nfs] Dec 07 19:46:39 <vm hostname> kernel: nfs4_file_flush+0x7e/0xe0 [nfsv4] Dec 07 19:46:39 <vm hostname> kernel: filp_flush+0x35/0x90 Dec 07 19:46:39 <vm hostname> kernel: __x64_sys_close+0x34/0x90 Dec 07 19:46:39 <vm hostname> kernel: x64_sys_call+0x1a20/0x24b0 Dec 07 19:46:39 <vm hostname> kernel: do_syscall_64+0x81/0x170 Dec 07 19:46:39 <vm hostname> kernel: ? do_syscall_64+0x8d/0x170 Dec 07 19:46:39 <vm hostname> kernel: ? syscall_exit_to_user_mode+0x86/0x260 Dec 07 19:46:39 <vm hostname> kernel: ? do_syscall_64+0x8d/0x170 Dec 07 19:46:39 <vm hostname> kernel: ? do_syscall_64+0x8d/0x170 Dec 07 19:46:39 <vm hostname> kernel: ? clear_bhb_loop+0x15/0x70 Dec 07 19:46:39 <vm hostname> kernel: ? clear_bhb_loop+0x15/0x70 Dec 07 19:46:39 <vm hostname> kernel: ? clear_bhb_loop+0x15/0x70 Dec 07 19:46:39 <vm hostname> kernel: entry_SYSCALL_64_after_hwframe+0x78/0x80 Dec 07 19:46:39 <vm hostname> kernel: RIP: 0033:0x7c085ef14f8b Dec 07 19:46:39 <vm hostname> kernel: RSP: 002b:00007c0838bff4a0 EFLAGS: 00000293 ORIG_RAX: 0000000000000003 Dec 07 19:46:39 <vm hostname> kernel: RAX: ffffffffffffffda RBX: 00005cc6f79e8130 RCX: 00007c085ef14f8b Dec 07 19:46:39 <vm hostname> kernel: RDX: 0000000000000000 RSI: 0000000000000000 RDI: 000000000000001d Dec 07 19:46:39 <vm hostname> kernel: RBP: 00005cc6f79e8130 R08: 0000000000000000 R09: 0000000000000000 Dec 07 19:46:39 <vm hostname> kernel: R10: 00005cc6f69d47c0 R11: 0000000000000293 R12: 000000000000001d Dec 07 19:46:39 <vm hostname> kernel: R13: 00007c0838bff500 R14: 0000000000000000 R15: 0000000000000000 Dec 07 19:46:39 <vm hostname> kernel: </TASK>
NFS share mounted from the vm to unraid with these options in autofs config:
-users,rw,auto,noatime,async,hard,rsize=32768,wsize=32768,fstype=nfs4
I looked through all the log files on unraid and couldn't find anything untoward about this issue aside from shfs stuck with some files open while unmounting.
After rebooting unraid everything came up happy again, but I think I'm going to downgrade again back to 6.12.10.