Hello all! Thank you for your wonderful Server OS. I cannot stop my array. I've spent the last hour digging threw logs and stuff and really am stupped. All the Array and Pool Devices are now unmounted and it still will not stop the Array. I manually unmounted Apps, Cache, Security, VM, Download pools. Manually unmounted everything else int he /mnt/ that was mounted. Which was only addons, disks, rootshare. The array stop seams to work fine but stops the process at emhttpd: shcmd (1754605): umount /mnt/disk16. It never does the rmdir /mnt/disk16. I tried doing this manually once and when it finally shutdown did it unclean because the /mnt/disk16 ect were already removed. I looked in Active Streams and nothing is there. I look in OpenFiles and it lists tons of files but they are all on Array Disks that are already stopped. The automated reboot goes without a hitch every week, no unclean shutdown. I'm at a loss and about to have to just reset the server. Here are my Diagnostics before I did anything manually.

What happened to being able to manually shutdown the array with CLI commands. It doesn't seam to work anymore.

rudder2-server-diagnostics-20251214-1039.zip

Edited December 14, 2025Dec 14 by Rudder2
Clarifying that the Diagnostics provided was before I performed any manual operations. I will attache a diagnostics after I performed manual operations trying to get array to stop.

Even worse news now. The server will no longer come up. Please help.

Got everything back up finally. The original problem probably still stands.

Diagnostics after I manually unmounted everything seamed to just vanish from the USB Flash log folder. Good thing I downloaded the 1st one before rebooting. Funny thing is I got the confirmation that it wrote to the Flash drive. Every time I rebooted the server it created a new Diagnostics file so maybe there is a only 2 diagnostics files at a time in flash log folder? It seams the server will not come back on-line without internet.

I think the server quit booting up because the pfSense Router sees the UPS threw the server. The server shuts down then pfSense sometimes shuts down because the UPS went off line with less than time remaining to delay, The server is programmed to be the last to shut down turning off the UPS till it's charged enough to perform a clean shutdown again just incase the power comes on for an hour and goes off again, my power is not reliable. I will be getting a dedicated UPS for the Network equipment so a fix for the sometimes internet problem is in the works. Once the internet was back on-line the server booted up and there was an unclean shutdown...arrg!

I forgot that the Array has to be started for BTRFS Balance so I accidentally took down the array instead of just Docker, doh! Leading my to the unmounting drives problem which never occurred at an opportune time to diagnose...LOL! I'm still having the problem I was about to start researching. I can't get 121 Gig of my Apps Pool to enter RAID1. This is why I accidentally stopped the Array. I was trying to balance that 121 Gig in to RAID1 and thought that Docker might of been holding it open. I just added another drive for redundancy and to give more overhead for databases. The server is running faster than ever now.

So it's possible I have multiple problems creating the issue.

Performed the lsof on all the devices still mounted in the /mnt/ folder and nothing showed up.

Performed a BTRFS Check in Maintenance mode and no errors come back.

Scrub on the other hand:

Error summary: read=384 Corrected: 368 Uncorrectable: 16 Unverified: 0

Tried to balance again and fount these errors in the logs:

Dec 14 14:15:43 Rudder2-Server ool www[895550]: /usr/local/emhttp/plugins/dynamix/scripts/btrfs_balance 'start' '/mnt/apps' '-dconvert=raid1,soft -mconvert=raid1,soft'

Dec 14 14:15:43 Rudder2-Server kernel: BTRFS info (device nvme2n1p1): balance: start -dconvert=raid1,soft -mconvert=raid1,soft -sconvert=raid1,soft

Dec 14 14:15:43 Rudder2-Server kernel: BTRFS info (device nvme2n1p1): relocating block group 326439534592 flags data

Dec 14 14:15:44 Rudder2-Server kernel: nvme2n1: I/O Cmd(0x2) @ LBA 629985752, 2560 blocks, I/O Error (sct 0x2 / sc 0x81) MORE

Dec 14 14:15:44 Rudder2-Server kernel: critical medium error, dev nvme2n1, sector 629985752 op 0x0:(READ) flags 0x84700 phys_seg 6 prio class 0 Dec 14 14:15:44 Rudder2-Server kernel: nvme2n1: I/O Cmd(0x2) @ LBA 629986824, 8 blocks, I/O Error (sct 0x2 / sc 0x81) MORE DNR

Dec 14 14:15:44 Rudder2-Server kernel: critical medium error, dev nvme2n1, sector 629986824 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0

Dec 14 14:15:45 Rudder2-Server kernel: BTRFS info (device nvme2n1p1): balance: ended with status: -5

Guess it's time for a copy all data reformat and copy all data back...Still don't know if this has anything to do with the Array shutdown problem.

OK, I copied all data off my apps pool, erased apps pool, formatted apps pool to RAID1, copied all data back, performed BTRFS Scrub. All BTRFS errors gone.

So, that rabbit hole completed. Is there anything else in the Diagnostics that I should do to fix the Array stopping hang?

Here is another Diagnostics after fixing the BTRFS errors of my Array hanging on stopping. In the logs is actually said and repeated for 1 minute:

Dec 14 15:13:06 Rudder2-Server emhttpd: Retry unmounting disk share(s)...

Dec 14 15:13:11 Rudder2-Server emhttpd: Unmounting disks...

Dec 14 15:13:11 Rudder2-Server emhttpd: shcmd (20047): umount /mnt/disk16

Dec 14 15:13:11 Rudder2-Server root: umount: /mnt/disk16: target is busy.

Dec 14 15:13:11 Rudder2-Server emhttpd: shcmd (20047): exit status: 32

This time instead of saying nothing about retrying unmounting. Docker and VMS systems are still offline at this point.

rudder2-server-diagnostics-20251214-1515.zip

Thank you again to whoever has a moment to review this Data. I've done everything I can this of. Still love unRAID...It was time for some love and 128 gig more RAM, been running reliably with out any intervention for 3 years.

Disable Docker and VM service and reboot in safe mode, start the array and see if it stops, if yes, retest again after booting in normal mode to rule out a plugin issue, leave the services still disabled, if still OK enable one of the services and retest, then the other, note that if the issue happens with the docker service enabled only, you may need to try enabling one or a few containers at a time to find the culprit.

The last diagnostics was generated with the Disk 16 hang on shutdown with the Docker and VM Services disabled then rebooted. There is defiantly something hanging the array stop that is outside the VM and Docker Services. I will have to try safe mode this weekend. The good news is the server is able to shutdown in the 8 - 10 minutes the UPS provides as back up power now. My server is angry when the memory usage gets above 72% and now that I added 128 gig more ram (192 Gig Total) it seams WAY more stable and faster than it has been in years. I no longer have the Array Never Stops problem so that was solved somewhere in my fixing problems as I found them during diagnostics.

I've read there was optimizations in 7.2.x that might address array disks hanging on array stop, so now that my server is running acceptably and we are past the .1 mark I will upgrade. Don't upgrade until server is stable. Also this upgrade will get rid of the log spam from my unRAID Home Assistance integration because of the new API. Looking forward to using the new API.

I also discovered that my Server was trying to write out the logs to the Array on shutdown instead of Cache Drive. Changing this sped up the process of shutdown.

Does the nVidia Driver plugin make the server require internet to boot? When I boot without internet my Server name becomes Tower instead of the standard given name. It also names the Diagnostic Files as Tower instead of given name also when booted without internet at this time also.

59 minutes ago, Rudder2 said:

Does the nVidia Driver plugin make the server require internet to boot?

I don't know; recommend asking in the driver support thread.