January 26, 20251 yr Unraid Version: 7 Motherboard: Super Micro MBD-H11SSL-I302P-MA015-O CPU: AMD Epyc 7302P Ram: 128 GB Hard Drives: 8 10tb (5 Seagate, 3 western digital) - 2 parity, all on the array, xfs encrypted 6 x 4tb (Western digital) - zfs mirror, 3 vdev 2 x 256gb SSDs - Cache 2 x 2tb NVME - Cache (PCIE Card) Networking: 2 x 1Gbpss fail over (Static - 192.168.0.0/24 onboard) 1 x 10Gbps (Static - 192.168.1.0/30 isolated pcie card) I've gone over the unclean shutdown thread and nothing in there helped. I can SSH into the server and access the Web UI via ikvm, however it just tries to load the web page and fails - 500 internal server error. I've attempted to boot in safemode with gui and without - no plugins both times - and neither mode allowed access to the web GUI. VM & Docker were also stopped at the time. SMB shares are also inaccessible however, the Root Global Share does show up in NETMAN (windows 10) but does no load any folders when you open it. I don't have syslog available prior to the unclean shutdown. Nginx will not start on boot and when I start it manually, the engine gives out a 500 internal server error. I've re-downloaded the Unraid OS and compared the USB stick to the Downloaded archive and can't see any differences side from the /config directory, which is to be expected. I've been able to start Nginx and PHP-FM as per below. LSOF shows not files being accessed on Disks though - which I think it why the server isn't starting properly. root@UNRAID-SERVER:/var/log# lsof | grep disk root@UNRAID-SERVER:/var/log# /etc/rc.d/rc.nginx start rc.nginx: Starting Nginx server daemon... rc.nginx: Nginx server daemon... Started. root@UNRAID-SERVER:/var/log# /etc/rc.d/rc.nginx status Nginx server daemon is currently running. root@UNRAID-SERVER:/var/log# ls btmp dmesg libvirt/ nfsd/ pkgtools/ removed_scripts@ scripts@ spooler vfio-pci cron faillog maillog nginx/ plugins/ sa/ secure swtpm/ vfio-pci-errors debug lastlog messages packages@ removed_packages@ samba/ setup@ syslog wtmp root@UNRAID-SERVER:/var/log# cd nginx root@UNRAID-SERVER:/var/log/nginx# ls error.log root@UNRAID-SERVER:/var/log/nginx# cat error.log root@UNRAID-SERVER:/var/log/nginx# tail error.log root@UNRAID-SERVER:/var/log/nginx# /etc/rc.d/rc.php-fpm start rc.php-fpm: Starting PHP-fpm daemon... rc.php-fpm: PHP-fpm daemon... Started. root@UNRAID-SERVER:/var/log/nginx# root@ANC-S-01:/var/log/nginx# tail error.log 2025/01/27 01:30:52 [crit] 4910#4910: *2 connect() to unix:/var/run/php-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: x.x.x.CLIENT, server: , request: "GET /Dashboard HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php-fpm.sock:", host: "x.x.x.UNRAID:8443" 2025/01/27 01:30:52 [error] 4910#4910: *2 auth request unexpected status: 502 while sending to client, client: x.x.x.CLIENT, server: , request: "GET /Dashboard HTTP/2.0", host: "x.x.x.UNRAID:8443" 2025/01/27 01:30:52 [crit] 4910#4910: *2 connect() to unix:/var/run/php-fpm.sock failed (2: No such file or directory) while connecting to upstream, client: x.x.x.CLIENT, server: , request: "GET /favicon.ico HTTP/2.0", subrequest: "/auth-request.php", upstream: "fastcgi://unix:/var/run/php-fpm.sock:", host: "x.x.x.UNRAID:8443", referrer: "https://x.x.x.UNRAID:8443/Dashboard" 2025/01/27 01:30:52 [error] 4910#4910: *2 auth request unexpected status: 502 while sending to client, client: x.x.x.CLIENT, server: , request: "GET /favicon.ico HTTP/2.0", host: "x.x.x.UNRAID:8443", referrer: "https://x.x.x.UNRAID:8443/Dashboard" After checking the /var/run/php-fpm.sock file root@UNRAID-SERVER:/var/run# ls acpid.pid blkid/ elogind.pid nginx.pid runlevel systemd@ acpid.socket= cgroup2-unraid.pid emhttpd.socket= nginx.socket= samba/ udev/ agetty.reload cron/ faillock/ nmbd.pid samba-dcerpcd.pid user/ atd.pid cryptsetup/ inetd.pid php-fpm.pid seedrng.lock utmp avahi-daemon/ dbus/ lock/ php-fpm.sock= smbd.pid winbindd.pid avahi-dnsconfd.pid elogind/ mount/ rsyslogd.pid sshd.pid xtables.lock System has been stable for over a year, completely stable since the upgrade to v7 (which had no issues). Any help you might be able to give would be appreciated! tower-diagnostics-20250126-2330.zip Edited January 26, 20251 yr by Little_Jon
January 26, 20251 yr Community Expert Solution Jan 26 23:20:50 ANC-S-01 emhttpd: import 30 pool device: (sdb) Samsung_SSD_850_PRO_256GB_S251NXAH116032A Jan 26 23:20:50 ANC-S-01 emhttpd: import 31 pool device: (sdc) Samsung_SSD_850_PRO_256GB_S251NXAH116940Y There's a problem with this pool config, delete /config/pools/pool_name.cfg and reboot, then reimport the pool: create a new pool with the same name and number of slots assign the pool device(s), leave the filesystem set to auto start the array to import the pool
January 26, 20251 yr Author 1 minute ago, JorgeB said: Jan 26 23:20:50 ANC-S-01 emhttpd: import 30 pool device: (sdb) Samsung_SSD_850_PRO_256GB_S251NXAH116032A Jan 26 23:20:50 ANC-S-01 emhttpd: import 31 pool device: (sdc) Samsung_SSD_850_PRO_256GB_S251NXAH116940Y There's a problem with this pool config, delete /config/pools/pool_name.cfg and reboot, then reimport the pool: create a new pool with the same name and number of slots assign the pool device(s), leave the filesystem set to auto start the array to import the pool Are you a wizard?
January 26, 20251 yr Author I went searching in the syslog.txt file for Error, Cache and Pool - I couldn't see anything
January 26, 20251 yr Author I've added the SSDs into a pool and it's showing a third drive, even though I selected the pool as 2 devices. diskFsType="luks:btrfs" diskUUID="eab4777f-2033-473c-ae67-a392aa7474a5" diskAutotrim="on" diskCompression="on" diskWarning="" diskCritical="" diskShareEnabled="yes" diskShareFloor="" diskComment="" diskExport="-" diskFruit="no" diskSecurity="public" diskReadList="" diskWriteList="" diskVolsizelimit="" diskCaseSensitive="auto" diskExportNFS="-" diskExportNFSFsid="0" diskSecurityNFS="public" diskHostListNFS="" diskFsProfile="raid1" diskFsWidth="3" diskFsGroups="1" diskSpindownDelay="-1" diskSpinupGroup="" diskId="Samsung_SSD_850_PRO_256GB_S251NXAH116032A" diskIdSlot="-" diskSize="250059064" diskId.1="Samsung_SSD_850_PRO_256GB_S251NXAH116940Y" diskIdSlot.1="-" diskSize.1="250059064" diskId.2="" diskIdSlot.2="-" diskSize.2="0" I've attempted to remove the Pool which basically just stalls the GUI. + The pool can't be removed via GUI, has to be removed via deleting the pool cfg file. There appears to be something wrong with the config somewhere else? Edited January 26, 20251 yr by Little_Jon
January 26, 20251 yr Community Expert 4 hours ago, Little_Jon said: I went searching in the syslog.txt file for Error, Cache and Pool - I couldn't see anything It was this line: Jan 26 23:20:50 ANC-S-01 emhttpd: pool_state_btrfs 3166 rdevStatus.32 not found 4 hours ago, Little_Jon said: I've added the SSDs into a pool and it's showing a third drive That is likely what caused the issue, pool is missing a device, v7.0 supports importing a degraded pool, but not removing the missing device, please post output from: btrfs fi usage -T /mnt/ssds I can then post the command to remove the device manually, and then you just need to reimport the pool once more.
January 27, 20251 yr Author Thanks Jorge The drives are empty, I moved everything off of them whilst I was looking into the ZFS pool mechanics and ZIL/L2ARC. root@UNRAID:~# btrfs fi usage -T /mnt/ssds Overall: Device size: 709.79GiB Device allocated: 4.06GiB Device unallocated: 705.73GiB Device missing: 232.87GiB Device slack: 0.00B Used: 288.00KiB Free (estimated): 353.86GiB (min: 353.86GiB) Free (statfs, df): 237.43GiB Data ratio: 2.00 Metadata ratio: 2.00 Global reserve: 5.50MiB (used: 0.00B) Multiple profiles: no Data Metadata System Id Path RAID1 RAID1 RAID1 Unallocated Total Slack -- ---------- ------- --------- -------- ----------- --------- ----- 1 /dev/dm-12 1.00GiB 1.00GiB 32.00MiB 236.43GiB 238.46GiB - 2 /dev/dm-13 - 1.00GiB 32.00MiB 237.43GiB 238.46GiB - 3 missing 1.00GiB - - 231.87GiB 232.87GiB - -- ---------- ------- --------- -------- ----------- --------- ----- Total 1.00GiB 1.00GiB 32.00MiB 705.73GiB 709.79GiB 0.00B Used 0.00B 128.00KiB 16.00KiB Edited January 27, 20251 yr by Little_Jon
January 27, 20251 yr Community Expert With the array started, type on the CLI btrfs dev del missing /mnt/ssds Once the device is removed, it should take a few seconds, stop the array and reimport the pool again: on main click on the first device for that pool and then "remove pool" back on main, create a new pool with the same name and number of slots assign the pool device(s), leave the filesystem set to auto start the array to import the pool
January 28, 20251 yr Author Whelp.. you made that pretty easy tbh. If you're not getting paid for supporting all of us, you should be. In your experience, is there any benefit of setting a higher block size for storing video files? Edited January 28, 20251 yr by Little_Jon
January 28, 20251 yr Community Expert 46 minutes ago, Little_Jon said: is there any benefit of setting a higher block size for storing video files? AFAIK there's no block size setting for btrfs, for zfs it does help if you set a higher record size with that type of files, default is 128K, 1M usually performs better with large fields, especially with raidz, not so much difference with mirrors, at least for me.
January 28, 20251 yr Author That's really good info. Does the same advise follow for the array running e-xfs?
January 28, 20251 yr Community Expert xfs also doesn't have a block setting, so just use it for any type of files.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.