October 15, 20241 yr Hi all, I have a problem currently that prevents me from removing certain directories/ files in my appdata directory for my dockers. In particular, I have a problem in removing this specific dir; "/mnt/user/appdata/bbergle-jellyfin/metadata/library". I first noticed a problem when doing anything with docker after power loss (I turned my house breaker off installing some lights... multiple times... without UPS... oops), then confirmed it was a problem after disabling docker. I tried fixing it by deleting docker .img file, and then restoring from my appdata backup plugin. It always hanged up the entire system, so I decided to clean it all. I tried rm -rf /mnt/user/appdata, but it hanged the same way as if I tried to restore. I then tried rm -rf --verbose and individually removed each docker dir, then found out it always hung on that specific jellyfin dir. Is there a way to forcibly remove this appdata share or this problematic dir? Currently when it hangs, I cannot even kill the rm -rf process via open files. /mnt/user/appdata (working directory) /mnt/user/appdata/bbergle-jellyfin/metadata/library/0f /mnt/user/appdata/bbergle-jellyfin/metadata /mnt/user/appdata/bbergle-jellyfin/metadata/library /mnt/user/appdata/bbergle-jellyfin/metad...rary/0f/0f206794f8026ce960d2a1ef94b57379 /mnt/user/appdata/bbergle-jellyfin/metad...f206794f8026ce960d2a1ef94b57379/chapters Ending process 4171... Checking... Process 4171 could not be gently killed... will use SIGKILL... Process 4171 ()... FAILED.... I cannot even shut down system with the shutdown command/ button, and must cycle power to restart. Thanks. System specs if it helps; Unraid OS Starter 6.12.13 ASRock B760M PG Lightning/D4, 12th Gen Intel® Core™ i3-12100 @ 4059 MHz, 16 GiB DDR4 Array in hybrid ZFS Parity - 16TB Re-certified Seagate Exos X20 Disk 1 - 10TB Re-certified HGST Ultrastar He10 Disk 2 - 16TB Re-certified Seagate Exos X20 Pool in RAID 0 ZFS Cache 1 - 512GB SK PC601 Cache 2 - 2TB Seagate Firecuda 530
October 15, 20241 yr Author Hi, TL,DR; Fucked the system even more (system hangs now while mounting disks) while trying to reproduce error, but plenty of diag after reboots and dialogue on diag capture while system is hanging. Timeline after trying to gather diag after delete attempt. Booting up the system as I usually do ended up in system hang, after inputting encryption key and mounting disks. I suspect due to me cycling power after another deletion attempt yesterday, while the parity check was ongoing (and I didn't abort it before attempting deletion). Had to cycle power (by turning off/ on PSU), then immediately captured diagnostics before mounting disks. (Included, 1320) Then I tried mounting disks again, but it hanged in the exact same way. Attempted to capture diag, but it hung as well. (Screenshot + dialogue? pasted) mkdir -p /boot/logs mkdir -p '/zulualpha-diagnostics-20241015-1326/system' '/zulualpha-diagnostics-20241015-1326/config' '/zulualpha-diagnostics-20241015-1326/logs' '/zulualpha-diagnostics-20241015-1326/shares' '/zulualpha-diagnostics-20241015-1326/smart' '/zulualpha-diagnostics-20241015-1326/qemu' '/zulualpha-diagnostics-20241015-1326/xml' top -bn1 -o%CPU 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/top.txt' tail /boot/bz*.sha256 >> '/zulualpha-diagnostics-20241015-1326/unraid-6.12.13.txt' uptime nproc lscpu 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lscpu.txt' lsscsi -vgl 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lsscsi.txt' lspci -knn 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lspci.txt' lsusb 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lsusb.txt' free -mth 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/memory.txt' ps -auxf --sort=-pcpu 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/ps.txt' lsof -Pni 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lsof.txt' lsmod|sort 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/lsmod.txt' df -h 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/df.txt' ip -br a|awk '/^(eth|bond)[0-9]+ /{print $1}'|sort dmidecode -qt2|awk -F: '/^ Manufacturer:/{m=$2};/^ Product Name:/{p=$2} END{print m" -"p}' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/motherboard.txt' dmidecode -qt0 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/motherboard.txt' cat /proc/meminfo 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/system/meminfo.txt' dmidecode --type 17 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/meminfo.txt' ethtool 'bond0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/ethtool.txt' ethtool -i 'bond0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/ethtool.txt' ethtool 'eth0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/ethtool.txt' ethtool -i 'eth0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/ethtool.txt' ip -br a|todos >'/zulualpha-diagnostics-20241015-1326/system/ifconfig.txt' sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1326/system/ifconfig.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/system/ifconfig.txt' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/system/ifconfig.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1326/system/ifconfig.txt' 2>/dev/null find /sys/kernel/iommu_groups/ -type l 2>/dev/null|sort -V|todos >'/zulualpha-diagnostics-20241015-1326/system/iommu_groups.txt' todos '/zulualpha-diagnostics-20241015-1326/system/cmdline.txt' echo -ne ' /boot ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/boot'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /boot/config ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/boot/config'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /boot/config/plugins ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/boot/config/plugins'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /boot/syslinux ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/boot/syslinux'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /var/log ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/var/log'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /var/log/plugins ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/var/log/plugins'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /boot/extra folder does not exist ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /var/log/packages ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/var/log/packages'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /var/lib/pkgtools/packages ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/var/lib/pkgtools/packages'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' echo -ne ' /tmp ' >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt';ls -l '/tmp'|todos >>'/zulualpha-diagnostics-20241015-1326/system/folders.txt' cp /boot/config/*.{cfg,conf,dat} '/zulualpha-diagnostics-20241015-1326/config' 2>/dev/null cp /boot/config/go '/zulualpha-diagnostics-20241015-1326/config/go.txt' 2>/dev/null sed -i -e '/password/c ***line removed***' -e '/user/c ***line removed***' -e '/pass/c ***line removed***' '/zulualpha-diagnostics-20241015-1326/config/go.txt' sed -ri 's/^((disk|flash)(Read|Write)List.*=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/config/*.cfg' 2>/dev/null sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1326/config/network.cfg' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/config/network.cfg' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/config/network.cfg' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1326/config/network.cfg' 2>/dev/null /usr/local/emhttp/webGui/scripts/show_interfaces ip|tr -d ' '|tr '#' ' '|tr ',' ' ' >'/zulualpha-diagnostics-20241015-1326/config/listen.txt' /usr/local/emhttp/webGui/scripts/error_interfaces|sed 's///' >>'/zulualpha-diagnostics-20241015-1326/config/listen.txt' sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1326/config/listen.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/config/listen.txt' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1326/config/listen.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1326/config/listen.txt' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/appdata.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/A-----e.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/B-----s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/domains.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/D-------s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/G---s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/isos.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/M---a.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/system.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/VM.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1326/shares/Z---------------y.cfg' 2>/dev/null ls -l /dev/disk/by-id/[asun]* 2>/dev/null|sed '/-part/d;s|^.*/by-id/[^-]*-||;s|-> ../../||;s|:|-|' smartctl -x '/dev/sdd' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/HUH721010ALE601_2YGA4JMD-2024-07-16 disk1 (sdd).txt' smartctl -x '/dev/sdb' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/ST16000NM000D-3PC101_ZVTBE47N-2024-07-16 parity (sdb).txt' smartctl -x '/dev/sdc' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/ST16000NM000D-3PC101_ZVTBJQK2-2024-07-16 disk2 (sdc).txt' smartctl -x '/dev/nvme1' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/PC601_NVMe_SK_hynix_512GB_SN94N235312604R21-2024-07-16 cache (nvme1).txt' smartctl -x '/dev/nvme0' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/Seagate_FireCuda_530_ZP2000GM30013_7VR0086Q-2024-07-16 cache2 (nvme0).txt' smartctl -x '/dev/nvme0' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/eui.6479a74f7f00124c-2024-07-16 cache2 (nvme0).txt' smartctl -x '/dev/nvme1' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/eui.ace42e00901df3522ee4ac0000000001-2024-07-16 cache (nvme1).txt' smartctl -x '/dev/sda' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1326/smart/Flash_USB_Disk_3727023E4182456931317-0-0-2024-07-16 flash (sda).txt' /usr/sbin/zpool status 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1326/system/zfs-info.txt' Cycled power again, another diag. capture before mounting (1334). Did not mount disks, used GUI for safe reboot, as this has fixed the problem before while it hanged due to parity check (suspected). Another diag captured, before mounting, in safe mode (1337). System hung again, after attempting to mount. Same dialogue as above, with last entry being /usr/sbin/zpool status 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1340/system/zfs-info.txt' Restarted (by long pressing power button), attempted diag capture before mounting. Dialogue; mkdir -p /boot/logs /usr/local/emhttp/plugins/fix.common.problems/scripts/scan.php diagnostics Fix Common Problems Scan: Downloading Support Files Fix Common Problems Scan: 1% HPApresent Fix Common Problems Scan: 2% checkServerDate Fix Common Problems Scan: 3% eth0NoIP Fix Common Problems Scan: 4% isArrayStarted Fix Common Problems Scan: 6% impliedArrayFilesOnCache Fix Common Problems Scan: 7% cacheOnlyFilesOnArray Fix Common Problems Scan: 8% arrayOnlyFilesOnCache Fix Common Problems Scan: 9% pluginUpdateCheck Fix Common Problems Scan: 11% dockerUpdateCheck Fix Common Problems Scan: 12% autoUpdateCheck Fix Common Problems Scan: 13% sameShareDifferentCase Fix Common Problems Scan: 14% outsideCommunication Fix Common Problems Scan: 16% dockerAppdataCacheOnly Fix Common Problems Scan: 17% disabledDisksPresent Fix Common Problems Scan: 18% missingDisksPresent Fix Common Problems Scan: 19% readErrorsPresent Fix Common Problems Scan: 20% fileSystemErrors Fix Common Problems Scan: 22% pluginsUpToDate Fix Common Problems Scan: 23% dockerUpToDate Fix Common Problems Scan: 24% dockerConfigUserShare Fix Common Problems Scan: 25% varLogFilled Fix Common Problems Scan: 27% dockerImageFull Fix Common Problems Scan: 28% rootfsFull Fix Common Problems Scan: 29% scheduledParityChecks Fix Common Problems Scan: 30% shareIncludeExcludeSet Fix Common Problems Scan: 32% shareIncludeExcludeSameDisk Fix Common Problems Scan: 33% UDmountedSlaveMode Fix Common Problems Scan: 34% checkNotifications Fix Common Problems Scan: 35% blacklistedPluginsInstalled Fix Common Problems Scan: 37% illegalShareName Fix Common Problems Scan: 38% writeToDriveTest Fix Common Problems Scan: 39% flashDriveFull Fix Common Problems Scan: 40% checkForHack Fix Common Problems Scan: 41% checkForModeration Fix Common Problems Scan: 43% pluginNotCompatible Fix Common Problems Scan: 44% cacheOnlyNoCache Fix Common Problems Scan: 45% shareNameSameAsDiskName Fix Common Problems Scan: 46% extraParamInRepository Fix Common Problems Scan: 48% outOfMemory Fix Common Problems Scan: 49% mceCheck Fix Common Problems Scan: 50% inotifyExhausted Fix Common Problems Scan: 51% reiserCache Fix Common Problems Scan: 53% SSDcacheNoTrim Fix Common Problems Scan: 54% marvelControllerTest Fix Common Problems Scan: 55% breadTest Fix Common Problems Scan: 56% lessThan2G Fix Common Problems Scan: 58% checkDockerCompatible Fix Common Problems Scan: 59% CPUoverheat Fix Common Problems Scan: 60% statsButNoPreclear Fix Common Problems Scan: 61% moverLogging Fix Common Problems Scan: 62% phpWarnings Fix Common Problems Scan: 64% invalidIncludedDisk Fix Common Problems Scan: 65% CPUSet Fix Common Problems Scan: 66% isolatedCPUdockerCollision Fix Common Problems Scan: 67% testXML Fix Common Problems Scan: 69% writeCacheDisabled Fix Common Problems Scan: 70% updatePluginSupport Fix Common Problems Scan: 71% flashSyslog Fix Common Problems Scan: 72% unassignedDevicesPlus Fix Common Problems Scan: 74% sysdream Fix Common Problems Scan: 75% caNotifications Fix Common Problems Scan: 76% legacyVFIO Fix Common Problems Scan: 77% checkBonding Fix Common Problems Scan: 79% checkSameNetwork Fix Common Problems Scan: 80% jumboFrames Fix Common Problems Scan: 81% authorizedKeysInGo Fix Common Problems Scan: 82% reservedUserName Fix Common Problems Scan: 83% rootPassword Fix Common Problems Scan: 85% xmrig Fix Common Problems Scan: 86% shareSpace69 Fix Common Problems Scan: 87% testTLD Fix Common Problems Scan: 88% testDockerOptsIp Fix Common Problems Scan: 90% wrongCachePoolFiles Fix Common Problems Scan: 91% corruptFlash Fix Common Problems Scan: 92% dockerUpdatePatch Fix Common Problems Scan: 93% macvlan Fix Common Problems Scan: 95% legacyMyServers Fix Common Problems Scan: 96% macvlanfix Fix Common Problems Scan: 97% dockerPatch6812 Fix Common Problems Scan: 98% oldStyleSSDTrim Fix Common Problems Scan: 100% syslinuxRoot mkdir -p '/zulualpha-diagnostics-20241015-1343/system' '/zulualpha-diagnostics-20241015-1343/config' '/zulualpha-diagnostics-20241015-1343/logs' '/zulualpha-diagnostics-20241015-1343/shares' '/zulualpha-diagnostics-20241015-1343/smart' '/zulualpha-diagnostics-20241015-1343/qemu' '/zulualpha-diagnostics-20241015-1343/xml' top -bn1 -o%CPU 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/top.txt' tail /boot/bz*.sha256 >> '/zulualpha-diagnostics-20241015-1343/unraid-6.12.13.txt' uptime nproc lscpu 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lscpu.txt' lsscsi -vgl 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lsscsi.txt' lspci -knn 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lspci.txt' lsusb 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lsusb.txt' free -mth 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/memory.txt' ps -auxf --sort=-pcpu 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/ps.txt' lsof -Pni 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lsof.txt' lsmod|sort 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/lsmod.txt' df -h 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/df.txt' ip -br a|awk '/^(eth|bond)[0-9]+ /{print $1}'|sort dmidecode -qt2|awk -F: '/^ Manufacturer:/{m=$2};/^ Product Name:/{p=$2} END{print m" -"p}' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/motherboard.txt' dmidecode -qt0 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/motherboard.txt' cat /proc/meminfo 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/system/meminfo.txt' dmidecode --type 17 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/meminfo.txt' ethtool 'bond0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/ethtool.txt' ethtool -i 'bond0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/ethtool.txt' ethtool 'eth0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/ethtool.txt' ethtool -i 'eth0' 2>/dev/null|todos >>'/zulualpha-diagnostics-20241015-1343/system/ethtool.txt' ip -br a|todos >'/zulualpha-diagnostics-20241015-1343/system/ifconfig.txt' sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1343/system/ifconfig.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/system/ifconfig.txt' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/system/ifconfig.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1343/system/ifconfig.txt' 2>/dev/null find /sys/kernel/iommu_groups/ -type l 2>/dev/null|sort -V|todos >'/zulualpha-diagnostics-20241015-1343/system/iommu_groups.txt' todos '/zulualpha-diagnostics-20241015-1343/system/cmdline.txt' echo -ne ' /boot ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/boot'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /boot/config ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/boot/config'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /boot/config/plugins ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/boot/config/plugins'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /boot/syslinux ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/boot/syslinux'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /var/log ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/var/log'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /var/log/plugins ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/var/log/plugins'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /boot/extra folder does not exist ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /var/log/packages ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/var/log/packages'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /var/lib/pkgtools/packages ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/var/lib/pkgtools/packages'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' echo -ne ' /tmp ' >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt';ls -l '/tmp'|todos >>'/zulualpha-diagnostics-20241015-1343/system/folders.txt' cp /boot/config/*.{cfg,conf,dat} '/zulualpha-diagnostics-20241015-1343/config' 2>/dev/null cp /boot/config/go '/zulualpha-diagnostics-20241015-1343/config/go.txt' 2>/dev/null sed -i -e '/password/c ***line removed***' -e '/user/c ***line removed***' -e '/pass/c ***line removed***' '/zulualpha-diagnostics-20241015-1343/config/go.txt' sed -ri 's/^((disk|flash)(Read|Write)List.*=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/config/*.cfg' 2>/dev/null sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1343/config/network.cfg' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/config/network.cfg' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/config/network.cfg' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1343/config/network.cfg' 2>/dev/null /usr/local/emhttp/webGui/scripts/show_interfaces ip|tr -d ' '|tr '#' ' '|tr ',' ' ' >'/zulualpha-diagnostics-20241015-1343/config/listen.txt' /usr/local/emhttp/webGui/scripts/error_interfaces|sed 's///' >>'/zulualpha-diagnostics-20241015-1343/config/listen.txt' sed -ri 's/(["\[ ])(127|10|172\.1[6-9]|172\.2[0-9]|172\.3[0-1]|192\.168)((\.[0-9]{1,3}){2,3}([/" .]|$))/\1@@@\2\3/g; s/(["\[ ][0-9]{1,3}\.)([0-9]{1,3}\.){2}([0-9]{1,3})([/" .]|$)/\1XXX.XXX.\3\4/g; s/@@@//g' '/zulualpha-diagnostics-20241015-1343/config/listen.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/config/listen.txt' 2>/dev/null sed -ri 's/(\[([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\])([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\6/g' '/zulualpha-diagnostics-20241015-1343/config/listen.txt' 2>/dev/null sed -ri 's/(([0-9a-f]{1,4}:){4})(([0-9a-f]{1,4}:){3}|:)([0-9a-f]{1,4})(\/[0-9]{1,3})([ .:/]|$)/\1XXXX:XXXX:XXXX:\5\7/g' '/zulualpha-diagnostics-20241015-1343/config/listen.txt' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/appdata.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/A-----e.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/B-----s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/domains.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/D-------s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/G---s.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/isos.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/M---a.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/system.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/VM.cfg' 2>/dev/null sed -ri 's/^(share(Comment|ReadList|WriteList)=")[^"]+/\1.../' '/zulualpha-diagnostics-20241015-1343/shares/Z---------------y.cfg' 2>/dev/null ls -l /dev/disk/by-id/[asun]* 2>/dev/null|sed '/-part/d;s|^.*/by-id/[^-]*-||;s|-> ../../||;s|:|-|' smartctl -x '/dev/sdd' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/smart/HUH721010ALE601_2YGA4JMD-2024-07-16 disk1 (sdd).txt' smartctl -x '/dev/sdb' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/smart/ST16000NM000D-3PC101_ZVTBE47N-2024-07-16 parity (sdb).txt' smartctl -x '/dev/sdc' 2>/dev/null|todos >'/zulualpha-diagnostics-20241015-1343/smart/ST16000NM000D-3PC101_ZVTBJQK2-2024-07-16 disk2 (sdc).txt' Force restart (power button again), then immediately went for GUI safe restart. Captured diag before mounting. (1350) System hangs while mounting. Force restart - GUI safe reboot - maintenance mode - diag. (1407) GUI reboot, normal. Tried mounting, system hang. Gave up. zulualpha-diagnostics-20241015-1320.zip zulualpha-diagnostics-20241015-1334.zip zulualpha-diagnostics-20241015-1337.zip zulualpha-diagnostics-20241015-1350.zip zulualpha-diagnostics-20241015-1407.zip
October 15, 20241 yr Community Expert Get the syslog when if gets unresponsive during mount: cp /var/log/syslog /boot/syslog.txt
October 15, 20241 yr Author The syslogs when I tried mounting it after force reboot, normal mode. syslog.txt syslog1.txt
October 15, 20241 yr Community Expert The zfs pool is crashing on import, see if you can import it read-only, but note that since you are using encryption, you will need to manually decrypt all the devices before, then type: zpool import -o readonly=on cache If it works, backup any data under /mnt/cache and recreate the pool
October 15, 20241 yr Author So I have no clue what I'm doing, but Started system (first without the boot USB and was wondering what now), gave password and started in maintenance mode, then in web terminal gave in the command. It didn't return any error, so I'm assuming that it worked. Then the following; root@ZuluAlpha:~# cp -r /mnt/cache/ /boot/cachebackup/ which gave cp: cannot create special file '/boot/cachebackup/cache/appdata/binhex-qbittorrentvpn/qBittorrent/config/ipc-socket': Operation not permitted I then tried to move individual dirs, but I'm not sure if it froze or just my USB is slow as shit (USB 2.0) and isn't responding in time. Next steps? It's just slow, manually moving other files now. Edited October 15, 20241 yr by clambake-stolid
October 15, 20241 yr Community Expert You can use something like Midnight Commander, mc on the console, it would be easier to use.
October 15, 20241 yr Author Managed to back up any relevant data (although nuking them all is also fine), how do I recreate pool? Is there data loss (apart from the /mnt/cache)? Should I disable encryption for all before attempting this? Edited October 15, 20241 yr by clambake-stolid
October 15, 20241 yr Community Expert 17 minutes ago, clambake-stolid said: how do I recreate pool? with the array stopped, click on the pool, then "Erase", then start the array and re-format. 17 minutes ago, clambake-stolid said: Is there data loss (apart from the /mnt/cache? Not for this operation. 18 minutes ago, clambake-stolid said: Should I disable encryption for all before attempting this? Up to you, I recommend only using it if you really need it, it's another layer that can cause issues and/or make it more difficult to recover if there are other problems
October 15, 20241 yr Author Solution After a bit of finagling, managed to nuke the 2 pool NVMes, and after checking /mnt/user, there is no /appdata dir anymore. It seemed to have a little problem mounting, with an LUKS error 5? and system was hung up again while formatting. But I've restarted, nuked the SSDs again, and it mounted and formatted successfully. Any clue what the source of the problem was? Probably the SSD having a bad... sector? or something between power cycles? Thanks.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.