Yivey_unraid

Hi! I have another thread in the forum, but as that started out regarding nginx crashes for the webUI I feel that a new thread was necessary to get proper support in this issue. Hopefully is this the source of the problems I've had.... This is my last post in that thread: Does any one have any ideas on how to solve this? Is my only option to buy a new MB and CPU? How does something like this happen? EDIT: The specs for my system at the moment is: Intel i5-11600 MSI Z590 Torpedo with built-in 2.5 GbE and 1 GbE Corsair Vengeance LPX Black DDR4 3200MHz 32GB (2x16GB) (New RAM since 2023-10-04) G.Skill Aegis DDR4-3200 DIMM CL16 32GB (2x16GB) (Original RAM, now removed from the system) Adaptec ASR-71605 RAID card in HBA mode Dual NVMe PCIe adapter card with ASMedia ASM2812 chipset (problems started long before this was added) 9 x 3.5" HDDs (HBA) 2 x 2.5" SATA SSDs (HBA) 1 x 2.5" SATA SSDs (MB) 2 x M.2 NVMe SSDs (MB) 2 x M.2 NVMe SSDs (ASM) Also tried using a TP-link TG-3468 PCIe 1 GbE card to rule out the onboard NICs

Update again... I found an old NUC that I ran HA on earlier. Connected it to the same switch as my main server and booted the same flash from the previous post that gave errors in my main system. Everything works on this NUC system as far as I can stresstest it just by installing Docker containers, plugins, removing all of them, clearing the Docker image and installing again. As it looks now, I'm pretty certain that it's the main servers hardware that are faulty in some way. As far as I understand SHA256 is a hash crypto function carried out by the CPU, but it could also be . Something is corrupting the data somewhere, or the algorithm isn't working (highly unlikely). I was still not sure that the problem wasn't related to the network parts on the motherboard. Going through the MB manual I found that all networking functionality is going through the chipset in all the ways I've tested. So I tested the PCIe Ethernet card in PCI_E1 socket to rule out that it's the chipset that's faulty. At first I got my hopes up, as it seemed to actually be working, but no bueno. 🙈 I don't know if I can rule the chipset out completely since the USB is still connected via it, and I don't know a way around that short of buying a PCIe card with both ethernet and USB. Now, the only thing I can think of to do further is to disassemble the PC and reseat the CPU in the socket. Don't really see what difference that'll make but I'll give it a try. I'm out of ideas, and buying a new MB and CPU on a hunch is really the last resort... Any other suggestions of what to look for or try out?

Tried some more. I installed a TP-Link PCIe Ethernet card and tried with a new fresh install of 6.12.4 on a new flash. Still same problem. So either it's a problem in my network, or it's a problem in the MB/CPU. This is the BIOS screen for the security settings on the MB. It does mention SHA256, which is what is erroring most frequently during my tries. Should I have this setup in any other way? This is settings I've never touched before. Should I start a new thread with these problems since my initial questions wasn't really about this SHA256 problem? The initial problem perhaps was caused by this problem though..

I will start to try and narrow down what HW is causing the issue. Also maybe move the server and physically try it on another network. Don't really have any other system to try it on, except for my old HP N40L Microserver. That doesn't have room for the HBA etc. But I can give the trial flash a go in it. Would you say I need to recreate the flash every time I do a new HW test? Like in the sense that faulty HW would corrupt the flash in some way so it'll present faults even when run on good HW?

Ok, I'm soon about to give up.... 😫 I've tried so many things now without getting it to work properly again. Every time I try something new it creates a new problem. I've done this (I'm sorry if it's not in chronological order, I should've kept a log on all my tests): Started with the bare minimum like @JorgeB suggested on the old flash but with a new fresh install. Started installing stuff back manually. Kept having nginx problems. Started with the bare minimum but on a new flash drive that I bought. Same deal, kept having nginx problems. And often the Docker engine couldn't start. If it started I wasn't able to install any containers due to to various errors but the most common was "docker: error pulling image configuration: image config verification failed for digest sha256". Started a completely new trial version on another new flash drive but kept having the same issues as above, minus the nginx one. Can install most plugins without an error, but no containers at all. Most installs look like this (just picked a random container in CA): During all these trial and error sessions I've deleted the docker directory multiple times, tried switching back to docker image. Moving the image or dir to a different pool with different FS. No luck. Same result. Tried reverting back to not use SSL. nginx still produced these type of errors in the log: Multiple times CA has had problems loading and suggested me to change the DNS settings. I've done that and tried both the suggested DNS settings as well as others. Both setting them static on the server as well as on the router. No change on either the "rebuild" system or the trial. I've tried using both my NIC's on my MB (MSI Z590 Torpedo). Both are Intel, one is 1 GbE and the other is 2.5 GbE. Both in bonding mode and separate. No change. I even changed out my switch to another since I had one laying around. No change. I bought two new 16 GB RAM to test out. No change. I guess I'll have 64 GB now if I ever get this going again... I tried using an USB to ethernet adapter to see if it was the onboard NIC's but couldn't get unraid to see the adapter and trying to boot with it connected to the system failed as I guess it tries to boot from the adapter as it's USB. Changed out the ethernet cable between the switch and the server. No change. I got desperate and updated the FW on both the MB and the HBA (Adaptec ASR-71605). No change. Really starting to suspect it's a hardware issue. Especially since a completely new trial version doesn't work either. I've attached two diagnostics. One is from the last time I ran my "rebuild" flash and one is the trial version. Unfortunately I've done so many restarts but these at least should show the problems I have with the Docker engine and how I'm not able to install any containers. What should I do here?!? I've put 25+ hours into this hunt for a needle in a haystack now. PLEASE HELP! define-7-diagnostics-20231005-2328_pro_version.zip tower-test-diagnostics-20231006-0119_trial_version.zip

@JorgeB What is your take on the fact that the problems keep occurring even though I restored the flash from a 1 week old backup? Back then I had no issues at all. Off course something could’ve been problematic in the setup back then already, just not presenting symptoms, but shouldn’t this work like a snapshot? I’ve found this bug report and my symptoms are very similar, but the problem is that I haven’t had IPv6 activated at all, only IPv4. But I tried without any plugins at all earlier, still same issues? I will try this again though. I guess the connect plugin is still named dynamix.my.servers in the plugins folder, because I can’t find any plugin named Connect?

nginx: 2023/09/28 20:07:12 [error] 19782#19782: SUB:WEBSOCKET:ws_recv NOT OK when receiving payload This error message is coming up a lot in the logs.

Waking up a dead thread here, but did you get anywhere with this?

I've tried a bunch of stuff now trying to narrowing it down. I did a fresh install off 6.12.4 on the old flash and copied over the bare minimum that you suggested. Tried to see if I could provoke an nginx failure. Everything seemed to work fine and I couldn't find any faults. Started moving over more stuff and trying. But eventually grew impatient since every try involved starting/stopping the server and handling the USB back and forth. Tried to only remove the plugins from the config folder. That wasn't the problem either since nginx kept crashing. Managed to grab a log from one of the times it was happening. Look around 20:14 in the attached syslog. Don't know if that's something that can point somewhere? After that I restarted nginx but eventually it stopped working all together. Also tried restoring to a week old flash backup, but that was also presenting the exact same issues with nginx crashing the unraid API. Even though I had none of those issues back then? What does that mean? I will keep trying to restore just parts of the config since the first stuff I tried didn't work. Feels like I'm stumbling blind here and looking for a needle in a haystack... define7-diagnostics-20230928-2024.zip

Does this approach mean I'll have to reconfigure all the plugins, modprobe.d, go-file, ssh keys, shares, network configs etc etc? Not possible to start with something less "destructive"? Sort of the other way around, and start by removing some parts of the install on the old flash? If so, what parts should I start removing? I appreciate the help!

OK, I tried now and started up a completely fresh install on a new flash drive. It work's but I haven't started the array or anything as I don't want to screw anything up. So now you suggest to copy some flies from the old install, make a fresh install on the old flash, then copy back those previous files to the new install? What files should I keep and what will I end up having to reconfigure?

I'll give that a go! What is it that points to the USB drive being faulty?

Any takers? I’ve also tried booting into safe mode but the webUI still crashes.

Hi! Started getting problems with my server the other day. By then I was on 6.12.3 and it all started with some Docker containers acting up. Went into the Unraid WebUI and the Docker page said that Docker was unable to start. Restarted Docker and got it working again. Then today it acted up again, and I also started getting problems with Unraid WebUI crashing on me. Restarted nginx over SSH using /etc/rc.d/rc.nginx restart That got the WebUI up again, but despite restarting the whole server Docker won't restart at all. SSH worked all the time though. In a futile attempt to perhaps fix it, I thought that I would try to upgrade to 6.12.4. So I pulled the USB stick and did a manual update. Everything started up ok, but now I couldn't start the array because the WebUI crashed on me again. Restarted nginx multiple times until it stopped working all together. I SSH:ed into the server and pulled the Diagnostics before shutting everything down. Any ideas on what's causing this? I can't figure it out in the logs. Unfortunately I don't have a remote syslog server. define7-diagnostics-20230926-2316.zip

Hmm, I was recommending this Vorta container to someone and they didn’t find it. I can’t either anymore, where did it go?

I have relatively little experience with proxmox, but I do like it better as a hypervisor than unraid. And I've run unraid for more than a decade. Not that I'm a power user in the VM area by any means, but I did find proxmox much easier to use for that. So I'd go with alternative nr 2, but that is without having any experience running unraid under proxmox. I just know it've been done. What specific is it you need Mac for, and what is the reason to make it relatively complicated with running macOS as a VM? Just curious.

Uninstalled and reinstalled the Mover tuning plugin, and that seem to have done the trick. I reused all the same settings and then manually activated Mover. Must have been some hiccup with the plugin after the upgrade to 6.12.3 that kept it from functioning properly. I'll monitor it the following days to see that it follows the set schedule. Thanks for the help @JorgeB 🙌

Mover logging is (was) enabled and Mover was run prior to downloading the diagnostics.

I only have one share that is suppose to move from “cache_downloads” to array, it’s named “unraid_data”.

Hi! Recently updated to 6.12.3 and ever since I've got this problem that my cache (named "cache_download") is filling up, which is sort of normal, but Mover isn't moving the files accordingly. The cache is 2 TB (1.87 TB used) and the torrent section of it is excluded from moving in Mover Tuning. Have had these settings for probably a year and it works just like I want it to, except now. The torrent section is roughly 800 GB and that means it's around 1 TB of data that Mover isn't moving. I've tried manually activating Mover and the logs just shows it starting and finishing in the same second. Something is prohibiting Mover from moving the files and I can't figure it out, please help! Perhaps it's a permission problem that some of the 'arrs are creating? define7-diagnostics-20230820-2309.zip

If you haven't installed it yet, install "Dynamix File Manager" from CA and using that you can view the shares content and it also gives you a Location column where each directory/file is located on your system. You can also use the file manager to move those files into your fastcache pool.

OK, seems like Unpackerr is the problem and not my setup. I didn't think it was Unpackerr at first because I have three more instances that is running :latest just fine. Perhaps that's because they have Sonarr connections and the non working ones haven't.. I downgraded to version 0.10.1 for those with "[PANIC] runtime error" and that worked.

With Unpackerr or some other container? I have other instances of it still running though..

Anyone have any clue here? 🙂

Hi! I'm having troubles getting two containers to start. They flat out refuse. It's two separate instances of Unpackerr that has worked flawlessly for a long time. Log output: 2023/01/18 01:07:11 Unpackerr v0.11.1 Starting! PID: 1, UID: 0, GID: 0, Now: 2023-01-18 01:07:11 +0100 CET 2023/01/18 01:07:11 Missing Lidarr URL in one of your configurations, skipped and ignored. 2023/01/18 01:07:11 Missing Sonarr URL in one of your configurations, skipped and ignored. 2023/01/18 01:07:11 ==> GoLift Discord: https://golift.io/discord <== 2023/01/18 01:07:11 ==> Startup Settings <== 2023/01/18 01:07:11 => Sonarr Config: 0 servers 2023/01/18 01:07:11 [PANIC] runtime error: index out of range [0] with length 0 I've tried restarting the server and starting in Safe Mode. No difference unfortunately. Any thoughts? I've recently had problems with getting "Error: filesystem layer verification failed for digest sha256" intermittently when installing or updating containers. That problem isn't really solved, but I can't recall ever getting that problem with these two containers. Don't know if this is relevant in any way but I though I'd mention it. define7-diagnostics-20230118-0118.zip