Bmalone

I added 2 new 18TB disks to my array yesterday and pre-clear started, and it shows as running. It's been 24 hours and it's only 9.7% completed. Also, it hasn't progressed all night and this morning and Unraid is showing 0% CPU usage, which I assume is a bug because I'm writing this from a VM hosted on the Unraid server and all my containers are running fine. In the syslog I see the events below. I have parity tuning set to run almost all day except from 16:00-00:00. Is this normal? I don't recall my original drives taking this long to clear. Apr 15 12:24:22 SmashySmash crond[2362]: exit status 255 from user root /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php "monitor" &>/dev/null Apr 15 12:30:23 SmashySmash crond[2362]: exit status 255 from user root /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php "monitor" &>/dev/null Apr 15 12:36:33 SmashySmash crond[2362]: exit status 255 from user root /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php "monitor" &>/dev/null Apr 15 12:42:24 SmashySmash crond[2362]: exit status 255 from user root /usr/local/emhttp/plugins/parity.check.tuning/parity.check.tuning.php "monitor" &>/dev/null

Today I tried to install a new container, deleted all the appdata from the previous install, and rolled back to older versions and I'm seeing the errors below consistently. Any idea why this all of a sudden stopped working? time=2024-04-11T16:27:59.961-05:00 comm=netdata source=health level=warning tid=709 thread=HEALTH msg_id=9ce0cb58ab8b44df82c4bf1ad9ee22de node=Goathead instance=app.dhcp_fds_open_limit context=app.fds_open_limit code=0 alert_id=1712871056 alert_unique_id=1712871195 alert_event_id=2 alert_transition_id=94e16a2fc9464250ad2e77411cf2c4bf alert_config=27ec43a6f27349f2808a959928f59920 alert=apps_group_file_descriptors_utilization alert_class=Utilization alert_component=Process alert_type=System alert_exec=/usr/libexec/netdata/plugins.d/alarm-notify.sh alert_recipient=sysadmin alert_duration=1 alert_value=500 alert_value_old=null alert_status=WARNING alert_value_old=UNINITIALIZED alert_units=% alert_summary="App group dhcp file descriptors utilization" alert_info="Open files percentage against the processes limits, among all PIDs in application group" alert_notification_timestamp=2024-04-11T16:27:59-05:00 msg="ALERT 'apps_group_file_descriptors_utilization' of instance 'app.dhcp_fds_open_limit' on node 'Goathead', transitioned from UNINITIALIZED to WARNING" time=2024-04-11T16:43:25.329-05:00 comm=cgroup-network source=collector level=error tid=1548 thread=cgroup-network msg="child pid 1549 exited with code 1." time=2024-04-11T16:43:25.329-05:00 comm=cgroup-network source=collector level=error tid=1548 thread=cgroup-network msg="Cannot find a cgroup PID from cgroup '/host/sys/fs/cgroup/docker/43c4a2d485238a93adc627e2338ecf01733bb284dc08a13e0bfe9cc7033a11b9'" time=2024-04-11T16:43:25.330-05:00 comm=netdata source=daemon level=error tid=761 thread=P[cgroups] msg="child pid 1548 exited with code 1."

I recently updated my Netdata Docker containers on my production and backup server which have an almost identical configuration, or as close as I can keep them, and after the update my production instance doesn't work and my backup server is fine. The app will start fine, but won't render any visualizations of the data and I'm getting the error below. I've tried deleting the container and reinstalling, but the error is the same. Any idea what could be causing this? time=2024-04-10T10:13:00.425-05:00 comm=apps.plugin source=collector level=error errno="3, No such process" tid=718 thread=apps.plugin msg="Cannot process /host/proc/12131/cmdline (command 'z_wr_iss')" time=2024-04-10T10:13:27.413-05:00 comm=apps.plugin source=collector level=error errno="3, No such process" tid=718 thread=apps.plugin msg="Cannot process /host/proc/36316/status (command 'zfs')" time=2024-04-10T10:19:58.407-05:00 comm=apps.plugin source=collector level=error errno="3, No such process" tid=718 thread=apps.plugin msg="Cannot process /host/proc/29752/limits (command 'zpool')"

Got it. Thanks for clarifying.

I would like to use this plugin but I have a question about it's configuration. In order to avoid conflicts, I understand that the scheduled parity check must be disabled if one wants to use the plugin. Therefore, I disabled it. When I try to turn on the parity check tuning, I get the error below and I'm unable to use it, despite not having any other parity check scheduled. I must be misunderstanding something. Can someone explain why I cannot use the plugin if I don't have any parity check scheduled and I've not selected the cumulative parity check option?

So I've left SSH off for a few days. Today, I turned it on again to try and test it again. What I noticed is as soon as I turned it on, performance was significantly impacted and the server hung, and the dashboard wouldn't render. When I looked in the syslog I can see that my server was absolutely flooded with Nginx events non-stop right after I turned it on. I tried to include the diagnostics, but there are so many events it won't complete. It won't even retrieve my folder directory in my media folder it's so busy. Does anyone know what might be causing this? Thousands upon thousands of these events... Apr 3 16:30:52 Goathead nginx: 2024/04/03 16:30:52 [alert] 6493#6493: worker process 31293 exited on signal 6 Apr 3 16:30:54 Goathead nginx: 2024/04/03 16:30:54 [alert] 6493#6493: worker process 31462 exited on signal 6 Apr 3 16:30:56 Goathead nginx: 2024/04/03 16:30:56 [alert] 6493#6493: worker process 31563 exited on signal 6 Apr 3 16:30:58 Goathead nginx: 2024/04/03 16:30:58 [alert] 6493#6493: worker process 31757 exited on signal 6 Apr 3 16:31:00 Goathead nginx: 2024/04/03 16:31:00 [alert] 6493#6493: worker process 31852 exited on signal 6 Apr 3 16:31:02 Goathead nginx: 2024/04/03 16:31:02 [alert] 6493#6493: worker process 32112 exited on signal 6 Apr 3 16:31:04 Goathead nginx: 2024/04/03 16:31:04 [alert] 6493#6493: worker process 32252 exited on signal 6 Apr 3 16:31:06 Goathead nginx: 2024/04/03 16:31:06 [alert] 6493#6493: worker process 32353 exited on signal 6 Apr 3 16:31:08 Goathead nginx: 2024/04/03 16:31:08 [alert] 6493#6493: worker process 32694 exited on signal 6 Apr 3 16:31:10 Goathead nginx: 2024/04/03 16:31:10 [alert] 6493#6493: worker process 32777 exited on signal 6 Apr 3 16:31:12 Goathead nginx: 2024/04/03 16:31:12 [alert] 6493#6493: worker process 32906 exited on signal 6 Apr 3 16:31:14 Goathead nginx: 2024/04/03 16:31:14 [alert] 6493#6493: worker process 33093 exited on signal 6 Apr 3 16:31:16 Goathead nginx: 2024/04/03 16:31:16 [alert] 6493#6493: worker process 33204 exited on signal 6 Apr 3 16:31:18 Goathead nginx: 2024/04/03 16:31:18 [alert] 6493#6493: worker process 33300 exited on signal 6 Apr 3 16:31:20 Goathead nginx: 2024/04/03 16:31:20 [alert] 6493#6493: worker process 33403 exited on signal 6 Apr 3 16:31:22 Goathead nginx: 2024/04/03 16:31:22 [alert] 6493#6493: worker process 33521 exited on signal 6 Apr 3 16:31:24 Goathead nginx: 2024/04/03 16:31:24 [alert] 6493#6493: worker process 33712 exited on signal 6 Apr 3 16:31:26 Goathead nginx: 2024/04/03 16:31:26 [alert] 6493#6493: worker process 33773 exited on signal 6 Apr 3 16:31:28 Goathead nginx: 2024/04/03 16:31:28 [alert] 6493#6493: worker process 34002 exited on signal 6 Apr 3 16:31:30 Goathead nginx: 2024/04/03 16:31:30 [alert] 6493#6493: worker process 34150 exited on signal 6 Apr 3 16:31:32 Goathead nginx: 2024/04/03 16:31:32 [alert] 6493#6493: worker process 34307 exited on signal 6 Apr 3 16:31:34 Goathead nginx: 2024/04/03 16:31:34 [alert] 6493#6493: worker process 34464 exited on signal 6 Apr 3 16:31:36 Goathead nginx: 2024/04/03 16:31:36 [alert] 6493#6493: worker process 34516 exited on signal 6 Apr 3 16:31:38 Goathead nginx: 2024/04/03 16:31:38 [alert] 6493#6493: worker process 34647 exited on signal 6 Apr 3 16:31:40 Goathead nginx: 2024/04/03 16:31:40 [alert] 6493#6493: worker process 34835 exited on signal 6 Apr 3 16:31:42 Goathead nginx: 2024/04/03 16:31:42 [alert] 6493#6493: worker process 34947 exited on signal 6 Apr 3 16:31:44 Goathead nginx: 2024/04/03 16:31:44 [alert] 6493#6493: worker process 35210 exited on signal 6 Apr 3 16:31:46 Goathead nginx: 2024/04/03 16:31:46 [alert] 6493#6493: worker process 35349 exited on signal 6 Apr 3 16:31:48 Goathead nginx: 2024/04/03 16:31:48 [alert] 6493#6493: worker process 35492 exited on signal 6 Apr 3 16:31:50 Goathead nginx: 2024/04/03 16:31:50 [alert] 6493#6493: worker process 35676 exited on signal 6 Apr 3 16:31:52 Goathead nginx: 2024/04/03 16:31:52 [alert] 6493#6493: worker process 35776 exited on signal 6 Apr 3 16:31:54 Goathead nginx: 2024/04/03 16:31:54 [alert] 6493#6493: worker process 35853 exited on signal 6 Apr 3 16:31:56 Goathead nginx: 2024/04/03 16:31:56 [alert] 6493#6493: worker process 36259 exited on signal 6 Apr 3 16:31:58 Goathead nginx: 2024/04/03 16:31:58 [alert] 6493#6493: worker process 36347 exited on signal 6

Yes of course.

Doe anyone have anything I can try to unlock the SSH capability? I assume that it is blocked and something needs to be done to unblock it, rather than it being a bug, but I don't know. Seems like a pretty fundamental feature to not work out of the box (albeit disables in the GUI which makes sense).

I still have it. Hence, my question above about a reliable way to delete a share using terminal. "The only thing the Fix Common Problems plugin is telling me now is to delete one of the Guacamole shares. However, every time I tried that, the server became unresponsive so I haven't tried it in a while. I also haven't added the parity drive back yet as I'm syncing my media to a backup server and will add it again after that completes. Is there a more reliable way to delete a share (they're empty) from the terminal?"

I don't know what is causing that. The IP address is my Mac, but I have no idea why it would be flooding Unraid with requests. I had to stop using Finder with Unraid a while ago because it basically crashes the Mac if I try to do anything. I can only use Windows now for any file operations. I don't know whether it's something to do with NFS or whether it's something to do with the way it's connecting to the GUI.

'Guacamole'

Sorry for the delay. Had some other tasks I needed to work on. So eventually the disk errors went away. The only thing the Fix Common Problems plugin is telling me now is to delete one of the Guacamole shares. However, every time I tried that, the server became unresponsive so I haven't tried it in a while. I also haven't added the parity drive back yet as I'm syncing my media to a backup server and will add it again after that completes. Is there a more reliable way to delete a share (they're empty) from the terminal? goathead-diagnostics-20240326-1210.zip

Easy enough. What about the file system on the drives? Just a simple format to wipe that? Nothing funny Unraid is going to try and preserve?

I had a weird issue today where I suddenly couldn't access my Unraid server from any device. Ended up having to log into IPMI and restart the server manually. So, I decided to configure SSH. However, I can connect fine, but Unraid is disabling my keyboard inputs so I cannot enter a password. Same behavior and syslog entry on both servers. SSH is enabled, but is there something else I have to do in order to enable it? Doesn't work from Putty on Windows or a Mac. Postponed keyboard-interactive for root from 192.168.30.252 port 58839 ssh2 [preauth]

I found a disk.cfg file. Since disk 6 is one of the disks it is looking for which I'v removed, would it make sense to delete everything between, and including, lines 134-156? Then do that for all the missing disks?

Not sure I follow. There doesn't seem to be any information regarding the disk configuration. is it perhaps in another folder?

No, only related to the downloads_cache.

I'll give that a try in the morning and report back.

Sorry if that wasn't clear. All my comments on that post was inline under the comments you provided and I was addressing your comment about my downloads cache. I included the name of the cache. What other labels would be helpful to include?

Is that somewhere on the flash drive?

I have that set to move from array to cache. That's where my torrent downloads land, and I want to move them manually to the ingestion cache which feeds the shares when I want. If it fills up, I expect it will spill over to the array, and then when I free up space, my expectation is mover will move as much as it can back to the cache. Is that not correct? I've added screen shots of the GUI settings too.

What is the best way to start with a new Unraid install? I've seen different advice on this and not found anything authoritative on this topic. I assume it's solely wiping the USB and keeping the key. However, I've seen different information on which file to keep, and some references to a key file that doesn't exist in my USB, so I'm assuming some of that is out of date. I have a license.text file which seems to be the license key. If I want to start an install like it was a brand new server, what exactly are the steps?

It's not clear how to fix the shares looking for drives that aren't there. I haven't selected any specific disks and checking them, they are all set to all. The Guacamole share was one I created, but then when I mapped a Windows drive to that in Guacamole, it created an additional one. I tried using Dynamix File Manager to delete it and that made everything unresponsive, so no, I've been afraid to touch it again. At least all my containers are working now and I can read/write files, which is good enough to move all the data onto the new server while I wipe it and start afresh.

Some of the errors are back. a/master/web/gui/dashboard/images/ms-icon-310x310.png Mar 21 13:51:26 Goathead kernel: TCP: request_sock_TCP: Possible SYN flooding on port 56574. Sending cookies. Check SNMP counters. Mar 21 13:53:01 Goathead root: Fix Common Problems Version 2024.03.09 Mar 21 13:53:01 Goathead root: Fix Common Problems: Error: Same share (guacamole) exists in a different case Mar 21 13:53:01 Goathead root: Fix Common Problems: Error: Same share (Guacamole) exists in a different case Mar 21 13:53:02 Goathead root: Fix Common Problems: Warning: Docker Application homarr has an update available for it ** Ignored Mar 21 13:53:02 Goathead root: Fix Common Problems: Warning: Docker Application Unraid-Cloudflared-Tunnel has an update available for it Mar 21 13:53:07 Goathead root: netdata-glibc: Could not download icon https://raw.githubusercontent.com/netdata/netdata/master/web/gui/dashboard/images/ms-icon-310x310.png Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk6 set in its included disk settings Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk7 set in its included disk settings Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk8 set in its included disk settings Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk9 set in its included disk settings Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk10 set in its included disk settings Mar 21 13:53:09 Goathead root: Fix Common Problems: Error: Share unraiddata has disk11 set in its included disk settings Mar 21 13:53:09 Goathead root: netdata-glibc: Could not download icon https://raw.githubusercontent.com/netdata/netdata/master/web/gui/dashboard/images/ms-icon-310x310.png Mar 21 13:53:09 Goathead root: netdata-glibc: Could not download icon https://raw.githubusercontent.com/netdata/netdata/master/web/gui/dashboard/images/ms-icon-310x310.png Mar 21 13:53:11 Goathead root: Fix Common Problems: Warning: Share appdata set to use pool containers_cache, but files / folders exist on the vm_cache pool Mar 21 13:53:11 Goathead sSMTP[21970]: Creating SSL connection to host Mar 21 13:53:11 Goathead sSMTP[21970]: SSL connection using TLS_AES_256_GCM_SHA384 Mar 21 13:53:13 Goathead sSMTP[21970]: Sent mail for [email protected] (221 2.0.0 closing connection f9-20020a814149000000b00609f8c5dfd3sm47462ywk.130 - gsmtp) uid=0 username=root outbytes=1541

I ran the 2 commands, stopped the array, rebooted, checked the syslog and didn't see a flood of errors, so I then started the array. Plugins have appeared again and I was able to run the diagnostics utility which I've attached. goathead-diagnostics-20240321-1343.zip