unw1red

I figured out what the issue was. I did not have default selected for the Prometheus data source in Grafana. The data was populating the database, but not being displayed in Grafana.

I am not running keepalived. They are individual instances of PiHole with both designated in my DHCP scope.

I am running two instances of Pi-Hole on two external servers. Can you monitor both? The one I have setup is not populating any data. Would the port be different for external servers?

As long as your server is wired, data will not traverse the router unless it being used by a wireless client or if the client is on a different subnet than the server. Meaning, if you are trying to read from InfluxDB container from Grafana container, via the server IP, the data will not traverse your router. It will be horseshoed through the eth0 interface. Make sense?

I have my server connected to a router with a NATed IP to the Internet. It is not on a DMZ. I have some port forwarding in place for Let's Encrypt. That is working as it should. I also have noticed that I can reach the internal ports of the Docker containers via the Internet to the Public IP address. Does this have anything to do with using both an Ethernet and Bridge interface on the server. I have br0 bridging eth0. If I turn off bridging, will the Docker containers stop being accessible? I do not have any VMs or docker containers using alternate, local IP addresses. On my router, I see two occurrences of my UNRaid server with the same IP address. Thanks for any help.

Additional question on this topic. After making the above changes, should you then set your appdata share to Cache: ONLY?

I just found this thread as well. I having been dealing with the SHFS problem for the last several weeks about every other day. It required a hard reset to clear. My server is currently in this state and I was trying to troubleshoot the situation. I found a couple of things: 1. The CPU load on my dual Xeon 2.0GHz 12 core/24 thread is climbing steadily and at about >=30.00, certain Docker containers become unusable 2. If I SSH into the server and try to do any dick I/O in the /mnt disk structure (except for /mnt/cache/appdata), the terminal session locks up and has to be killed. 3. The CPU usage in UNRAID is still within normal levels (<=25% CPU) while there are a few individual cores that are peaked at 100% and do not go down. I am going to hard reboot the server and make this change in all of my Docker containers to see if this alleviates my issues. Thank you for the recommendation.

I may have figured this one out, but I was wondering if anyone else had seen this situation. I have a Supermicro 12 HDD chassis with a Supermicro dualXeon motherboard, 12 cores and 24 threads and 128Gb of RAM. I am running about 10 Docker containers with Home Assistant, MariaDB, Radarr, Sonarr, Plex, and Zoneminder (at the time this was happening). After about 1-2 days after a clean boot, my server would not allow me to connect or use any of the Docker containers. I couldn't kill, restart or upgrade them from the UI or CLI. The processor load would hit about 40-50 when this started happening. UNRAID would still report that the processor usage was about 2-10%. I was unable to reboot the server or kill processes from the CLI and would eventually have to do a hard reboot which makes me cringe. I finally turned down Zoneminder and recorded my CCTV to an unassigned drive so that the load would not be duplicated to the parity drive. It has been stable for almost 48 hours and seems to be working fine. When it was in this state, I could do top, htop or ps to see what was going on and there were a huge amount (>100) S or D processes. A large amount of them were disk operations. Do you think that the duplication of writes could have caused this zombie state with UNRAID?

What procedure did you use to upgrade the disk? Did you use the data rebuild procedure? Did you preclear the disks?

As the primary network connection or as an access point? This is from the wiki Realtek - RTL8169S, RTL8111B, RTL8111C, probably others too ***NOTE - MANY PEOPLE REPORT ISSUES WITH REALTEK, SO CHECK THE FORUMS BEFORE ASSUMING YOUR BOARD WILL WORK***

You need to make sure both sides of the link are configured the same. Sometimes, a LAGG can be configured to LOAD BALANCING on one side and 802.3ad on the other and it will bounce interfaces until the end of time.

Can you post the config for your switch for the UNRaid interfaces?

NextCloud updated last night and successfully started. I tried my NextCloud app from my phone this morning and noticed that I could do no writes to my account. I can read the files, but I can't create a folder or upload. I then looked at the logs for NextCloud and all seems correct. I then tried to reboot the server with to no avail. I opened an UNRaid console and ran htop. It shows that the processor is running away (40.0 and growing on a 12-core, 24 thread server). The CPU usage is steadily climbing and the prevalent process is "/usr/local/sbin/shfs /mnt/user -disks 127 2048000000". The diagnostics don't reveal anything out of the ordinary to me. Anyone else experiencing the same thing this morning? musial-diagnostics-20191119-1444.zip

After two and a half days, and a reboot, all went back to normal. Thanks for the help.

Once the rebuild completed, I rebooted the system. All came back fat and happy. The error messages are no longer appearing in the syslog. I will now move onto replacing a 3TB drive with a 12TB drive and allowing parity to rebuild the data drive. Cross your fingers.

I just found those messages as well. I am at 93% on my data rebuild. After it is done, I will reboot and report back.

The only thing that stands out in the logs, barring the backups that are trying to be performed but can't, is this block. Nov 11 05:30:16 Musial emhttpd: shcmd (4942): /usr/sbin/hdparm -y /dev/nvme1n1 Nov 11 05:30:16 Musial root: HDIO_DRIVE_CMD(standby) failed: Inappropriate ioctl for device Nov 11 05:30:16 Musial root: Nov 11 05:30:16 Musial root: /dev/nvme1n1: Nov 11 05:30:16 Musial root: issuing standby command Nov 11 05:30:16 Musial emhttpd: shcmd (4942): exit status: 25

I used this procedure for this operation... https://wiki.unraid.net/The_parity_swap_procedure

Here you go... musial-diagnostics-20191111-1554.zip

Here is the screenshot of my array.

I did some upgrades this weekend. I had a 10TB parity drive and swapped it out for a 12TB in preparation for some HD size upgrades. I replaced a 3TB with the new 12TB. I then changed the parity from the 10TB to the new 12TB. The parity copy took about 24 hours, but it all completed. I then reassigned the 10TB that was my parity to be a data drive. I did the reassignments and then started the array so the 10TB could go through data rebuild. From reading numerous posts, while this operation was taken place, my server should have function normally, albeit slow, while this was happening. My shares were not available and Docker and VMs could not start because they had missing paths (appdata and domains). Is this normal? I still need to replace two 3TB drives with two new 12TB drives, but I am afraid to start this procedure for fear that my server will be useless for another two to three days. Can anyone comment or guide me on what I may be doing wrong?

@t5800512 And then you rebooted your server?