jena

Crashed again today. on July 27, I followed the third option: "Keep docker containers in host or bridge mode which use the server IP address and ports as needed" I changed all dockers to host or bridge mode. Turned off "AdGuard" because it cannot run under host or bridge mode and disabled auto start. My unraid ran for a week and crashed again (same symptom, no response) last night. I attached the syslog. Please start from July 27, the day that I rebooted and made the change. syslog_20210803.txt

Great. A relief for me finally. Thank you so much.

Thank you very much. I also just attached syslog in my last reply. Could you help to take a look?

After a few days of running pretty stable. This morning. It froze again. This time there is video display (probably in the video buffer) right before it froze. syslog_0726_1045.txt neo-diagnostics-20210726-1051.zip

Yes after the crash. The syslog of the before crash (original problem) couldn't be found after the third crash despite I already enabled mirror syslog before that (after first crash).

I did that when 2nd time happened. I have seen the syslog under /boot/logs/syslog once, which contained logs from Jul 12. This time, before hard shut down, I pulled my data HDD out while they were spun own (I felt them, no spin vibration) in an attempt to prevent parity check again. I then hard shut down and put the HDD back in and boot it up again. Now I got error saying that my flash drive is not read/write. I guess it might have a temporary lock up due to hard shut down or huge file copy from /var/log/syslog. The UNRAID web UI and SMB service all seem to be working and I canceled parity check. I can see the file in /boot but log directory shows like this in ls -l command and not accessible. d????????? ? ? ? ? ? logs/ The var/log/syslog is huge at 120-ish MB and I manually copied it to a share. I attached part of "syslog_0717_1014"(the rest are all the same error "FAT-fs (sda1): error, corrupted directory (invalid entries)") and diagnose zip of this time (10:16am) I looked into the var/log/syslog, they are all Jul 17 (today). I did normal shut down via webUI and unplugged the flash drive and attempt to read it off it. There is logs folder, but nothing in it. Plug flash drive back it, UNRAID boots up fine, I attached another syslog at 10:44am and and diagnose zip of this time (10:59am). I don't know if I could recover the syslog that contains Jul 12-Jul16. How can I permanently cancel parity check due to hard shut down? I will do a full parity check after all the diagnose. neo-diagnostics-20210717-1016.zip syslog_0717_1014_part.txt syslog_0717_1044.txt neo-diagnostics-20210717-1059.zip

Hi UNRAID community, Setup I don't have GPU passthrough. I have plex docker with standard intel iGPU hardware transcoding setting (like modprobe i915). Once array and plex docker is running, at least for a couple hours, there is video out for UNRAID command line and response to keyboard input. I have plugin a Mellanox MCX311a to UNRAID made it to be primary in bond and On board 1G to be secondary in active backup config. Current switch is Netgear MX510TX 10G/multigig switch (I returned Mikrotik CRS305) These have been stable and I got full speed to 2.5G with 0 Retr for most of the time. I have got 7-8Gbps and 9.2 Gbps between MCX311a <-> Netgear MX510TX <-> HP 533FLR 10Gbe Problem I had this happen to my UNRAID server recently. First time, UNRAID be come unreachable in early morning, no reply to ping, no video out, no response to key board. The power indicator was lit and fans were running. HDD busy indicator was NOT lit or blink at all. Have to do hard shut down and it booted fine and just need to do parity check. Second morning, same exact story, UNRAID is not responsive and not reachable in the morning. Hard shut down. I did 4 pass of full memtest with my memtest DOS flash drive, no error. Somehow, memtest in the UNRAID boot menu doesn't work and it will just reboot. After memtest, it booted fine and parity check. Third morning, UNRAID is not reachable. The power indicator was lit and fans were running. HDD busy indicator blinked occasionally. ping response is very wired. Notice the third time is 3.7s. Pinging 192.168.x.x with 32 bytes of data: Reply from 192.168.x.x: bytes=32 time=14ms TTL=64 Request timed out. Reply from 192.168.x.x: bytes=32 time=3704ms TTL=64 Reply from 192.168.x.x: bytes=32 time=62ms TTL=64 If I unplug SPF+ cable, and have 1G NIC plugged to switch or router directly, ping reply: Destination host unreachable. This time, the monitor has video out but not responsive, and also not responsive to keyboard input. Last night keyboard was working and RGB lighting is on. This morning, keyboard numlock doesn't lit on when numlock is pressed, also RGB lighting doesn't work. Unplugged and plugged to different USB port, no response, no lights. At this point how can I diagnose? Do I have to hard shut down and do another parity check?

That I don't know. My unraid is under-powered old 4-core i5, not that useful for plotting. I ran Mint20 (Ubuntu) and windows as plotter. That way I can use swar plot manager (or plotman).

This could be why it broke my previous settings. I didn't recall that I updated the docker today (but maybe UNRAID auto updated it). Mine is also "up to date" version 1.1.7.dev0

https://forums.unraid.net/profile/119008-partition-pixel/ Took me an hour and got it working. I would suggest to put it in the docker template. Looking at here https://github.com/Chia-Network/chia-docker/blob/main/entrypoint.sh The docker needs a variable called ca So I added a variable for the docker (called Key below) called ca to add: press "Add another Path, Port, Variable, Label or Device" Config Type:Variable Name:ca Key:ca Value:/config/ssl/ca/ Make sure you also put these in (which should not be needed according to config.yaml in the original post "Farming on many machines" farmer_address: YOUR.FARMER.IP farmer_port: 8447

Hello, Thank you for this awesome docker template. I got the harvester only working for a day and it was connecting to the main full node ok (can see passing filter). My initial setting was: harvester only = true farmer_address: XXX.XXX.XXX.XXX farmer_port: 8447 I had to put farmer address in here, otherwise docker will fail to start, which should not be necessary. I believe the value here changes the "full node -> farmer_peer" section, which is not needed according to https://github.com/Chia-Network/chia-blockchain/wiki/Farming-on-many-machines Anyway, the initial setting worked. Then, all of a sudden, the docker might have restarted due to a parity disk problem (no change to docker settings). appdata is on cache disk, which should not be affected. How the docker log says: farmer address required. no matter whether I put farmer address and port here, it complaints about not having farmer address.

UPDATE: borrowed some hardware so I can do more isolated test. First I took UNRAID out of equation. PC2: I borrowed a buddy's i5-7500 and B150M motherboard. 10G NIC is plugged in slot 1 x16. PC1: (ASUS B550 Intel I225V 2.5G), TX/RX buffer = 1024, flow control enabled. Server2: OMV, RTL8125 2.5G NIC, just uses putty to SSH. Treat it as just a Linux. Adapter settings: see detail at the end All default, MTU 1500. iperf3 host Server2 (RTL8125) to client PC1 (Intel I225V) 283 MBytes/sec, no Retr displayed in output (maybe issue with windows version of iperf3?) host Server2 (RTL8125) to client PC2 (10G NIC) 259 MBytes/sec, total Retr = ~60000 host PC1 (I225V) to client PC2 (10G NIC) 166 MBytes/sec (interesting...), total Retr = ~30000 This was "host PC, client UNRAID (10G), 30MB/s". My guess there was multiple factors led to this. host PC1 (I225V) to client Server2 (RTL8125) 221 MBytes/sec, total Retr = 28 host PC2 (10G NIC) to client PC1 (I225V) 283 MBytes/sec, no Retr displayed in output host PC2 (10G NIC) to client Server2 (RTL8125) 229 MBytes/sec, Retr = 0 So it looks like I215V has some problem (there are quite a bit report on Rev1 and 2 of I225, dropping packets). Mine is Rev3, which is "supposed" to be free of previous issue. But seems to be ok from 2.5G Intel I215V to RTL8125. I am seeking diagnose directions. Thanks! ####### Server2 2.5G RTL8125 Settings for enp5s0: Supported ports: [ TP MII ] Supported link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseT/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: Yes Supported FEC modes: Not reported Advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Full 2500baseT/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: Yes Advertised FEC modes: Not reported Link partner advertised link modes: 10baseT/Half 10baseT/Full 100baseT/Half 100baseT/Full 1000baseT/Half 1000baseT/Full 10000baseT/Full 2500baseT/Full 5000baseT/Full Link partner advertised pause frame use: No Link partner advertised auto-negotiation: Yes Link partner advertised FEC modes: Not reported Speed: 2500Mb/s Duplex: Full Port: MII PHYAD: 0 Transceiver: internal Auto-negotiation: on Supports Wake-on: pumbg Wake-on: d Link detected: yes ####### PC2 10G NIC ethtool enp1s0 Settings for enp1s0: Supported ports: [ FIBRE ] Supported link modes: 1000baseKX/Full 10000baseKR/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 1000baseKX/Full 10000baseKR/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Port: Direct Attach Copper PHYAD: 0 Transceiver: internal Auto-negotiation: off Cannot get wake-on-lan settings: Operation not permitted Current message level: 0x00000014 (20) link ifdown Link detected: yes

I will test that. Also test on a different MB and report back.

"from PC-2.5G NIC to another server with 2.5G NIC" this was tested on 10G switch with two RJ45-SFP+ adapter. The adapter has auto negotiation and can work at 1/2.5/5/10G. Also PC and server have default 1500MTU, no jumbo frame.

Router: EdgeRouter X Switch 1 (10G): MikroTik CRS305-1G-4S+IN, Switch 2: NetGear GS305 PC - Switch1: 2.5G RJ45 - SFP+ adapter, CAT6 cable UNRAID - Switch 1: 10G Tek DAC cable I currently don't have extra DAC cable to try PC MB: ASUS B550i On Board NIC: Intel I225-V UNRAID Recently migrated from a old DELL Optiplex 3010 (with default NIC offloading and flow setting, i.e. ON). It has no problem to reach sustained 110MB/s. MB: HP Z240 (C236 chipset) Bond0 (fail-over) eth0 - 10G NIC: Mellanox MCX-311A eth1 - On board NIC: Intel I219-LM Symptoms: (using Switch 1) iperf3 -c IP-address -f M UNRAID side test file is on NVME SSD cache only. PC side test file is on SATA SSD. iperf3 speed for 10G MCX-311A NIC host PC, client UNRAID, 30MB/s (~240Mbps) host PC, client UNRAID, 220MB/s File copy UNRAID to PC 30MB/s PC to UNRAID 280MB/s iperf3 speed for for onboard Intel I219-LM NIC iperf3 speed at 80MB/s (~640Mbps) File copy UNRAID to PC 106MB/s, almost steady. PC to UNRAID 100MB/s, with momentary speed drop during copy What I have tried: [2.5G NIC seems good, Switch 1 and adaptor works] iperf3 from PC-2.5G NIC to another server with 2.5G NIC, achieved 230MB/s, almost full speed. File transfer speed are similar to iperf. [No improvement] Swap DAC cable in different ports in Switch 1 [No improvement] Unplug 10G DAC cable (using eth1, 1G), Swap different CAT6 cable in different ports in Switch 1, still 80MB/s, Windows to SMB file transfer speed is also around 40-80MB/s. Later NIC offloading fixed some of this issue. [No improvement] change eth0 to onboard 1G Intel I219-LM, unplug 10G DAC cable [Some improvement] NIC offloading OFF, NIC flow OFF, also typed in "ethtool -K eth0 tso off" "ethtool -K eth0 gso off" for both eth0 and eth1. iperf3 onboard 1G Intel I219-LM, 112MB/s, seems normal, expect for the first data point ~40MB/s in the "host UNRAID, client PC" test (this behavior has been consistent). But PC to UNRAID file transfer speed (with single 100GB file) starts at 110MB/s and drops to 40MB/s in the half way through and come back up to 110MB/s in a minute, and later drops again and come back. [No improvement] (with NIC offloading OFF, NIC flow OFF, buffer size 256 for TX and RX) change eth0 back to 10G NIC, still 30MB/s (UNRAID to PC) [No change] test with another PC with its onboard intel 1G NIC, still 30MB/s for 10G NIC. [No change] test all 1G test with Switch 2. 1. 10G - Host PC, Client UNRAID Connecting to host PC-2.5G-NIC-IP, port 5201 [ 5] local UNRAID-10G-IP port 59950 connected to PC-IP port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 34.6 MBytes 34.6 MBytes/sec 12221 106 KBytes [ 5] 1.00-2.00 sec 23.7 MBytes 23.7 MBytes/sec 4609 103 KBytes [ 5] 2.00-3.00 sec 27.8 MBytes 27.8 MBytes/sec 6516 211 KBytes [ 5] 3.00-4.00 sec 25.4 MBytes 25.4 MBytes/sec 6037 150 KBytes [ 5] 4.00-5.00 sec 24.0 MBytes 24.0 MBytes/sec 4322 97.0 KBytes [ 5] 5.00-6.00 sec 25.7 MBytes 25.7 MBytes/sec 6001 99.8 KBytes [ 5] 6.00-7.00 sec 34.4 MBytes 34.4 MBytes/sec 8015 200 KBytes [ 5] 7.00-8.00 sec 36.6 MBytes 36.6 MBytes/sec 10178 106 KBytes [ 5] 8.00-9.00 sec 29.8 MBytes 29.8 MBytes/sec 9167 111 KBytes [ 5] 9.00-10.00 sec 24.8 MBytes 24.8 MBytes/sec 6076 91.2 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 287 MBytes 28.7 MBytes/sec 73142 sender [ 5] 0.00-10.00 sec 285 MBytes 28.5 MBytes/sec receiver 2. 10G - Host UNRAID, Client PC Connecting to host UNRAID-10G-NIC-IP, port 5201 [ 4] local PC-2.5G-IP port 60951 connected to UNRAID-10G-NIC-IP port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 283 MBytes 283 MBytes/sec [ 4] 1.00-2.00 sec 283 MBytes 283 MBytes/sec [ 4] 2.00-3.00 sec 283 MBytes 283 MBytes/sec [ 4] 3.00-4.00 sec 283 MBytes 283 MBytes/sec [ 4] 4.00-5.00 sec 283 MBytes 283 MBytes/sec [ 4] 5.00-6.00 sec 283 MBytes 283 MBytes/sec [ 4] 6.00-7.00 sec 283 MBytes 283 MBytes/sec [ 4] 7.00-8.00 sec 283 MBytes 283 MBytes/sec [ 4] 8.00-9.00 sec 281 MBytes 281 MBytes/sec [ 4] 9.00-10.00 sec 283 MBytes 283 MBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 2.76 GBytes 283 MBytes/sec sender [ 4] 0.00-10.00 sec 2.76 GBytes 283 MBytes/sec receiver 3. 1G - Host PC, Client UNRAID Connecting to host PC, port 5201 [ 5] local UNRAID port 59964 connected to PC port 5201 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 112 MBytes 112 MBytes/sec 0 314 KBytes [ 5] 1.00-2.00 sec 112 MBytes 112 MBytes/sec 0 308 KBytes [ 5] 2.00-3.00 sec 112 MBytes 112 MBytes/sec 0 305 KBytes [ 5] 3.00-4.00 sec 112 MBytes 112 MBytes/sec 0 297 KBytes [ 5] 4.00-5.00 sec 112 MBytes 112 MBytes/sec 0 297 KBytes [ 5] 5.00-6.00 sec 113 MBytes 113 MBytes/sec 0 305 KBytes [ 5] 6.00-7.00 sec 111 MBytes 111 MBytes/sec 0 299 KBytes [ 5] 7.00-8.00 sec 112 MBytes 112 MBytes/sec 0 302 KBytes [ 5] 8.00-9.00 sec 112 MBytes 112 MBytes/sec 0 297 KBytes [ 5] 9.00-10.00 sec 112 MBytes 112 MBytes/sec 0 299 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 1.10 GBytes 112 MBytes/sec 0 sender [ 5] 0.00-10.00 sec 1.09 GBytes 112 MBytes/sec receiver 4. 1G - Host UNRAID, Client PC Connecting to host UNRAID, port 5201 [ 4] local PC port 49534 connected to UNRAID port 5201 [ ID] Interval Transfer Bandwidth [ 4] 0.00-1.00 sec 44.4 MBytes 44.4 MBytes/sec [ 4] 1.00-2.00 sec 112 MBytes 112 MBytes/sec [ 4] 2.00-3.00 sec 112 MBytes 112 MBytes/sec [ 4] 3.00-4.00 sec 111 MBytes 111 MBytes/sec [ 4] 4.00-5.00 sec 111 MBytes 111 MBytes/sec [ 4] 5.00-6.00 sec 110 MBytes 110 MBytes/sec [ 4] 6.00-7.00 sec 111 MBytes 111 MBytes/sec [ 4] 7.00-8.00 sec 111 MBytes 111 MBytes/sec [ 4] 8.00-9.00 sec 112 MBytes 112 MBytes/sec [ 4] 9.00-10.00 sec 112 MBytes 112 MBytes/sec - - - - - - - - - - - - - - - - - - - - - - - - - [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 1.02 GBytes 105 MBytes/sec sender [ 4] 0.00-10.00 sec 1.02 GBytes 105 MBytes/sec receiver