prytzen

So it appears that my ISP is to blame. There were some changes around that April timeline and it appears they are now blocking Port 80 inbound on residential connections (oddly everything else they seem to be fine with). But that means that i cannot use HTTP as the method of certificate authentication. I use Dreamhost as my registrar and hosting platform. Is there a way to use DNS as the authentication? I know that they have an API, but am not sure how to make it work with the SWAG container.

I am having issues with my SWAG install and it not updating certificates. Seems as though the last time it updated was back in April some time and has failed ever since. It was working fine before then and i cannot find anything that has changed. The errors that it keeps throwing are about timeouts and port 80 not being available, but i have verified that i am forwarding Port 80 to the container like always. I have a Unifi Security Gateway Pro and have forwarded port 80 on the outside to the appropriate IP address and port 80 on the inside (the IP and Port of the SWAG container). This automatically sets up a firewall rule allowing that traffic, but for some reason either this is not working, or the container is not responding to those HTTP requests. Any Ideas? Certbot output (anonymized domain and IP address) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Processing /etc/letsencrypt/renewal/plex.domain.tld.conf - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Renewing an existing certificate for plex.domain.tld and 4 more domains Certbot failed to authenticate some domains (authenticator: standalone). The Certificate Authority reported these problems: Domain: homeassistant.domain.tld Type: connection Detail: aa.bb.cc.dd: Fetching http://homeassistant.domain.tld/.well-known/acme-challenge/yh5SakI-iluZRukPZrhi7DAzwPVnN09r6q-S4OwIO7c: Timeout during connect (likely firewall problem) Domain: nextcloud.domain.tld Type: connection Detail: aa.bb.cc.dd: Fetching http://nextcloud.domain.tld/.well-known/acme-challenge/kq8c_9WqjEfhmmixjCJnMLShXxp1I21T42Nv3Sou1gM: Timeout during connect (likely firewall problem) Domain: ombi.domain.tld Type: connection Detail: aa.bb.cc.dd: Fetching http://ombi.domain.tld/.well-known/acme-challenge/cOCO5wmZM8N-WcLYwUjUjvDJh2cGbMh66s2VePAJ0fs: Timeout during connect (likely firewall problem) Domain: plex.domain.tld Type: connection Detail: aa.bb.cc.dd: Fetching http://plex.domain.tld/.well-known/acme-challenge/a8xsYJhw1CRGMyUwPhDQ4opxJRXr4AfGvbm2vqmHhC8: Timeout during connect (likely firewall problem) Domain: unifi.domain.tld Type: connection Detail: aa.bb.cc.dd: Fetching http://unifi.domain.tld/.well-known/acme-challenge/jU2jB7sLax3NxOcVBHdygQ1f-PJLWH1UJjXvKSq7KbM: Timeout during connect (likely firewall problem) Hint: The Certificate Authority failed to download the challenge files from the temporary standalone webserver started by Certbot on port 80. Ensure that the listed domains point to this machine and that it can accept inbound connections from the internet. Failed to renew certificate plex.domain.tld with error: Some challenges have failed. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - All renewals failed. The following certificates could not be renewed: /etc/letsencrypt/live/plex.domain.tld/fullchain.pem (failure) - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 1 renew failure(s), 0 parse failure(s) Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

As always - you are both correct. Thank you for the advice. I was able to manually update the Flash drive with 6.9 and everything seems to work. Thanks!

I have upgraded my server with a new motherboard and CPU. I have everything working, but there is no network connection. I usually have it set as Static, but it didnt work so i booted into GUI mode and switched it to DHCP - still not working. I thought maybe the onboard WiFi was getting in the way so i disabled it in BIOS, but still the same issue. I have not changed the Network cable or the switch port, these are known good and working. I can see in the devices list, an ethernet controller listed, so i am assuming that the device is being picked up (but maybe i just dont know what i am looking at). The only other thing that i can think of is that the onboard ethernet is 2.5Gbps being plugged into a 100/1000 mbps switchport. I can see on the switch side that autonegotiate has picked up 1000mbps - but still no IP address or network connection. Any Ideas you can provide would be greatly appreciated. Diagnostic ZIP attached. unraid-diagnostics-20210423-0507.zip

Great. Thanks. Looks like i am up and running.

That should be easy since the array is already stopped. Everytime I have done this in the past it automatically formats as btrfs. Is there an option that I am missing somewhere?

24 hours in and no errors. I do not believe that the RAM is an issue at this point. Is there a way that i can force a new CACHE drive to be XFS rather than BTRFS?

attached unraid-diagnostics-20201031-1054.zip

And how would i go about determining that?

I have had this issue a few times in the past and do not know how to resolve it. Symptoms: The server will become unresponsive to anything and everything and has to be hard booted. When it comes back up and i start the array (not set to start automatically), it will immediately lock up again. Problem: I have diagnosed it to being an issue with the CACHE drive being corrupted in some way. In the past I have just swapped out the cache drive with another one, formatted it and let it rebuild (I have backups and don't really lose anything). It is just a pain. I would rather not have to go through that process and just fix the issue. I believe that the file system on the cache drive is getting corrupted but don't know why. It is a BTFRS drive, which I understand can be temperamental. Want: To fix the error on the current cache drive and get it back up and running as is. Or, if possible, use the drive that is current there and having issues, reformat it to XFS and utilize that as the cache drives in hopes that XFS will be a more stable filesystem. Below are logs that i was able to capture when starting the array that show the cache drive (sdh) having issues. Oct 27 15:22:21 UNRAID emhttpd: shcmd (505): mkdir -p /mnt/cache Oct 27 15:22:21 UNRAID emhttpd: cache uuid: f5cd23ae-1a51-4349-a01d-65b2335b7b7c Oct 27 15:22:21 UNRAID emhttpd: cache TotDevices: 1 Oct 27 15:22:21 UNRAID emhttpd: cache NumDevices: 1 Oct 27 15:22:21 UNRAID emhttpd: cache NumFound: 1 Oct 27 15:22:21 UNRAID emhttpd: cache NumMissing: 0 Oct 27 15:22:21 UNRAID emhttpd: cache NumMisplaced: 0 Oct 27 15:22:21 UNRAID emhttpd: cache NumExtra: 0 Oct 27 15:22:21 UNRAID emhttpd: cache LuksState: 0 Oct 27 15:22:21 UNRAID emhttpd: shcmd (506): mount -t btrfs -o noatime,nodiratime -U f5cd23ae-1a51-4349-a01d-65b2335b7b7c /mnt/cache Oct 27 15:22:21 UNRAID kernel: BTRFS info (device sdh1): disk space caching is enabled Oct 27 15:22:21 UNRAID kernel: BTRFS info (device sdh1): has skinny extents Oct 27 15:22:21 UNRAID kernel: BTRFS info (device sdh1): enabling ssd optimizations Oct 27 15:22:21 UNRAID kernel: BTRFS info (device sdh1): start tree-log replay Oct 27 15:22:21 UNRAID kernel: ------------[ cut here ]------------ Oct 27 15:22:21 UNRAID kernel: kernel BUG at fs/btrfs/extent-tree.c:6862! Oct 27 15:22:21 UNRAID kernel: invalid opcode: 0000 [#1] SMP PTI Oct 27 15:22:21 UNRAID kernel: CPU: 0 PID: 13450 Comm: mount Tainted: P O 4.19.107-Unraid #1 Oct 27 15:22:21 UNRAID kernel: Hardware name: To Be Filled By O.E.M. To Be Filled By O.E.M./Z77 Extreme4, BIOS P2.90 07/11/2013 Oct 27 15:22:21 UNRAID kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f Oct 27 15:22:21 UNRAID kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff Oct 27 15:22:21 UNRAID kernel: RSP: 0018:ffffc9000210f828 EFLAGS: 00010202 Oct 27 15:22:21 UNRAID kernel: RAX: 0000000004ca6098 RBX: 0000000000000001 RCX: ffff8883e3100000 Oct 27 15:22:21 UNRAID kernel: RDX: 0000000000002000 RSI: 0000000000000002 RDI: ffff8883e3126230 Oct 27 15:22:21 UNRAID kernel: RBP: ffff8883e25a3070 R08: ffffc9000210f7a8 R09: ffffc9000210f7b0 Oct 27 15:22:21 UNRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Oct 27 15:22:21 UNRAID kernel: R13: ffff8883e7a69f08 R14: ffff8883e3126230 R15: 0000000072ac2000 Oct 27 15:22:21 UNRAID kernel: FS: 000015436552b480(0000) GS:ffff88843f400000(0000) knlGS:0000000000000000 Oct 27 15:22:21 UNRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 27 15:22:21 UNRAID kernel: CR2: 000015311239b3a0 CR3: 00000003e4388003 CR4: 00000000001606f0 Oct 27 15:22:21 UNRAID kernel: Call Trace: Oct 27 15:22:21 UNRAID kernel: __btrfs_run_delayed_refs+0xa77/0xbf4 Oct 27 15:22:21 UNRAID kernel: ? generic_bin_search.constprop.0+0x163/0x1ab Oct 27 15:22:21 UNRAID kernel: btrfs_run_delayed_refs+0x5d/0x16d Oct 27 15:22:21 UNRAID kernel: ? btrfs_set_path_blocking+0x20/0x44 Oct 27 15:22:21 UNRAID kernel: btrfs_commit_transaction+0x54/0x79c Oct 27 15:22:21 UNRAID kernel: btrfs_recover_log_trees+0x332/0x39b Oct 27 15:22:21 UNRAID kernel: ? replay_one_dir_item+0x15c/0x15c Oct 27 15:22:21 UNRAID kernel: open_ctree+0x182e/0x1c94 Oct 27 15:22:21 UNRAID kernel: btrfs_mount_root+0x388/0x509 Oct 27 15:22:21 UNRAID kernel: ? pcpu_alloc_area+0xeb/0xff Oct 27 15:22:21 UNRAID kernel: ? cpumask_next+0x15/0x16 Oct 27 15:22:21 UNRAID kernel: ? pcpu_alloc+0x37d/0x489 Oct 27 15:22:21 UNRAID kernel: mount_fs+0x10/0x77 Oct 27 15:22:21 UNRAID kernel: vfs_kern_mount+0x66/0x100 Oct 27 15:22:21 UNRAID kernel: btrfs_mount+0x16b/0x7f9 Oct 27 15:22:21 UNRAID kernel: ? pcpu_block_update_hint_alloc+0x73/0x183 Oct 27 15:22:21 UNRAID kernel: ? pcpu_chunk_relocate+0x8/0x5a Oct 27 15:22:21 UNRAID kernel: ? pcpu_alloc_area+0xeb/0xff Oct 27 15:22:21 UNRAID kernel: ? pcpu_next_unpop+0x31/0x3c Oct 27 15:22:21 UNRAID kernel: ? cpumask_next+0x15/0x16 Oct 27 15:22:21 UNRAID kernel: ? pcpu_alloc+0x37d/0x489 Oct 27 15:22:21 UNRAID kernel: ? mount_fs+0x10/0x77 Oct 27 15:22:21 UNRAID kernel: ? btrfs_remount+0x3eb/0x3eb Oct 27 15:22:21 UNRAID kernel: mount_fs+0x10/0x77 Oct 27 15:22:21 UNRAID kernel: vfs_kern_mount+0x66/0x100 Oct 27 15:22:21 UNRAID kernel: do_mount+0x7b5/0xa22 Oct 27 15:22:21 UNRAID kernel: ? _copy_from_user+0x2f/0x4d Oct 27 15:22:21 UNRAID kernel: ? memdup_user+0x3a/0x57 Oct 27 15:22:21 UNRAID kernel: ksys_mount+0x71/0x99 Oct 27 15:22:21 UNRAID kernel: __x64_sys_mount+0x1c/0x1f Oct 27 15:22:21 UNRAID kernel: do_syscall_64+0x57/0xf2 Oct 27 15:22:21 UNRAID kernel: entry_SYSCALL_64_after_hwframe+0x44/0xa9 Oct 27 15:22:21 UNRAID kernel: RIP: 0033:0x15436566525a Oct 27 15:22:21 UNRAID kernel: Code: 48 8b 0d 39 7c 0c 00 f7 d8 64 89 01 48 83 c8 ff c3 66 2e 0f 1f 84 00 00 00 00 00 0f 1f 44 00 00 49 89 ca b8 a5 00 00 00 0f 05 <48> 3d 01 f0 ff ff 73 01 c3 48 8b 0d 06 7c 0c 00 f7 d8 64 89 01 48 Oct 27 15:22:21 UNRAID kernel: RSP: 002b:00007fffca32e4c8 EFLAGS: 00000202 ORIG_RAX: 00000000000000a5 Oct 27 15:22:21 UNRAID kernel: RAX: ffffffffffffffda RBX: 00001543657ecf64 RCX: 000015436566525a Oct 27 15:22:21 UNRAID kernel: RDX: 000000000040d500 RSI: 000000000040d6c0 RDI: 000000000040d740 Oct 27 15:22:21 UNRAID kernel: RBP: 000000000040d2f0 R08: 0000000000000000 R09: 0000000000000003 Oct 27 15:22:21 UNRAID kernel: R10: 0000000000000c00 R11: 0000000000000202 R12: 0000000000000000 Oct 27 15:22:21 UNRAID kernel: R13: 000000000040d740 R14: 000000000040d500 R15: 000000000040d2f0 Oct 27 15:22:21 UNRAID kernel: Modules linked in: xfs nfsd lockd grace sunrpc md_mod nct6775 hwmon_vid bonding nvidia_drm(PO) nvidia_modeset(PO) nvidia(PO) crc32_pclmul intel_rapl_perf intel_uncore pcbc aesni_intel aes_x86_64 glue_helper crypto_simd ghash_clmulni_intel cryptd kvm_intel kvm intel_cstate coretemp drm_kms_helper crct10dif_pclmul intel_powerclamp crc32c_intel x86_pkg_temp_thermal drm mxm_wmi tg3 syscopyarea sysfillrect sysimgblt fb_sys_fops agpgart i2c_i801 i2c_core ahci libahci pcc_cpufreq video button ie31200_edac wmi backlight Oct 27 15:22:21 UNRAID kernel: ---[ end trace b4dd9901cbdd221e ]--- Oct 27 15:22:21 UNRAID kernel: RIP: 0010:__btrfs_free_extent+0x558/0x90f Oct 27 15:22:21 UNRAID emhttpd: shcmd (506): exit status: 139 Oct 27 15:22:21 UNRAID emhttpd: /mnt/cache mount error: No file system Oct 27 15:22:21 UNRAID emhttpd: shcmd (507): umount /mnt/cache Oct 27 15:22:21 UNRAID kernel: Code: ba c8 1a 00 00 e9 36 fd ff ff 48 81 3c 24 ff 00 00 00 76 18 48 8b 74 24 48 48 89 ef e8 a8 79 ff ff 3b 84 24 b8 00 00 00 74 02 <0f> 0b 48 83 7c 24 48 00 8b 45 40 74 0c 39 44 24 08 0f 84 0c ff ff Oct 27 15:22:21 UNRAID kernel: RSP: 0018:ffffc9000210f828 EFLAGS: 00010202 Oct 27 15:22:21 UNRAID kernel: RAX: 0000000004ca6098 RBX: 0000000000000001 RCX: ffff8883e3100000 Oct 27 15:22:21 UNRAID kernel: RDX: 0000000000002000 RSI: 0000000000000002 RDI: ffff8883e3126230 Oct 27 15:22:21 UNRAID kernel: RBP: ffff8883e25a3070 R08: ffffc9000210f7a8 R09: ffffc9000210f7b0 Oct 27 15:22:21 UNRAID kernel: R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000 Oct 27 15:22:21 UNRAID kernel: R13: ffff8883e7a69f08 R14: ffff8883e3126230 R15: 0000000072ac2000 Oct 27 15:22:21 UNRAID kernel: FS: 000015436552b480(0000) GS:ffff88843f400000(0000) knlGS:0000000000000000 Oct 27 15:22:21 UNRAID kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Oct 27 15:22:21 UNRAID kernel: CR2: 000015311239b3a0 CR3: 00000003e4388003 CR4: 00000000001606f0 Oct 27 15:22:21 UNRAID root: umount: /mnt/cache: not mounted. Oct 27 15:22:21 UNRAID emhttpd: shcmd (507): exit status: 32 Oct 27 15:22:21 UNRAID emhttpd: shcmd (508): rmdir /mnt/cache Oct 27 15:22:21 UNRAID emhttpd: shcmd (509): sync

Yes i was able to restore the AppData, and after watching a few videos i was able to learn i could just redownload the Docker executable code and make sure it was pointing to my AppData and everything is working. Odd that a server crashing and having to essentially rebuild it has given me more confidence in the product and how it works. Thank you all for your help.

I swapped the CACHE drive with another on hand and was able to get the array to mount properly. Now i need to understand how to restore my Docker images. I use CA Backup/Restore and have a backup of the appdata and have restored that backup, but none of the containers are showing up under the Docker tab. What am i not doing right or missing?

I had to rebuild my boot disk and replace my CACHE drive and am trying to get things back working again. thankfully i have a backup of my appdata with CA BAckup/Restore, but i guess i am not using it right. I have Restored the backup, but none of my docker containers show under the docker tab. what am i doing wrong or what am i missing?

I looked at it again this morning. I started the machine in safemode and was able to bring the array online in MAINTENANCE mode. I looked at the log and didnt see anything and with the log open i took the array offline and then tried to bring it online in normal mode (with mounting the drives). In the log (attached) it looks like all the drives mount fine, but there are errors btrfs with the CACHE drive. I had to restart the server and i brought the array back on in MAINTENANCE mode. and then opened the CACHE drive to do the btfrs Check status and here is the output. [1/7] checking root items [2/7] checking extents ref mismatch on [15157989376 159744] extent item 1099503797234, found 1 incorrect local backref count on 15158149120 root 5 owner 1597264 offset 950272 found 1 wanted 3827212289 back 0x3d892c0 backpointer mismatch on [15158149120 8192] ERROR: errors found in extent allocation tree or chunk allocation [3/7] checking free space cache [4/7] checking fs roots [5/7] checking only csums items (without verifying data) [6/7] checking root refs [7/7] checking quota groups skipped (not enabled on this FS) Opening filesystem to check... Checking filesystem on /dev/sdh1 UUID: 02bb4712-df62-48cc-9654-1e32a7629104 found 14323052544 bytes used, error(s) found total csum bytes: 5855024 total tree bytes: 96059392 total fs tree bytes: 69353472 total extent tree bytes: 15826944 btree space waste bytes: 22216609 file data blocks allocated: 91474538496 referenced 8596598784 log.txt

Ok. So i have created a new flash drive and gotten it licensed properly. I copied the configuration files necessary (following this: https://wiki.unraid.net/Files_on_v6_boot_drive) leaving out plugins or docker files. The server boots, and webpage displays fine. However, i am unable to bring the array online. I have let it run for about an hour and it has not finished. I tried to run "diagnostics" from the command line, but it as well has not finished (as i suspect the array mounting is getting in the way). So i would suspect there is something up with my array.... or my disks? I did notice that when i started the server my Disk4 had a SMART error. I acknowledged the error and tried to bring the array online. Would this be causing it all? Is Disk4 just dead? How can i test this theory.

I tried this and let it go for 10 minutes but didnt seem to complete. I turned off the server and inspected the logs folder and nothing was there. Plugged the USB back into the computer and now it appears the USB is no longer bootable. Do i need to re-create my boot disk?

Is there a way to run the diagnostics from a the command line directly on the server and save it to the USB stick so i can provide this?

I disabled Docker, and started the array. the message Array Starting. Mounting disks is displayed, but it does not seem to be bringing the array online. I have tried to open a new browser window to the server to pull the diagnostics again, but it is unavailable. Pinging the server, it responded for about a minute, but now is not pinging.

Diagnostics Attached unraid-diagnostics-20200528-0816.zip

Config/go file attached. Yes - SafeMode GUI No Plugins works fine still. go

I have renamed all the PLG files in the Config/Plugin folder and the symptoms are the same. The server will start up, become pingable, but the web interface will never load. Eventually the server becomes unpingable. How can i find out what is going on here?

Dont really know where to begin on troubleshooting so any assistance would be very helpful. Currently, my server will not boot properly in normal mode. I can start the server, it will get to the login prompt and be pingable on the network but the web interface will not load. After about 1-2 minutes, the server will no longer be pingable. During the time that the server is available, i can SSH to it without issue. If i boot the server in GUI mode, the same symptoms occur and i am still unable to get to the web interface directly on the server via "localhost" I have started the server in Safe Mode GUI No Plugins and it will start up no problem and be stable with the web interface available. This lead me to believe that it is an issue with a plugin. I looked on the forms and saw that you could remove the files in Config/Plugins so i have (Before and after shots attatched). After doing this, the server is still responding with the same behavior. What i did notice, is after turning the server back on, the folder Config/plugins/dynamix was recreated somehow. What do i do?!