I noticed my Nextcloud stopped working, and i cant figure out why?!
It may have been when Nextcloud, Swag or MariaDB updated (i use autoupdate-plugin).
But i cant find any errors in either of them?
Other sites works, like Heimdall, so its probably not SWAG, and i can connect to MariaDB with adminer and see nextcloud-db so probable not MariaDB either.
root@kapten:/mnt/user/appdata/mariadb/databases# cat 91b6971974d5.err
211029 11:09:29 mysqld_safe Starting mariadbd daemon with databases from /config/databases
2021-10-29 11:09:29 0 [Note] /usr/bin/mariadbd (mysqld 10.5.12-MariaDB-log) starting as process 496 ...
2021-10-29 11:09:29 0 [Note] InnoDB: Uses event mutexes
2021-10-29 11:09:29 0 [Note] InnoDB: Compressed tables use zlib 1.2.11
2021-10-29 11:09:29 0 [Note] InnoDB: Number of pools: 1
2021-10-29 11:09:29 0 [Note] InnoDB: Using crc32 + pclmulqdq instructions
2021-10-29 11:09:29 0 [Note] mariadbd: O_TMPFILE is not supported on /var/tmp (disabling future attempts)
2021-10-29 11:09:29 0 [Note] InnoDB: Using Linux native AIO
2021-10-29 11:09:29 0 [Note] InnoDB: Initializing buffer pool, total size = 268435456, chunk size = 134217728
2021-10-29 11:09:29 0 [Note] InnoDB: Completed initialization of buffer pool
2021-10-29 11:09:29 0 [Note] InnoDB: 128 rollback segments are active.
2021-10-29 11:09:29 0 [Note] InnoDB: Creating shared tablespace for temporary tables
2021-10-29 11:09:29 0 [Note] InnoDB: Setting file './ibtmp1' size to 12 MB. Physically writing the file full; Please wait ...
2021-10-29 11:09:29 0 [Note] InnoDB: File './ibtmp1' size is now 12 MB.
2021-10-29 11:09:29 0 [Note] InnoDB: 10.5.12 started; log sequence number 7078025644; transaction id 26259672
2021-10-29 11:09:29 0 [Note] InnoDB: Loading buffer pool(s) from /config/databases/ib_buffer_pool
2021-10-29 11:09:29 0 [Note] Plugin 'FEEDBACK' is disabled.
2021-10-29 11:09:29 0 [Note] Server socket created on IP: '::'.
2021-10-29 11:09:29 0 [Note] Reading of all Master_info entries succeeded
2021-10-29 11:09:29 0 [Note] Added new Master_info '' to hash table
2021-10-29 11:09:29 0 [Note] /usr/bin/mariadbd: ready for connections.
Version: '10.5.12-MariaDB-log' socket: '/run/mysqld/mysqld.sock' port: 3306 MariaDB Server
2021-10-29 11:09:29 0 [Note] InnoDB: Buffer pool(s) load completed at 211029 11:09:29
I have the exact same problem with the exact same setup. Nextcloud + mariaDB + Swag. Recent update by auto-app-updates killed nextcloud. I've confirmed swag is fine, there was a new /config/nginx/proxy.conf file pushed but i tested out old & new one. No difference (the change was to set a header to blank, this was to fix a security issue). Based on logs that this person and I am seeing I assume its something to did with PHP. I know there was a PHP8 version in the works but checking the default/normal nextcloud php version shows its PHP 7.4.25 so it shouldn't be a bad PHP upgrade? I looked through https://github.com/linuxserver/docker-nextcloud/releases and couldn't find anything that stood out, seems everything is just re-basing to latest alpine release. In the Nextcloud GUI I was already up-to-date as of at least 1-2 weeks ago. I tried downgrading the docker image from latest (== 22.2.0-ls160) to 22.2.0-ls156 which is from about 1 month ago but that didn't help either. I'm out of idea's. PHP error logs don't show anything useful.
Update 1) changing log level to "debug" for php7 does not show any more information. I guess crash/shutdown happens too quickly. I looked at logs from 1 month ago and they are the same as today/OP in regards to fpm shutting down. Nextcloud was working at that time. I saw a warning regarding increasing max children in those older logs. I just increased pm.max_children = 300 from the default 5. Its the only thing i could see in the past logs. Didn't make a difference.
Update 2) I think it might be caused by DNS but still investigating. I use pihole for DNS. /etc/resolve.conf in nextcloud has the correct local DNS server IPv4 address. When i try to ping local servers using their local DNS entries in pihole (from within nextcloud docker container), it fails to resolve. I know pihole DNS entries are working in my web browser it should not be caused by pihole. I also see HTTP 302's in nextcloud logs (as there should be) but no errors.
Update 3) Probably not a DNS issue. Ran NSLookup from within containers and they can resolve DNS address just fine. If i go to Nextcloud WebUI from within unraid-->docker menu it goes to unraid IPv4:<nextcloud port>. This is good. This then redirects to correct DNS address as configured in nextcloud. That is a SWAG docker container within unraid. This container runs on custom network using macvlan driver (so it can get its own IPv4 static address from home router). Swag acts as reverse-proxy for nextcloud. Looking at its logs it says "2021/11/01 09:49:43 [error] 497#497: *1 connect() failed (111: Connection refused) while connecting to upstream," for <unraid Ipv4>:<nextcloud_port-mapping>. All address and DNS stuff looks correct. I tried curl from SWAG container to unraid GUI, it failed. I tried to <unraid Ipv4>:<nextcloud_port-mapping> and it failed. Possibly issue with networking between SWAG & unraid.
Update 4) Logs and tests seem to show that my swag container cannot reach unraid IPv4 address. I thinking the custom/special network I created for the swag container will probably need to be recreated. Maybe there was some docker or unraid update that modifying some default settings? Because Swag cannot reach the UnRaid host IPv4 address it cannot act as a reverse proxy.
Update 5) Turned docker on/off from UnRaid GUI. This seemed to fix whatever weird issue caused swag not to be able to access UnRaid IPv4 address. I guess the solution of turn it off & on is still the best solution that exists. I was just turning off/on the wrong things. (Note i did enable the docker option of preserving user networks at startup. This shouldn't matter because i didn't create any networks).
[Support] Linuxserver.io - Nextcloud
in Docker Containers
Posted · Edited by WeDevs
I have the exact same problem with the exact same setup. Nextcloud + mariaDB + Swag. Recent update by auto-app-updates killed nextcloud. I've confirmed swag is fine, there was a new /config/nginx/proxy.conf file pushed but i tested out old & new one. No difference (the change was to set a header to blank, this was to fix a security issue). Based on logs that this person and I am seeing I assume its something to did with PHP. I know there was a PHP8 version in the works but checking the default/normal nextcloud php version shows its PHP 7.4.25 so it shouldn't be a bad PHP upgrade? I looked through https://github.com/linuxserver/docker-nextcloud/releases and couldn't find anything that stood out, seems everything is just re-basing to latest alpine release. In the Nextcloud GUI I was already up-to-date as of at least 1-2 weeks ago. I tried downgrading the docker image from latest (== 22.2.0-ls160) to 22.2.0-ls156 which is from about 1 month ago but that didn't help either. I'm out of idea's. PHP error logs don't show anything useful.
Update 1) changing log level to "debug" for php7 does not show any more information. I guess crash/shutdown happens too quickly. I looked at logs from 1 month ago and they are the same as today/OP in regards to fpm shutting down. Nextcloud was working at that time. I saw a warning regarding increasing max children in those older logs. I just increased pm.max_children = 300 from the default 5. Its the only thing i could see in the past logs. Didn't make a difference.
Update 2) I think it might be caused by DNS but still investigating. I use pihole for DNS. /etc/resolve.conf in nextcloud has the correct local DNS server IPv4 address. When i try to ping local servers using their local DNS entries in pihole (from within nextcloud docker container), it fails to resolve. I know pihole DNS entries are working in my web browser it should not be caused by pihole. I also see HTTP 302's in nextcloud logs (as there should be) but no errors.
Update 3) Probably not a DNS issue. Ran NSLookup from within containers and they can resolve DNS address just fine. If i go to Nextcloud WebUI from within unraid-->docker menu it goes to unraid IPv4:<nextcloud port>. This is good. This then redirects to correct DNS address as configured in nextcloud. That is a SWAG docker container within unraid. This container runs on custom network using macvlan driver (so it can get its own IPv4 static address from home router). Swag acts as reverse-proxy for nextcloud. Looking at its logs it says "2021/11/01 09:49:43 [error] 497#497: *1 connect() failed (111: Connection refused) while connecting to upstream," for <unraid Ipv4>:<nextcloud_port-mapping>. All address and DNS stuff looks correct. I tried curl from SWAG container to unraid GUI, it failed. I tried to <unraid Ipv4>:<nextcloud_port-mapping> and it failed. Possibly issue with networking between SWAG & unraid.
Update 4) Logs and tests seem to show that my swag container cannot reach unraid IPv4 address. I thinking the custom/special network I created for the swag container will probably need to be recreated. Maybe there was some docker or unraid update that modifying some default settings? Because Swag cannot reach the UnRaid host IPv4 address it cannot act as a reverse proxy.
Update 5) Turned docker on/off from UnRaid GUI. This seemed to fix whatever weird issue caused swag not to be able to access UnRaid IPv4 address. I guess the solution of turn it off & on is still the best solution that exists. I was just turning off/on the wrong things. (Note i did enable the docker option of preserving user networks at startup. This shouldn't matter because i didn't create any networks).