Hi,

 

I'm slowly migrating my dockers from old hardware (tower) to newer hardware (towerpve). Memory usage is not even close to 50% (of 16G) and the system it self is not that busy but - since last week I had to hard reset the new server. Before I did that the led of the flask drive was full on - no blinks at all. I dunno if it was writing of reading (or both). Because the syslog is discarded and in this case I did not want to write it to flask I set the server as its own syslog. I also run a tail of the syslog on my workstation as well as the dashboards.

Just before the system became unresponsive again I had a ton of winbindd stacktrace kinda messages (also a lot of php-fpm)

 

Migrating the dockers is one thing - i make backups of the data every evening - I can revert back to my old tower if I have to but it's intel gen4 -and rather old. I really hope someone knows what the heck is going on.

syslog-tower.txt

Edited May 23, 20242 yr by sjoerd

This looks more like a web server docker error that you are running, than unraid itself having errors.
You would need to delete the docker temp data maybe delete the docker image file and recreate all dockers by click add template and re-selecting the running dockers. 

unraid has been fickle with board changes. there are some side things to do to make sure it is ready for new hardware especail when you chagne chipsets.

From recreating the network config, to stopping vm/dockers before the move, from plugin cleanup and data integrity.
Make sure all disk get transferd. Not sure what docker/service this is:

May 23 09:26:15 towerpve root: mediawiki-kiwi: Could not download icon /mnt/user/appdata/mediawiki-kiwi/images/mw-logo.png
May 23 09:54:02 towerpve emhttpd: spinning down /dev/sdk
May 23 10:44:27 towerpve root: K1600GT_ONTW: Could not download icon /mnt/user/appdata/wp-k1600gt-ontw/wp-content/uploads/2022/07/favico-k1600gt-nl.jpg
May 23 10:44:27 towerpve root: mediawiki-kiwi: Could not download icon /mnt/user/appdata/mediawiki-kiwi/images/mw-logo.png
May 23 11:14:31 towerpve root: K1600GT_ONTW: Could not download icon /mnt/user/appdata/wp-k1600gt-ontw/wp-content/uploads/2022/07/favico-k1600gt-nl.jpg
May 23 11:14:31

That is most likely not causing a servercrash  - just a file missing - also. those two dockers are running on that hardware for over a year.

The mediawiki-kiwi is the official mediawiki docker from mediawiki.org - I make a unraid docker config for it - totally nothing special

The k1600gt_ontw is the wordpress docker from Kru-X (as found in CA) - also nothing special

 

I hope someone from lime could take a look at it

 

 

Please post diagnostic file

towerpve-diagnostics-20240524-0829.zip

Hey guyz - server crashed again yesterday - logging to the syslog mount stopped at 18:46 as you can see

 

May 31 17:54:34 towerpve sshd[10701]: pam_unix(sshd:session): session closed for user root
May 31 18:04:36 towerpve emhttpd: read SMART /dev/sdk
May 31 18:09:10 towerpve winbindd[13985]: [2024/05/31 18:09:10.198407,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
May 31 18:09:10 towerpve winbindd[13985]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED
May 31 18:09:44 towerpve winbindd[13985]: [2024/05/31 18:09:44.175119,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
May 31 18:09:44 towerpve winbindd[13985]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED
May 31 18:46:04 towerpve emhttpd: spinning down /dev/sdk
May 31 18:46:59 towerpve winbindd[13985]: [2024/05/31 18:46:58.895567,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
May 31 18:46:59 towerpve winbindd[13985]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_IO_TIMEOUT
May 31 20:11:32 towerpve root: Delaying execution of fix common problems scan for 10 minutes
May 31 20:11:32 towerpve unassigned.devices: Mounting 'Auto Mount' Devices...
May 31 20:11:32 towerpve unassigned.devices: Mounting partition 'sdb1' at mountpoint '/mnt/disks/WD2T'...
May 31 20:11:33 towerpve unassigned.devices: Mount cmd: /sbin/mount -t 'ext4' -o rw,relatime,nodev,nosuid '/dev/sdb1' '/mnt/disks/WD2T'
May 31 20:11:33 towerpve kernel: EXT4-fs (sdb1): recovery complete
May 31 20:11:33 towerpve kernel: EXT4-fs (sdb1): mounted filesystem with ordered data mode. Quota mode: disabled.

 

Edited June 1, 20242 yr by sjoerd

And again

Jun  4 13:32:15 towerpve winbindd[13990]: [2024/06/04 13:32:14.819476,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
Jun  4 13:32:15 towerpve winbindd[13990]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_DISCONNECTED
Jun  4 13:32:43 towerpve winbindd[13990]: [2024/06/04 13:32:43.808444,  0] ../../source3/winbindd/winbindd_samr.c:111(open_internal_lsa_conn)
Jun  4 13:32:43 towerpve winbindd[13990]:   open_internal_lsa_conn: Could not connect to lsarpc pipe: NT_STATUS_IO_TIMEOUT
Jun  4 13:52:24 towerpve emhttpd: read SMART /dev/sdk
Jun  4 13:52:28 towerpve winbindd[13990]: [2024/06/04 13:52:28.642050,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
Jun  4 13:52:28 towerpve winbindd[13990]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_IO_TIMEOUT
Jun  4 13:56:36 towerpve winbindd[13990]: [2024/06/04 13:56:32.901265,  0] ../../source3/winbindd/winbindd_samr.c:71(open_internal_samr_conn)
Jun  4 13:56:36 towerpve winbindd[13990]:   open_internal_samr_conn: Could not connect to samr pipe: NT_STATUS_CONNECTION_RESET

 

Can please someone from the staff/devs look into this -

 

Try rebooting in safe mode, also, what is the custom script you have running in your go file?

#!/bin/bash
# custom
echo "copying custom root profile settings.."
cp -rvf /boot/config/custom/structure/. /.
chmod +x /etc/profile.d/*.sh

echo "adding aliasses.."
# aliassing vim
if $(which vim > /dev/null 2>&1) ; then
  alias vi='vim'
fi

echo "done.."

 

|-- go
`-- structure
    |-- etc
    |   `-- profile.d
    |       `-- extravars.sh
    |-- root
    |   |-- .bash_profile
    |   |-- .mysql
    |   |   |-- mysqldump_ontw-k1600gt.cnf
    |   |   |-- mysqldump_prod-k1600gt.cnf
    |   |   `-- mysqldump_towerpve.cnf
    |   `-- .vimrc
    `-- usr
        `-- local
            |-- bin
            |   `-- netdev_stats.sh
            `-- etc
                `-- rc.d
                    `-- rc.netdev_stats

 

(the rc.netdev_stats is not active)

 

#/bin/bash

# /etc/profile.d/extravars.sh

export BACKUP_PATH=/mnt/user/backup
export BACKUP_PATH_DB=$BACKUP_PATH/mariadb
export BACKUP_PATH_VM=$BACKUP_PATH/virtual_machines

export APPDATA=/mnt/user/appdata
export BASHLIB=$APPDATA/scripts/bshlib

 

 

# /root/.bash_profile

# console coloring for kool kids
PS1='\[\033[01;32m\]\u@\h\[\033[00m\]:\[\033[01;34m\]\w\[\033[00m\]\$ '

# if this is an xterm set the title to user@host:dir
case "$TERM" in
xterm*|rxvt*)
    PS1="\[\e]0;\u@\h: \w\a\]$PS1"
    ;;
*)
    ;;
esac

# impersonate a user
alias user="su -ls /bin/bash"
alias v="ls -lA"

# custom
alias vi="vim"

# fix linedrawing
export LANG=C.UTF-8

 

 

Soo nothing fancy really - just some extra environments variables and a altered .bash_profile for root and an customized .vimrc - the /root/.mysql/* is for backupscript i got on /mnt/user/appdata/scripts

 

 

Edited June 4, 20242 yr by sjoerd

18 minutes ago, JorgeB said:

Try rebooting in safe mode

and may be worth commenting out the script, just in case, code is not my thing so not sure it can be a problem, but it won't hurt to try.

I can comment out the rc but leaving out the extra env vars is going to break my backupsccripts (and they run not at the time the server hangs with the winbindd thing - infact - those scripts are pure bash and not winbindd on anything (thats samba only isnt it?)

 

Like mentioned it's beyond my knowledge to say if the scripts can be a problem, but if you think they are not, leave them for now.

happend again yesterday - to be sure I just replaced the flashdrive since I got a lot of errors on it (signs of corruption kinda errors)

So.... the server is running now without issues (sofar :D) for 10 days.. I changed two things: I replaced the flaskdrive which was reporting quite a lot of bad blocks and I change the docker network from macvlan to ipvlan (coz the system check plugin was whining about it). Personally I suspect the flaskdrive was the main issue here but that is just an assumption (my other server also has macvlan on the docker and is up for like 5 months orso

Edited June 17, 20242 yr by sjoerd