SSH Issue with specific client (also Unraid)

[Xutho]

Hi folks,

maybe you can give me a hint.. i´ve a strange issue with SSH.

I´ve 2 Unraid Systems up and running
Live-System and Backup System

Backup System is connected via Wiregurad to my opnsense Router (successfully i have fully access to that machine)
Last years i made backups with rsync to that Backup system on a daily basis.

But somehow now rsync has connection problems

 

rsync -avzse "ssh -i /root/.ssh/siteA-rsync-key" --bwlimit=50000 --delete /mnt/user/Backup/ [email protected]:/mnt/user/Data/Xutho-Unraid/Backup/
Connection closed by 10.6.0.3 port 22
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: unexplained error (code 255) at io.c(231) [sender=3.2.7]

 

as well if i try from my live-system to open a SSH connection
 

root@Xutho-Unraid:~/.ssh# ssh [email protected]

Connection closed by 10.6.0.3 port 22
root@Xutho-Unraid:~/.ssh#

 

 

But i can see that try on my Backup-System:
 

Feb 25 20:08:31 Xutho-Backup  sshd[4935]: Connection from 192.168.178.100 port 58776 on 10.6.0.3 port 22 rdomain ""
Feb 25 20:09:02 Xutho-Backup flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update
Feb 25 20:10:02 Xutho-Backup flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update
Feb 25 20:10:31 Xutho-Backup  sshd[4935]: fatal: Timeout before authentication for 192.168.178.100 port 58776

 

Funny thing: all my clients on the same network  as my Live-System can flawless connect via SSH to Backup-System except my Live-System......

Is there any kind of blacklist? Is maybe my Live-System kinda locked out?

 

[JorgeB]

ssh -vvv

might help see what the problem is.

[Xutho]

21 minutes ago, JorgeB said:

ssh -vvv

might help see what the problem is.

Hi Jorge
it stucks at "debug1: expecting SSH2_MSG_KEX_ECDH_REPLY"   and after ~30 sec´s  the connection will be closed.


 

root@Xutho-Unraid:~# ssh [email protected] -vvv
OpenSSH_9.1p1, OpenSSL 1.1.1s  1 Nov 2022
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 10.6.0.3 is address
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts' -> '/root/.ssh/known_hosts'
debug3: expanded UserKnownHostsFile '~/.ssh/known_hosts2' -> '/root/.ssh/known_hosts2'
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug3: ssh_connect_direct: entering
debug1: Connecting to 10.6.0.3 [10.6.0.3] port 22.
debug3: set_sock_tos: set socket 3 IP_TOS 0x48
debug1: Connection established.
debug1: identity file /root/.ssh/id_rsa type 0
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: identity file /root/.ssh/id_ecdsa_sk type -1
debug1: identity file /root/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: identity file /root/.ssh/id_ed25519_sk type -1
debug1: identity file /root/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /root/.ssh/id_xmss type -1
debug1: identity file /root/.ssh/id_xmss-cert type -1
debug1: identity file /root/.ssh/id_dsa type -1
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_9.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_9.1
debug1: compat_banner: match: OpenSSH_9.1 pat OpenSSH* compat 0x04000000
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 10.6.0.3:22 as 'root'
debug3: record_hostkey: found key type ECDSA in file /root/.ssh/known_hosts:1
debug3: load_hostkeys_file: loaded 1 keys from 10.6.0.3
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug3: order_hostkeyalgs: prefer hostkeyalgs: [email protected],ecdsa-sha2-nistp256
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256,ext-info-c
debug2: host key algorithms: [email protected],ecdsa-sha2-nistp256,[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],ssh-ed25519,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,[email protected],[email protected],rsa-sha2-512,rsa-sha2-256
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected],zlib
debug2: compression stoc: none,[email protected],zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: [email protected],curve25519-sha256,[email protected],ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group14-sha256
debug2: host key algorithms: rsa-sha2-512,rsa-sha2-256,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: ciphers stoc: [email protected],aes128-ctr,aes192-ctr,aes256-ctr,[email protected],[email protected]
debug2: MACs ctos: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: [email protected],[email protected],[email protected],[email protected],[email protected],[email protected],[email protected],hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,[email protected]
debug2: compression stoc: none,[email protected]
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: [email protected]
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: [email protected] MAC: <implicit> compression: none
debug1: kex: client->server cipher: [email protected] MAC: <implicit> compression: none
debug3: send packet: type 30
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY


Connection closed by 10.6.0.3 port 22
root@Xutho-Unraid:~# 

 

[Xutho]

I´ve managed to get it working again....

Somehow Wireguard didn´t run correctly on that Backup-Machine.
So i deleted all of the config and created a whole new profile in Wireguard for that machine (but same IP)

After i established my connection SSH was working fine....  strange behaviour that only SSH was kinde screwed up but nevermind ... it works again

@JorgeB thanks for that quick reply i´ve learned again something new for future "ssh root@ip -vvv" never saw that paramenter for debugging




Edit: I´ve found the exact issue! There seems to be a dependency with my MTU Value - I set it to 1440 and the issue was reproducable - after setting it back to AUTO the issue with SSH from Unraid-Server was gone.
Hopefully this helps someone in future

Edited March 2, 2023 by Xutho