Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

[dlandon]

5 hours ago, root1657 said:

Hello all.... I am plagued by the SAMBA mount issue, and it has me offline since yesterday. I swear on 3 decades of sysad that the credentials did not change, that all other systems are able to see the share, that there is network connectivity between unraid and the share, etc. My great error was daring to reboot my unraid machine, which I always fear to do, because samba is always an absolute battle to mount... is it set to auto mount? yes. Has it ever in the history of ever managed to auto mount? no. Typical workflow is to add new samba, scan to discover shares, and when it finds the share as an option, cancel it, and there's a 25% chance that the mount button lights up, else try again over and over until it does. Well, this time that isn't working anymore. The mount button is active, and it just keep timing out. I'm about at my wits end with this (it wasn't a long trip.... ), and throw myself at the mercy of the community to beg assistance before I have to purchase a different solution, but I cant keep living like this, its killing production.

Post diagnostics and then go to a command line and type "ud_diagnostics" and also post the /flash/logs/ud_diagnostics.zip file.

[root1657]

7 hours ago, dlandon said:

Post diagnostics and then go to a command line and type "ud_diagnostics" and also post the /flash/logs/ud_diagnostics.zip file.

Fingers crossed... thank you!

ud_diagnostics-20240808-085840.zip tower-diagnostics-20240808-0857.zip

[zhuzm]

Hi, I'm a newbie and I just started playing with unraid. I use unraid as the underlying system and then run Synology in a virtual machine. Synology uses NFS to share folders, and unraid can add them, but I can't mount them on unraid anyway. Is there something wrong with my settings?

[dlandon]

8 hours ago, zhuzm said:

Hi, I'm a newbie and I just started playing with unraid. I use unraid as the underlying system and then run Synology in a virtual machine. Synology uses NFS to share folders, and unraid can add them, but I can't mount them on unraid anyway. Is there something wrong with my settings?

Do the following:

• Go to Settings->Unassigned Devices and set the "Debug Log Level" to "None".  The UD debug will flood the log.
• Go to Settings->Unassigned Devices and set the "NFS Version to use when Mounting Remote Shares" to "Auto".

[dlandon]

17 hours ago, root1657 said:

Fingers crossed... thank you!

ud_diagnostics-20240808-085840.zip 33.31 kB · 1 download tower-diagnostics-20240808-0857.zip 116.14 kB · 1 download

I'm still looking at your diagnostics, but for the moment review your network configuration.  You have 4 NICs set up in a Bond as active backup.  That's probably not a good use of the 4 NICs.

 

I would also set your server network with a specified gateway IP and a static IP address for the server.  DHCP may be causing delays in acquiring an IP address on boot of your server.

 

Any router changes lately?  Upgraded firmware causing new problems?

 

Based on what I've seen so far, your issues are probably network related.

[root1657]

6 hours ago, dlandon said:

I'm still looking at your diagnostics, but for the moment review your network configuration.  You have 4 NICs set up in a Bond as active backup.  That's probably not a good use of the 4 NICs.

 

I would also set your server network with a specified gateway IP and a static IP address for the server.  DHCP may be causing delays in acquiring an IP address on boot of your server.

 

Any router changes lately?  Upgraded firmware causing new problems?

 

Based on what I've seen so far, your issues are probably network related.

The nics are as they have been since the installation of unraid in the before times. The network remains unchanged, and doesn't explain why every other application is fine, but mounting has been a pain, always. Even when going through the steps to add it as new, and its discovered, and that lights up the mount button, so seemingly it now knows where to find it, it times out..... What is the tolerable delay in a dns request for this plugin? I'll fire up a packet analyzer and see how long its taking, but the IP's are permanent reservations, and haven't changed literally in years.... (looks like dns requests from unraid to the dns for names inside the room are coming back approx 1ms) 

 

There is no configurable router or firewall between this unraid and this storage device, they are on a 8 port cisco dumb switch (SG-110D-08) with the computer used to access them both (this computer still has no issues with that storage device or access to the unraid, suggesting all three have good connection to that same switch) with one port to uplink to a different room (and eventually out to the world, I'm using that link to write this message actually). This device has been power cycled multiple times in troubleshooting this event.

 

Based on your feedback that you believe this is network related, should I open a support request over in general, and what should I tell them you think the network issue is?

Edited August 9 by root1657
added attachment

[zhuzm]

21 hours ago, dlandon said:

Do the following:

• Go to Settings->Unassigned Devices and set the "Debug Log Level" to "None".  The UD debug will flood the log.
• Go to Settings->Unassigned Devices and set the "NFS Version to use when Mounting Remote Shares" to "Auto".

Thank you very much. I know that I need to change the log to none. I turned it on to see where the problem occurred. I tried changing those settings, but none of them worked. Finally, I reinstalled the unraid system and it solved the problem. I guess it was a bug in the system, not a setting problem. Thank you again for your reply!

[dlandon]

17 hours ago, root1657 said:

The nics are as they have been since the installation of unraid in the before times. The network remains unchanged, and doesn't explain why every other application is fine, but mounting has been a pain, always. Even when going through the steps to add it as new, and its discovered, and that lights up the mount button, so seemingly it now knows where to find it, it times out..... What is the tolerable delay in a dns request for this plugin? I'll fire up a packet analyzer and see how long its taking, but the IP's are permanent reservations, and haven't changed literally in years.... (looks like dns requests from unraid to the dns for names inside the room are coming back approx 1ms) 

 

There is no configurable router or firewall between this unraid and this storage device, they are on a 8 port cisco dumb switch (SG-110D-08) with the computer used to access them both (this computer still has no issues with that storage device or access to the unraid, suggesting all three have good connection to that same switch) with one port to uplink to a different room (and eventually out to the world, I'm using that link to write this message actually). This device has been power cycled multiple times in troubleshooting this event.

 

Based on your feedback that you believe this is network related, should I open a support request over in general, and what should I tell them you think the network issue is?

What is the remote server?

[root1657]

14 hours ago, dlandon said:

What is the remote server?

It is a drobo, that Ironically, I was going to use this mount to migrate off of soon (its a massive risk keeping so much data in an unsupported device anymore). Its been configured as-is for a very long time, and has been 'working' with this unraid since it was built a year or two ago, with the exception of unraid occasionally refusing to mount it (all other network devices can access the share as per usual). I use this mount for tdarr to reduce file sizes, and was just about at the point that I can start migrating drives into the unraid array proper in hopes of retiring the drobo (which is why I currently have an unlimited storage license and just a 1tb nvme to hold docker containers to do automated work on the samba share before migration, having reduced storage requirements nearly in half). If this mount continues to not work anymore, I'm going to have to rethink the entire approach to the project, and have to flip it on its head, offering a share from unraid, and using the windows workstation to move files between 2 shares, which is BW inefficient, and will nearly double the transfer time I've calculated at 2 weeks up to nearly a month.... Good times.... Not a 'normal' use case, I know, but if they were all the same, it wouldnt be any fun!  

[dlandon]

15 hours ago, root1657 said:

It is a drobo, that Ironically, I was going to use this mount to migrate off of soon (its a massive risk keeping so much data in an unsupported device anymore). Its been configured as-is for a very long time, and has been 'working' with this unraid since it was built a year or two ago, with the exception of unraid occasionally refusing to mount it (all other network devices can access the share as per usual). I use this mount for tdarr to reduce file sizes, and was just about at the point that I can start migrating drives into the unraid array proper in hopes of retiring the drobo (which is why I currently have an unlimited storage license and just a 1tb nvme to hold docker containers to do automated work on the samba share before migration, having reduced storage requirements nearly in half). If this mount continues to not work anymore, I'm going to have to rethink the entire approach to the project, and have to flip it on its head, offering a share from unraid, and using the windows workstation to move files between 2 shares, which is BW inefficient, and will nearly double the transfer time I've calculated at 2 weeks up to nearly a month.... Good times.... Not a 'normal' use case, I know, but if they were all the same, it wouldnt be any fun!  

Here is some more information on the error you are getting:

 

The mount error you are encountering (cifs_mount failed w/return code = -112) indicates that the mount operation failed due to a network-related issue. Specifically, the return code -112 corresponds to a connection timeout or a problem reaching the SMB server. This typically occurs when:

• The server is unreachable: The network connection to the SMB server (FLATSTAR.LOCAL) might be down or misconfigured.
• DNS resolution issues: The system may have trouble resolving FLATSTAR.LOCAL to an IP address.
• Firewall issues: A firewall could be blocking the connection between your Unraid server and the SMB server.

To troubleshoot this issue, you can try the following steps:

• Check network connectivity: Verify that the SMB server is reachable from your Unraid server using a simple ping or nslookup command.
• Check firewall settings: Ensure that there are no firewall rules blocking the connection between your Unraid server and the SMB server.
• You might also want to examine the logs on the SMB server for any signs of connection attempts or errors.

[root1657]

On 8/11/2024 at 10:45 AM, dlandon said:

Here is some more information on the error you are getting:

 

The mount error you are encountering (cifs_mount failed w/return code = -112) indicates that the mount operation failed due to a network-related issue. Specifically, the return code -112 corresponds to a connection timeout or a problem reaching the SMB server. This typically occurs when:

• The server is unreachable: The network connection to the SMB server (FLATSTAR.LOCAL) might be down or misconfigured.
• DNS resolution issues: The system may have trouble resolving FLATSTAR.LOCAL to an IP address.
• Firewall issues: A firewall could be blocking the connection between your Unraid server and the SMB server.

To troubleshoot this issue, you can try the following steps:

• Check network connectivity: Verify that the SMB server is reachable from your Unraid server using a simple ping or nslookup command.
• Check firewall settings: Ensure that there are no firewall rules blocking the connection between your Unraid server and the SMB server.
• You might also want to examine the logs on the SMB server for any signs of connection attempts or errors.

The name resolves, 100% ping test, there is no firewall between these devices, they are connected by a cisco sg110d-08 unmanaged switch. Samba on drobo configuration remains unchanged 'forever' and all other devices are able to use it. Drobo logs not available.

 

Am I understanding these logs correctly that the 112 is due to a protocol mismatch, and unraid steps down to samba 1.0, and then actually gets a 512 error on samba 1.0? Whats a 512?

 

Edited August 12 by root1657

[dlandon]

3 hours ago, root1657 said:

The name resolves, 100% ping test, there is no firewall between these devices, they are connected by a cisco sg110d-08 unmanaged switch. Samba on drobo configuration remains unchanged 'forever' and all other devices are able to use it. Drobo logs not available.

UD is seeing the remote server online.  This is the online status UD detects when checking the remote server being online.  This is from the ud_diagnostics.

{
    "FLATSTAR.LOCAL.SMB": {
        "no_pings": 0,
        "online": "yes",
        "changed": "no"
    }
}

What this means is the remote server SMB port (445) is open and the remote server is acting as a SMB server.

 

Is there a conflict with port 445?

3 hours ago, root1657 said:

Am I understanding these logs correctly that the 112 is due to a protocol mismatch, and unraid steps down to samba 1.0, and then actually gets a 512 error on samba 1.0?

UD polls the remote server to determine which protocol the server supports.  The -112 means the remote server connection failed for network reasons.  It doesn't seem to be a protocol issue.

 

3 hours ago, root1657 said:

Whats a 512?

The mount command timed out:

Aug  7 10:02:49 Tower kernel: CIFS: VFS: cifs_mount failed w/return code = -512
Aug  7 10:02:49 Tower unassigned.devices: Warning: shell_exec(/sbin/mount -t 'cifs' -o rw,relatime,noserverino,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=1.0,credentials='/tmp/unassigned.devices/credentials_FLATSTAR_PlexMedia' '//FLATSTAR.LOCAL/PlexMedia' '/mnt/remotes/FLATSTAR_PlexMedia' 2>&1) took longer than 15s!
Aug  7 10:02:49 Tower unassigned.devices: SMB mount failed: 'command timed out'.
Aug  7 10:02:49 Tower unassigned.devices: Remote Share '//FLATSTAR.LOCAL/PlexMedia' failed to mount.

SMB 1.0 may not be supported on your drobo server.

 

Try this:

Go to Settings->SMB and enable NetBIOS.

It shouldn't be needed unless Unraid is acting as a SMB v1.0 server.

[dlandon]

3 hours ago, root1657 said:

The name resolves, 100% ping test, there is no firewall between these devices, they are connected by a cisco sg110d-08 unmanaged switch. Samba on drobo configuration remains unchanged 'forever' and all other devices are able to use it. Drobo logs not available.

I just did a test to a server that only supports SMB v1.0 and I got the -112 errors when UD tried to mount a remote share.  Check the SMB settings on the drobo server.

[root1657]

23 minutes ago, dlandon said:

I just did a test to a server that only supports SMB v1.0 and I got the -112 errors when UD tried to mount a remote share.  Check the SMB settings on the drobo server.

its. not. the. drobo. We are troubleshooting symptoms I'm not having, chasing rabbits. this drobo has not changed configuration in forever, an all other devices on the network are currently using it. The mount was working in unraid for many months (years?). 

 

I'm sorry that the plugin doesn't work anymore, but the only thing that changed in this environment was that the unraid was rebooted, Ironically, because there is a historical issue with unraid leaving files on the share locked, and having to reboot unraid to allow anything else to access them.... 

 

My network and smb server are fine, and this is going in circles. Sorry if I sound frustrated. Is there a different forum where I might be able to obtain assistance with the plugin issue I'm having? Else I'll have to declare this plugin option unsuitable for my systems and reengineer the entire environment. Just sad that it worked for so long, and now it wont.

[dlandon]

19 minutes ago, root1657 said:

The mount was working in unraid for many months (years?). 

I do see one thing that is different when mounting a CIFS share.  It may be a samba change.  Try this command to mount the remote share:

/sbin/mount -t 'cifs' -o rw,relatime,noserverino,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,credentials='/tmp/unassigned.devices/credentials_FLATSTAR_PlexMedia' '//FLATSTAR.LOCAL/PlexMedia' '/mnt/remotes/FLATSTAR_PlexMedia'

I removed the 'uid' and 'gid' options.

 

I am seeing this in your log that I've not seen before:

Aug  7 10:02:04 Tower kernel: CIFS: enabling forceuid mount option implicitly because uid= option is specified
Aug  7 10:02:04 Tower kernel: CIFS: enabling forcegid mount option implicitly because gid= option is specified

 

You might also have to specify a SMB version in the mount command if the drobo needs it.

[root1657]

4 minutes ago, dlandon said:

I do see one thing that is different when mounting a CIFS share.  It may be a samba change.  Try this command to mount the remote share:

/sbin/mount -t 'cifs' -o rw,relatime,noserverino,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,credentials='/tmp/unassigned.devices/credentials_FLATSTAR_PlexMedia' '//FLATSTAR.LOCAL/PlexMedia' '/mnt/remotes/FLATSTAR_PlexMedia'

I removed the 'uid' and 'gid' options.

 

I am seeing this in your log that I've not seen before:

Aug  7 10:02:04 Tower kernel: CIFS: enabling forceuid mount option implicitly because uid= option is specified
Aug  7 10:02:04 Tower kernel: CIFS: enabling forcegid mount option implicitly because gid= option is specified

 

You might also have to specify a SMB version in the mount command if the drobo needs it.

[dlandon]

2 minutes ago, root1657 said:

Run this command first:

mkdir /mnt/remotes/FLATSTAR_PlexMedia

 

[dlandon]

1 hour ago, root1657 said:

This is going to be a problem because I see you are using credentials and they are not available unless UD does a mount.

 

I am sort of out of ideas for the moment.

 

Edit: Try removing the remote share and then add it back.  Let UD search for the server and enter the credentials.  Then tell UD to search for the shares.  If the shares don't show, you have a credentials issue you need to sort out.  Then try to mount the share again and see if it works.

 

I am seeing some information that credentials can cause the -112 and -512 errors you are seeing.

 

[root1657]

3 hours ago, dlandon said:

This is going to be a problem because I see you are using credentials and they are not available unless UD does a mount.

 

I am sort of out of ideas for the moment.

 

Edit: Try removing the remote share and then add it back.  Let UD search for the server and enter the credentials.  Then tell UD to search for the shares.  If the shares don't show, you have a credentials issue you need to sort out.  Then try to mount the share again and see if it works.

 

I am seeing some information that credentials can cause the -112 and -512 errors you are seeing.

 

Tried mounting with the new procedure (image attached), no joy. Tried adding a new mount, and it cant see any shares (I swear the credentials are correct and working on every other device, and they haven't changed, the problem is in unraid or the plugin somewhere, lets not blame the smb server again...)

I have made peace with the fact that unassigned devices is no longer capable of mounting SMB. I'm just going to have to engineer another array and let the windows workstation handle the migration.

 

[dlandon]

7 hours ago, root1657 said:

Tried mounting with the new procedure (image attached), no joy. Tried adding a new mount, and it cant see any shares (I swear the credentials are correct and working on every other device, and they haven't changed, the problem is in unraid or the plugin somewhere, lets not blame the smb server again...)

I have made peace with the fact that unassigned devices is no longer capable of mounting SMB. I'm just going to have to engineer another array and let the windows workstation handle the migration.

 

Special characters can be a problem for php.  Be sure the password is characters and numbers only and see if it works.

[root1657]

5 hours ago, dlandon said:

Special characters can be a problem for php.  Be sure the password is characters and numbers only and see if it works.

Never minding that this is the same password that has been in place for a while now, are you saying the plugin can't handle proper passwords? I'm a cybersecurity engineer and find that unsettling....

[dlandon]

26 minutes ago, root1657 said:

Never minding that this is the same password that has been in place for a while now, are you saying the plugin can't handle proper passwords? I'm a cybersecurity engineer and find that unsettling....

Did you not say that you removed and re-entered the remote share on every reboot?  If so, I am wanting you to redo it in case there is a typo or the config file got corrupted.  I asked you to be sure to use a password without special characters as a test to be sure there are no problems with special characters.

 

Also, look at the Settings->PHP Settings log and see if we have any php errors.

[root1657]

9 hours ago, dlandon said:

Did you not say that you removed and re-entered the remote share on every reboot?  If so, I am wanting you to redo it in case there is a typo or the config file got corrupted.  I asked you to be sure to use a password without special characters as a test to be sure there are no problems with special characters.

 

Also, look at the Settings->PHP Settings log and see if we have any php errors.

I did not say I remove it on every reboot, I said I have to start to add a new remote smb to get it to search, then cancel it, so the one that's already configured will allow the mount button to light up. 

 

I have no settings | php settings. Checking tools| web ui | php setting shows log size 0, image attached.

[root1657]

On 8/13/2024 at 8:01 AM, dlandon said:

Did you not say that you removed and re-entered the remote share on every reboot?  If so, I am wanting you to redo it in case there is a typo or the config file got corrupted.  I asked you to be sure to use a password without special characters as a test to be sure there are no problems with special characters.

 

Also, look at the Settings->PHP Settings log and see if we have any php errors.

Its mounted again now. Nothing in the environment changed except that unraid was turned off for a time while I was frustrated with it.... so, yeah, smb server was fine, network was fine, creds were fine, there is something else that happens in unraid during shutdown/boot. I have no answer, only more questions.....

[dlandon]

5 hours ago, root1657 said:

Its mounted again now. Nothing in the environment changed except that unraid was turned off for a time while I was frustrated with it.... so, yeah, smb server was fine, network was fine, creds were fine, there is something else that happens in unraid during shutdown/boot. I have no answer, only more questions.....

I guess your server needed a rest.  Maybe there is something in your hardware that reset with a power off.