Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


dlandon

5568 posts in this topic Last Reply

Recommended Posts

16 minutes ago, bondoo0 said:

I'm not sure if anyone else has upgraded to 6.9.0 RC 35 with this installed, but it seems to create an issue for the web console (nginx).  When I upgraded to RC 35, the console wouldn't come up, and after some trial and error I determine that it was due to something in unassigned devices plugin (I don't have the plus installed if that helps narrow it down).  I could get the Web UI to start by restarting nginx, and waiting about 5 minutes.  I verified this by re-adding the unassigned devices plugin, and once again the problem showed up, remove the plugin and the problem goes away.    Any suggestions would be appreciated.  

 

Here is the link (hopefully this comes through) to the thread I started with my diagnostics.zip attached:  

 

Can you post a screen shot so we can see what you are talking about.

Link to post
  • Replies 5.6k
  • Created
  • Last Reply

Top Posters In This Topic

Top Posters In This Topic

Popular Posts

New release of UD.  Changes: When changing the mount point (which is also the share name), the mount point is checked for a duplicate of a user share or another UD device.  Samba cannot handle

While I appreciate your interest in esthetics, there is more to consider than just 'aligning' disk drive mounts: This disk is mounted without a UD script, it is probably better to put it in th

<step on soap box> Keep in mind that the Unraid array disk configuration is static and doesn't change until it is stopped.  UD has to deal with hot plugged disks, devices being dynamically

Posted Images

1.) Why is Unassigned Devices sending so many POST requests to "http://thoth/plugins/unassigned.devices/UnassignedDevices.php" while the MAIN tab is open? Regarding my network monitor it connects to the server every second:

1572667596_2020-11-1710_44_58.thumb.png.653cd4d07cb92cc733d9d3a7d7f5f5a0.png

 

POST content:

action=detect&csrf_token=blabla

Response:

{"reload":false,"diskinfo":0}

2.) And I have/had an additional problem. I start at the beginning.  Yesterday I decided to replace my SSD. So I changed all cached shares to "prefer" and executed the mover to move all data to the HDD. But the mover was extremely slow. I stopped the mover and enabled the mover logs. Result: New log entries popped up really slow and the mover tried to move files that were already moved?! Didn't make sense. And the even more strange thing was, that there was still action on the HDD. So I used inotifywait to check the traffic on the HDD and finally tried to use lsof to check which process was accessing this HDD and after 20 seconds (!) it returned this:

lsof
lsof: WARNING: can't stat() cifs file system /mnt/disks/DESKTOP-QQ89TOG_Desktop
      Output information may be incomplete.

After additional 20 seconds a WARNING for the next SMB share path appeared. I decided to stop the command and unmount all SMB shares. But it was not possible through the Dashboard. The logs returned that it took more than 10 seconds:

Nov 17 08:56:22 Thoth unassigned.devices: Removing Remote SMB/NFS share '//DESKTOP-/Pictures'...
Nov 17 08:56:22 Thoth unassigned.devices: Unmounting Remote SMB/NFS Share '//DESKTOP-/Pictures'...
Nov 17 08:56:22 Thoth unassigned.devices: Unmounting '//DESKTOP-/Pictures'...
Nov 17 08:56:22 Thoth unassigned.devices: Unmount cmd: /sbin/umount -t cifs '//DESKTOP-/Pictures' 2>&1
Nov 17 08:56:32 Thoth unassigned.devices: Error: shell_exec(/sbin/umount -t cifs '//DESKTOP-/Pictures' 2>&1) took longer than 10s!
Nov 17 08:56:32 Thoth unassigned.devices: Unmount of '//DESKTOP-/Pictures' failed. Error message: command timed out

Finally I unmounted manually ALL SMB shares by the WebTerminal which took each 30 to 120 seconds (!), until I was able to (fast) mount and unmount through the dashboard again. I think it had something to do with some mounted SMB shares that were offline (shutdown Windows PCs). That's how it looks now (before all SMB shares were mounted):

1943678678_2020-11-1710_43_28.png.a300150b60da1ef327299d86507bef56.png

 

Its only a guess, but I think this delay caused, that the mover has been executed twice (race condition) and so started to move files, that were already moved. Or the mover suffers from the huge timeout which is similar to what happend to me with the lsof command?!

 

P.S. After unmounting all the SMB shares the mover is fast as usual and does not return any errors. Even after I mounted the SMB shares that are online at the moment.

 

 

Conclusions:

- the timeout of 10 seconds should be raised or changeable through the UD settings

- something needs to be changed with the handling of mounted, but offline SMB shares. Maybe an auto unmount after x seconds and if the Auto Mount option has been selected it checks by interval (ping or similar) if it is able to remount the share.

 

Link to post

You have a lot of remote mounted shares.  Is it really necessary to do things that way?

16 minutes ago, mgutt said:

the timeout of 10 seconds should be raised or changeable through the UD settings

What would this do?  If the remote server is not reachable, increasing the delay will only slow down things further.  The reason for the timeouts is to keep some commands like 'df' from completely hanging the Unraid server.  If it can't complete in 10 seconds, it probably won't happen at all.

18 minutes ago, mgutt said:

something needs to be changed with the handling of mounted, but offline SMB shares. Maybe an auto unmount after x seconds and if the Auto Mount option has been selected it checks by interval (ping or similar) if it is able to remount the share.

I've given this a lot of thought because of other users having the same issues.  Unmounting the remote share whenever the remote server is off-line will create more problems than it solves.  CIFS does not work as cleanly as you might think when it comes to unmounting and re-mounting a share when the remote server is off-line.

 

Conclusion:

- Keep the remote server online and available, or only mount the remote shares when actually needed.

Link to post
15 minutes ago, dlandon said:

What would this do?  If the remote server is not reachable, increasing the delay will only slow down things further. 

I meant only the timeout of the unmount command, not the general timeout. As long the command is interrupted after 10 seconds its not possible to unmount through the Dashboard. And shouldn't the plugin allow the user to unmount something which was mounted through the plugin? 🤔

18 minutes ago, dlandon said:

Is it really necessary to do things that way?

Yes. I use the mounted shares to backup the client PCs. Works really nice and does not need any backup software.

20 minutes ago, dlandon said:

Unmounting the remote share whenever the remote server is off-line will create more problems than it solves

What are these problems? I mean what is different compared to a disconnected USB drive?

21 minutes ago, dlandon said:

only mount the remote shares when actually needed.

Of course I could use scripts only, but this is nothing I prefer and nothing I could easily explain to other people. I mean thats the reason why I'm using a Plugin. GUI power 😅

Link to post
3 minutes ago, mgutt said:

I meant only the timeout of the unmount command, not the general timeout. As long the command is interrupted after 10 seconds its not possible to unmount through the Dashboard. And shouldn't the plugin allow the user to unmount something which was mounted through the plugin?

CIFS does not work the same as a hard drive unmount.  Unmounting a remote share that is off-line is messy and shouldn't be done.  The best bet is to get the remote server back on-line.  I'll take a look at some options.

5 minutes ago, mgutt said:

What are these problems? I mean what is different compared to a disconnected USB drive?

Unmounting a remote share that is off-line is not a good idea for several reasons.  CIFS will do a recovery once the remote server is back on line.  Unmounting a remote share when the remote server is off-line will pull the rug out from under any application using that share and is a really bad idea.  What would be the criteria for unmounting the share?  There are so many different servers I don't know where to even start on the time to wait to unmount, and whether or not the remote server would handle it properly.

Link to post
27 minutes ago, mgutt said:

I meant only the timeout of the unmount command, not the general timeout. As long the command is interrupted after 10 seconds its not possible to unmount through the Dashboard. And shouldn't the plugin allow the user to unmount something which was mounted through the plugin?

The unmount of a CIFS mount is supposed to be forced when the server is off-line.  Looking into the code, I see a logic error that I will correct.  I'll spend a little time testing before including it in UD.

Link to post
14 hours ago, dlandon said:

Can you post a screen shot so we can see what you are talking about.

I'm not clear what the screen shot you are asking for would be. In my case when I have unassigned devices plugin installed, after a reboot, there is no unraid web page to display, and when I telnet in nginx isn't running, I can manually start nginx, and after a couple minutes unraid's web page comes up, and all appears normal, until the next reboot.

Link to post
2 hours ago, bondoo0 said:

I'm not clear what the screen shot you are asking for would be. In my case when I have unassigned devices plugin installed, after a reboot, there is no unraid web page to display, and when I telnet in nginx isn't running, I can manually start nginx, and after a couple minutes unraid's web page comes up, and all appears normal, until the next reboot.

You said you could see Dockers.  What webpage does not show?  Just UD?  Do you get a blank screen?

Link to post
12 minutes ago, dlandon said:

You said you could see Dockers.  What webpage does not show?  Just UD?  Do you get a blank screen?

Sorry, I should have been clear.  I get a timeout page going to the unraid home page (or other unraid pages such as Dashboard, UD, Main, etc).  However, I have dockers, such as Plex that publish their own web page outside of unraid, and those pages are available.

 

The only way to get the unraid pages to come back up is to login via SSH, and start nginx, which shows as not running on system boot when I have UD installed, and I believe is what provides the unraid web front end.

Link to post
4 hours ago, bondoo0 said:

Sorry, I should have been clear.  I get a timeout page going to the unraid home page (or other unraid pages such as Dashboard, UD, Main, etc).  However, I have dockers, such as Plex that publish their own web page outside of unraid, and those pages are available.

 

The only way to get the unraid pages to come back up is to login via SSH, and start nginx, which shows as not running on system boot when I have UD installed, and I believe is what provides the unraid web front end.

Post a screen shot of the UD page when it is working.  Also post your diagnostics.

Link to post
21 hours ago, dlandon said:

Unmounting a remote share that is off-line is not a good idea for several reasons.  CIFS will do a recovery once the remote server is back on line.

Can't confirm that. "mount -t cifs" returns nothing, although both SMB servers are online again (which is shown in the UD dashboard). "cat /etc/mtab" does not return shares, too.

 

21 hours ago, dlandon said:

Unmounting a remote share when the remote server is off-line will pull the rug out from under any application using that share and is a really bad idea.

Again: What is the difference compared to a USB drive (applications could rely on that, too)? Its up the the user to decide if he needs permanent connection. And the Unraid mover is a main unraid component which is in my opinion more important than a backup script which fails because of a unmounted server, which is not reachable anyway. Why not adding a new option?

460718735_2020-11-1808_33_26.png.322a5a06c169f4107481a0640ac43550.png

 

20 hours ago, dlandon said:

The unmount of a CIFS mount is supposed to be forced when the server is off-line.

Regarding my experience not needed. It takes longer, but it disconnects through the usual command, too.

 

21 hours ago, dlandon said:

What would be the criteria for unmounting the share?

 

That's how I solved it at the moment:

#!/bin/bash
# make script race condition safe
if [[ -d "/tmp/${0///}" ]] || ! mkdir "/tmp/${0///}"; then exit 1; fi; trap 'rmdir "/tmp/${0///}"' EXIT;
# auto unmount offline smb shares
while read line; do
    smb_server=$(echo $line | grep -oP '^//.*?/')
    smb_server=${smb_server:2:-1}
    smb_share=$(echo $line | grep -oP '^//.*? on ')
    smb_share=${smb_share:0:-4}
    timeout 1 ping -c 1 $smb_server
    if [[ $? -eq 124 ]]; then
        echo "Automatically unmount offline $smb_share"
        /sbin/umount -t cifs $smb_share
    fi
done < <(mount -t cifs)

Only auto-re-mount is missing as I'm searching for the config file of UD at the moment ;)

Edited by mgutt
Link to post

When I make changes/enhancements to UD I have to take into consideration the application to the wider community.  It appears you have a unique need and have come up with a solution for your situation.  At this time, I don't feel it appropriate to add an automatic unmount/re-mount of remote shares.  We've discussed some of the technical issues.

 

I do continually look for better ways to handle CIFS shares.  CIFS mounts depend on a reliable network connection and when the remote server goes off-line, the Linux commands to query things like used and free space will hard hang.  'df' is the worst.  That's why the commands are timed out by UD.  Every remote server also seems to do things differently and it's quite a challenge to keep up with every server's quirks.  Some older NAS devices are still using SMBv1, which has known security issues.

Link to post
20 hours ago, dlandon said:

Post a screen shot of the UD page when it is working.  Also post your diagnostics.

Here are my diagnostics and a screen shots.  I wasn't sure if you wanted the UD config page (I believe I left everything at the default) or the page showing the UD tab from main.  I have one drive that is used for offline backups, so it isn't typically connected, but is still shown/remembered.1602694758_2020-11-18(1).thumb.png.c341c9c09ce8647095b0de174f3d35e8.png

2020-11-18 (2).png

unraid-server1-diagnostics-20201116-1320.zip

Link to post
2 hours ago, bondoo0 said:

Here are my diagnostics and a screen shots.

I see this in the log:

Nov 16 13:14:06 unraid-server1 unassigned.devices: Mounting 'Auto Mount' Remote Shares...
...
Nov 16 13:19:36 unraid-server1 unassigned.devices: Cannot 'Auto Mount' Remote Shares.  Network not available!

The startup check for the network to be available before auto mounting remote shares is taking over 5 minutes to determine the network is not available.  There is an issue on the network available script that causes this lengthy time, and because it is in an Unraid startup event, the Unraid GUI will not be available until it completes the network check.  I have a fix applied, but it is not released yet.

 

When you reboot Unraid, the GUI will eventually come up after the 5 minute delay caused by this issue.

Link to post
24 minutes ago, dlandon said:

I see this in the log:


Nov 16 13:14:06 unraid-server1 unassigned.devices: Mounting 'Auto Mount' Remote Shares...
...
Nov 16 13:19:36 unraid-server1 unassigned.devices: Cannot 'Auto Mount' Remote Shares.  Network not available!

The startup check for the network to be available before auto mounting remote shares is taking over 5 minutes to determine the network is not available.  There is an issue on the network available script that causes this lengthy time, and because it is in an Unraid startup event, the Unraid GUI will not be available until it completes the network check.  I have a fix applied, but it is not released yet.

 

When you reboot Unraid, the GUI will eventually come up after the 5 minute delay caused by this issue.

Okay, that makes sense, thanks for looking into this for me, and thanks for all your work on this plugin (and also explains why when I manually restarted nginx I still had to wait 5 minutes or so for the GUI to come up).

Link to post

@dlandon:

 

I rechecked the error code today, tried around with the command UD gives in the webterminal and concluded that there must be something wrong with the share.

 

Then i discovered that my Win machines could not connect to the share too today (checked it right at the beginning with UD, not after)

 

Checked into my openwrt Router and bumm, it lost the user account i build and used for the share. Dont know why, but it happened.

 

TL;DR: It works now and i am sorry wasting your time

 

Conclusion: Mount Error Code 13 or 95: Check your share at the outside server

Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.