Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


Recommended Posts

1 hour ago, geonerdist said:

Good catch, thanks....better brush up on my smart report skills. Would it matter if I pulled the diagnostics after I had rebooted and uninstalled -UD? I'm going to see if it'll hang again and get diagnostics downloaded before it becomes completely unresponsive. Thanks again!

Read the prior post.  I'd like you to try that to see if that helps keep UD from hanging.  After you copy the file, work with UD a bit to see if it hangs.  Post diagnostics so I can see where things go wrong.

Link to comment

Hello @dlandon,

 

Happy Holidays one and all!

 

So, I just had a power outage and the UPS shutdown executed as expected. However, (and I neglected to address this way back when I discovered it, ugh) my Unassigned Devices (UD) do not disconnect during shutdown, if those devices are already off (power outage or not), and the primary halts shutting down.

 

I was able to complete the shutdown and reboot after restarting the other servers UD was connected to prior to power outage. My question is, and please bear with me as I am trying to learn, in an effort to stop this from happening in the future, do I open Putty and copy (cp) the ‘lib.php’ to the ‘/usr/local/emhttp/plugins/unassigned.devices/include/’ directory and restart the primary server?

 

I attached the Diag log and my setup is as follows:

I have a dual Mellanox 10Gb NIC in the primary server and single Mellanox 10Gb NIC in two other servers. There is an NFS connection through UD on the primary server to each of the other two servers. If the connections in UD, on the primary server, are not disconnected before the other two servers shutdown, the primary will not execute a clean shutdown and hangs until the other two servers are back up.

 

I am open to any directions, suggestions, solutions.

Thanks in advance for your patience and any assistance!

 

tower01-diagnostics-20191129-0019.zip

Link to comment
3 minutes ago, TechMed said:

My question is, and please bear with me as I am trying to learn, in an effort to stop this from happening in the future, do I open Putty and copy (cp) the ‘lib.php’ to the ‘/usr/local/emhttp/plugins/unassigned.devices/include/’ directory and restart the primary server?

Just copy it and that's all.  If you reboot, the file will be replaced by the UD plugin when the array is started.

 

The lib.php file is a test version I want tested by people that have issues with UD hanging.  I'll release it in an updated plugin once I get some feedback on if it is helping and not creating problems.

Link to comment

 

4 minutes ago, dlandon said:

The lib.php file is a test version I want tested by people that have issues with UD hanging.  I'll release it in an updated plugin once I get some feedback on if it is helping and not creating problems.

Got it!

 

I do not want to dirty the thread, but for the benefit of the student, do you know where can I research how to write a (in DOS parlance) ".bat" file that can be run before the actual 'shutdown' command is sent?

 

(and thanks for the quick reply!)

Link to comment
19 minutes ago, TechMed said:

Hello @dlandon,

 

Happy Holidays one and all!

 

So, I just had a power outage and the UPS shutdown executed as expected. However, (and I neglected to address this way back when I discovered it, ugh) my Unassigned Devices (UD) do not disconnect during shutdown, if those devices are already off (power outage or not), and the primary halts shutting down.

 

I was able to complete the shutdown and reboot after restarting the other servers UD was connected to prior to power outage. My question is, and please bear with me as I am trying to learn, in an effort to stop this from happening in the future, do I open Putty and copy (cp) the ‘lib.php’ to the ‘/usr/local/emhttp/plugins/unassigned.devices/include/’ directory and restart the primary server?

 

I attached the Diag log and my setup is as follows:

I have a dual Mellanox 10Gb NIC in the primary server and single Mellanox 10Gb NIC in two other servers. There is an NFS connection through UD on the primary server to each of the other two servers. If the connections in UD, on the primary server, are not disconnected before the other two servers shutdown, the primary will not execute a clean shutdown and hangs until the other two servers are back up.

 

I am open to any directions, suggestions, solutions.

Thanks in advance for your patience and any assistance!

 

tower01-diagnostics-20191129-0019.zip 121.61 kB · 1 download

When there is an unclean shutdown, a diagnostics fie is created if possible at /flash/logs/.  Please post it if there is one.

Link to comment
18 minutes ago, TechMed said:

Some log entries:

Nov 28 18:09:56 Tower01 unassigned.devices: Unmount cmd: /bin/umount -fl '10.69.69.240://mnt/user/black_hole' 2>&1

Nov 28 18:19:59 Tower01 kernel: nfs: server 10.69.69.240 not responding, still trying
### [PREVIOUS LINE REPEATED 1 TIMES] ###

Nov 28 19:00:20 Tower01 unassigned.devices: Successfully unmounted '10.69.69.240://mnt/user/black_hole'

It looks like the unount of the nfs share is stuck.  The unmount '-fl' parameter is supposed to force an unmount.  Looks like that is taking a long time.  The new timeout on the 'umount' should help.

 

 

Link to comment

That is the assumption I was running with; stumbled across it by shear luck! haha

 

Despite the occasional head scratching, I really appreciate the tool.

For me it makes transferring files between servers, regardless if it between floors or cities, much easier.

 

Thanks again for the assist.

Link to comment

I set up a Backup server sitting right next to my Primary server. From Primary, I used UD to SMB mount a share on Backup, and I'm using Duplicati to run the backups from Primary to Backup. Backup had been assigned a static IP address and all was good. In preparation for moving Backup off-site I switched it from static to DHCP (I'm not sure of networking at its new home-to-be) and now Primary cannot access it via the UD mount point. This makes some sense to me since its IP address changed (though I'd specified the UD mount by name, not IP) - I believe this is because the DNS cache has not yet updated to recognize Backup at its new IP address. 

 

Having done this yesterday evening, this morning (about 12 hours later) it still wasn't properly recognizing the server at its new IP. (in UD, I clicked the mount point to browse and all it showed me was "parent folder"). This morning I unmounted and remounted the share (it took quite a while to unmount, but the remount went very quickly), and now I can properly browse the directory structure via UD's mount point share. 

 

A couple of questions:

1) Is it normally necessary to unmount/remount when an IP address changes or should I normally just have to wait out the DNS cache update?

2) Was it something else totally that was the issue here?

3) Once the server goes to its remote location, it will be on a totally different subnet as I will be using the great ZeroTier plug in to get the two of them to talk to each other, I presume that going from a 192.168.* address to a 172.29.* address will not have any impact on UD's ability to map this, correct?

4) Is SMB the best option for having the 2 servers talk to each other, or would I be better off using an NFS mount? (If so, I'd probably keep the SMB mount in order to be able to browse via Windows, just because I can.)

 

Thanks (again, I hope) for picking this up and continuing to run with it! 

Link to comment

Hi @FreeMan,

 

First I am FAR from an expert, but coincidentally I am actually backing up systems right now.

There are folks here significantly more knowledgeable about UD than myself. That said I can speak to #3 & #4 from personal experience.

 

#3 - No problems. Have  read through the link in #4 and you should be golden!

 

#4 - I think most everyone will agree that a NFS connection is better.

       With respect to the connection, I HIGHLY recommend looking at this link as @tr0910 (and others) put in a great deal of work to make backups a breeze. I will not lie, it is a bit of work, but not only is it worth the work, it is a fast efficient and @#$%ed near fool-proof method to doing backups. I too started with Duplicati and bailed after the second multi-day backup; Rsync is WAY better.

 

Have fun! This is actually an enjoyable project!

Link to comment

A plugin update fails with the following lines:

plugin: updating: unassigned.devices.plg
plugin: downloading: "https://github.com/dlandon/unassigned.devices/raw/master/unassigned.devices-2019.11.29.tgz" ... done
plugin: bad file MD5: /boot/config/plugins/unassigned.devices/packages/unassigned.devices-2019.11.29.tgz

Somebody else having this issue?

Link to comment

I'm also having the same issues. If you want to fix it yourself right now, download the .plg to somewhere on your server, change the 12th line so that the md5 sum is:

 

8796a7b713b0d121385f1452eb7880b1

 

and then tell unRAID to install from that downloaded plg file. Otherwise, I expect it will be fixed shortly.

  • Thanks 1
Link to comment
34 minutes ago, bamhm182 said:

I'm also having the same issues. If you want to fix it yourself right now, download the .plg to somewhere on your server, change the 12th line so that the md5 sum is:

 

8796a7b713b0d121385f1452eb7880b1

 

and then tell unRAID to install from that downloaded plg file. Otherwise, I expect it will be fixed shortly.

Where i find the .plg ? After Reboot  my Server the unassigend device / plugin its gone 😞 

 

PS: Find it ... Hope it will help

Edited by KinGSiZ3
Find it
Link to comment

You can run these commands:

cd $(mktemp -d)
wget https://raw.githubusercontent.com/dlandon/unassigned.devices/master/unassigned.devices.plg
sed -i s/fc6f30a2f824f5b56e367ed73edea275/8796a7b713b0d121385f1452eb7880b1/g unassigned.devices.plg

 

Then you can go to the Plugins tab > Install Plugin, and then paste the file location in there, for example: /tmp/tmp.tivBjv7aFZ/unassigned.devices.plg and click install.

 

Edited by bamhm182
Link to comment
4 minutes ago, bamhm182 said:

You can run these commands:


cd $(mktemp -d)
wget https://raw.githubusercontent.com/dlandon/unassigned.devices/master/unassigned.devices.plg
sed -i s/fc6f30a2f824f5b56e367ed73edea275/8796a7b713b0d121385f1452eb7880b1/g unassigned.devices.plg

 

Then you can go to the Plugins tab > Install Plugin, and then paste the file location in there, for example: /tmp/tmp.tivBjv7aFZ/unassigned.devices.plg and click install.

 

Really you are my daily Hero today ... THX for the help. All working now :-*

Edited by KinGSiZ3
Link to comment

I was finally able to reproduce some of the hangs people have experienced.  I had to add code to handle some Linux commands that hang when a remote CIFS share server goes offline.  Some commands that didn't even have anything to do with the CIFS mounts.

 

All Linux commands now have time outs to prevent hangs.  You'll see error messages in the log when a command times out.  I may have to extend some time outs for different systems.

 

Anyone having the UD hang, please test and then give me a diagnostics regardless if you have issues or not.  I need to see if the time outs are appropriate for all cases.

Link to comment

Thanks, @TechMed. I've looked through that thread a couple of times. I've wanted the "Oh, carp. I just realized I made changes and now I need to go back to an older version" version of backup. But, in the last 10 years or so, I don't think I've ever gone back to recover an old version of a file, so maybe it's time to give rsync a 2nd look. :) 

Link to comment
  • trurl pinned this topic

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.