Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


Recommended Posts

New release of UD.  Highlights:

  • Don't allow unmounting a disk that was not mounted by UD.  If you manually mount an unassigned disk at a mount point other than /mnt/disks/, UD will not be able to unmount it.
  • Change disk size query to try to keep from spinning up disks.
  • Cut down on tmp file accesses.
  • Fix a situation where permission settings from UD Settings would not update the shares.
  • Some more code cleanup.
Link to comment
1 hour ago, dlandon said:

New release of UD.  Highlights:

  • Don't allow unmounting a disk that was not mounted by UD.  If you manually mount an unassigned disk at a mount point other than /mnt/disks/, UD will not be able to unmount it.
  • Change disk size query to try to keep from spinning up disks.
  • Cut down on tmp file accesses.
  • Fix a situation where permission settings from UD Settings would not update the shares.
  • Some more code cleanup.

I just updated it and tested the new release with my external USB drive. Sadly, a refresh on the main tab still wakes up the drive. The option "PASS THRU" is enabled. Found this in my logs when I refresh the tab:

May 31 23:27:23 Zeus unassigned.devices: Error: shell_exec(/usr/bin/dd bs=446 count=1 if=/dev/sdb 2>/dev/null | /bin/sum | /bin/awk '{print }') took longer than 5s!

Note that the device is formatted as ZFS and reported as "zfs_member" by UD. Can confirm that the command wakes up the drive:

root@Zeus:~# /usr/bin/dd bs=446 count=1 if=/dev/sdb 2>/dev/null | /bin/sum | /bin/awk '{print }'
23591     1

 

Edit: Solved with 2020.05.31a. Big thanks @dlandon!

Edited by T0a
Link to comment
On 5/30/2020 at 1:15 AM, dlandon said:

You set per user access in the UD settings.  This is the same way that Unraid does user access.

Yes I get that you can set samba user access via the UI (for array shares and UD). The array shares won't force the "nobody" user on me tho (I attached a video for reference).  

All I'm wondering why UD has to behave differently in this regard? Why limiting the options for advanced users for no good reason? (or is there a good reason?) The unix group is "users" for everyone accessing via SMB if you want common permissions, why have a redundant "nobody" user that is of no use and just behaves differently than shares on the array? Why make administration more difficult? Why do I need a custom script that removes this unnecessary option and why can't I just check a box in the UI to do so?  

 

Don't get me wrong, I really appreciate your plugin. I just haven't found a good reason (nor did you give any) for why it has to be different.

2020-06-01 13-50-08.mkv

Edited by Thx And Bye
Link to comment
5 hours ago, Thx And Bye said:

Yes I get that you can set samba user access via the UI (for array shares and UD). The array shares won't force the "nobody" user on me tho (I attached a video for reference).  

All I'm wondering why UD has to behave differently in this regard? Why limiting the options for advanced users for no good reason? (or is there a good reason?) The unix group is "users" for everyone accessing via SMB if you want common permissions, why have a redundant "nobody" user that is of no use and just behaves differently than shares on the array? Why make administration more difficult? Why do I need a custom script that removes this unnecessary option and why can't I just check a box in the UI to do so?  

 

Don't get me wrong, I really appreciate your plugin. I just haven't found a good reason (nor did you give any) for why it has to be different.

2020-06-01 13-50-08.mkv 1.2 MB · 3 downloads

I would appreciate it if you would take your intensity level down a notch.  I understand you are frustrated because UD dosn't work exactly as you want.  There are many users each with different needs and I try to address as many of the needs as I can for everyone and not customize UD for a particular situation.  If I provided a check box for every off the wall request in UD, there would not be enough UI space for them all.

 

I didn't explain why the force nobody user was done on UD remote shares because you didn't ask.  CIFS mounts are a real can of worms.  There are three diferent versions of the SMB protocol, one of which is becoming obsolete.  There are two levels of security.  Each host seems to have its own way of doing things which makes it a bit difficult to manage with UD.  Some older servers only support SMB v1.  Some require different levels of security for each version.  Permissions are handled by the host and there have been issues in the past with access to a remote share without the force nobody parameter in the samba config file.  That's why it was added.

 

I guess I am a bit confused as to why you are mounting a remote SMB share only to then share it again on Unraid and then demand UD do the permissions as you desire.  Why not access the remote share directly and set permissions on the host as you like?

Link to comment

I've come across a new (to me) issue that started a couple days ago. I had one old 2TB drive get marked as bad and emulated via parity. I did a clean shutdown and then swapped out the drive with a new 8TB that had successfully been precleared. Alas when I restarted the system, it saw the new 8TB but 2 more old 2TB drives were listed as missing (yes, I had the array set so it didn't start automatically upon reboot).

 

I tested all 3 of the failed 2TB drives and they spin up but are not recognized, i.e. the Disks utility on my Ubuntu system doesn't even list/show them at all. So no big concern - they were 3 of 7 old 2TB drives that I've been using as scratch space on my backup unRAID box. I'm not overly concerned about lost data as the system is my backup system for items that I want to keep from my main unRAID and it's still running fine.

 

So I decided to do a 'New Config' with the 3 x 2TB drives removed and the new 8TB installed in one of the hotswap bays of one of my Startech hot-swap enclosures (3 x 3.5" in 2 x 5.25" space; https://www.amazon.ca/gp/product/B00HS23QZO/ ). When the parity build/sync started all seemed good but when I checked on it about 2hrs later, the new 8TB drive is throwing constant read errors. I also noticed that the drive is listed as part of the unRAID array but also showed up in UD as per the following screenshot (highlighted in yellow):

 

EnclosureFailure.thumb.jpg.6b1b8b191cb11458b1c941554ea54b1d.jpg

 

I did grab diagnostics (had to use terminal and type 'diagnostics' as the unRAID webgui for the System Log showed an error like this user reported back in April:

 

I'm now going to remove the 3 bay hot-swap enclosure and try direct cabling to the new 8TB drive. A 2nd attempt like the 1st failed as well when the 8TB drive appeared to drop off and then got 're-claimed' by UD and the array simultaneously. I thought that UD would ignore devices that are part of the array but the same drive appears in both the array and in UD when the read errors occur.

 

I'm fairly certain that direct cabling will solve the issue and that I have a defective hot-swap enclosure. Regardless I wanted to report it here just in case there's something that you @dlandon want me to look into. I have the diagnostics from both 'failures' but it's certain that the issue was caused when the drive(s) dropped off the bus. Somehow that killed the 3 x 2TB drives but again, no worry as it was backups of data on my main unRAID setup.

 

Let me know if you need further details. Thanks!

Edited by AgentXXL
Link to comment
16 hours ago, dlandon said:

I guess I am a bit confused as to why you are mounting a remote SMB share only to then share it again on Unraid and then demand UD do the permissions as you desire.  Why not access the remote share directly and set permissions on the host as you like?

I already did ask for a reason but that's not really something we have to argue about because it's pointless and leads to nowhere.

Our communication didn't quite work out I'd guess, my apologies for that. So lets start fresh.


I just have some local non-array drives that are mounted via UD. No shenanigans with mounting and re-exporting some other network shares.

The array works just as expected, new files are written with the accessing username and the 'users' group. So it's easy to set user-specific permissions, permissions for all registered users and for everything else (e.g. stuff running in a docker). So all three sets of permission work as intended (Owner, Group, Other) work independently and give great control over individual files without needing to restrict a user completely via the SMB access permissions.

 

Now for the non-array drives in the same physical and logical system that are mounted via UD (/mnt/disks/) and exported via a SMB share ("SHARE") there is one dimension less of control over the permissions. As in the Owner "nobody" is effectively equal to Group "users" for those shares. So all SMB access to those shares are operations with only two sets of permission since Owner and Group are essentially equal, if that makes sense. (Taking away the control over the Owner permission)

 

Why is that, and would it be possible to implement a option to gain back that additional control that the Unix permission system allows without any hackery? As this is a artificial restriction imposed specifically by UD and I don't see a technical reason to do so. Simply removing the "Force User = nobody" from the SMB config files created by UD and then restarting samba works just fine. I understand that removing this per default makes little sense, because it could breaks a lot of peoples setups, but I'd really appreciate a option to have the UD shared drives operate in the same way on the Unix file system as the shares on the array.

 

 

Link to comment
12 hours ago, Thx And Bye said:

I already did ask for a reason but that's not really something we have to argue about because it's pointless and leads to nowhere.

Our communication didn't quite work out I'd guess, my apologies for that. So lets start fresh.


I just have some local non-array drives that are mounted via UD. No shenanigans with mounting and re-exporting some other network shares.

The array works just as expected, new files are written with the accessing username and the 'users' group. So it's easy to set user-specific permissions, permissions for all registered users and for everything else (e.g. stuff running in a docker). So all three sets of permission work as intended (Owner, Group, Other) work independently and give great control over individual files without needing to restrict a user completely via the SMB access permissions.

 

Now for the non-array drives in the same physical and logical system that are mounted via UD (/mnt/disks/) and exported via a SMB share ("SHARE") there is one dimension less of control over the permissions. As in the Owner "nobody" is effectively equal to Group "users" for those shares. So all SMB access to those shares are operations with only two sets of permission since Owner and Group are essentially equal, if that makes sense. (Taking away the control over the Owner permission)

 

Why is that, and would it be possible to implement a option to gain back that additional control that the Unix permission system allows without any hackery? As this is a artificial restriction imposed specifically by UD and I don't see a technical reason to do so. Simply removing the "Force User = nobody" from the SMB config files created by UD and then restarting samba works just fine. I understand that removing this per default makes little sense, because it could breaks a lot of peoples setups, but I'd really appreciate a option to have the UD shared drives operate in the same way on the Unix file system as the shares on the array.

I have been trying to figure out your use case and you seem to want to just explain permissions over and over.  I'd like to see if there is another option for you.  I'm not a fan of additional UD settings that can be confusing to users and potentially create support issues.  You are the first user that has had an issue with this.

On 6/1/2020 at 7:55 AM, Thx And Bye said:

The unix group is "users" for everyone accessing via SMB if you want common permissions, why have a redundant "nobody" user that is of no use and just behaves differently than shares on the array? Why make administration more difficult? Why do I need a custom script that removes this unnecessary option and why can't I just check a box in the UI to do so?

The "force users = nobody" was added for good reasons.  I put a lot of thought into implementing features in UD  to be sure the feature applies to the greatest number of users.  My goal is to minimize user issues to be sure their experience is positive and limit support issues.  One of the downsides of this approach is a potential loss of some features that advanced users might want.

 

You are a newbie and haven't had much experience with the forum.  Spend a minute and search the forum for "force user = nobody".  You will see a lot of discussion about permission issues.  That is why I implemented "force users = nobody".  It solved more problems than it created.

Link to comment

Loving this plugin so far, real nice to be able to try out unraid and get it setup without committing XFS / transferring all my data to this format and preventing me from going back if needed. Also it will take a long time to transfer everything over since I will have to move things to another drive, and back again.

 

I am having an issue though, my old setup was windows and thus I used spaces in the share names without an issue. A lot of devices are mapped to these names and changing them all would be a royal pain at this point.

 

Is there a way to use spaces in the share names with UD?

 

Everytime I try it just turns the space into an underscore?

 

Although unraid itself appears to be able to handle spaces fine.

 

Any tips would be great.

Link to comment
2 hours ago, TexasUnraid said:

Loving this plugin so far, real nice to be able to try out unraid and get it setup without committing XFS / transferring all my data to this format and preventing me from going back if needed. Also it will take a long time to transfer everything over since I will have to move things to another drive, and back again.

 

I am having an issue though, my old setup was windows and thus I used spaces in the share names without an issue. A lot of devices are mapped to these names and changing them all would be a royal pain at this point.

 

Is there a way to use spaces in the share names with UD?

 

Everytime I try it just turns the space into an underscore?

 

Although unraid itself appears to be able to handle spaces fine.

 

Any tips would be great.

I'm looking into the possibility of spaces in the share name.

Link to comment
10 hours ago, TexasUnraid said:

Loving this plugin so far, real nice to be able to try out unraid and get it setup without committing XFS / transferring all my data to this format and preventing me from going back if needed. Also it will take a long time to transfer everything over since I will have to move things to another drive, and back again.

 

I am having an issue though, my old setup was windows and thus I used spaces in the share names without an issue. A lot of devices are mapped to these names and changing them all would be a royal pain at this point.

 

Is there a way to use spaces in the share names with UD?

 

Everytime I try it just turns the space into an underscore?

 

Although unraid itself appears to be able to handle spaces fine.

 

Any tips would be great.

Update UD and you should be able to use spaces in the share names.

Link to comment
14 hours ago, dlandon said:

The "force users = nobody" was added for good reasons.  I put a lot of thought into implementing features in UD  to be sure the feature applies to the greatest number of users. 

Just wondering if this is some kind of legacy fix for permissions? There seems to be a "fix permissions" script that does the same and set everything to nobody:users. Also the consensus seems to set 777 permissions for some reason? I personally "grew up" with just setting the permissions that are actually needed and not making the file system "free-for-all" just because it's more convenient.  

Since I'm fairly new to unraid and the regular array shares are simply using the <username>:users permissions (at least for new installations), I don't really see a reason to incorporate nobody into the unix permissions. For me this just seems like adding unnecessary abstraction to the permissions.

 

14 hours ago, dlandon said:

My goal is to minimize user issues to be sure their experience is positive and limit support issues.

I totally understand this approach and at this point this surely shouldn't be just removed for everyone since it would probably cause lots of problems for many people. But imo there should be at least a kinda hidden away "advanced settings" that could display a warning if the setting is changed to give the option to unify SMB behavior with the regular unraid array shares. Taking away options just to make something more fool proof isn't always the right approach it just needs to be clear what settings can cause massive problems and just make those less accessible.

Link to comment

Hi all, hope you can help!

 

I have a clean nvidia unraid install and I have unassigned devices installed.

When I first installed UD it worked great. However after a reboot of the server I now get the below errors:

 

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1106

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1298

 

I have uninstalled UD and deleted /flash/config/plugins/unassigned.devices/ and then installed UD again as @dlandon instructed another user to do and this worked after installing but keeps showing that same error every time the server is rebooted.

 

Hope someone can help!!!

Link to comment
2 hours ago, Solverz said:

Hi all, hope you can help!

 

I have a clean nvidia unraid install and I have unassigned devices installed.

When I first installed UD it worked great. However after a reboot of the server I now get the below errors:

 

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1106

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1298

 

I have uninstalled UD and deleted /flash/config/plugins/unassigned.devices/ and then installed UD again as @dlandon instructed another user to do and this worked after installing but keeps showing that same error every time the server is rebooted.

 

Hope someone can help!!!

You didn't get it quite right.  Delete /flash/config/plugins.unassigned.devices/*.cfg, then immediately reboot.  There are working copies in memory that will overwrite the ones on the flash unless you reboot right away.

Link to comment

I am getting the same error as Solverz:

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1106

I tried uninstalling the plugin, rebooting, etc.  The error keeps coming back.

Thanks

Joe

Link to comment
5 hours ago, Joe said:

I am getting the same error as Solverz:

Warning: Invalid argument supplied for foreach() in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 1106

I tried uninstalling the plugin, rebooting, etc.  The error keeps coming back.

Thanks

Joe

This is generally from a corrupted file on the flash.  Remove the /flash/config/plugins/unassigned.devices/*.cfg files and reboot.

Link to comment
4 hours ago, dlandon said:

This is generally from a corrupted file on the flash.  Remove the /flash/config/plugins/unassigned.devices/*.cfg files and reboot.

Thank you.  That worked.  I could have sworn that I checked that the directories were deleted after removing the plugin.  

I deleted them through terminal using MC and instead of /flash I found the files through /boot.  I was wondering why in the Krusader docker I see a /flash and /boot but can not get into either, but using terminal and Midnight Comander to navigate, there is no /flash, only a /boot and I can see and delete the files.  

Link to comment

Hello,

 

I'm currently migrating data from my old array disks and encountered some problems with disks spinning up without reason. And I found it's caused by UD reading data from precleared disks.

unraid-ud-preclear-01.jpg.01e0c109af0864566b546b8d8afd3fba.jpg

It happen when UD reloads (probably) its device list when switching to 'Main' tab and on UD plugin configuration page. Preclear disk page (containing device list too) itself doesn't cause this disk read so I suspect it's from some UD request.

 

What's bit misleading is that it looks like there are 2 kinds of 'precleared' status in UD

  • Disks precleared in previous session (before restart) - marked as 'precleared' in FS column (see picture below) - these are affected by disk read and spin up
  • Disks precleared in this session - marked as '-' in FS column (see picture below) - these are unaffected

unraid-ud-preclear-02.jpg.d9c8c7d433284065be66855d320dd203.jpg

 

Link to comment
3 hours ago, bambi73 said:

I'm currently migrating data from my old array disks and encountered some problems with disks spinning up without reason. And I found it's caused by UD reading data from precleared disks.

UD checks for the preclear signature on a disk when the UI is loaded.  That's how it knows to mark the disk as 'Precleared'.  You can stop this by setting the disk as 'Pass Thru'.  You will of course get the no file system indicator.

4 hours ago, bambi73 said:

What's bit misleading is that it looks like there are 2 kinds of 'precleared' status in UD

When a disk is precleared, there is no file system.  This is what '-' indicator means.  If UD finds the preclear signature it marks the file system as 'Preclared'.  I suspect that the sanning to look for the signature times out at times and fails to find the signature, and marks the disk as having no file system.

 

If you'll post your diagnostics zip I can see if that is the case.

Link to comment
On 6/3/2020 at 9:43 AM, Thx And Bye said:

But imo there should be at least a kinda hidden away "advanced settings" that could display a warning if the setting is changed to give the option to unify SMB behavior with the regular unraid array shares.

I've added a configuration option in the next release.

  • Thanks 1
Link to comment
17 hours ago, dlandon said:

UD checks for the preclear signature on a disk when the UI is loaded.  That's how it knows to mark the disk as 'Precleared'.  You can stop this by setting the disk as 'Pass Thru'.  You will of course get the no file system indicator.

Thanks, 'Pass Thru' helped.

I kind of expected UD (or Preclear) caches state because it doesn't change during session (unless you format it) but it doesn't matter too much. 'Pass Thru' workaround is working fine and 'Precleared' disks staying in UD for long time are probably quite corner case :)

17 hours ago, dlandon said:

When a disk is precleared, there is no file system.  This is what '-' indicator means.  If UD finds the preclear signature it marks the file system as 'Preclared'.  I suspect that the sanning to look for the signature times out at times and fails to find the signature, and marks the disk as having no file system.

 

If you'll post your diagnostics zip I can see if that is the case.

Added diagnostics. But these drives didn't cause any problems for me so you don't have to check it. Of course unless you want to improve plugin :)

optimus-diagnostics-20200605-0220.zip

Link to comment

after lots of reading and not understanding why my unraid was acting strange - it is possible that a quirk of UD may be causing my issue.

 

overview of unraid issue:

To protect the data on my drives I enabled encryption.  To enable autostart I inserted into /boot/config/go  the line "cp /boot/config/keyfile /root"

 

This seemed to go well with the drive autostarting on bootup.

Then I bought a giant usb drive to use as an array backup, so I installed unassigned devices and UD plus so that I could also encrypt the usb drive.

 

I do a reboot and all is not well.   The UD luks encrypted usb drive does auto-mount... but my array no longer auto starts its asking me for a keyfile or keyphrase.

 

If I open a console window and go into /root the keyfile is not there.  If I pass in a keyfile from the webGui then it start and the keyfile is in /root

 

does anyone have any idea what is going on? why is the keyfile not getting copied from the go script (or why is it getting deleted after emhttp starts)?

is UD somehow doing something with the encrypted UD that touches /root/keyfile in some way?

 

 

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.