Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


Recommended Posts

15 hours ago, dlandon said:

No.

How you are copying the files?  What kind of disk?  SSD or spinner?

Thats what I thought but was trying to explain the slow performance I have seen with UD since day one.

 

I had both SSD and spinning rust on UD.

 

The transfer yesterday was an HDD, I know for a fact that drive should be doing ~120-150mb/s with the files I was copying based on prior copies of the same files when I was using windows. Pretty much the slowest that drive ever goes except with small files is ~100mb/s.

 

There is some kind of bottleneck as watching the performance in netdata will show bandwidth is limited to a fixed number, generally between 40-60mb/s. CPU usage is fine and the drive is only at 10-30% utilization time (aka, it is only at 10-30% of total bandwidth).

 

I know at one point I tested copies with krusader and saw some really strange results.

 

When writing to an NTFS drive my READ speeds would be capped at roughly half the speed it should be but writes would be fine (they would burst at full speed thanks to the ram buffer).

 

When writing to a BTRFS drive reading from an NTFS drive, the speeds would be fine on both ends IIRC.

 

Only thing I could explain is some kind of NTFS driver issue.

Link to comment

I cannot add 2 drives of the same brand because they have the same IDs

When I change the Device Moint Point of the second drive, that (stupid thing) changes the DMP of the first disk

 

I cannot be the 1st one who ran out on this issue

 

vs=2020.07.03a

Link to comment
10 minutes ago, flic said:

I cannot add 2 drives of the same brand because they have the same IDs

Each disk has a unique ID.  How do you have them connected?

10 minutes ago, flic said:

When I change the Device Moint Point of the second drive, that (stupid thing) changes the DMP of the first disk

That (stupid thing) has to have a unique device ID in order to identify each disk.  What do you expect UD to do if the disks don't have unique IDs?

13 minutes ago, flic said:

I cannot be the 1st one who ran out on this issue

This can happen if you have the disks in an external bay.  That can present both disks to UD with the same ID.

Link to comment

both drives are connected via USB 3.0 interface. I know each drive must have unique IDs, but they are put there (I cannot change them)

So, you seem to tell me that your plugin is not compatible with external bays like USB?

 

I wanted to get away from Microsoft Space Storage but it looks like I will have to go back until something better comes along

Edited by flic
Link to comment
9 hours ago, flic said:

So, you seem to tell me that your plugin is not compatible with external bays like USB?

This will be a limitation of the USB external bay you are using.  Many external bays DO present each drive with a unique ID to Unraid and so work fine.

 

Having said that it is worth mentioning that is is not recommended that you have array drives connected via USB as USB connections tend to not be reliable enough for array drives.  USB is typically used for removable drives that are only temporarily connected to an Unraid system (typically for back-up or data transfer purposes)

Link to comment

Remote SMB Share Won't Mount

 

When I add a remote SMB share, the GUI walks me through the steps and lets me search for and select the Samba share on my network by "name" (not IP address).

image.png.7fefe697f181f4aa9422a6087673a721.png

 

I then select the share I want from the pull down and end up with this:

 

image.thumb.png.ed75193fb70fdbec1fea451286320560.png

 

 

However, when I hit the "MOUNT" button, I get the error below. It can't find the SMB share by the name that it gave me to use:

 

Jul  9 15:50:20 Tower unassigned.devices: Mount SMB share '//VIDEO/ContaCam' using SMB3 protocol. 
Jul  9 15:50:20 Tower unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=3.0,credentials='/tmp/unassigned.devices/credentials_ContaCam' '//VIDEO/ContaCam' '/mnt/disks/VIDEO_ContaCam' 
Jul  9 15:50:20 Tower unassigned.devices: SMB3 mount failed: mount error: could not resolve address for VIDEO: Unknown error

 

It won't let me modify it afterwards to set the IP address. 

To make it work, I have to delete the share, add a new SMB share and search by IP address. Except now, the location is this ugly:

 

/mnt/disks/192.168.xxx.xxx_ContaCam

 

Instead of the much nicer:

/mnt/disks/VIDEO_ContaCam

 

By the way, I am running version 6.8.3 of Unraid. I just updated to the latest version of this plugin (2020.07.10) and installed the 'plus' version but the error persists:

image.thumb.png.8f09a303ada6a82314cba90b5e20d533.png

 

This used to work a while back. I don't know what changed other than the version of Unraid.

 

 

 

tower-diagnostics-20200709-1641.zip

Link to comment
21 minutes ago, frakman1 said:

This used to work a while back. I don't know what changed other than the version of Unraid.

There have been changes to Samba that affects the lookup of servers and shares.  NetBios is basically being phased out because of security issues and it is being replaced by Windows Discovery.  Whether you have NetBios and/or Windows DIscocery (WSD) enabled will affect the lookup and mounting of remote servers and shares.  The most reliable way is to use the IP address and then change the mount point to something more meaningful.

 

Change the mount point by clicking on the + symbol and then click on the partition mount point and change it to somethig more meaningful. 

Link to comment

New release of UD.  Changes:

  • When changing the mount point (which is also the share name), the mount point is checked for a duplicate of a user share or another UD device.  Samba cannot handle two shares with the same name.
  • When mounting a UD device, the mount point is checked for a duplicate and if found will not mount the device.  You will need to resolve the duplicate in order to mount the device.
  • Add '--allow-discards' to luks open when an encrypted disk is a SSD so discard and trim will work on the disk.
  • Like 3
  • Thanks 2
Link to comment
13 hours ago, dlandon said:

New release of UD.  Changes:

  • When changing the mount point (which is also the share name), the mount point is checked for a duplicate of a user share or another UD device.  Samba cannot handle two shares with the same name.
  • When mounting a UD device, the mount point is checked for a duplicate and if found will not mount the device.  You will need to resolve the duplicate in order to mount the device.
  • Add '--allow-discards' to luks open when an encrypted disk is a SSD so discard and trim will work on the disk.

Dude you're a beast. Thank you for your dedication.

Link to comment

Story time.... 🤣  I think I have a lot of UDs. 16 drives to be exact managed by UD. Pretty cool!

 

Quite the growth since 2011 when I first started with UnRAID. I have all 28 data disks in UnRAID utilized. So now with another 16 disks managed by UD, it'd given me some experiences to consider how UD works with this managing this many devices and where do I want to go next.

 

The recent enhancement of 'None' or 'All' to show the mount points has been awesome. Really improved the usability of UD with this many devices. I can easily find mount point names.

 

As I keep adding devices, managing mount point names is getting more of a chore. Thankfully UD has a great feature that it uses the disk serial number ID by default as a mount point. This works great with MergerFS, as I can keep this default, and just add the parent directories I wish on the drive and MergerFS takes care of the rest. Super easy! 😀

 

Two enhancements I think about sometimes based on this large number of UD devices:

 

  1. 1With this many devices in UD, it can take about 10 or more seconds to load the Main UnRAID page with the UD GUI. It has me wondering if there is a way to speed this up. Like if disabling temperature (hdparm) or some other info UD queries to improve UD loading speed.
     
  2. I find of all the switched on the UD GUI I use the 'Auto-Mount' switch the most. Each time it triggers the 'refresh animation' and has me wondering if there is a way to make it work similar to UnRAID Docker's 'autostart'. Those switches don't trigger a GUI refresh when switched, so I'm wondering if a similar method could be leveraged by UD.

 

I really love UD has enabled all this to be possible via the GUI. While I could do it via the console, it wouldn't be as much fun. Really appreciate all the work you've put into UD over the years @dlandon.

 

Link to comment
1With this many devices in UD, it can take about 10 or more seconds to load the Main UnRAID page with the UD GUI. It has me wondering if there is a way to speed this up. Like if disabling temperature (hdparm) or some other info UD queries to improve UD loading speed.

Is this delay loading the main page or for the UD animation? I have 35 UDs on one of my servers and the main page loads instantly, UD animation takes a couple of seconds before it displays the devices.

 

I find of all the switched on the UD GUI I use the 'Auto-Mount' switch the most. Each time it triggers the 'refresh animation' and has me wondering if there is a way to make it work similar to UnRAID Docker's 'autostart'. Those switches don't trigger a GUI refresh when switched, so I'm wondering if a similar method could be leveraged by UD.

I would also like to see this, the less the refresh animation shows the better, if possible of course.

 

My main request for current UD would be to update the disks used/free space in real time, like the GUI does, currently you need to refresh the GUI (and wait for the animation), again if possible to do.

 

 

 

 

Link to comment
4 hours ago, Lev said:

1With this many devices in UD, it can take about 10 or more seconds to load the Main UnRAID page with the UD GUI. It has me wondering if there is a way to speed this up. Like if disabling temperature (hdparm) or some other info UD queries to improve UD loading speed.

Do you have any remote smb/nfs mounts?  If so, switch the 'Complete' switch to 'Simple' and see if that improves the load time.  Also check your log for any UD log entries about commands timing out.

 

4 hours ago, Lev said:

I find of all the switched on the UD GUI I use the 'Auto-Mount' switch the most. Each time it triggers the 'refresh animation' and has me wondering if there is a way to make it work similar to UnRAID Docker's 'autostart'. Those switches don't trigger a GUI refresh when switched, so I'm wondering if a similar method could be leveraged by UD.

Fixed in the next release.

Link to comment
2 hours ago, qba said:

Hi there need some help get error after removing a external usb drive, and creating a new config.

 

Warning: file_get_contents(/sys/block/serie/queue/rotational): failed to open stream: No such file or directory in /usr/local/emhttp/plugins/unassigned.devices/include

 

I still see the drive in:

root@Tower:/mnt/disks# ls
/bin/ls: cannot access 'ST2000DM001-1CH164_Z1E6HZ0H': Input/output error

192.168.2.250_Serie/  192.168.2.81_serie/  ST2000DM001-1CH164_Z1E6HZ0H/

 

How can i Fix this?

 

It is an issue with UD not handling this situation very well, but it also looks to be something with your system.  Please post diagnostics.

Link to comment

There is a change in SSD device partition alignment.  The alignment on SSD devices partitioned with UD is now 1MiB.  What this means is SSD devices formated with UD will not be compatible as either a cache or pool device on any version earlier that 6.9.  If you format a SSD with UD, it will be compatible with 6.9 as a cache or pool device, but you cannot roll back to an earlier version of Unraid without Unraid wanting to format the device.

 

This is also a limitation wth Unraid.  Any SSD device partitioned with 6.9 Beta 25 will not be compatible with erlier versions of Unraid.

 

Any SSD partitioned with UD as a UD disk only will still mount and work properly, although eventually it would be best to re-format the SSD to the new partition alignment for performance reasons.

  • Thanks 1
Link to comment
5 hours ago, dlandon said:

Do you have any remote smb/nfs mounts?

No smb / nfs mounts are in use or created. I think I also keep the disks spun up to see if that helps. Sometimes it's a longer wait for the refresh than other times. I suspect this is when there is activity on one or more of the drives and UD is having to wait in the queue to get the info from the disks that it needs for the refresh.

 

5 hours ago, dlandon said:

Fixed in the next release.

You rock, awesome.

Link to comment
50 minutes ago, dlandon said:

There is a change in SSD device partition alignment.  The alignment on SSD devices partitioned with UD is now 1MiB.  What this means is SSD devices formated with UD will not be compatible as either a cache or pool device on any version earlier that 6.9.  If you format a SSD with UD, it will be compatible with 6.9 as a cache or pool device, but you cannot roll back to an earlier version of Unraid without Unraid wanting to format the device.

 

This is also a limitation wth Unraid.  Any SSD device partitioned with 6.9 Beta 25 will not be compatible with erlier versions of Unraid.

 

Any SSD partitioned with UD as a UD disk only will still mount and work properly, although eventually it would be best to re-format the SSD to the new partition alignment for performance reasons.

does this also affect nvme m2 devices ? or ssd only ?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.