Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array


dlandon

Recommended Posts

I'm having issues passing through a 4tb hdd using UD to a VM. I've tried formatting the drive as NTFS using UD and Windows without luck. I'm only passing through 1 partition, but when I do, the VM detects multiple partitions. I've tried GPT, MBR and dynamic to no luck. I've also tried formatting in Windows then changing the 2nd vDisk location to partition 1 after.

 

The reason I need to pass through the partition only is because if I pass through the entire drive my dockers won't see new files. 

 

When I pass through the entire drive it works just fine.

 

image.thumb.png.0a504c79ebe73468ef51533e2fc2bdbf.png

 

I can click on Storage and navigate the directories using the web

image.png.71e55ab159e6f0b8c6a4b0f64138f675.png

 

 

/dev/disk/by-id/ata-ST4000DM004-2CV104_ZFN0FS4D-part1

image.png.2a8145fe93d2456e243f6303436bfab9.png

 

This is what it looks like in the VM

image.png.29caabe3fed0482a99b9ddf52bab810d.png

Edited by iPenguin02
Link to comment
On 10/4/2020 at 7:11 AM, dlandon said:

@Danno93093 supplied me with the diagnostics I needed to find and resolve the issue.  This is why I ask for the diagnostics.  I found an extra space character from some disks information in the df command that was causing a parsing problem.

 

Problem fixed in the latest release.

Can confirm -- it is now showing correctly on my server:
338510492_Image2020-10-06at11_24_29PM.thumb.png.01f005e8a67590ef11ddd964432dab84.png

Thanks @dlandon!

Link to comment

I have a problem.  Last night, PLEX was updated.  The Docker container wasn't updated till today.  I installed it just fine.  PLEX runs on an nvme designated as an Unassigned Device.  After I updated the Docker this morning, I found that Unraid had also updated to 6.9.0beta30.  After I installed the update and rebooted, I no longer have the Unassigned Devices section in my "Main" tab of Unraid.  However, if I go to the "Dashboard" tab, it does display that section along with the drives.  But, it doesn't indicate my partitions on those drives.  So in a sense, it's not really working.  PLEX is unable to start because of it's location on one of the Unassigned Devices partitions.  Any ideas?

 

====== UPDATE ======

 

The problem is not related to the version of Unraid.  I just down graded back to 6.9.0beta29 and the problem is still there.  I think it's the Plugin.  Can I reinstall it without losing the information about my partitions?

 

====== UPDATE ======

 

It worked.  I took the leap of faith and uninstalled/reinstalled it.  Seems to be working now.  Must have been a settings corruption somewhere.

Edited by hansolo77
Link to comment
On 10/5/2020 at 6:48 PM, trurl said:

For future reference, please assume we want the complete zip. It has many things in it that can help make much more sense of things than just the syslog. I often don't even look at the syslog until I have looked at other things in diagnostics just to get some context of what I might be looking for in those hundreds of lines of syslog. And many things syslog doesn't even tell us that are in other parts of the diagnostics.

 

Save everyone time if we don't even have to ask for it. For example, it was 4 hours after your initial question before we finally got the diagnostics.

 

I apologize and certainly see what you're saying. I will be proactive in the future with the diagnostics files. I really appreciate the help and support on the forums in aggregate. This is the the first time I think I have ever posted an issue. 99% of the time I can solve any issue in looking through these forums and spaceinvaderone videos...

 

On 10/5/2020 at 7:21 PM, dlandon said:

I'm not an expert with disk controllers, but it appears your disk controller caused Linux to have trouble assigning drive designations for Unraid and Unraid assigned the wrong designations to the array disks.  UD checks the vars for the disks not assigned to the array and designates any that are not as unassigned disks.  In this case all disks look to UD to be unassigned.

This isn't really a UD issue.  UD is doing exactly what it is supposed to do.

 

I would report this in the 6.8.3 support forum.

 

On 10/6/2020 at 12:14 AM, JorgeB said:

Unraid doesn't support SAS multipath. 

 

Ah... I think I might know the issue then. I was having some other issues a while back and in the process of troubleshooting I replaced the cable between my server and the SAS enclosure. When I worked out that it wasn't the cable, or the controller modules in the drive enclosure (an MD1000), I left both cables attached. I assume that this could be the cause. I'll try unplugging one and restarting and see what comes up. I have new processors to install as well, so I might install those while its down this weekend... Of course I guess the alternative culprit could be the HBA card, which is also due to be replaced soon, so hopefully one of the two fixes things.

 

Thank you guys for the help. I really can't convey how appreciated it is. This place is a gold mine of information.

Link to comment
On 10/6/2020 at 5:23 PM, dlandon said:

This has come up at times.  I am pondering a solution.

I haven't had time to test the new update, but I really appreciate the time you've put into this plugin. I'll know for certain the next time my server boots, but I'm fairly certain that the check for network connectivity before mounting remote shares will resolve my issue.

Edit: Tested and working as expected! Thank you @dlandon

Edited by ryanhaver
Updated status
Link to comment

Hopefully this is the right thread for this:

 

Potential bug with Historical Device reporting SN#s:

 

Synopsis:

Replaced 2 4TB drives for 1 16TB drive

Now under Historical Devices it is reporting the 16TB drive is missing

 

Steps to reproduce:

 

In my case:

Disk 7 had a 4TB SN# V3F9

Unassigned Device 4TB SN# F8R8

 

Powered system down and swapped out F8R8 for 16TB SN# TRZP

Formatted SN# TRZP

Once completed swapped out SN# V3F9 (Disk 7 slot) with SN# TRZP which moved V3F9 down to unassigned

At some point it is now reporting SN# TRZP is missing even thou it is sitting in DISK 7 Slot. I wasn't paying attention to the various menus during this process.

 

Thoughts:

What I think is happening is when I swapped out F8R8 with TRZP during the power down it thinks TRZP was F8R8

Wrong info.png

Link to comment
1 hour ago, Draffutt said:

Disk 7 had a 4TB SN# V3F9

Unassigned Device 4TB SN# F8R8

 

Powered system down and swapped out F8R8 for 16TB SN# TRZP

Formatted SN# TRZP

Once completed swapped out SN# V3F9 (Disk 7 slot)

Are you saying you formatted a disk with Unassigned Devices, then used that disk to rebuild disk7 in the array? Why? Rebuild completely overwrites the disk with the formatted filesystem of the original disk with all its files, so formatting it before using it as a replacement is completely pointless.

 

I always worry when someone seems to misunderstand "format" because a lot of people have mistakenly thought they could format a disk in the array then restore its data from parity.

Link to comment

I am still on Unraid 6.8.3

 

I just added a new unassigned disk - precleared no problem (first time I used JoeL's script, normally I use gfjardim when preclearing).  Now I went to delete the partition so it'll format the disk but after typing Yes I'm getting a failed operation and its not doing it.  Can I use the CLI to force a format to XFS to move it to the next step to be an unassigned device?

 

I wonder if its an issue w/ UD and if I should uninstall and reinstall?  I"ve tried rebooting a few times still no luck.  If I uninstall and reinstall it won't delete the stuff from the other UD Drive will it?  

Edited by cbr600ds2
added new thought
Link to comment

First off, awesome plugin! Been using it for close to a year now without issue. Really, really useful.

 

Currently I'm using it for an NVME drive to run my VMs (and a couple of dockers) off of. Everything was fine until I changed my array (and UD) encryption to a keyfile, now the array starts on reboot but the UD NVMe drive will no longer auto mount (it's unlocked successfully via the keyfile but just isn't mounted).

 

Has anyone come accross this before and/or have a fix?

Link to comment
5 hours ago, cferra said:

I am not sure if this was asked, but would it be possible to integrate mounting remote SSH shares into UD?  That would go along way to make things easier for those that are using sonarr and radarr and I think would be really helpful.

 

Thanks!

No plans for that at this time.

Link to comment
29 minutes ago, DaveDoesStuff said:

First off, awesome plugin! Been using it for close to a year now without issue. Really, really useful.

 

Currently I'm using it for an NVME drive to run my VMs (and a couple of dockers) off of. Everything was fine until I changed my array (and UD) encryption to a keyfile, now the array starts on reboot but the UD NVMe drive will no longer auto mount (it's unlocked successfully via the keyfile but just isn't mounted).

 

Has anyone come accross this before and/or have a fix?

You'll need to post your diagnostics for us to help you.

Link to comment
2 hours ago, DaveDoesStuff said:

Of course, thank you (didn't want to bombard you with diagnostics straight away incase it was a common issue my searches had just failed to find).

 

Attached.

ibstorage-diagnostics-20201019-0854.zip 153.72 kB · 0 downloads

Here is the problem.

Oct 17 14:16:07 iBstorage unassigned.devices: Adding disk '/dev/mapper/UD_SSD'...
Oct 17 14:16:10 iBstorage kernel: BTRFS: device fsid e3a85b22-09ae-4d31-a525-5782f992e1b0 devid 1 transid 5640954 /dev/dm-11
Oct 17 14:16:10 iBstorage unassigned.devices: luksOpen error: WARNING: Locking directory /run/cryptsetup is missing! 
Oct 17 14:16:10 iBstorage unassigned.devices: Partition 'TOSHIBA_THNSNK128GVN8_M.2_2280_128GB_871B53DKKLNU' could not be mounted.

I have no idea why this happened.  Post this on the 6.8.3 support and see if Limetech has a solution.

Link to comment

To start with positiv things, your plugin is one of my favorites. I use it as long as I'm running my unraid server. But unfortunately I have an issue and I don't know what happened. Since the last update of the plugin, I can't mount my smb share from one of my unraid linux vms. 

The smb share is mountable with my windows machines wihtout a problem.

 

Here ist the log output with the errors:
 

Oct 19 16:45:46 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB3 protocol.
Oct 19 16:45:46 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=3.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:45:56 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:45:56 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=3.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:45:56 HomeServer unassigned.devices: SMB3 mount failed: command timed out.
Oct 19 16:45:56 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB2 protocol.
Oct 19 16:45:56 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=2.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:46:06 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:46:06 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=2.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:46:06 HomeServer unassigned.devices: SMB2 mount failed: command timed out.
Oct 19 16:46:06 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB1 protocol.
Oct 19 16:46:06 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=1.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:46:16 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:46:16 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=1.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:46:16 HomeServer unassigned.devices: SMB1 mount failed: command timed out.
Oct 19 16:46:16 HomeServer unassigned.devices: Mount of '//192.168.178.205/Down-01' failed. Error message: 'command timed out'.

 

Hope you can help me to sort this out.

Link to comment
11 hours ago, dlandon said:

Here is the problem.


Oct 17 14:16:07 iBstorage unassigned.devices: Adding disk '/dev/mapper/UD_SSD'...
Oct 17 14:16:10 iBstorage kernel: BTRFS: device fsid e3a85b22-09ae-4d31-a525-5782f992e1b0 devid 1 transid 5640954 /dev/dm-11
Oct 17 14:16:10 iBstorage unassigned.devices: luksOpen error: WARNING: Locking directory /run/cryptsetup is missing! 
Oct 17 14:16:10 iBstorage unassigned.devices: Partition 'TOSHIBA_THNSNK128GVN8_M.2_2280_128GB_871B53DKKLNU' could not be mounted.

I have no idea why this happened.  Post this on the 6.8.3 support and see if Limetech has a solution.

When cryptsetup opens a LUKS2 (newer format) header, it needs this /run/cryptsetup directory to place some locks for atomic operations. If this directory is missing, it spits out that warning message, creates the directory and continues. In the case above, it completed successfully.

 

The UD code seems to assume that if cryptsetup luksOpen generated any output on stdout/stderr, it means it had an error - and assumes failure (.../include/lib.php). Therefore this warning message from cryptsetup causes it to give up.

 

@DaveDoesStuff may have recently formatted this particular drive, so that the header format is LUKS2. Hence, this chain of events.

 

You may want to consider changing UD to look at cryptsetup's exit code, rather than stdout content, for a reliable indication of success/failure.

Link to comment
19 hours ago, kimdu said:

To start with positiv things, your plugin is one of my favorites. I use it as long as I'm running my unraid server. But unfortunately I have an issue and I don't know what happened. Since the last update of the plugin, I can't mount my smb share from one of my unraid linux vms. 

The smb share is mountable with my windows machines wihtout a problem.

 

Here ist the log output with the errors:
 


Oct 19 16:45:46 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB3 protocol.
Oct 19 16:45:46 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=3.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:45:56 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:45:56 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=3.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:45:56 HomeServer unassigned.devices: SMB3 mount failed: command timed out.
Oct 19 16:45:56 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB2 protocol.
Oct 19 16:45:56 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=2.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:46:06 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:46:06 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=2.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:46:06 HomeServer unassigned.devices: SMB2 mount failed: command timed out.
Oct 19 16:46:06 HomeServer unassigned.devices: Mount SMB share '//192.168.178.205/Down-01' using SMB1 protocol.
Oct 19 16:46:06 HomeServer unassigned.devices: Mount SMB command: /sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=1.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01'
Oct 19 16:46:16 HomeServer kernel: CIFS VFS: cifs_mount failed w/return code = -512
Oct 19 16:46:16 HomeServer unassigned.devices: Error: shell_exec(/sbin/mount -t cifs -o rw,nounix,iocharset=utf8,file_mode=0777,dir_mode=0777,uid=99,gid=100,vers=1.0,credentials='/tmp/unassigned.devices/credentials_Down-01' '//192.168.178.205/Down-01' '/mnt/disks/192.168.178.205_Down-01' 2>&1) took longer than 10s!
Oct 19 16:46:16 HomeServer unassigned.devices: SMB1 mount failed: command timed out.
Oct 19 16:46:16 HomeServer unassigned.devices: Mount of '//192.168.178.205/Down-01' failed. Error message: 'command timed out'.

 

Hope you can help me to sort this out.

It looks like the remote server is not available to Unraid.

Link to comment

It definitly is, unraid is able to load the share I want to connect to. I chossed the share from the drop down menue.

Both machines are on the same network. And as mentioned, other windows pcs on the same network are able to connect to this smb share without a problem. 

Are there any tests I can do, to get more information from unraid?

Link to comment
3 hours ago, kimdu said:

It definitly is, unraid is able to load the share I want to connect to. I chossed the share from the drop down menue.

Both machines are on the same network. And as mentioned, other windows pcs on the same network are able to connect to this smb share without a problem. 

Are there any tests I can do, to get more information from unraid?

Delete and recreate the remote shares.

  • Thanks 1
Link to comment

Hi, I discovered the script (at the opening post) to make a copy of several shares to an external hdd. In my case I would like to copy several shares (actually all of them on only 1 disk [disk1] ) to an external hdd mounted using UD in /mnt/disks/UD_hdd2.

In my scenario what should I change in the script to rsync "disk1" with the external drive?

Apart from the latter anything else I should take into account?

Rgds

Link to comment
14 minutes ago, luca2 said:

In my scenario what should I change in the script to rsync "disk1" with the external drive?

Are the shares you want to backup from disk1 stored entirely on disk1?

 

I modified the script to backup several shares, but, not by disk as the shares are spread across several disks.

 

If you really only want to copy all of disk1, change the rsync line to copy from /mnt/disk1

 

Edited by Hoopster
Link to comment

How do you mount an SMB share that contains a $ character (a hidden share running on Windows)?  When I try to add the share using unassigned devices it fails with the following error in the log: 

 

unassigned.devices: Share '//blah/blah$' contains a '$' character.#011It cannot be added.

 

I tried manually adding it to samba_mount.cfg, but it still doesn't work there either.

 

Adding some keywords because this issue is hard to search for: dollar sign

 

 

EDIT:  I found an old chain about this issue from 2016 where @dlandon incorrectly believed that 'mount -t cifs' does not support hidden windows shares (shares ending with $).  I can fill in some missing details  - the issue you were experiencing is that you were trying to connect to the default admin shares on Windows (C$, D$, admin$, etc) - in order to mount those you either need to disable UAC on the Windows side or change a registry value (add DWORD LocalAccountTokenFilterPolicy=1 to HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Policies\System).  My point is that hidden shares are certainly supported by the cifs mount command, and in my case I'm not trying to mount one of these 'admin' shares - I'm just trying to mount a regular hidden share which are not affected by that policy.  It would be great if you can add support for this.

 

 

EDIT2: @dlandon - I did some testing with the unassigned devices plugin code and I think the fix is as simple as removing the '$' check and changing line 1195 in include/lib.php from

$samba_mounts = @parse_ini_file($config_file, true);

to

$samba_mounts = @parse_ini_file($config_file, true, INI_SCANNER_RAW);

It would be great if you can test/implement this minor change. Thanks!

Edited by ksarnelli
possible solution
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.