Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

[dlandon]

Anyone having an issue with the smb mounts, please provide me with more information on your setup and a copy of the UD log.  Be sure to let me know what version of unRAID you are using.  Some changes were made to UD to better handle the situation when the remote smb share goes off-line, but there may be more areas where changes need to be made.

 

From the reports in this thread, the issue seems to be with UD hanging when a remote smb share goes off-line.

[Spies]

On 06/09/2017 at 2:57 PM, dlandon said:

Look at Settings->Unassigned Devices.  You can control SMB permissions there.

I set it to "Yes (hidden)" but the share still appears (albeit inaccessible).

[dlandon]

14 minutes ago, Spies said:

I set it to "Yes (hidden)" but the share still appears (albeit inaccessible).

That's by design.

[Spies]

5 minutes ago, dlandon said:

That's by design.

How does that differ to setting just "Yes" then?

[dlandon]

33 minutes ago, Spies said:

How does that differ to setting just "Yes" then?

Yes is visible.  Yes (Hidden) is not visible.  Both have permissions as set in the Settings->Unassigned Devices.

[dlandon]

On 9/3/2017 at 1:08 PM, dlandon said:

I have implemented partitioning xfs and btrfs disks in UD to match the setting of Settings->Disk Settings->Default Partition Format.  The partition will start at sector 63 or 64 based on that setting.  This should allow disks formatted and partitioned with UD to properly go into the array.

 

With the release of 6.4.0-rc8q and the change to be compatible with 4Kn disks, there seem to be more issues with disks formatted with UD and put into the array.  I had to change out my cache disk on rc7 and unRAID would not format it.  I formatted the new cache disk with UD and installed it as my new cache disk.  unRAID accepted the disk and went happily on its way.  When I upgraded to rc8, the cache disk became un-mountable.  I would have thought that the original formatted cache disk starting at sector 2048 would be rejected by unRAID and force me to format it.  It did not and accepted that format.  I'm seeing others that have upgraded that are probably having the same problem.

 

Please consider this feature beta until it can be tested beyond what I could do with my limited supply of disks.  For those of you with a tolerance to possibly having problems, please check it out for me and provide feedback.  I'm not an expert on unRAID disk formatting and partitioning, but I think what I did will accomplish the job.

 

Note: I reluctantly put the time into this feature because of the issues that have come up and I don't see LT folding UD into unRAID in this release.  Hopefully this is all that is needed.

As it turns out this is not the total answer to the format and partition incompatibility.  After some testing, I can't format a disk with UD and have it accepted into the array.  I'm awaiting some help from LT or others knowledgeable on the format and partition layout for unRAID so I can work this out.

[Spies]

But I'm saying that "Yes (hidden)" still shows under \\tower

 

In my mind this should mean that you cannot browse to it under \\tower but typing the absolute path \\tower\duplicati would mean you could still access it?

[dlandon]

19 minutes ago, Spies said:

But I'm saying that "Yes (hidden)" still shows under \\tower

 

In my mind this should mean that you cannot browse to it under \\tower but typing the absolute path \\tower\duplicati would mean you could still access it?

The way it works is:

'Yes' - share shows when 'Network' is clicked in Windows.  The samba parameter 'browseable' is set to 'yes'

'Yes  (hidden)' - share does not show when 'Network' is clicked in Windows.  The samba parameter 'browseable' is set to 'no'.

 

Both are accessible with '\\Tower\slare'.

 

This is how it works for me.

[Spies]

Any way i can diagnose why it isn't working? setting hidden on the main unraid shares works fine.

 

Could it be SMB caching somewhere?

 

Edited September 7, 2017 by Spies

[Squid]

20 minutes ago, Spies said:

Could it be SMB caching somewhere?

Probably.  I just did this, and one share disappeared immediately.  The others took up to 10 minutes to disappear

[dlandon]

22 minutes ago, Spies said:

Any way i can diagnose why it isn't working? setting hidden on the main unraid shares works fine.

 

Could it be SMB caching somewhere?

 

Go to a command line and type:

'nano /etc/samba/unassigned.devices/duplicati' and see what the 'browseable' parameter is set to.  If set to 'no' it should not be browseable.  i.e it should not show in the 'Network'.

 

@Squid is right.  It can take a while for the setting to take effect.

 

I don't see any users listed under the smb settings.  Do you have any users set up for your server?

[dlandon]

I just looked at the code and I see the situation you are seeing.  If you don't have any users defined, the share is always browseable and you'll see in the UD log a message about no valid smb users defined and the share can't be accessed.  I will change this so the 'Yes (hidden)' setting will in fact make the share not browseable.

[Spies]

6 minutes ago, dlandon said:

I just looked at the code and I see the situation you are seeing.  If you don't have any users defined, the share is always browseable and you'll see in the UD log a message about no valid smb users defined and the share can't be accessed.  I will change this so the 'Yes (hidden)' setting will in fact make the share not browseable.

Thanks.

[dlandon]

A new version of UD is available.  The major feature update is the ability to partition a disk in UD compatible with the array partitioning.  I have tested this with a 2TB disk, but I don't have a disk larger than that to test gpt partitions.  If anyone is willing, I would appreciate it if you could format a disk larger than 2TB and let me know if unRAID accepts it as properly partitioned.

 

The other change is to add some timeouts to the 'df' commands on remote mounted smb and nfs shares to try to alleviate the hangs when a remote share goes off-line.

 

The smb 'browseable' parameter is now set properly when no smb users are found.

[dlandon]

1 hour ago, dlandon said:

A new version of UD is available.  The major feature update is the ability to partition a disk in UD compatible with the array partitioning.  I have tested this with a 2TB disk, but I don't have a disk larger than that to test gpt partitions.  If anyone is willing, I would appreciate it if you could format a disk larger than 2TB and let me know if unRAID accepts it as properly partitioned.

 

The other change is to add some timeouts to the 'df' commands on remote mounted smb and nfs shares to try to alleviate the hangs when a remote share goes off-line.

 

The smb 'browseable' parameter is now set properly when no smb users are found.

 

Initial testing of the partition compatibility was done by me on a 2TB disk (msdos), and on a 3TB disk (gpt) by @johnnie.black.  Looks like the format compatibility is working.

 

Be careful though until some more feedback is provided that the partition incompatibility is solved.

[wgstarks]

1 hour ago, dlandon said:

 

Initial testing of the partition compatibility was done by me on a 2TB disk (msdos), and on a 3TB disk (gpt) by @johnnie.black.  Looks like the format compatibility is working.

 

Be careful though until some more feedback is provided that the partition incompatibility is solved.

I've got an 8TB that just finished preclear. I'll try it on that as soon as I get some free time this afternoon.

[wgstarks]

5 hours ago, dlandon said:

 

Initial testing of the partition compatibility was done by me on a 2TB disk (msdos), and on a 3TB disk (gpt) by @johnnie.black.  Looks like the format compatibility is working.

 

Be careful though until some more feedback is provided that the partition incompatibility is solved.

Formatted an 8TB Seagate ST8000DM004 to XFS via UD and SMB 3. Installed the drive internally into my server and added to array as a replacement without having to reformat. Data rebuild is in progress. Looks like no issues with the formatting.

[dlandon]

7 minutes ago, wgstarks said:

Formatted an 8TB Seagate ST8000DM004 to XFS via UD and SMB 3. Installed the drive internally into my server and added to array as a replacement without having to reformat. Data rebuild is in progress. Looks like no issues with the formatting.

Excellent.  Thank you for the feedback.

[trurl]

8 minutes ago, wgstarks said:

Formatted an 8TB Seagate ST8000DM004 to XFS via UD and SMB 3. Installed the drive internally into my server and added to array as a replacement without having to reformat. Data rebuild is in progress. Looks like no issues with the formatting.

Maybe I'm missing something here, but if a disk is used to replace/rebuild another disk in the array then its history, such as whether or not it was formatted in UD, doesn't matter, does it?

[Squid]

2 minutes ago, trurl said:

Maybe I'm missing something here, but if a disk is used to replace/rebuild another disk in the array then its history, such as whether or not it was formatted in UD, doesn't matter, does it?

AFAIK There's issues in rc8 rebuilding onto a drive that doesn't have the proper partitioning already set.

[dlandon]

8 minutes ago, Squid said:

AFAIK There's issues in rc8 rebuilding onto a drive that doesn't have the proper partitioning already set.

UD is now creating the proper partition format for the array and if it was accepted into the array without formatting, it is good to go.

 

What you might be confused about is previously a disk would be accepted in pre rc8 versions that weren't the proper partition format.  LT says that because of the encryption on disks, the format is now more strict and disks not properly partitioned will not be accepted into the array without a format.

[wgstarks]

Just to be clear, I'm still running 6.3.5, so if I'm understanding correctly this didn't fully test the new formatting.

[Squid]

15 minutes ago, dlandon said:

UD is now creating the proper partition format for the array and if it was accepted into the array without formatting, it is good to go.

 

What you might be confused about is previously a disk would be accepted in pre rc8 versions that weren't the proper partition format.  LT says that because of the encryption on disks, the format is now more strict and disks not properly partitioned will not be accepted into the array without a format.

 

[dlandon]

29 minutes ago, trurl said:

Maybe I'm missing something here, but if a disk is used to replace/rebuild another disk in the array then its history, such as whether or not it was formatted in UD, doesn't matter, does it?

Correct.  It doesn't matter.  UD formats exactly like unRAID.

[dlandon]

2 minutes ago, wgstarks said:

Just to be clear, I'm still running 6.3.5, so if I'm understanding correctly this didn't fully test the new formatting.

It did.