Jump to content
dlandon

Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

3120 posts in this topic Last Reply

Recommended Posts

19 hours ago, awilderaid said:

Has anyone been able to figure this out? I've tried a different NIC, a different cable, and moved the PCI slot the NICs are in on both servers and made no progress. Is there a way to monitor Samba or spit out some logs for it to see if that's maybe crashing and causing the issue? 

Go to a command line and type this command and post the output:

/usr/bin/timeout 1 /bin/df

 

Share this post


Link to post
Posted (edited)
8 hours ago, dlandon said:

I have spent a lot of time on the UD plugin to keep it from appearing to 'hang'.  The issue comes from some commands UD uses to get the mounted devices information.  Remote mounted devices using CIFS mounts are particularly messy because of the 'df' command used to get remote mounted devices information and the way servers going off-line are handled.  I've set timeouts on all commands that I can to keep UD from hanging, but there may be more that needs to be done.  I can't reproduce the problem with the setup I have, so it is very difficult to fix the issues you are seeing because it is probably related to your particular setup.

 

You can't unmount a remote mounted share when it goes off-line because CIFS reports the mount as busy.  Unless UD does a forced unmount, the device won't unmount.  It is not a good idea for UD to do a forced unmount, because the device could actually be busy.

 

My suggestion is to unmount any remote mounted shares before you take the remote server off-line.

 

Also, any of you having any issues with UD, please remove the preclear plugin and reboot your server.  The preclear plugin is partially integrated with UD (that's why you can see preclear information in UD), and there is a background preclear task that collects device information that may have some issues.  I don't have the time or desire to troubleshoot preclear.  There has been a lot of discussion about the need to preclear disks.  It really isn't necessary and the preclearing has been a source of problems.  I know all you preclear fanatics will want to start a debate about it, but for troubleshooting these issues, please get it out of the way.

 

Post your diagnostics from normal operation.

I'll see what happens using NFS later today for comparison.
NFS publicly mounted share going offline resulted in both servers responding without throwing errors or locking up. when the NFS shares returned online, they also continued on like nothing had happened.

 

let me know if there is any other info you need or methods to try.

 

much appreciated!

twins-diagnostics-20190629-1128.zip

 

Edited by 1812

Share this post


Link to post
3 hours ago, dlandon said:

You need to post this as an feature request.  The UD plugin doesn't have any control over the dashboard page.

Thanks man.

I just thought that this place on the dashboard was controlled by your plugin.

image.png.d6c81361d9fc0c0fd3d9c1e8b30fa69a.png

Share this post


Link to post
On 6/26/2019 at 7:43 PM, Marshalleq said:

you could just copy to the network share within unraid.  

I simply ended up running the command:

rsync -arPv --no-perms /mnt/user/Avid\ 101/ /mnt/disks/SPJ

Files seemed to copy over with a few of those permission errors/noticies early on, but eveything seemed to transfer over just fine. I will find out later when I get back to working with the files.

 

Thanks for the help!

Share this post


Link to post
2 hours ago, Zer0Nin3r said:

I simply ended up running the command:


rsync -arPv --no-perms /mnt/user/Avid\ 101/ /mnt/disks/SPJ

Files seemed to copy over with a few of those permission errors/noticies early on, but eveything seemed to transfer over just fine. I will find out later when I get back to working with the files.

 

Thanks for the help!

Awesome, I have to admit, I do love how flexible rsync is - definitely check those errors though, there shouldn't really be any.  Good luck!

Share this post


Link to post
13 hours ago, 1812 said:

I'll see what happens using NFS later today for comparison.
NFS publicly mounted share going offline resulted in both servers responding without throwing errors or locking up. when the NFS shares returned online, they also continued on like nothing had happened.

 

let me know if there is any other info you need or methods to try.

 

much appreciated!

twins-diagnostics-20190629-1128.zip 87.52 kB · 0 downloads

 

That’s how it should work.  SMB should work the same.

Share this post


Link to post
25 minutes ago, dlandon said:

That’s how it should work.  SMB should work the same.

the only difference in SMB config between the 2 Unraid servers is that "server B" has "max protocol = smb2" added to it. 

Share this post


Link to post
Posted (edited)
On 6/29/2019 at 11:28 PM, Marshalleq said:

I have to admit, I do love how flexible rsync is

I'm new to it, but from what I've been reading here in the forums, it seems way better than Apple Finder's "merge" function. I just wish exFAT played better with Unix & Linux.

 

Happy Sunday fun day and have a great 4th of July (for those that celebrate!)

Edited by Zer0Nin3r

Share this post


Link to post

Well, one would think that Unraid's / linux's mounting of the exfat system would be enough to tell the OS to ignore file permissions.  However when you're running a tool that is explicitly trying to change permissions on a target file system that simply does not support ANY permissions, it's hard to blame Unraid.  I'm not sure, but it's possible there's an extra mount flag that isn't being run that would get around this issue - which would have to be a feature / bug request with unraid.  Perhaps you could try that.

Share this post


Link to post

As long as the data transfers in both directions to the exFAT drive and the errors are benign — I can live with that.

Share this post


Link to post

Essential plugin that's given me years of hassle-free use...

 

I have an external (USB) Segate drive that I use with Duplicati for backups. It is mounted via Unassigned Devices and spins up when a backup is executed and down again after. All is well.

 

I have a new WD MyBook (8tb USB) drive I want to use in the same way. However, it is being temperamental in spinning down when not required. By default the WD has its own power management and APM is set to off. If I use hdparm to set the timeout and enable APM it works for a bit, then resets itself (seemingly randomly after an hour or so, I know it would on server reboot). If I use the built in WD power management it spins down as instructed, but up again at random times. The drive is not being accessed as I only use it for backups at 4am. If I go to the web UI  "main" page at any time I always get a pause on the Unassigned Devices section while the drive spins up. The Segate drive behaves perfectly and doesn't do this.


Anyway, I'm not expecting a magical solution to this, but with a bit of help I think I have a workaround.

 

If I unmount the device it seems to stay spun-down.

 

I've searched but cannot find a way to mount (and unmount) via the command line. I'm sure it must be possible. I could then add a pre (and post) Duplicati script to mount before a backup and unmount after, which should sort things.

 

Any help with the command I could use to mount (a previously configured drive in Unassigned Devices) would be really helpful. Thanks.

Share this post


Link to post
2 hours ago, page3 said:

Essential plugin that's given me years of hassle-free use...

 

I have an external (USB) Segate drive that I use with Duplicati for backups. It is mounted via Unassigned Devices and spins up when a backup is executed and down again after. All is well.

 

I have a new WD MyBook (8tb USB) drive I want to use in the same way. However, it is being temperamental in spinning down when not required. By default the WD has its own power management and APM is set to off. If I use hdparm to set the timeout and enable APM it works for a bit, then resets itself (seemingly randomly after an hour or so, I know it would on server reboot). If I use the built in WD power management it spins down as instructed, but up again at random times. The drive is not being accessed as I only use it for backups at 4am. If I go to the web UI  "main" page at any time I always get a pause on the Unassigned Devices section while the drive spins up. The Segate drive behaves perfectly and doesn't do this.


Anyway, I'm not expecting a magical solution to this, but with a bit of help I think I have a workaround.

 

If I unmount the device it seems to stay spun-down.

 

I've searched but cannot find a way to mount (and unmount) via the command line. I'm sure it must be possible. I could then add a pre (and post) Duplicati script to mount before a backup and unmount after, which should sort things.

 

Any help with the command I could use to mount (a previously configured drive in Unassigned Devices) would be really helpful. Thanks.

Click on the help button while on the UD page and you will find the answer.

Share this post


Link to post
8 hours ago, dlandon said:

Click on the help button while on the UD page and you will find the answer.

Ah, thanks!

Share this post


Link to post
Posted (edited)

Hello all,

I apologize up front if this has been asked already. I searched for it and found nothing.

 

Is there a possibility AFP will be a supported as a mount protcol for unassigned devices?

Not that its a huge issue but my OCD is killing me when I see this.....

 

Thanks for the great work on this!!!

 

 

Screen Shot 2019-07-07 at 11.48.47 AM.png

 

 

EDIT:

After posting I did just realize I could mount everything via SMB. I just hate how SMB operates on MAC. Never seems stable to me.

Edited by biggeek

Share this post


Link to post

Here is a strange one.  I added a 4T drive to my setup, used UD to format it as XFS, gave it a custom mount point, mounted it, and turned on auto mount.  After I started using it I noticed that the OS was only seeing it as a 2T drive.  I checked the partition table and there was just one partition and it was 4T.  UD was showing it was 2T in the GUI.  After I rebooted, it did not auto mount and the mount button was greyed out just for that drive(destructive mode is on) 

 

I tried manually mounting it:

 

mount -t xfs /dev/sdd1 /mnt/disks/tmp/

mount: /mnt/disks/tmp: wrong fs type, bad option, bad superblock on /dev/sdd1, missing codepage or helper program, or other error.

 

I removed the lines about it in the .cfg file and rebooted,  same issue.

 

Tried running an xfs_repair, but no superblock was found.

 

Any ideas would be much appreciated.

Share this post


Link to post
Posted (edited)

What disk controller is it attached to?    Some older controllers were limited to 2.2GB

Edited by itimpi

Share this post


Link to post

This disk is connected directly to a SATA port on the motherboard.  The motherboard is a current generation X11 Supermicro

Share this post


Link to post

Just tried it on an identical 4T drive also hooked up directly to the motherboard and this time it did mount as a 4T mount

Share this post


Link to post
1 hour ago, biggeek said:

Hello all,

I apologize up front if this has been asked already. I searched for it and found nothing.

 

Is there a possibility AFP will be a supported as a mount protcol for unassigned devices?

Not that its a huge issue but my OCD is killing me when I see this.....

 

Thanks for the great work on this!!!

 

 

Screen Shot 2019-07-07 at 11.48.47 AM.png

 

 

EDIT:

After posting I did just realize I could mount everything via SMB. I just hate how SMB operates on MAC. Never seems stable to me.

I wouldn’t expect too much regarding AFP. Apple has deprecated it in favor of APFS and LT has indicated they will likely do the same.

 

SMB with Enhanced osX enabled has been working nicely for me in unRAID 6.7+.

Share this post


Link to post
4 hours ago, wgstarks said:

I wouldn’t expect too much regarding AFP. Apple has deprecated it in favor of APFS and LT has indicated they will likely do the same.

 

SMB with Enhanced osX enabled has been working nicely for me in unRAID 6.7+.

 

Thanks for that info. I haven't kept up much on AFPS but I'll need to look into it. SO far SMB is working ok with enhanced osX enabled.  I'll see how well it continues to operate.

 

Share this post


Link to post

APFS is a containerised file system.  I think you mean SMB.  Apple still operates AFP, however it is clear it is moving to SMB.  It still works and I imagine it will for some time, but new features and development will likely be across SMB.  Yes, that's the Windows networking protocol being used by apple. :D

Share this post


Link to post

Hi all, I want to report a problem I have with unraid 6.7.2 and the latest version of the plugin available as of today.

I have 3 disks in unassigned devices and I'd want to be able to local mount 2 of them to be able to share them in my vm.

Both disks are western digital, one of 2TB (sdc), the second one of 6TB (sde), both are ntfs.

I have no problem with the 2TB disk, I can mount and share.

In unraid webGui the mount button is greyed out for the 6TB disk, so I cannot mount it through the plugin.

What I did:

Through firefox "inspect element" I tried to force the button to be active, by deleting in the html code disabled="": the button is now enabled, by clicking the mount button the disk is not mounted (obviously), but i have one more row in the log file: no filesystem detected.

In fact, in the fs column in the webGUI there's no ntfs for this disk.

The disk seems to be ok, it can be accessible in windows 10, no errors when checked.

Strange thing is that it is mounted through unraid if i mount the disk in the terminal with this command (mount the disk after creating the folder in /mnt/disks):

mkdir /mnt/disks/6TB
/sbin/mount -t ntfs -o auto,async,noatime,nodiratime,nodev,nosuid,umask=000 '/dev/sde1' '/mnt/disks/6TB'

The disk is mounted and through the webGUI I can share it (however the fs column for this disk remains empty).

I can also unmount the disk through the webGUI by clicking on the unmount button.

 

I cannot track why the filesystem of this disk is not recognised by the plugin.

 

Any idea?

 

Thanks

Share this post


Link to post
On 6/29/2019 at 8:42 PM, dlandon said:

That’s how it should work.  SMB should work the same.

bump because this happens with remotely mounted smb shares off local network as well locking up my server. 

Share this post


Link to post
12 hours ago, 1812 said:

bump because this happens with remotely mounted smb shares off local network as well locking up my server. 

Post diagnostics.

Share this post


Link to post
14 minutes ago, dlandon said:

Post diagnostics.

 

As I mentioned before, the system does not respond to diagnostic requests when this occurs via SMB. I had posted diagnostics from NFS mounts being unexpectedly removed, but the expected behavior occurred and the system responded normally without issue. But in this instance, an SMB share went offline, limiting system functions ( as shown on the previous page) To regain full functionality the server has to be rebooted. After initiating shutdown, and monitoring the syslog via web GUI, this is where it hangs :

 

Jul 13 07:09:14 Brahms1 avahi-daemon[6409]: Withdrawing address record for fe80::10a6:baff:fea2:ea5d on zt0.
Jul 13 07:09:14 Brahms1 avahi-daemon[6409]: Withdrawing address record for 192.168.2.253 on zt0.
Jul 13 07:09:47 Brahms1 root: Status of all loop devices
Jul 13 07:09:47 Brahms1 root: /dev/loop1: [2049]:4 (/boot/bzfirmware)
Jul 13 07:09:47 Brahms1 root: /dev/loop2: [0032]:53108 (/mnt/cache/system/libvirt/libvirt.img)
Jul 13 07:09:47 Brahms1 root: /dev/loop0: [2049]:3 (/boot/bzmodules)
Jul 13 07:09:47 Brahms1 root: /dev/loop3: [0032]:1807716 (/mnt/cache/docker/docker.img)
Jul 13 07:09:47 Brahms1 root: Active pids left on /mnt/*
Jul 13 07:10:15 Brahms1 kernel: CIFS VFS: Server 192.168.2.254 has not responded in 120 seconds. Reconnecting...

 

You had previously said that I should try to make sure remote shares are online or unmount before they go offline. While that is possible for those on my LAN, it is not possible to ensure when they are connected via vpn or other tunnel to offsite shares, which is the case with the above message.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.