Jump to content
dlandon

Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

3223 posts in this topic Last Reply

Recommended Posts

Posted (edited)

I'm having issues with my UD no longer mounting my 2-disk BTRFS RAID1 pool ("ssd" = nvme0n1p1 set to automount + "ssd2" = nvme1n1p1). The 2nd disk [mounted to "ssd2"] of the pool (not set to automount) doesn't get luksOpen'ed and therefore is "missing," so the automount of the 1st disk fails.

Jul 22 12:33:02 Tower unassigned.devices: Mounting 'Auto Mount' Devices...
Jul 22 12:33:02 Tower unassigned.devices: Adding disk '/dev/mapper/ssd'...
Jul 22 12:33:03 Tower kernel: BTRFS: device fsid 4939602d-ea6f-4dd0-a535-252580b60aac devid 1 transid 11909816 /dev/dm-13
Jul 22 12:33:03 Tower unassigned.devices: Mount drive command: /sbin/mount  '/dev/mapper/ssd' '/mnt/disks/ssd'
Jul 22 12:33:03 Tower kernel: BTRFS info (device dm-13): disk space caching is enabled
Jul 22 12:33:03 Tower kernel: BTRFS info (device dm-13): has skinny extents
Jul 22 12:33:03 Tower kernel: BTRFS error (device dm-13): devid 2 uuid f66ef13b-8a93-45da-b73e-0d6a478729b2 is missing
Jul 22 12:33:03 Tower kernel: BTRFS error (device dm-13): failed to read chunk tree: -2
Jul 22 12:33:03 Tower kernel: BTRFS error (device dm-13): open_ctree failed
Jul 22 12:33:03 Tower emhttpd: Warning: Use of undefined constant luks - assumed 'luks' (this will throw an Error in a future version of PHP) in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 634
Jul 22 12:33:03 Tower unassigned.devices: Mount of '/dev/mapper/ssd' failed. Error message: mount: /mnt/disks/ssd: wrong fs type, bad option, bad superblock on /dev/mapper/ssd, missing codepage or helper program, or other error. 
Jul 22 12:33:03 Tower unassigned.devices: Partition 'Samsung_SSD_970_PRO_1TB_S462NF0M300357X' could not be mounted...
Jul 22 12:33:03 Tower unassigned.devices: Disk with serial 'Samsung_SSD_970_PRO_1TB_S462NF0M311892X', mountpoint 'ssd2' is not set to auto mount and will not be mounted...

After every array start, I have to execute luksOpen on "ssd2" and mount "ssd" manually (see below). Then after this, unmount/mount works again in the GUI.

root@Tower:~# /usr/sbin/cryptsetup luksOpen /dev/nvme1n1p1 ssd2 --allow-discards --key-file /root/keyfile
root@Tower:~# mkdir /mnt/disks/ssd
root@Tower:~# /sbin/mount '/dev/mapper/ssd' '/mnt/disks/ssd'

I think this happened after I renamed the mounts and physically changed slots, though it seemed to work after one restart, but then broke after the next restart. Have restarted a few times since and it never works anymore, even if I change the names.

 

If I switch the nvme1n1 to automount instead of nvme0n1, it still results in the same error.

tower-diagnostics-20190722-1649.zip

Edited by golli53
fixed manual command

Share this post


Link to post

Hi,

 

I have two HDD's in an external enclosure which I want to connect to my server to act as backups for certain shares. What filesystem would be best to format these as so Unraid can read and write to them but also I can connect them directly to my Mac and it can read and write to them.

Share this post


Link to post
2 hours ago, rctneil said:

Hi,

 

I have two HDD's in an external enclosure which I want to connect to my server to act as backups for certain shares. What filesystem would be best to format these as so Unraid can read and write to them but also I can connect them directly to my Mac and it can read and write to them.

You could test with APFS, but I doubt unRAID supports that. Probably HFS+ is going to be your best bet. 

 

Paragon Software offers extFS for Mac which would also allow you to use EXT. I don’t think there is any sort of free version though.

Share this post


Link to post
1 minute ago, wgstarks said:

You could test with APFS, but I doubt unRAID supports that. Probably HFS+ is going to be your best bet. 

 

Paragon Software offers extFS for Mac which would also allow you to use EXT. I don’t think there is any sort of free version though.

Thanks.  I'd rather stay clear of extra software just to read a drive.  I'm assuming both Unraid and MacOS support HFS+?     I have previously used XFS but the issue I had was that MacOS couldn't read or write it

Share this post


Link to post
10 minutes ago, rctneil said:

assuming both Unraid and MacOS support HFS+?

Yes

Share this post


Link to post

Due to limited SATA Ports on my Motherboard, I used a WD MyBook 8 TB via USB and UD-plugin to offload not very important data.

Before I have precleared this drive with the appropriate plugin and formatted to XFS with UD-plugin destructive mode.

 

Now I have replaced 2x SATA SSD with 2x NVME SSD and would like to add this WD Whitelabel 8 TB with 5 TB of data to my array.

Therefore I followed the advices here in forums to stop array, make new config with keeping the current assignment.

Then added the WD Whitelabel 8 TB as Disk 4 and hit start array.

 

However the new Disk 4 comes up with "Unmountable: Unsupported partition layout" and Unraid offers me to format it, which would delete my existing data.

The drive can be mounted with UD without problems and all files are accessible, which should rule out any corruption of the XFS partition.

 

From my understanding drives formatted with UD should have compatible partition layout with the array.

I have checked the syslog, but could not find anything helpful.

I have attached the diagnostics file, so somebody can hopefully point me into the right direction.

tower-diagnostics-20190725-2004.zip

Share this post


Link to post
10 hours ago, Forusim said:

I used a WD MyBook 8 TB via USB

The problem is likely the USB enclosure where the disk was formatted, as it didn't expose the full partition, you'd need to reformat the disk outside it.

Share this post


Link to post
13 hours ago, johnnie.black said:

The problem is likely the USB enclosure where the disk was formatted, as it didn't expose the full partition, you'd need to reformat the disk outside it.

Wow, that's a bummer. Is there a command, I could use to verify a difference in the partition layout between formatting over USB or SATA?

Share this post


Link to post
10 hours ago, Forusim said:

Wow, that's a bummer. Is there a command, I could use to verify a difference in the partition layout between formatting over USB or SATA?

You can use fdisk to compare the number of sectors before and after.

Share this post


Link to post
1 hour ago, johnnie.black said:

You can use fdisk to compare the number of sectors before and after.

Thanks johnnie.

@dlandon Is my issue posted on Thursday at 10:18 PM a general one for all USB attached drives?

Then it would make sense to give a warning at the beginning, when you are talking about "DIsks formated XFS or BTRFS will be partitioned compatible with the array disks and can be installed in the array without a re-format."

Share this post


Link to post
a general one for all USB attached drives?

It's not general, but it's not uncommon either, depends on the USB enclosure used.

 

 

Share this post


Link to post
4 hours ago, Forusim said:

Thanks johnnie.

@dlandon Is my issue posted on Thursday at 10:18 PM a general one for all USB attached drives?

Then it would make sense to give a warning at the beginning, when you are talking about "DIsks formated XFS or BTRFS will be partitioned compatible with the array disks and can be installed in the array without a re-format."

The only addition that would make sense to me would be to add "as long as the disk is not moved to a different type controller."

It's not just USB that's the issue, if you were to format it on a RAID controller with the disk set to a single volume RAID0 so it will show up, the same thing would likely happen. Some USB enclosures pass disks through unmolested, some don't, as johnnie.black said.

 

While not recommended for reliability reasons, it would work to leave your USB drive the way it was in the enclosure and add it to the array. It would be accepted without formatting. So the statement that disks can be installed in the array without reformatting is technically correct.

Share this post


Link to post
1 hour ago, jonathanm said:

The only addition that would make sense to me would be to add "as long as the disk is not moved to a different type controller."

It's not just USB that's the issue, if you were to format it on a RAID controller with the disk set to a single volume RAID0 so it will show up, the same thing would likely happen. Some USB enclosures pass disks through unmolested, some don't, as johnnie.black said.

 

While not recommended for reliability reasons, it would work to leave your USB drive the way it was in the enclosure and add it to the array. It would be accepted without formatting. So the statement that disks can be installed in the array without reformatting is technically correct.

Thanks for the clarification.

Good to know, that it would have worked in array via USB, but as you said it´s less reliable.

I have copied all the data on Disk4 to various disks with free space, added it to the array, formatted it, moved back everything and now rebuilding the parity.

Took me 2 additional days for all the transfers, but hopefully someone else will read it and know in advance.

Share this post


Link to post

Hi All

 

I have a problem accessing files on a mounted unassigned drive

(But only for data containing the Appdata for Emby)

Also I can not unmount the drive? But the mount dosent work correctly because I cannot access the drive as a SMB share?

 

The drive is full:

image.thumb.png.5cfe1b5efa32fa70415fed4e0d4d2056.png

 

Trying to access the files so I can delete these file (Initial thought is was Plex) in Crusader I can see that the files are locked

And that Emby have generated a temp file that is 235G !!! because of this I can not run my servers

Normally I would correct any file rights in Unraid but that does not work for assigned devices?

Please help, my last reboot hang up the system and I had to cut the power....

image.thumb.png.0821b95507a1da370ae73356f335b7ce.png

Share this post


Link to post
12 hours ago, casperse said:

Hi All

 

I have a problem accessing files on a mounted unassigned drive

(But only for data containing the Appdata for Emby)

Also I can not unmount the drive? But the mount dosent work correctly because I cannot access the drive as a SMB share?

 

The drive is full:

image.thumb.png.5cfe1b5efa32fa70415fed4e0d4d2056.png

 

Trying to access the files so I can delete these file (Initial thought is was Plex) in Crusader I can see that the files are locked

And that Emby have generated a temp file that is 235G !!! because of this I can not run my servers

Normally I would correct any file rights in Unraid but that does not work for assigned devices?

Please help, my last reboot hang up the system and I had to cut the power....

image.thumb.png.0821b95507a1da370ae73356f335b7ce.png

Can't you just ssh in and delete the files?

Share this post


Link to post
3 hours ago, saarg said:

Can't you just ssh in and delete the files?

Your right!!! of course as root user you can do anything.... Didnt think about unraid root being the solution

Deleted all temp files (After a preclear I could format the SSD - I know preclear is not for SSD but I couldnt get UAD to show the format option

I then was able to format the drive and copy the files to this seperate SSD for my appdata so hope this solves this for the time being

Share this post


Link to post

Wondering if someone might have insight, having trouble finding what I'm looking for.  I had written a script previously that I was using for some automation with an unassigned device.  Everything was working perfectly but with the move to this plugin, now not so much.  I was using the snap.sh script with certain parameters to safely work with the drive.  I'm wondering if I can gather the some way programmatically from the new plugin.  Here are the functions I was using and am looking to replace:

 

#get the serial number of a drive based on share name

/boot/config/plugins/snap/snap.sh -getSerialNumberFromSharename $share_name

 

#verify if the drive by serial number is in the array

/boot/config/plugins/snap/snap.sh -isDeviceInUnraidArray $drive_serial

 

#mount the drive by its share name

/boot/config/plugins/snap/snap.sh -m $share_name

 

#unmount the drive by its share name

/boot/config/plugins/snap/snap.sh -M $share_name

 

#spin down the drive by its serial number

/boot/config/plugins/snap/snap.sh -spindown $drive_serial

 

My end game is that I have an external drive that may or may not be connected to my unraid server.  Nightly, if the drive is present, it mounts and does a sync to that drive, and then unmounted and spins down so it can be removed.  This gives me a drive (or drives) that I can pull whenever I am travelling and it has been synced with a bunch of files I have listed in my script.

 

Share this post


Link to post
2 hours ago, jfrancais said:

Wondering if someone might have insight, having trouble finding what I'm looking for.  I had written a script previously that I was using for some automation with an unassigned device.  Everything was working perfectly but with the move to this plugin, now not so much.  I was using the snap.sh script with certain parameters to safely work with the drive.  I'm wondering if I can gather the some way programmatically from the new plugin.  Here are the functions I was using and am looking to replace:

 

#get the serial number of a drive based on share name

/boot/config/plugins/snap/snap.sh -getSerialNumberFromSharename $share_name

 

#verify if the drive by serial number is in the array

/boot/config/plugins/snap/snap.sh -isDeviceInUnraidArray $drive_serial

 

#mount the drive by its share name

/boot/config/plugins/snap/snap.sh -m $share_name

 

#unmount the drive by its share name

/boot/config/plugins/snap/snap.sh -M $share_name

 

#spin down the drive by its serial number

/boot/config/plugins/snap/snap.sh -spindown $drive_serial

 

My end game is that I have an external drive that may or may not be connected to my unraid server.  Nightly, if the drive is present, it mounts and does a sync to that drive, and then unmounted and spins down so it can be removed.  This gives me a drive (or drives) that I can pull whenever I am travelling and it has been synced with a bunch of files I have listed in my script.

 

Ud is very different from SNAP.  You can accomplish the same thing, but UD handles things differently.  UD detects a device when it is plugged in and runs your script automatically.  Read the first post for details.

Share this post


Link to post
1 minute ago, dlandon said:

Ud is very different from SNAP.  You can accomplish the same thing, but UD handles things differently.  UD detects a device when it is plugged in and runs your script automatically.  Read the first post for details.

My device is almost always plugged in.  I need it to run in a scheduled basis to ensure when I go off site it is no more than 24hrs out of sync.  And I need to run it based of serial number and not the /dev it isn’t guaranteed to be in the same dev position.

Share this post


Link to post
57 minutes ago, jfrancais said:

My device is almost always plugged in.  I need it to run in a scheduled basis to ensure when I go off site it is no more than 24hrs out of sync.  And I need to run it based of serial number and not the /dev it isn’t guaranteed to be in the same dev position.

Use the user scripts plugin to set your script to run on a cron.  You name your device mount point with UD and it is mounted with that name.  You can then Copy files to that mount point.  You don’t need the dev.  As I said, read the first post and you will see some examples.

Share this post


Link to post
33 minutes ago, dlandon said:

Use the user scripts plugin to set your script to run on a cron.  You name your device mount point with UD and it is mounted with that name.  You can then Copy files to that mount point.  You don’t need the dev.  As I said, read the first post and you will see some examples.

 

From the plugin help:

'/usr/local/sbin/rc.unassigned mount auto' - all devices and SMB/NFS mounts set to auto mount will be mounted.

'/usr/local/sbin/rc.unassigned umount auto' - all devices and SMB/NFS mounts set to auto mount will be unmounted.

'/usr/local/sbin/rc.unassigned umount all' - all devices and SMB/NFS mounts are unmounted in preparation for shutting down the array.

'/usr/local/sbin/rc.unassigned mount /dev/sdX' - mount device sdX where 'X' is the device designator.

'/usr/local/sbin/rc.unassigned umount /dev/sdX' - unmount device sdX where 'X' is the device designator. You can use this command in a UD script to unmount the device when the script has completed.

 

I don't want the drives to be configured to automount, and I don't want to mount/unmount all.

Share this post


Link to post
42 minutes ago, jfrancais said:

 

From the plugin help:

'/usr/local/sbin/rc.unassigned mount auto' - all devices and SMB/NFS mounts set to auto mount will be mounted.

'/usr/local/sbin/rc.unassigned umount auto' - all devices and SMB/NFS mounts set to auto mount will be unmounted.

'/usr/local/sbin/rc.unassigned umount all' - all devices and SMB/NFS mounts are unmounted in preparation for shutting down the array.

'/usr/local/sbin/rc.unassigned mount /dev/sdX' - mount device sdX where 'X' is the device designator.

'/usr/local/sbin/rc.unassigned umount /dev/sdX' - unmount device sdX where 'X' is the device designator. You can use this command in a UD script to unmount the device when the script has completed.

 

I don't want the drives to be configured to automount, and I don't want to mount/unmount all.

Read the first post of this thread.

Share this post


Link to post

I did, multiple times.  It says specifically to leave the disk mounted so Cron has access.  This is specifically NOT what I want to do.  The reason I’m asking is that I read it and can’t figure out how to accomplish what I did in SNAP after reading the limited documentation(ie the first post)

Share this post


Link to post
On 7/23/2019 at 12:55 PM, wgstarks said:

Yes

@wgstarks I've just got around to formatting the drive using Unassigned Devices.  What option do I choose for HFS+?  I don't see it in the list?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.