Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

[wgstarks]

Can I use UD to create a drive pool? I use a UD disk as a download location for torrents but it often fills up and I'm wondering if it's possible to create a pool from several USB attached external drives? It wouldn't be used for permanent storage. Only a temporary location for torrents being seeded. Not sure how much of a headache it would be though.

[JorgeB]

38 minutes ago, wgstarks said:

Can I use UD to create a drive pool?

You can using the CLI, but why not use an Unraid pool?

[itimpi]

38 minutes ago, wgstarks said:

Can I use UD to create a drive pool? I use a UD disk as a download location for torrents but it often fills up and I'm wondering if it's possible to create a pool from several USB attached external drives? It wouldn't be used for permanent storage. Only a temporary location for torrents being seeded. Not sure how much of a headache it would be though.

As I understand it you have to create such a pool manually using the relevant CLI commands.  Once that is done mounting any of the pool members within UD results in the pool being mounted.  It is always possible the recent spate of UD updates have changes this though.

[dlandon]

26 minutes ago, itimpi said:

It is always possible the recent spate of UD updates have changes this though.

The only issue that has come up is in the duplicate share names check recently added to UD that won't mount devices with duplicate names.  There is a setting that can be turned off so UD won't check for duplicate UD names.

 

Can someone post a link on how to do this using the cli?  I'm considering adding the ability to create pools to UD.

[JorgeB]

8 minutes ago, dlandon said:

Can someone post a link on how to do this using the cli?  I'm considering adding the ability to create pools to UD.

I hid the FAQ entry about that when Unraid started supporting multiple pools, here it is:

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=462135

 

[wgstarks]

55 minutes ago, JorgeB said:

You can using the CLI, but why not use an Unraid pool?

Maybe I need to check into this more. I’ve always thought that external drives shouldn’t be assigned to the array. Or maybe cache pools aren’t assigned?

[JorgeB]

3 minutes ago, wgstarks said:

I’ve always thought that external drives shouldn’t be assigned to the array. Or maybe cache pools aren’t assigned?

It's not recommended to use USB drives as array or pool members, but it's allowed.

[trurl]

2 minutes ago, wgstarks said:

Maybe I need to check into this more. I’ve always thought that external drives shouldn’t be assigned to the array. Or maybe cache pools aren’t assigned?

USB connections aren't recommended for the parity array because disconnects will make them out-of-sync and require rebuild.

 

Also not generally a good idea to have USB connections in a pool, but you said the data wasn't important.

[wgstarks]

20 minutes ago, trurl said:

USB connections aren't recommended for the parity array because disconnects will make them out-of-sync and require rebuild.

 

Also not generally a good idea to have USB connections in a pool, but you said the data wasn't important.

The data is all temporary but I don’t want usb disconnects knocking the array offline or causing parity checks and such. Not sure what sort of problems I would be inviting into my life.

[trurl]

9 minutes ago, wgstarks said:

The data is all temporary but I don’t want usb disconnects knocking the array offline or causing parity checks and such. Not sure what sort of problems I would be inviting into my life.

The pool we are talking about is separate from the parity array.

[wgstarks]

46 minutes ago, trurl said:

The pool we are talking about is separate from the parity array.

So disk events would only effect the docker that it was slaved to?

[dlandon]

47 minutes ago, trurl said:

The pool we are talking about is separate from the parity array.

What we are talking about is a pool in ud, which would not be a part of the array.  It would just be another ud device, but pooled.

[trurl]

57 minutes ago, dlandon said:

What we are talking about is a pool in ud

That would be appropriate for this thread, of course, but whether it is appropriate to do it that way or with an actual Unraid pool is the question.

 

Unraid pools of course have the advantage of being part of user shares with all that functionality.

 

Is it is any worse to have USB in an Unraid pool or in a UD pool? Maybe. I have certainly seen user shares broken due to corrupt member disks.

[wgstarks]

Would a usb disconnect break the pool (assuming the disconnect was just a transient) or would it just cause possible file corruption?

 

It would be fairly easy to recheck a torrent but I wouldn’t want to lose all the data on the pool.

[JorgeB]

4 minutes ago, wgstarks said:

Would a usb disconnect break the pool (assuming the disconnect was just a transient) or would it just cause possible file corruption?

It's difficult to say for sure, it might, it might not, also depends on the USB hardware, if they respect write barriers or not, if they don't and writes are lost the pool will go south quickly.

[R__]

On 12/13/2021 at 9:55 AM, dlandon said:

Post a UD screen shot and your diagnostics.

Both are attached, thanks. I can't mount the highlighted disk anymore at the moment, unless I unplug Dev 5 from the enclosure. Dev 3 and Dev 4 are in the same enclosure, and they mount/unmount as expected.

 

rvknas-diagnostics-20211214-1010.zip

[dlandon]

2 hours ago, R__ said:

Both are attached, thanks. I can't mount the highlighted disk anymore at the moment, unless I unplug Dev 5 from the enclosure. Dev 3 and Dev 4 are in the same enclosure, and they mount/unmount as expected.

 

rvknas-diagnostics-20211214-1010.zip 106.33 kB · 0 downloads

UD tracks a disk by its serial number as that is the only reliable thing to use because other things like the sdX device designation can change.  Because UD tracks devices by serial number, if you have duplicates, UD thinks they are all one disk and any thing done on one disk, is also done the the other as you've seen.

 

Your external USB bay is presenting several disks with the same serial number.  UD can't handle this situation.

[R__]

51 minutes ago, dlandon said:

UD tracks a disk by its serial number as that is the only reliable thing to use because other things like the sdX device designation can change.  Because UD tracks devices by serial number, if you have duplicates, UD thinks they are all one disk and any thing done on one disk, is also done the the other as you've seen.

 

Your external USB bay is presenting several disks with the same serial number.  UD can't handle this situation.

That makes sense, however, like I mentioned the other two disks (who also share the same serial) can be independently mounted/unmounted successfully. Is there a way to reset the mount point name for the one that now both have "movies-archive" assigned to their defaults, so I can use them independently?

[dlandon]

6 hours ago, R__ said:

Is there a way to reset the mount point name for the one that now both have "movies-archive" assigned to their defaults, so I can use them independently?

Because the serial numbers are the same, UD can't handle changing the mount point on one disk.  It will change both.

 

You need to figure out how to get your enclosure to provide a unique serial number for each device.

[dlandon]

I put together a simple test of a UD Pool and made changes to the UI to handle disk pools a little better.  It also fixes the issue with checking for duplicate share names on pool disks without the setting to override the check.  Mounting two different disks that are not in a pool to the same mount point can cause loss of data and the duplicate check offers some protection against that.

 

 

I wanted to do this so UD at least handled disk pools better.  This allows those that have legacy pools to mount them with UD and work with them.  You should also be able to unassign a pool from the array and work with it in UD.

 

The array pool offers a lot of advantages over UD and you should really use that instead of UD for pools.

[trurl]

Terminology is complicated.

 

I have always referred to the parity array as the array, and considered cache as not part of the array. Array and cache are part of user shares.

 

Then we got multiple pools, and I also considered those not part of the array. Array and pools are part of user shares

 

And, in a more generic sense of the word "pool", the array is also a pool. In fact, the user shares make the array and pools all one "pool".

 

Now we get UD pools. Of course those aren't part of the array or user shares.

[hansolo77]

I'm having trouble accessing my Unassigned Devices.  I have a Pro Licence, and have been able to access the drives listed in the plugin in the past.  However, recently I've noticed I can't even see the drives anymore.  I've tried a couple of things, like whitelisting my server in ABP and MalwareBytes, and using alternate browsers (I typically use FireFox or Chrome but Edge has the same results).  The closest I can get is with my tablet, where I get "nothing" on the MAIN tab, but "something" on the DASHBOARD tab, where it at least shows the devices as "Dev 1", etc, but I can't access the files.  This all started because I wanted to see how much storage was left on the drives.  I'm not sure what else to do or try.  The last thing I did with my array was add in the final 2x12tb drives.  I currently use Unassigned Devices to hold a 1tb SSD for all my Plex App/Meta data.  PLEX still works, so I know it's accessing.  I just can't SEE it from an admin point of view.  It's really disconcerting not being able to even see any stats.

 

Anything I've missed?

 

I actually just ran a scan using "Fix Common Problems" and it says

Quote

/var/log is getting full (currently 100 % used)

and that I should report the diagnostics.  Maybe it's related?

 

kyber-diagnostics-20211216-1737.zip

 

 

--------

EDIT:

 

So I took the advice of the Fix Common Problems plugin and rebooted.  Once, up, everything is displaying like it should.  So my question now is, can somebody help me identify where my problem might be?

Edited December 16, 2021 by hansolo77

[dlandon]

17 hours ago, hansolo77 said:

I'm having trouble accessing my Unassigned Devices.  I have a Pro Licence, and have been able to access the drives listed in the plugin in the past.  However, recently I've noticed I can't even see the drives anymore.  I've tried a couple of things, like whitelisting my server in ABP and MalwareBytes, and using alternate browsers (I typically use FireFox or Chrome but Edge has the same results).  The closest I can get is with my tablet, where I get "nothing" on the MAIN tab, but "something" on the DASHBOARD tab, where it at least shows the devices as "Dev 1", etc, but I can't access the files.  This all started because I wanted to see how much storage was left on the drives.  I'm not sure what else to do or try.  The last thing I did with my array was add in the final 2x12tb drives.  I currently use Unassigned Devices to hold a 1tb SSD for all my Plex App/Meta data.  PLEX still works, so I know it's accessing.  I just can't SEE it from an admin point of view.  It's really disconcerting not being able to even see any stats.

 

Anything I've missed?

 

I actually just ran a scan using "Fix Common Problems" and it says

and that I should report the diagnostics.  Maybe it's related?

 

kyber-diagnostics-20211216-1737.zip 395.71 kB · 1 download

 

 

--------

EDIT:

 

So I took the advice of the Fix Common Problems plugin and rebooted.  Once, up, everything is displaying like it should.  So my question now is, can somebody help me identify where my problem might be?

Your log is flooded with:

Dec 12 11:59:56 Kyber nginx: 2021/12/12 11:59:56 [alert] 17877#17877: worker process 15211 exited on signal 6
Dec 12 11:59:57 Kyber nginx: 2021/12/12 11:59:57 [alert] 17877#17877: worker process 15212 exited on signal 6
Dec 12 11:59:57 Kyber nginx: 2021/12/12 11:59:57 [alert] 17877#17877: worker process 15218 exited on signal 6

This could cause a problem with the GUI.  I don't understand this well enough to help, but others that are more knowledgeable about nginx might be able to help.

 

This is why rebooting worked - it cleared up the problem, but I suspect it will come back.

[dlandon]

On 12/15/2021 at 8:02 AM, trurl said:

Now we get UD pools. Of course those aren't part of the array or user shares.

UD will not be creating pools.  The only thing I wanted to do here is for UD to handle pooled disks a little better.  Pooled disks are better done in 'Pool Devices' in Unraid.

[trurl]

56 minutes ago, dlandon said:

UD will not be creating pools.  The only thing I wanted to do here is for UD to handle pooled disks a little better.  Pooled disks are better done in 'Pool Devices' in Unraid.

Not against the idea, just wondering about consistent ways to talk about it all.