Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

[oh-tomo]

11 minutes ago, dlandon said:

The mount button indicators are documented in the first post.  For me to investigate further, please supply the ud diagnostics.  Go to a command line and type 'ud_diagnostics'.  Then post the /flash/logs/ud_diagnostics.zip file here.

 

 

ud_diagnostics posted!

ud_diagnostics-20240508-141826.zip

[dlandon]

21 minutes ago, oh-tomo said:

ud_diagnostics posted!

Both sdl and sdm have the same mount point: "EFI".  Each mount point has to be unique.  Either don't mount these partitions at the same time, remove them if they are not needed, or rename them to be unique (best choice).

 

UD thinks the disk (sdl) was not unmounted and then reinserted, and marked the mount button "Reboot" because it thinks the disk was not properly unmounted.

[Amane]

6 hours ago, dlandon said:

That does pretty much the same thing as the port check commands I posted.  I do find them to be slightly faster.

Oh ok, i tested agin. you are right, the command was not wrong, sry

[zoggy]

Unraid 6.12.10

UD 2024.05.07 

UD+ 2024.04.01 

UD Preclear 2024.04.23a 

 

I was comparing some data from my current cache drive and my old cache drive which I've mounted with UD.

I noticed that even though in UD I have the option "Mount SSDs with 'discard' option?" set to  yes, it is not doing it for my nvme drive:

# mount | grep nvme
/dev/nvme0n1p1 on /mnt/cache type btrfs (rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/)
/dev/nvme1n1p1 on /mnt/disks/S59ANMFNA03845Z type btrfs (rw,relatime,ssd,space_cache=v2,subvolid=5,subvol=/)

 

 

Is there anything special I need to do after setting that option (tried to unmount/re-mount via UD), but still does not show discard being set.

# cat /sys/block/nvme1n1/queue/rotational
0

 

 

checking unraid syslogs... looks like it is however telling it to do discard

May  9 00:39:15 husky unassigned.devices: Mounting partition 'nvme1n1p1' at mountpoint '/mnt/disks/S59ANMFNA03845Z'...
May  9 00:39:15 husky unassigned.devices: Mount cmd: /sbin/mount -t 'btrfs' -o rw,relatime,space_cache=v2,discard '/dev/nvme1n1p1' '/mnt/disks/S59ANMFNA03845Z'
May  9 00:39:15 husky unassigned.devices: Successfully mounted '/dev/nvme1n1p1' on '/mnt/disks/S59ANMFNA03845Z'.

 

so makes me wonder if it should have discard=async or if discard isnt showing up for some other reason.. cant reboot right now but will try that tomorrow in case something is just stuck.

 

rebooting unraid, and now mounting the drive with ud.. i see discard

# mount | grep nvme
/dev/nvme0n1p1 on /mnt/cache type btrfs (rw,noatime,ssd,discard=async,space_cache=v2,subvolid=5,subvol=/)
/dev/nvme1n1p1 on /mnt/disks/S59ANMFNA03845Z type btrfs (rw,relatime,ssd,discard,space_cache=v2,subvolid=5,subvol=/)

 

Edited May 10 by zoggy

[csimpson]

Hello, new Unraid user here. I'm trying to use RSYNC to sync an external Seagate drive mounted with unassigned devices.
 

rsync -av --delete "/mnt/user/Shared/" "/mnt/user/syncTest/"

Works perfectly fine. However, when trying to copy to my mounted external drive, I use the command:
 

rsync -av --delete "/mnt/user/Shared/" "/mnt/disks/Expansion/"

And I get the error message "Operation not permitted (1)" for all of the files.
 

Krusader works when trying to copy files to the External Drive. Not sure what I may be overlooking with rsync?

ls -al "/mnt/disks/Expansion/" gives me the following:

There's some garbage files in there that came with the Seagate drive that I'll be getting rid of.  But it's mainly the directories 
 

Archive
Family
Finances
Media 
Other
 

That are going to stay.  Not sure what I may be overlooking?

Edited May 9 by csimpson

[dlandon]

15 minutes ago, csimpson said:

That are going to stay.  Not sure what I may be overlooking?

What file system is the UD disk?

[csimpson]

46 minutes ago, dlandon said:

What file system is the UD disk?

 
vfat - it's how it came formatted and I didn't change it.

[dlandon]

Just now, csimpson said:

 
vfat - it's how it came formatted and I didn't change it.

Rsync is trying to write extended attributes and the vfat file system does not support extended attributes,  You have several options:

• Reformat the drive to another file system that supports extended attributes.
• Tell rsync to not write extended attributes.

[csimpson]

6 minutes ago, dlandon said:

Rsync is trying to write extended attributes and the vfat file system does not support extended attributes,  You have several options:

• Reformat the drive to another file system that supports extended attributes.
• Tell rsync to not write extended attributes.

Ah hah!  That's funny because last night I created a couple scripts to delete the ._ and .ds_store and thumbs.db files that my mac creates to keep things tidy. So those have all been cleared up.

I also added

veto files=/._*/.DS_Store/thumbs.db/

in the SMB settings.

I ran the rsync script again with a bit more success.

However, an example of an error I receive is:
 

rsync: [generator] chown "/mnt/disks/Expansion/Finances/.luckybackup-snaphots" failed: Operation not permitted (1)

I have a bunch of these.  It looks like there's a bunch of files that start with a "." and then the filename.

Using the User Scripts plugin, would 
 

find /mnt/user -maxdepth 9999 -noleaf -type f -name ".*" -exec rm "{}" \;

 

delete these files?  I suspect so? But then, I also see that there's files called ".smbdeleteAAA02e74.4" for example.  Do those need to remain?  Sorry, still new to Unraid and am moving through my migration.

[trurl]

6 minutes ago, csimpson said:

created a couple scripts to delete the ._ and .ds_store

https://forums.unraid.net/topic/48707-additional-scripts-for-userscripts-plugin/?do=findComment&comment=480177

 

[csimpson]

1 minute ago, trurl said:

https://forums.unraid.net/topic/48707-additional-scripts-for-userscripts-plugin/?do=findComment&comment=480177

 

Yes, I already did that and created a couple more to delete thumbs.db and ._ files. I'm more curious about if the following script will delete only files beginning with a "."

I believe it will because there's no wildcard before the "." but this could be a big "oopsie" if I'm not right. But this script would also delete .smbdelete files as well and I don't know if they're important or not.
 

find /mnt/user -maxdepth 9999 -noleaf -type f -name ".*" -exec rm "{}" \;

 

[dlandon]

17 minutes ago, csimpson said:

I also added

veto files=/._*/.DS_Store/thumbs.db/

in the SMB settings.

I would try to find another way as this affects performance on all SMB shares if you do this globally in smb-extras.

[Kilrah]

2 hours ago, csimpson said:

I have a bunch of these.  It looks like there's a bunch of files that start with a "." and then the filename.

That means hidden, but it's not because they're hidden that they can't be important. Just deleting/skipping all dotfiles could be a really bad idea.

 

2 hours ago, csimpson said:

rsync: [generator] chown "/mnt/disks/Expansion/Finances/.luckybackup-snaphots" failed: Operation not permitted (1)

Again the problem is that rsync tries to change the owner of the file on the destination to match the source but fat doesn't support any kind of owner/perms so it fails. So either ignore the errors or use the correct rsync options to skip copying the perms and attributes.

Edited May 9 by Kilrah

[csimpson]

19 minutes ago, Kilrah said:

That means hidden, but it's not because they're hidden that they can't be important. Just deleting/skipping all dotfiles could be a really bad idea.

 

Again the problem is that rsync tries to change the owner of the file on the destination to match the source but fat doesn't support any kind of owner/perms so it fails. So either ignore the errors or use the correct rsync options to skip copying the perms and attributes.

Understood. I'll leave the "." files alone for now.  Is there any reason to keep the external drive as vFat? I want to be able to have an external drive that if unraid fails, I can take that external drive to my mac or PC and be able to access the data. As I understand it:

NTFS - will give me two file tables to protect from an accidental eject and can only be read by a Mac so not a bad option.
exFat - seems like a good choice, but with only a single file table can be prone to corruption from an accidental disconnect.
APFS - Great, but won't work with Windows.
vFat - don't know very much about it.

Is there a preferred format for Unraid and reliability along with maximum compatibility?

Also, would you mind sharing the sync options? I'm new to rsync

Edited May 9 by csimpson

[Kilrah]

NTFS or ExFAT, vfat also has the issue that it cannot support any file bigger than 4GB.

But they'll both exhibit the same issue, ExFAT also doesn't support any owner/perms, NTFS does but in a completely different way.

 

1 hour ago, csimpson said:

Also, would you mind sharing the sync options? I'm new to rsync

Sorry, don't know them as is, I just used XFS for my backup external, I can just boot a linux distro on any machine I have if I need to access it.

[csimpson]

2 hours ago, Kilrah said:

NTFS or ExFAT, vfat also has the issue that it cannot support any file bigger than 4GB.

But they'll both exhibit the same issue, ExFAT also doesn't support any owner/perms, NTFS does but in a completely different way.

 

Sorry, don't know them as is, I just used XFS for my backup external, I can just boot a linux distro on any machine I have if I need to access it.

My mistake, it's already exFat. Expansion is the drive.  Not sure what EFI is there for?

 

 

I'll take a look and see if I can find some scripts to accomplish the sync options.  

[csimpson]

Okay, I think I've got the script figured out.  
 

rsync -avz --no-perms --no-owner --no-group --delete "/mnt/user/source/" "/mnt/disks/destination"

Running it now and seems to be successful... So far...

[dlandon]

New release of UD:

• Change in remote server online check that should solve the situation where a remote share can be successfully added but then shows as being offline.
• Some code cleanup for robustness.

[d4man]

Hello folks, got some strange behaviors with an unassigned disk that is passed thru to an OMV VM as a backup disk on my unraid.

 

The cheap hitachi 4 tbb disk appears in the UA section, I successfully passed it thru using /dev/disk/by-id/ata-Hitachi_xxxxxxx in the OMV VM 2nd vdisk location section, then I created an SMB Share which works, both are mounted, but the hitachi disk appears empty when I use the file manager. The hitachi disk and the OMV/OMV Share are supposed to be the same disk. When I'd click the clickable links before it would open a window showing me both folders exactly the same. I deleted the old OMV VM to make a new, updated one and while that was happening I installed the new UA plugin. This is driving me nuts! I spent 7 hours fiddling around with this, and just gave up. Any help is appreciated.

[Kilrah]

2 minutes ago, d4man said:

, I successfully passed it thru using /dev/disk/by-id/ata-Hitachi_xxxxxxx in the OMV VM 2nd vdisk location section, then I created an SMB Share which works, both are mounted

Seems you are both mounting the drive in UD and in the VM, you must never do this. If you pass it to the VM mark it as "passed through" in UD so that you won't mount it by mistake. 

 

If you actually want a share that's accessible then you cannot pass it through, just have the VM access that share.

Edited May 12 by Kilrah

[d4man]

39 minutes ago, Kilrah said:

Seems you are both mounting the drive in UD and in the VM, you must never do this. If you pass it to the VM mark it as "passed through" in UD so that you won't mount it by mistake. 

 

If you actually want a share that's accessible then you cannot pass it through, just have the VM access that share.

Thanks for your quick reply, unmounted the disk in UA, detached it, reattched it, left it unmounted and now unraid wont complete the SMB share, saying to look at syslog. Gonna fiddle around with it more

 

EDIT: a quick reboot fixed it and I have it back to the way it was -- thanks again!

Edited May 12 by d4man

[Kilrah]

56 minutes ago, d4man said:

EDIT: a quick reboot fixed it and I have it back to the way it was -- thanks again!

Not sure of the current setup, but again it must NOT be mounted/shared from unraid/UD and accessed by a VM through /dev/disk/by-id... at the same time. You risk data loss and/or corruption if you do that.

[d4man]

14 minutes ago, Kilrah said:

Not sure of the current setup, but again it must NOT be mounted/shared from unraid/UD and accessed by a VM through /dev/disk/by-id... at the same time. You risk data loss and/or corruption if you do that.

That is weird, I have never heard that before, and I don't doubt you but I've always done it this way with no problems, this VM is a backup of a backup of a backup so data loss is no big deal, I might even borrow the drive and put it in my TrueNAS backup server. Thanks again!!!

[nickramsbottom]

I'm having an issue with changes to SMB mounted shares not being shown.

 

I am mounting an SMB share on a Synology NAS, I am unsure whether the drive has ever tracked updates to the network drive but it indexed the drive on first connect and the files it is showing are accessible.

 

I have gone to the mount directory of the share in terminal and looked for newer files and not seen them. This is the case for files made by the unraid server or other devices on the network.

 

Steps I have taken to try and fix this:

• delete the mounted drive in unassigned devices and re-add the drive. The re-added drive did not re-index the network drive and still doesn't update
• created a separate set of credentials for unassigned devices to access the share with

I'm slightly lost with what to do from there, the new files are definitely in the share. Other devices on the network can see the extra files/folders.

 

 

 

[dlandon]

28 minutes ago, nickramsbottom said:

I'm having an issue with changes to SMB mounted shares not being shown.

 

I am mounting an SMB share on a Synology NAS, I am unsure whether the drive has ever tracked updates to the network drive but it indexed the drive on first connect and the files it is showing are accessible.

 

I have gone to the mount directory of the share in terminal and looked for newer files and not seen them. This is the case for files made by the unraid server or other devices on the network.

 

Steps I have taken to try and fix this:

• delete the mounted drive in unassigned devices and re-add the drive. The re-added drive did not re-index the network drive and still doesn't update
• created a separate set of credentials for unassigned devices to access the share with

I'm slightly lost with what to do from there, the new files are definitely in the share. Other devices on the network can see the extra files/folders.

 

 

 

Post Unraid diagnostics.