Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

dlandon · June 5, 2022

Share issue fixed in the next release.

Nano · June 5, 2022

1 hour ago, dlandon said:

Share issue fixed in the next release.

Thanks for that and glad i helped find a bug then, cheers for everything

Renegade605 · June 5, 2022

Hi there, having an issue with a couple NFS remote mounted shares. Server 1 is a Dell workstation on v6.9.2 and server 2 is an HP DL380 G7 on v6.10.1.

Shared folders mounted on one Unraid box from the other one (in either direction) get mounted as read-only. Attempting to copy any file to it results in

cp: cannot create regular file '/mnt/remotes/tv/custom.list': Read-only file system

Server 2 has a different NFS share mounted from a Dell EMC SAN with no writing issues.

Logs from mounting:

Jun  5 15:01:42 EDI unassigned.devices: Mounting Remote Share '192.168.0.1:/mnt/user/tv'...
Jun  5 15:01:42 EDI unassigned.devices: Mount NFS command: /sbin/mount -t 'nfs' -o rw,noacl '192.168.0.1:/mnt/user/tv' '/mnt/remotes/tv'
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chmod(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2059
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chown(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2060
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chgrp(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2061
Jun  5 15:01:42 EDI unassigned.devices: Successfully mounted '192.168.0.1:/mnt/user/tv' on '/mnt/remotes/tv'.

Subnet 0 is a direct link between servers with no other devices or switches/routers. But, attempting to mount the share on subnets 1 or 2 has the same result.

dlandon · June 5, 2022

11 minutes ago, Renegade605 said:
Hi there, having an issue with a couple NFS remote mounted shares. Server 1 is a Dell workstation on v6.9.2 and server 2 is an HP DL380 G7 on v6.10.1.

Shared folders mounted on one Unraid box from the other one (in either direction) get mounted as read-only. Attempting to copy any file to it results in
cp: cannot create regular file '/mnt/remotes/tv/custom.list': Read-only file system
Server 2 has a different NFS share mounted from a Dell EMC SAN with no writing issues.

Logs from mounting:
Jun  5 15:01:42 EDI unassigned.devices: Mounting Remote Share '192.168.0.1:/mnt/user/tv'...
Jun  5 15:01:42 EDI unassigned.devices: Mount NFS command: /sbin/mount -t 'nfs' -o rw,noacl '192.168.0.1:/mnt/user/tv' '/mnt/remotes/tv'
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chmod(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2059
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chown(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2060
Jun  5 15:01:42 EDI unassigned.devices: PHP Warning: chgrp(): Read-only file system in /usr/local/emhttp/plugins/unassigned.devices/include/lib.php on line 2061
Jun  5 15:01:42 EDI unassigned.devices: Successfully mounted '192.168.0.1:/mnt/user/tv' on '/mnt/remotes/tv'.
Subnet 0 is a direct link between servers with no other devices or switches/routers. But, attempting to mount the share on subnets 1 or 2 has the same result.

Unraid 6.9 uses NFSv3. Unraid 6.10 uses NFSv4. You'll probably have to set the Unraid 6.10 to only mount using NFSv3.

Why don't you update both servers to 6.10? The NFS versions will then both be NFSv4. NFSv4 has a lot fewer issues that NFSv3.

Renegade605 · June 5, 2022

58 minutes ago, dlandon said:

Unraid 6.9 uses NFSv3. Unraid 6.10 uses NFSv4. You'll probably have to set the Unraid 6.10 to only mount using NFSv3.

Why don't you update both servers to 6.10? The NFS versions will then both be NFSv4. NFSv4 has a lot fewer issues that NFSv3.

I thought of this as I wrote it so I went and checked. Both servers are already set to mount with NFSv3. If you have no other ideas I could upgrade anyway, I just wanted to wait until my second server was set up and running (one project at a time, you know?).

Renegade605 · June 6, 2022

Update: Both servers are now on v6.10.2 and NFSv4 but there is no change in behaviour. Regardless, the servers can't stay on NFSv4 because the other network share server 2 is currently connected to doesn't support it. (Unless we get the option to mount different shares with different protocol versions.)

cferra · June 6, 2022

I am not sure what is happening, but here are the symptoms. I have mounted 2 SMB shares to a windows server 2022 box and have them mapped in docker containers as r/w slave as suggested.. for some reason, the mounts malfunction and the plugin shows 0tb on the mounts but shows as mounted in the UI. This has been continuing to happen more and more.. and I cant figure out a reason..

any help would be great!

cferra · June 6, 2022

i've included a screenshot and the syslog for reference.

ferranteapp1-syslog-20220606-0200.zip

dlandon · June 6, 2022

5 hours ago, Renegade605 said:

I thought of this as I wrote it so I went and checked. Both servers are already set to mount with NFSv3. If you have no other ideas I could upgrade anyway, I just wanted to wait until my second server was set up and running (one project at a time, you know?).

Post your diagnostics zip.

dlandon · June 6, 2022

2 hours ago, Renegade605 said:

Update: Both servers are now on v6.10.2 and NFSv4 but there is no change in behaviour. Regardless, the servers can't stay on NFSv4 because the other network share server 2 is currently connected to doesn't support it. (Unless we get the option to mount different shares with different protocol versions.)

You can set to mount UD remote shares by going to UD Settings and setting to NFSv3.

dlandon · June 6, 2022

1 hour ago, cferra said:

i've included a screenshot and the syslog for reference.

ferranteapp1-syslog-20220606-0200.zip 52.36 kB · 0 downloads

You have network or remote server issues that are keeping UD from geting the information it needs for the remote share:

Jun  5 21:47:45 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_PMS_DATA_UNRAID' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:45 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_PMS_DATA_UNRAID' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:46 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_PMS_DATA_UNRAID' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:48 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_PMS_DATA_UNRAID' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:50 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:50 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:51 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:51 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:53 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!
Jun  5 21:47:54 FERRANTEAPP1 unassigned.devices: Error: shell_exec(/bin/df '/mnt/remotes/FERRANTEMD1_Sync' --output=size,used,avail | /bin/grep -v '1K-blocks' 2>/dev/null) took longer than 5s!

The 'df' commands are timing out. This is the comand to determine the size, used, and free space on the remote share. That's why you see all zeroes.

Renegade605 · June 6, 2022

8 hours ago, dlandon said:

Post your diagnostics zip.

Here are both servers.

8 hours ago, dlandon said:

You can set to mount UD remote shares by going to UD Settings and setting to NFSv3.

Sorry, I meant choose v3/v4 independently for each remote mount share. ie. Servers talk to each other in v4, but to the SAN in v3.

global-dynamics-diagnostics-20220606-0639.zip edi-diagnostics-20220606-0640.zip

dlandon · June 6, 2022

Your edi-diagnostics server has some

issues you need to clean up:

Jun  6 04:40:02 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:02 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr
Jun  6 04:40:03 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:03 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr
Jun  6 04:40:04 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:04 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr

Jun  6 04:50:06 EDI kernel: Call Trace:
Jun  6 04:50:06 EDI kernel: <TASK>
Jun  6 04:50:06 EDI kernel: intel_idle+0x1b/0x21
Jun  6 04:50:06 EDI kernel: cpuidle_enter_state+0xc6/0x1db
Jun  6 04:50:06 EDI kernel: cpuidle_enter+0x2a/0x36
Jun  6 04:50:06 EDI kernel: do_idle+0x1b7/0x225
Jun  6 04:50:06 EDI kernel: cpu_startup_entry+0x1d/0x1f
Jun  6 04:50:06 EDI kernel: start_kernel+0x656/0x67b
Jun  6 04:50:06 EDI kernel: secondary_startup_64_no_verify+0xb0/0xbb
Jun  6 04:50:06 EDI kernel: </TASK>
Jun  6 04:50:06 EDI kernel: NMI: IOCK error (debug interrupt?) for reason 61 on CPU 0.
Jun  6 04:50:06 EDI kernel: CPU: 0 PID: 0 Comm: swapper/0 Tainted: P        W IO      5.15.43-Unraid #1
Jun  6 04:50:06 EDI kernel: Hardware name: HP ProLiant DL380 G7, BIOS P67 03/30/2010
Jun  6 04:50:06 EDI kernel: RIP: 0010:mwait_idle_with_hints.constprop.0+0x5f/0x86
Jun  6 04:50:06 EDI kernel: Code: 48 89 d1 65 48 8b 04 25 c0 bb 01 00 0f 01 c8 48 8b 00 a8 08 75 14 eb 07 0f 00 2d dc 6e 9f 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 c0 bb 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b
Jun  6 04:50:06 EDI kernel: RSP: 0018:ffffffff82203e48 EFLAGS: 00000046
Jun  6 04:50:06 EDI kernel: RAX: 0000000000000001 RBX: 0000000000000002 RCX: 0000000000000001
Jun  6 04:50:06 EDI kernel: RDX: 0000000000000000 RSI: ffffffff82311ca0 RDI: 0000000000000001
Jun  6 04:50:06 EDI kernel: RBP: ffffe8f6ff81b300 R08: 0000215f75e73451 R09: ffff888913a1eca0
Jun  6 04:50:06 EDI kernel: R10: 0000000000000020 R11: 00000000000005d8 R12: 0000000000000002
Jun  6 04:50:06 EDI kernel: R13: ffffffff82311d70 R14: ffffffff82311d88 R15: 0000000000000000
Jun  6 04:50:06 EDI kernel: FS:  0000000000000000(0000) GS:ffff888913a00000(0000) knlGS:0000000000000000
Jun  6 04:50:06 EDI kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun  6 04:50:06 EDI kernel: CR2: 000014e4467a4000 CR3: 000000016eeb6006 CR4: 00000000000226f0

Jun  6 04:53:33 EDI kernel: nfs: server 192.168.0.1 not responding, timed out

You are wanting the two servers to cross mount shares using NFSv4 and have a remote share mounted NFSv3?

Renegade605 · June 6, 2022

14 minutes ago, dlandon said:

Your edi-diagnostics server has some

issues you need to clean up:

Oh that's interesting. I'll check that out after work. Seems unlikely to be the cause though, as the NFS shares in either direction have the same problem.

14 minutes ago, dlandon said:

You are wanting the two servers to cross mount shares using NFSv4 and have a remote share mounted NFSv3?

I'm just saying if that's possible. Otherwise they all stay on v3.

dlandon · June 6, 2022

4 minutes ago, Renegade605 said:

Oh that's interesting. I'll check that out after work. Seems unlikely to be the cause though, as the NFS shares in either direction have the same problem.

The issue of the NFS server (192.168.0.1) not responding is a problem.

Renegade605 · June 6, 2022

1 minute ago, dlandon said:

The issue of the NFS server (192.168.0.1) not responding is a problem.

I missed that part. Any idea what could cause that? Especially since it successfully mounts the shares read-only.

dlandon · June 6, 2022

3 minutes ago, Renegade605 said:

I missed that part. Any idea what could cause that? Especially since it successfully mounts the shares read-only.

Both your servers are 6.10. I have two servers running 6.0 and I can cross mount NFSv3 and NFSv4 shares. Shares are mounted RW and not RO.

Don't minimize the errors on the one server. You need to clean those up. Not sure what effect those have on NFS.

The server not responding is probably a network or server configuration issue.

user12345678 · June 6, 2022

Greetings.

This is minor, however I thought I'd ask & see what could be done...

Any chance that the customized device name from the Main tab:

1552931954_MainTab.PNG.52f58bdc93fb0285d3d4f7e20f6d5b9e.PNG

could appear on the Dashboard tab (instead of the default 'Dev n'):

2023519154_DashboardTab.PNG.b8a6fc0953d8e3695f9570c88ce3f72f.PNG

Thanks in advance!

dlandon · June 6, 2022

1 hour ago, user12345678 said:

This is minor, however I thought I'd ask & see what could be done...

Any chance that the customized device name from the Main tab:

The Dashboard is stock Unraid and UD has no control over that page. Since UD is controlling the naming of devices, Unraid is not aware of those device names, and it is a minor display issue, it is not going to be viewed as a very important change.

user12345678 · June 6, 2022

I figured as much, thank you for the consideration/info!

Renegade605 · June 7, 2022

So I've figured it out, and it's entirely me not understanding NFS security settings. 🤦‍♂️

For anyone who should come across this later: when setting an Unraid share to export over NFS, setting it to "Privacy: Private" with no rule will cause every other machine to mount the share as read-only. This didn't come up in any of my searching for a solution, and given how simple it is it seems it should have been the first thing to come up. But the google is shrouded in mystery; who knows what's going on under the hood there.

The rule I went with is "192.168.0.2(sec=sys,rw)", which allows read/write access only to connections coming from that IP address and denies all other connections. This may be insecure depending on your use case. Since for me, 192.168.0.0/24 is a direct link between servers without internet access, an intruder would require physical access to the servers to overcome this, and if that were the case I have far larger concerns than the contents of my shares.

12 hours ago, dlandon said:

Your edi-diagnostics server has some

issues you need to clean up:

Jun  6 04:40:02 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:02 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr
Jun  6 04:40:03 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:03 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr
Jun  6 04:40:04 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/backup
Jun  6 04:40:04 EDI emhttpd: error: share_luks_status, 6158: Operation not supported (95): getxattr: /mnt/user/tdarr

Jun  6 04:50:06 EDI kernel: Call Trace:
Jun  6 04:50:06 EDI kernel: <TASK>
Jun  6 04:50:06 EDI kernel: intel_idle+0x1b/0x21
Jun  6 04:50:06 EDI kernel: cpuidle_enter_state+0xc6/0x1db
Jun  6 04:50:06 EDI kernel: cpuidle_enter+0x2a/0x36
Jun  6 04:50:06 EDI kernel: do_idle+0x1b7/0x225
Jun  6 04:50:06 EDI kernel: cpu_startup_entry+0x1d/0x1f
Jun  6 04:50:06 EDI kernel: start_kernel+0x656/0x67b
Jun  6 04:50:06 EDI kernel: secondary_startup_64_no_verify+0xb0/0xbb
Jun  6 04:50:06 EDI kernel: </TASK>
Jun  6 04:50:06 EDI kernel: NMI: IOCK error (debug interrupt?) for reason 61 on CPU 0.
Jun  6 04:50:06 EDI kernel: CPU: 0 PID: 0 Comm: swapper/0 Tainted: P        W IO      5.15.43-Unraid #1
Jun  6 04:50:06 EDI kernel: Hardware name: HP ProLiant DL380 G7, BIOS P67 03/30/2010
Jun  6 04:50:06 EDI kernel: RIP: 0010:mwait_idle_with_hints.constprop.0+0x5f/0x86
Jun  6 04:50:06 EDI kernel: Code: 48 89 d1 65 48 8b 04 25 c0 bb 01 00 0f 01 c8 48 8b 00 a8 08 75 14 eb 07 0f 00 2d dc 6e 9f 00 b9 01 00 00 00 48 89 f8 0f 01 c9 <65> 48 8b 04 25 c0 bb 01 00 f0 80 60 02 df f0 83 44 24 fc 00 48 8b
Jun  6 04:50:06 EDI kernel: RSP: 0018:ffffffff82203e48 EFLAGS: 00000046
Jun  6 04:50:06 EDI kernel: RAX: 0000000000000001 RBX: 0000000000000002 RCX: 0000000000000001
Jun  6 04:50:06 EDI kernel: RDX: 0000000000000000 RSI: ffffffff82311ca0 RDI: 0000000000000001
Jun  6 04:50:06 EDI kernel: RBP: ffffe8f6ff81b300 R08: 0000215f75e73451 R09: ffff888913a1eca0
Jun  6 04:50:06 EDI kernel: R10: 0000000000000020 R11: 00000000000005d8 R12: 0000000000000002
Jun  6 04:50:06 EDI kernel: R13: ffffffff82311d70 R14: ffffffff82311d88 R15: 0000000000000000
Jun  6 04:50:06 EDI kernel: FS:  0000000000000000(0000) GS:ffff888913a00000(0000) knlGS:0000000000000000
Jun  6 04:50:06 EDI kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Jun  6 04:50:06 EDI kernel: CR2: 000014e4467a4000 CR3: 000000016eeb6006 CR4: 00000000000226f0

Jun  6 04:53:33 EDI kernel: nfs: server 192.168.0.1 not responding, timed out

In case you're curious, the cause of these errors is:

IOCK error for reason 61 on CPU 0: According to my reading, this is a result of a weird glitch between the kernel and the BIOS and/or firmware on the integrated RAID controller on these machines. During boot this kernel panic might occur and prevent the system from finishing the boot. A BIOS update and firmware update for the RAID controller will supposedly fix this, but I've seen many people who are more experienced than I am brick systems trying to do such things, so I've decided to live with it. I have IPMI access, so if it does happen I can reboot that way from anywhere.

nfs: server 192.168.0.1 not responding: not sure, but only happens when the above error occurs, so I'm not concerned since in that case the machine hasn't booted anyway.

emhttpd: error: share_luks_status...: has to do with how I hacked together GUI administration of shares on my ZFS pool. I opted to create a symbolic link from /mnt/user/*share-name* to /mnt/*zfs-pool*/*share-name* which allows me to set the SMB and NFS security settings from the Unraid GUI even though they're ZFS datasets and don't normally show up in the GUI. Other than clogging up the logs, it doesn't seem to be hurting anything. I am open to hearing suggestions from others that present the same level of user-friendliness for administration without throwing this error every second, should anyone wish to share them.

user12345678 · June 9, 2022

Greetings.

Tried this in the General Support forum:

trying here as well...

I have 12 unassigned drives, three of them have one-time high CRC error counts due to bad cables which I've since replaced and everything is fine now except...

On every reboot they swap 'Dev n' numbers with other drives and I get new CRC error alerts (same three drives, CRC error counts never increase, they are just on a different 'Dev n' after reboot) and I have to re-acknowledge them.

Is there any way to get a drive to 'stick' to the same 'Dev n' number after reboot or otherwise get the 'acknowledge' that I do manually to follow the drive by its serial or something?

Thanks in advance!

dlandon · June 9, 2022

30 minutes ago, user12345678 said:

Is there any way to get a drive to 'stick' to the same 'Dev n' number after reboot or otherwise get the 'acknowledge' that I do manually to follow the drive by its serial or something?

Yes, click on the device designator ('Dev X') and make a change to the SMART settings and save the change. The warnings will follow the serial number and not the 'Dev X'.

user12345678 · June 9, 2022

9 minutes ago, dlandon said:

Yes, click on the device designator ('Dev X') and make a change to the SMART settings and save the change.

Got it, will try that, thank you!!

user12345678 · June 9, 2022

3 hours ago, dlandon said:

Yes, click on the device designator ('Dev X') and make a change to the SMART settings and save the change.

I rebooted 7 times (excessive, I know) and it looks like this did the trick.

I also noticed the drives aren't 'jumping around' anymore among 'Dev n' nor '/dev/sd[n]' assignments, does this also stabilize that in some way? Or perhaps that's just a 7x coincidence...

Unassigned Devices - Managing Disk Drives and Remote Shares Outside of The Unraid Array

Recommended Posts

Link to comment

Top Posters In This Topic

Popular Days

Top Posters In This Topic

Popular Days

Popular Posts

dlandon

dlandon

dlandon

Posted Images

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Link to comment

Join the conversation