BTRFS warning (device <unknown>): duplicate device

[comet424]

as i remote servicing my sisters servers.. i noticed  duplicate device..  dont know how or fix...  as i was trying to figure why docker was slow

and plex server had the  Gray Circle ?    but then when you refresh docker  it fast   and then icon is back.. im pretty sure Plex slows down docker like its slow..   as i mentioned in past docker slowness issues i found  topics i wrote

 

but here is the error

Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdg1 devid 1 generation 231168 scanned by udevd (7620)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdb1 devid 1 generation 1573 scanned by udevd (7617)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdf1 devid 1 generation 297299 scanned by udevd (7619)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdc1 devid 1 generation 23784 scanned by udevd (7622)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdd1 devid 1 generation 25146352 scanned by udevd (7615)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sde1 devid 1 generation 662393 scanned by udevd (7616)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdh1 devid 1 generation 822261 scanned by udevd (7614)
Feb  2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdi1 devid 1 generation 12853 scanned by udevd (7625)

 

 

how i fix that?

 a reboot didnt fix it..  

 

and fix common issues  didnt find no errors other then  no root password

 

i also checked my sisters other server.. and there is 1 warrently with the duplicate device... my main server i cant check  shows  no errors or warrnings

 

mitchflix-diagnostics-20220202-1913.zip

Edited February 3, 2022 by comet424

[JorgeB]

Not a big deal, it's just a warning, it appears to be caused by the UD plugin, reboot in safe mode if there's no more warning you can post in the UD support thread to see if anything can be done about it.

[comet424]

ah ok i tried googling how to boot into safe mode....  as i cant access it locallly i work on my sisters server remotely..

under the flash it has some features..  and if i boot into safe mode do i loose network access ?

[JorgeB]

7 minutes ago, comet424 said:

and if i boot into safe mode do i loose network access ?

Nope, just all plugins aren't loaded, alternatively you can uninstall just the UD plugin and reboot.

[comet424]

ah ok..  the safe mode didnt have the issue  i  rebooted back again  normal...  it didnt pop up the several  duplicates  just 1 at the moment..

 

ill post in there  

 

i appreciate the help 🙂

[JSE]

To add to this, it's best to remove the plugin until it is potentially addressed, as is warned here: https://btrfs.wiki.kernel.org/index.php/Gotchas#Block-level_copies_of_devices

 

While these two "copies" are in fact the same thing, the thing with unraid is the /dev/mdx devices correspond to the /dev/sdx devices. So if the kernel sees both all of a sudden, it just thinks they're unique devices. In normal filesystems, this is fine, but Btrfs isn't "normal", so it gets angry because there are two devices with the same Device ID (all btrfs devices in a pool share the same UUID). A btrfs pool stores unique devices in its own metadata with numerical device IDs which are incremental. If you have a pool, consider running the command:
 

btrfs device usage /mnt/cache

It will list the devices in the pool with an ID. However, if you run

blkid

 on your pool devices, you'll see they all have the same UUID. This is normal. This is how btrfs identifies devices belonging to a pool.

 

The thing is, it's the device ID in the btrfs metadata that is the issue. The btrfs driver should *never* see duplicate device IDs. When you run a btrfs device scan, it looks for these devices. This is how btrfs can work with pool devices without you needing to specify each device when it mounts, instead, mounting any of the devices in a pool will mount all of them. However, when duplicates are found, this warning is triggered on mount, since the btrfs driver talks to devices by their device ID.

 

It's like trying to mount two XFS devices with the same UUID, the kernel will get angry. Only in this case, btrfs has its own IDs. It's the same thing.

 

On old kernels, this would almost certainly cause corruption. While I think there has been effort to harden against this in later kernel versions, like is being used on unraid today, if the kernel ever did get confused and commits a write to the /dev/sdx device, while you may not have fs corruption in this case, it could cause the parity to go out of sync. Or it could cause corruption if there's any race condition or write barrier issues between the MDx devices and the SDx devices.

Honestly, I think it's just best to avoid that potential issue. The UD plugin must be doing a device scan, triggering the duplicate device detection.

 

This is probably intentional by the plugin, since of course, if you had a btrfs array that wasn't assigned as a pool in Unraid, it would still be possible to easily mount it without returning any errors (otherwise btrfs will return an error if the kernel hasn't "scanned" all the devices).

 

So as far as I can tell, since it would probably be undesirable to remove this feature from the plugin, you should either choose to only use Btrfs in pools only if you wanna use UD, or don't use the UD plugin.

Edited February 5, 2022 by JSE
Misspoke about UUID, derp. Clarify UUIDs.

[dlandon]

On 2/3/2022 at 1:53 AM, JorgeB said:

Not a big deal, it's just a warning, it appears to be caused by the UD plugin, reboot in safe mode if there's no more warning you can post in the UD support thread to see if anything can be done about it.

@JorgeB UD is using the following command to find which UD disks are in a pool,  It also tells me which one is the primary member.

/sbin/btrfs fi show mountpoint

 

Based on what I see in this post, it is causing these btrfs warnings?  I guess I'm very confused.  Btrfs pools have the same UUID, so what's this all about?

 

Is there another way to determine the btrfs devices in a pool?

[JorgeB]

41 minutes ago, dlandon said:

Based on what I see in this post, it is causing these btrfs warnings? 

I don't think that's problem, looks like it's triggering another device scan, and it will detect duplicates if there are btrfs file systens in the array, since mdX is a duplicate of sdX1, I believe that in this case these warnings can be ignored but to be honest not sure if they can result in issues, it would be better to avoid them, older UD releases didn't do this.

[dlandon]

1 hour ago, JorgeB said:

I don't think that's problem, looks like it's triggering another device scan, and it will detect duplicates if there are btrfs file systens in the array, since mdX is a duplicate of sdX1, I believe that in this case these warnings can be ignored but to be honest not sure if they can result in issues, it would be better to avoid them, older UD releases didn't do this.

This is a recent addition to UD to support pool devices better.  I think I see an issue with the way I am doing this and I'll make a change.

[SimonF]

Hi @dlandon

 

fi show -m will show just mounted and drives, without -m I noticed this spins up all drives even if not BTRFS.

 

Do you know the UUID?

 

root@unraid:~# btrfs fi show -m 
Label: none  uuid: 58ba2066-d3cb-4cbd-bff1-d495b96a65e8
        Total devices 5 FS bytes used 1.73TiB
        devid    2 size 238.47GiB used 15.00GiB path /dev/sdd1
        devid    4 size 931.51GiB used 573.03GiB path /dev/sde1
        devid    5 size 465.76GiB used 105.00GiB path /dev/sdf1
        devid    6 size 1.82TiB used 1.47TiB path /dev/sdk1
        devid    7 size 1.82TiB used 1.47TiB path /dev/sdj1

Label: none  uuid: 8e2e2002-103b-435d-989d-72914534959e
        Total devices 1 FS bytes used 43.99GiB
        devid    1 size 50.00GiB used 49.50GiB path /dev/loop2

Label: none  uuid: 730c3855-c3d0-4d31-aabf-e33b715990cf
        Total devices 1 FS bytes used 1.91MiB
        devid    1 size 1.00GiB used 126.38MiB path /dev/loop3

root@unraid:~# btrfs fi show -m 58ba2066-d3cb-4cbd-bff1-d495b96a65e8
Label: none  uuid: 58ba2066-d3cb-4cbd-bff1-d495b96a65e8
        Total devices 5 FS bytes used 1.73TiB
        devid    2 size 238.47GiB used 15.00GiB path /dev/sdd1
        devid    4 size 931.51GiB used 573.03GiB path /dev/sde1
        devid    5 size 465.76GiB used 105.00GiB path /dev/sdf1
        devid    6 size 1.82TiB used 1.47TiB path /dev/sdk1
        devid    7 size 1.82TiB used 1.47TiB path /dev/sdj1

root@unraid:~# 

 

[dlandon]

29 minutes ago, SimonF said:

fi show -m will show just mounted and drives, without -m I noticed this spins up all drives even if not BTRFS.

 

Do you know the UUID?

I use btrfs fi show on only a single mountpoint, so this should not be an issue.

[dlandon]

I'm working on cleaning up the way this is handled in UD. The problem is that I do the btrfs check on every refresh of the UD screen.  That's every 3 seconds!  I've changed to only look for the pool status when it is mounted.  This was some poor coding on my part.  Had a senior moment I supporse.

 

Once I have it tested and complete some other issues I'm working on, I'll issue an update.  Probably today.

[dlandon]

Update to the latest release of UD and let me know if it solved the problem.

[comet424]

no didnt solve it  for me.. updated  then re booted the 2 servers.. but those duplicate errors back

i used btfrs for all my arrays and hot swaps..   i know my other server  needs a reboot  been going 30 days..  it has a couple ntfs  hot swaps  but i use all btrfs  i kept it to 1 kind

as basiclly i read btrfs  supposed to be better then xfs

server with 1 array drive (mitchsserver)  has 1 Duplicate

server with 8 array drives (mitchflix) has 8 duplicates

 

right now these 2 servers dont use a parity drive.. but  i do use it on my main server that is basiclly a backup server to my sisters 2 servers

 

mitchsserver-diagnostics-20220204-1313.zip mitchflix-diagnostics-20220204-1313.zip

Edited February 4, 2022 by comet424

[dlandon]

1 hour ago, comet424 said:

no didnt solve it  for me

Do you have Pool devices in UD?

 

Try removing the UD plugin and see if it stops the messages.

[comet424]

for the old plugin  you had me remove the plugin on both servers and i rebooted  in the other topic.. and it removed the Duplicate issue..  but then when i reinstalled the Plugin and rebooted it was back

 

as for Pool Devices in UD  i not sure what you mean

 

i have Array  Drives, and I have   Pool Device Drives, which is for Downloading and VMs and Unraid Files appdatas stuff...   

 

i dont have a Pool Device UD....  as i dont know how you could make a Pool Device be a Pool Device and a UD...

 

but ill re uninstall Plugins   and do a reboot and check

 

 

[comet424]

so i did it for the 1 server  that has the 8 dup warnings

 

1..  uninstalled the pluggin..   rebooted computer.....  and dup errors didnt show

2.. then i reinstalled  UD plugin and UD Plus Plugin....    i checked the logs no issue

3.. then i rebooted the Server  and the dup errors are back  in the system log

 

so it seems it happens at bootup???   

 

[dlandon]

It happens at bootup when UD is trying to auto mount devices.  I'm not sure why that is happening, because UD s not doing anything since you don't have any auto mounted devices.

 

There is one thing I'm doing just before the auto mount that causes some udev action.  I'm going to do a little testing and then issue an update to let you try again.

[dlandon]

3 hours ago, comet424 said:

so i did it for the 1 server  that has the 8 dup warnings

Update UD and try once more to see if the messages stop.

[comet424]

didnt work still  showing up

and 

2022.02.04a   

is installed

 

should i try uninstalling Plus and see if that causes it?

 

Feb  4 19:42:25 MitchFlix kernel: Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA
Feb  4 19:42:25 MitchFlix kernel: ACPI: Early table checksum verification disabled
Feb  4 19:42:25 MitchFlix kernel: floppy0: no floppy controllers found
Feb  4 19:42:25 MitchFlix kernel: random: 6 urandom warning(s) missed due to ratelimiting
Feb  4 19:42:35 MitchFlix rpc.statd[1927]: Failed to read /var/lib/nfs/state: Success
Feb  4 19:42:38 MitchFlix kernel: ACPI Warning: SystemIO range 0x0000000000000295-0x0000000000000296 conflicts with OpRegion 0x0000000000000290-0x0000000000000299 (\AMW0.SHWM) (20200925/utaddress-204)
Feb  4 19:42:39 MitchFlix root: Interface "tunl0" added. Warning: no bandwidth limit has been set.
Feb  4 19:42:39 MitchFlix root: Interface "gre0" added. Warning: no bandwidth limit has been set.
Feb  4 19:42:39 MitchFlix root: Interface "gretap0" added. Warning: no bandwidth limit has been set.
Feb  4 19:42:39 MitchFlix root: Interface "erspan0" added. Warning: no bandwidth limit has been set.
Feb  4 19:42:39 MitchFlix root: Interface "ip_vti0" added. Warning: no bandwidth limit has been set.
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdg1 devid 1 generation 231191 scanned by udevd (7680)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdb1 devid 1 generation 1589 scanned by udevd (7676)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdf1 devid 1 generation 297322 scanned by udevd (7683)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdd1 devid 1 generation 25146375 scanned by udevd (7685)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdc1 devid 1 generation 23808 scanned by udevd (7682)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sde1 devid 1 generation 662416 scanned by udevd (7674)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdh1 devid 1 generation 822278 scanned by udevd (7678)
Feb  4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdi1 devid 1 generation 12870 scanned by udevd (7658)

 

Edited February 5, 2022 by comet424

[dlandon]

1 hour ago, comet424 said:

should i try uninstalling Plus and see if that causes it?

You can try that, but all UD+ does is install some packages.

 

I'm at a loss why this is happening.  You don't have any UD disks, so UD isn't doing anything.

 

Post your latest diagnostics,  I want to see where in the startup sequence it is happening.

[dlandon]

14 hours ago, JorgeB said:

older UD releases didn't do this

When do you think this started?  I can look back at my changes and see it there is anything that might cause this.

[comet424]

i didnt have a problem  around christmas  if that helps  i didnt notice an error as i setup my sisters servers back then i re built her servers and there wasnt errors then so before dec 24th    well there is always errors/warrnings  but not the duplicate..  i havent been checking...  i only checked when i mentioned my Tower(main server)  lost the UD drives  it was running  those days  and i mentioned like 25 days ago  they were gone.. they were physically there  but they disappeared..  

 

and the uninstalling  Plus didnt work..  and here is the current file...  i dont dare touch my tower as its working but it has a Parity   and its been idling  24 25 days  and that other guy said it could bugger up parity drive and array..   so i dont wanna reboot it..

as i use your UD plugin  its a great feature and i appreciate this program you wrote 🙂

 

and i cant see if the error happened with my Main Tower server.. due to it had alot of other errors  i forget which one..  and for some reason  it has no more errors or warnings  its actually just black..  but i know  u never have a perfect boot 

 

mitchflix-diagnostics-20220204-2130.zip

Edited February 5, 2022 by comet424

[comet424]

i dont think i noticed the issue  when you changed the the plugin when i mention the drives disappeared on my main server..  they were physically there but UD  showed they werent even plugged in..  i think  it didnt have the error then but i cant 100% say

 

if there is a way i can install  the older plugins version like one from xmas  and the one they day you said install it.. i could try that  and see if it happens    as i only doing it to my sisters 1 server..  as i wanna keep the other running and i dont wanna test it on my tower main server..  but if there is a few vers  i can copy to the flash  and reboot and test

Edited February 5, 2022 by comet424

[dlandon]

I believe this is coming from udev events in UD.  I've released a new version just now that has the udev rules reworked and I've added some debug logging to track the udev and hotplug activity.  What this will do is let me see what event(s) in UD are happening just before the btrfs log messages occur.

 

Please update to the latest version, and then go to the UD settings and set the debug level to '1'.  Do whatever you need to do to make the log messages happen and post the diagnostics zip file.

 

8 hours ago, comet424 said:

if there is a way i can install  the older plugins

This is a very difficult thing to do because I'd have to track down the differences between versions to determine what changed.

 

I can't reproduce the issue.  I've tried to force it with no luck, so the next best thing is to get enough information in the log for me to track this down.