comet424 Posted February 3, 2022 Share Posted February 3, 2022 (edited) as i remote servicing my sisters servers.. i noticed duplicate device.. dont know how or fix... as i was trying to figure why docker was slow and plex server had the Gray Circle ? but then when you refresh docker it fast and then icon is back.. im pretty sure Plex slows down docker like its slow.. as i mentioned in past docker slowness issues i found topics i wrote but here is the error Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdg1 devid 1 generation 231168 scanned by udevd (7620) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdb1 devid 1 generation 1573 scanned by udevd (7617) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdf1 devid 1 generation 297299 scanned by udevd (7619) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdc1 devid 1 generation 23784 scanned by udevd (7622) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdd1 devid 1 generation 25146352 scanned by udevd (7615) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sde1 devid 1 generation 662393 scanned by udevd (7616) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdh1 devid 1 generation 822261 scanned by udevd (7614) Feb 2 19:02:00 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdi1 devid 1 generation 12853 scanned by udevd (7625) how i fix that? a reboot didnt fix it.. and fix common issues didnt find no errors other then no root password i also checked my sisters other server.. and there is 1 warrently with the duplicate device... my main server i cant check shows no errors or warrnings mitchflix-diagnostics-20220202-1913.zip Edited February 3, 2022 by comet424 Quote Link to comment
JorgeB Posted February 3, 2022 Share Posted February 3, 2022 Not a big deal, it's just a warning, it appears to be caused by the UD plugin, reboot in safe mode if there's no more warning you can post in the UD support thread to see if anything can be done about it. Quote Link to comment
comet424 Posted February 3, 2022 Author Share Posted February 3, 2022 ah ok i tried googling how to boot into safe mode.... as i cant access it locallly i work on my sisters server remotely.. under the flash it has some features.. and if i boot into safe mode do i loose network access ? Quote Link to comment
JorgeB Posted February 3, 2022 Share Posted February 3, 2022 7 minutes ago, comet424 said: and if i boot into safe mode do i loose network access ? Nope, just all plugins aren't loaded, alternatively you can uninstall just the UD plugin and reboot. Quote Link to comment
comet424 Posted February 3, 2022 Author Share Posted February 3, 2022 ah ok.. the safe mode didnt have the issue i rebooted back again normal... it didnt pop up the several duplicates just 1 at the moment.. ill post in there i appreciate the help 🙂 Quote Link to comment
JSE Posted February 3, 2022 Share Posted February 3, 2022 (edited) To add to this, it's best to remove the plugin until it is potentially addressed, as is warned here: https://btrfs.wiki.kernel.org/index.php/Gotchas#Block-level_copies_of_devices While these two "copies" are in fact the same thing, the thing with unraid is the /dev/mdx devices correspond to the /dev/sdx devices. So if the kernel sees both all of a sudden, it just thinks they're unique devices. In normal filesystems, this is fine, but Btrfs isn't "normal", so it gets angry because there are two devices with the same Device ID (all btrfs devices in a pool share the same UUID). A btrfs pool stores unique devices in its own metadata with numerical device IDs which are incremental. If you have a pool, consider running the command: btrfs device usage /mnt/cache It will list the devices in the pool with an ID. However, if you run blkid on your pool devices, you'll see they all have the same UUID. This is normal. This is how btrfs identifies devices belonging to a pool. The thing is, it's the device ID in the btrfs metadata that is the issue. The btrfs driver should *never* see duplicate device IDs. When you run a btrfs device scan, it looks for these devices. This is how btrfs can work with pool devices without you needing to specify each device when it mounts, instead, mounting any of the devices in a pool will mount all of them. However, when duplicates are found, this warning is triggered on mount, since the btrfs driver talks to devices by their device ID. It's like trying to mount two XFS devices with the same UUID, the kernel will get angry. Only in this case, btrfs has its own IDs. It's the same thing. On old kernels, this would almost certainly cause corruption. While I think there has been effort to harden against this in later kernel versions, like is being used on unraid today, if the kernel ever did get confused and commits a write to the /dev/sdx device, while you may not have fs corruption in this case, it could cause the parity to go out of sync. Or it could cause corruption if there's any race condition or write barrier issues between the MDx devices and the SDx devices. Honestly, I think it's just best to avoid that potential issue. The UD plugin must be doing a device scan, triggering the duplicate device detection. This is probably intentional by the plugin, since of course, if you had a btrfs array that wasn't assigned as a pool in Unraid, it would still be possible to easily mount it without returning any errors (otherwise btrfs will return an error if the kernel hasn't "scanned" all the devices). So as far as I can tell, since it would probably be undesirable to remove this feature from the plugin, you should either choose to only use Btrfs in pools only if you wanna use UD, or don't use the UD plugin. Edited February 5, 2022 by JSE Misspoke about UUID, derp. Clarify UUIDs. Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 On 2/3/2022 at 1:53 AM, JorgeB said: Not a big deal, it's just a warning, it appears to be caused by the UD plugin, reboot in safe mode if there's no more warning you can post in the UD support thread to see if anything can be done about it. @JorgeB UD is using the following command to find which UD disks are in a pool, It also tells me which one is the primary member. /sbin/btrfs fi show mountpoint Based on what I see in this post, it is causing these btrfs warnings? I guess I'm very confused. Btrfs pools have the same UUID, so what's this all about? Is there another way to determine the btrfs devices in a pool? Quote Link to comment
JorgeB Posted February 4, 2022 Share Posted February 4, 2022 41 minutes ago, dlandon said: Based on what I see in this post, it is causing these btrfs warnings? I don't think that's problem, looks like it's triggering another device scan, and it will detect duplicates if there are btrfs file systens in the array, since mdX is a duplicate of sdX1, I believe that in this case these warnings can be ignored but to be honest not sure if they can result in issues, it would be better to avoid them, older UD releases didn't do this. Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 1 hour ago, JorgeB said: I don't think that's problem, looks like it's triggering another device scan, and it will detect duplicates if there are btrfs file systens in the array, since mdX is a duplicate of sdX1, I believe that in this case these warnings can be ignored but to be honest not sure if they can result in issues, it would be better to avoid them, older UD releases didn't do this. This is a recent addition to UD to support pool devices better. I think I see an issue with the way I am doing this and I'll make a change. 1 Quote Link to comment
SimonF Posted February 4, 2022 Share Posted February 4, 2022 Hi @dlandon fi show -m will show just mounted and drives, without -m I noticed this spins up all drives even if not BTRFS. Do you know the UUID? root@unraid:~# btrfs fi show -m Label: none uuid: 58ba2066-d3cb-4cbd-bff1-d495b96a65e8 Total devices 5 FS bytes used 1.73TiB devid 2 size 238.47GiB used 15.00GiB path /dev/sdd1 devid 4 size 931.51GiB used 573.03GiB path /dev/sde1 devid 5 size 465.76GiB used 105.00GiB path /dev/sdf1 devid 6 size 1.82TiB used 1.47TiB path /dev/sdk1 devid 7 size 1.82TiB used 1.47TiB path /dev/sdj1 Label: none uuid: 8e2e2002-103b-435d-989d-72914534959e Total devices 1 FS bytes used 43.99GiB devid 1 size 50.00GiB used 49.50GiB path /dev/loop2 Label: none uuid: 730c3855-c3d0-4d31-aabf-e33b715990cf Total devices 1 FS bytes used 1.91MiB devid 1 size 1.00GiB used 126.38MiB path /dev/loop3 root@unraid:~# btrfs fi show -m 58ba2066-d3cb-4cbd-bff1-d495b96a65e8 Label: none uuid: 58ba2066-d3cb-4cbd-bff1-d495b96a65e8 Total devices 5 FS bytes used 1.73TiB devid 2 size 238.47GiB used 15.00GiB path /dev/sdd1 devid 4 size 931.51GiB used 573.03GiB path /dev/sde1 devid 5 size 465.76GiB used 105.00GiB path /dev/sdf1 devid 6 size 1.82TiB used 1.47TiB path /dev/sdk1 devid 7 size 1.82TiB used 1.47TiB path /dev/sdj1 root@unraid:~# Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 29 minutes ago, SimonF said: fi show -m will show just mounted and drives, without -m I noticed this spins up all drives even if not BTRFS. Do you know the UUID? I use btrfs fi show on only a single mountpoint, so this should not be an issue. Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 I'm working on cleaning up the way this is handled in UD. The problem is that I do the btrfs check on every refresh of the UD screen. That's every 3 seconds! I've changed to only look for the pool status when it is mounted. This was some poor coding on my part. Had a senior moment I supporse. Once I have it tested and complete some other issues I'm working on, I'll issue an update. Probably today. Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 Update to the latest release of UD and let me know if it solved the problem. Quote Link to comment
comet424 Posted February 4, 2022 Author Share Posted February 4, 2022 (edited) no didnt solve it for me.. updated then re booted the 2 servers.. but those duplicate errors back i used btfrs for all my arrays and hot swaps.. i know my other server needs a reboot been going 30 days.. it has a couple ntfs hot swaps but i use all btrfs i kept it to 1 kind as basiclly i read btrfs supposed to be better then xfs server with 1 array drive (mitchsserver) has 1 Duplicate server with 8 array drives (mitchflix) has 8 duplicates right now these 2 servers dont use a parity drive.. but i do use it on my main server that is basiclly a backup server to my sisters 2 servers mitchsserver-diagnostics-20220204-1313.zip mitchflix-diagnostics-20220204-1313.zip Edited February 4, 2022 by comet424 Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 1 hour ago, comet424 said: no didnt solve it for me Do you have Pool devices in UD? Try removing the UD plugin and see if it stops the messages. Quote Link to comment
comet424 Posted February 4, 2022 Author Share Posted February 4, 2022 for the old plugin you had me remove the plugin on both servers and i rebooted in the other topic.. and it removed the Duplicate issue.. but then when i reinstalled the Plugin and rebooted it was back as for Pool Devices in UD i not sure what you mean i have Array Drives, and I have Pool Device Drives, which is for Downloading and VMs and Unraid Files appdatas stuff... i dont have a Pool Device UD.... as i dont know how you could make a Pool Device be a Pool Device and a UD... but ill re uninstall Plugins and do a reboot and check Quote Link to comment
comet424 Posted February 4, 2022 Author Share Posted February 4, 2022 so i did it for the 1 server that has the 8 dup warnings 1.. uninstalled the pluggin.. rebooted computer..... and dup errors didnt show 2.. then i reinstalled UD plugin and UD Plus Plugin.... i checked the logs no issue 3.. then i rebooted the Server and the dup errors are back in the system log so it seems it happens at bootup??? Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 It happens at bootup when UD is trying to auto mount devices. I'm not sure why that is happening, because UD s not doing anything since you don't have any auto mounted devices. There is one thing I'm doing just before the auto mount that causes some udev action. I'm going to do a little testing and then issue an update to let you try again. Quote Link to comment
dlandon Posted February 4, 2022 Share Posted February 4, 2022 3 hours ago, comet424 said: so i did it for the 1 server that has the 8 dup warnings Update UD and try once more to see if the messages stop. Quote Link to comment
comet424 Posted February 5, 2022 Author Share Posted February 5, 2022 (edited) didnt work still showing up and 2022.02.04a is installed should i try uninstalling Plus and see if that causes it? Feb 4 19:42:25 MitchFlix kernel: Warning: PCIe ACS overrides enabled; This may allow non-IOMMU protected peer-to-peer DMA Feb 4 19:42:25 MitchFlix kernel: ACPI: Early table checksum verification disabled Feb 4 19:42:25 MitchFlix kernel: floppy0: no floppy controllers found Feb 4 19:42:25 MitchFlix kernel: random: 6 urandom warning(s) missed due to ratelimiting Feb 4 19:42:35 MitchFlix rpc.statd[1927]: Failed to read /var/lib/nfs/state: Success Feb 4 19:42:38 MitchFlix kernel: ACPI Warning: SystemIO range 0x0000000000000295-0x0000000000000296 conflicts with OpRegion 0x0000000000000290-0x0000000000000299 (\AMW0.SHWM) (20200925/utaddress-204) Feb 4 19:42:39 MitchFlix root: Interface "tunl0" added. Warning: no bandwidth limit has been set. Feb 4 19:42:39 MitchFlix root: Interface "gre0" added. Warning: no bandwidth limit has been set. Feb 4 19:42:39 MitchFlix root: Interface "gretap0" added. Warning: no bandwidth limit has been set. Feb 4 19:42:39 MitchFlix root: Interface "erspan0" added. Warning: no bandwidth limit has been set. Feb 4 19:42:39 MitchFlix root: Interface "ip_vti0" added. Warning: no bandwidth limit has been set. Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdg1 devid 1 generation 231191 scanned by udevd (7680) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdb1 devid 1 generation 1589 scanned by udevd (7676) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdf1 devid 1 generation 297322 scanned by udevd (7683) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdd1 devid 1 generation 25146375 scanned by udevd (7685) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdc1 devid 1 generation 23808 scanned by udevd (7682) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sde1 devid 1 generation 662416 scanned by udevd (7674) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdh1 devid 1 generation 822278 scanned by udevd (7678) Feb 4 19:43:22 MitchFlix kernel: BTRFS warning (device <unknown>): duplicate device /dev/sdi1 devid 1 generation 12870 scanned by udevd (7658) Edited February 5, 2022 by comet424 Quote Link to comment
dlandon Posted February 5, 2022 Share Posted February 5, 2022 1 hour ago, comet424 said: should i try uninstalling Plus and see if that causes it? You can try that, but all UD+ does is install some packages. I'm at a loss why this is happening. You don't have any UD disks, so UD isn't doing anything. Post your latest diagnostics, I want to see where in the startup sequence it is happening. Quote Link to comment
dlandon Posted February 5, 2022 Share Posted February 5, 2022 14 hours ago, JorgeB said: older UD releases didn't do this When do you think this started? I can look back at my changes and see it there is anything that might cause this. Quote Link to comment
comet424 Posted February 5, 2022 Author Share Posted February 5, 2022 (edited) i didnt have a problem around christmas if that helps i didnt notice an error as i setup my sisters servers back then i re built her servers and there wasnt errors then so before dec 24th well there is always errors/warrnings but not the duplicate.. i havent been checking... i only checked when i mentioned my Tower(main server) lost the UD drives it was running those days and i mentioned like 25 days ago they were gone.. they were physically there but they disappeared.. and the uninstalling Plus didnt work.. and here is the current file... i dont dare touch my tower as its working but it has a Parity and its been idling 24 25 days and that other guy said it could bugger up parity drive and array.. so i dont wanna reboot it.. as i use your UD plugin its a great feature and i appreciate this program you wrote 🙂 and i cant see if the error happened with my Main Tower server.. due to it had alot of other errors i forget which one.. and for some reason it has no more errors or warnings its actually just black.. but i know u never have a perfect boot mitchflix-diagnostics-20220204-2130.zip Edited February 5, 2022 by comet424 Quote Link to comment
comet424 Posted February 5, 2022 Author Share Posted February 5, 2022 (edited) i dont think i noticed the issue when you changed the the plugin when i mention the drives disappeared on my main server.. they were physically there but UD showed they werent even plugged in.. i think it didnt have the error then but i cant 100% say if there is a way i can install the older plugins version like one from xmas and the one they day you said install it.. i could try that and see if it happens as i only doing it to my sisters 1 server.. as i wanna keep the other running and i dont wanna test it on my tower main server.. but if there is a few vers i can copy to the flash and reboot and test Edited February 5, 2022 by comet424 Quote Link to comment
dlandon Posted February 5, 2022 Share Posted February 5, 2022 I believe this is coming from udev events in UD. I've released a new version just now that has the udev rules reworked and I've added some debug logging to track the udev and hotplug activity. What this will do is let me see what event(s) in UD are happening just before the btrfs log messages occur. Please update to the latest version, and then go to the UD settings and set the debug level to '1'. Do whatever you need to do to make the log messages happen and post the diagnostics zip file. 8 hours ago, comet424 said: if there is a way i can install the older plugins This is a very difficult thing to do because I'd have to track down the differences between versions to determine what changed. I can't reproduce the issue. I've tried to force it with no luck, so the next best thing is to get enough information in the log for me to track this down. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.