phyzical

no the smart short self test failure is post reboot so to the system everything is "working" the newer one runs fine but the older one, it doesnt even go to a % just returns Completed: failed segments ill give the syslinux line a go then. Have you ever seen it cause a non faulty one go suspended? incase it really is just a close to death ssd? the cache pool nvme drive itself reports in the attributes - Available spare 100% - Available spare threshold 10% - Percentage used 107% - SSD endurance remaining -7 %

Hey so in the last 2 weeks i have noticed im getting a new weird issue. I have two cache drives as two separate pools "cache" and "generic" i use cache for mostly appdata and generic for downloads, transcodes ect. ive started getting " Pool has encountered an uncorrectable I/O failure and has been suspended." a reboot will bring it back but it can last a few days and sometimes only a few hours. 90% of the time it happens to the cache drive (nvme0n1) But i have seen it once affect my other one (nvme1n1) Does anyone have any idea what i can do to troubleshoot this? the cache nvme is a drive that has done alot in the last two years (~1.5PB read/write) and i just assumed it was on its way out but then it happend to the newer one which has only seen ~70TB R/W so far which makes me question if its something else. i do not believe it was an unraid update (i am running 7.1.4 atm but it was happening on 7.1.3 also) i have attached a diagnotics zip when the problem occurred and i can see logs ``` Jul 12 18:20:58 MEDIABOX kernel: pcieport 0000:00:06.0: AER: Correctable error message received from 0000:00:06.0 Jul 12 18:20:58 MEDIABOX kernel: pcieport 0000:00:06.0: PCIe Bus Error: severity=Correctable, type=Physical Layer, (Receiver ID) Jul 12 18:20:58 MEDIABOX kernel: pcieport 0000:00:06.0: device [8086:a74d] error status/mask=00000001/00002000 Jul 12 18:20:58 MEDIABOX kernel: pcieport 0000:00:06.0: [ 0] RxErr (First) Jul 12 18:21:29 MEDIABOX kernel: nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0x10 Jul 12 18:21:29 MEDIABOX kernel: nvme nvme0: Does your device have a faulty power saving mode enabled? Jul 12 18:21:29 MEDIABOX kernel: nvme nvme0: Try "nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off" and report a bug Jul 12 18:21:29 MEDIABOX kernel: nvme 0000:02:00.0: enabling device (0000 -> 0002) Jul 12 18:21:29 MEDIABOX kernel: nvme nvme0: Disabling device after reset failure: -19 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 559389704 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 42888448 op 0x1:(WRITE) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 160791304 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 1257939184 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 1780483096 op 0x0:(READ) flags 0x0 phys_seg 3 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=286406479872 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=21957836800 size=131072 flags=3162240 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=82324099072 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 1628102984 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=911606296576 size=45056 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=644063813632 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: I/O error, dev nvme0n1, sector 256392024 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 0 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=833587679232 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=781692096512 size=45056 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=131271667712 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=286409359360 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=874528718848 size=12288 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=911606173696 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=25036091392 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=911606902784 size=45056 flags=2148533376 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=911602991104 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=911606976512 size=16384 flags=2148533376 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=286409367552 size=45056 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=874528731136 size=12288 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=911606784000 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=1 offset=164190846976 size=4096 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: zio pool=cache vdev=/dev/nvme0n1p1 error=5 type=2 offset=874528743424 size=24576 flags=3145856 Jul 12 18:21:29 MEDIABOX kernel: WARNING: Pool 'cache' has encountered an uncorrectable I/O failure and has been suspended. ### [PREVIOUS LINE REPEATED 27 TIMES] ### Jul 12 18:21:56 MEDIABOX kernel: traps: vwmetrics[15027] general protection fault ip:15323630f50f sp:7ffecb3506c0 error:0 in libc.so.6[2650f,15323630f000+155000] Jul 12 18:21:58 MEDIABOX kernel: WARNING: Pool 'cache' has encountered an uncorrectable I/O failure and has been suspended. ``` i ran the general repair scrub ect, after reboot it doesn't say there is any data errors. but the drive does insta fail the quick smart test i did see Jul 12 18:21:29 MEDIABOX kernel: nvme nvme0: Try "nvme_core.default_ps_max_latency_us=0 pcie_aspm=off pcie_port_pm=off" and report a bug But i find it strange after 2 years of using this it would be a power saving issue right? cc @JorgeB (as i know you have a wealth of info on this based on other posts ive come across) Thanks! mediabox-diagnostics-20250712-1830.zip

thanks for taking the time to update the plugin (as someone who has made some very basic plugins i understand how frustrating the back end is to play with) i just blatted and redid my folders works like a charm ❤️

@ChirpyTurnipyou are not alone... i have a i7-14700K too was crashing daily multiple times a day every now and then id get a few days. Whent he bios updates came out it was okay but i would still find the system had crashed a couple times a week.. But now for some reason in the last few days ive been getting all kinds of segfaults. random containers crashing and rebooting. Ive even have had my jellyfin db just get corrupted. I have tried swapping around my ram and im still seeing segaults, no matter which sticks im using. im down to two things also, my MB or the cpu. Ive even tried stressing the hell out of the system with like https://github.com/Mysticial/y-cruncher ran 100% cpu cores running 100c using 90gb of ram for 30 minutes, while maintaining all my services and it was fine..... i have no idea what is causing this and it is driving me insaneeeeeee

Hey i am in the process of hooking all my services up to authelia. I was just wondering if there was anything for the unraid ui specifically for user management?

i feel like this should be solved by just setting some sort of timeout on nchan's? like if tab is in the background for more than x minutes unsubscribe all nchans? maybe some indicator on the dashboard saying all nchan's in hibernation, "click to force a refresh to resubscribe"? it deffs isnt that simple though, what if they are updating all docker containers that might take longer than x... idk not as easy as you think to solve. below is my userscript to run each boot to bandaid the issue #!/bin/bash sed -i "s/client_body_buffer_size 32k;/client_body_buffer_size 20m;/" "/etc/nginx/nginx.conf" if [ "$(cat /etc/nginx/nginx.conf | grep "nchan_shared_memory_size" || echo "")" = "" ]; then sed -i "/include \/etc\/nginx\/conf.d\/servers.conf;/i nchan_shared_memory_size 512M;" "/etc/nginx/nginx.conf" fi /etc/rc.d/rc.nginx restart i did notice tools like dynamix plugins talk ALOT so i wonder if we should be telling them to chill out a bit? or atleast let the user change the period at which say temp readings are communicated, so instead of spamming every few seconds i can just say hey once a minute? i feel like this issue may get worse with all the graph stuff they want to do in 7

can confirm since the latest wave of bios updates it hasnt died once

my workaround is to do this in an array startup userscript just replace `YOURDOMAINHERE` and you should be good #!/bin/bash # Path to the nginx.conf file NGINX_CONF="/etc/nginx/nginx.conf" # New Content-Security-Policy directive NEW_CSP="add_header Content-Security-Policy \"frame-ancestors 'self' https://connect.myunraid.net/ https://*.YOURDOMAINHERE\";" # Use sed to replace the line sed -i.bak "/add_header Content-Security-Policy/c\\ $NEW_CSP " $NGINX_CONF # Restart Nginx to apply changes nginx -s reload echo "Content-Security-Policy directive updated and Nginx reloaded."

neither the nvme drive itself, just based on what `default_ps_max_latency_us` does, from waht i could see is it disables power management of the drive

thanks for the tip ill give it a go. Out of curiosity any idea why it goes into permanent hibernation? does it just get stuck in "power saving mode" after the first reboot due to the interactions of unraid/ syslinux stuff?

hmm actually i think just a full power down restores the drive not the bios reset, but soft reboots persists the issue

okay so it was up for a day, i went to reboot and the system locked up on array shutting down. hard reset and once i loaded back in unraid was reporting the bind error again nvme stopped showing up until i reset my bios again attached diagnostics backupbox-diagnostics-20240902-1322.zip

lol.. aaand it all just works the second time around.. i must have just configured something wrong the first time Thanks again for the rubber ducking @JorgeB

Ah sorry to waste your time, honestly not sure whats happened. i tried updating my bios out of a whim and it reset my bios as part of it and it began reporting again its like unraid affected the bios config not sure. now i cant boot unraid as it only shows a usb partition not the uefi boot, haha time to find what new setting to disable to get that back.. i post a diagnostic if it happens again when i get it back up.

Hey, Was just trying to setup a windows install that could be both booted via unraid and single booted. It was an nvme drive with windows installed natively. I went into the system devices tool and choose to bind the nvme drive on boot, then rebooted. It showed up fine in the vm template i choose to attach it to a new vm and booted it with vnc. I then saw it loading the windows installer and assumed i did something wrong so tried to stop the vm, it didnt stop so i tried force killing the vm and my whole unraid became unresponsive. So i decided to hard reboot and then when i loaded the dashboard up i was getting VFIO binding errors saying the device couldnt be found? sure enough its not listed in system devices anymore and its not even showing up the in bios, i manually removed it from the binding and rebooted but its no change. Have i just killed the nvme drive? I don't have any logs from when it all went down but i have attached the latest boot any help would be appreciated backupbox-syslog-20240901-0704.zip

@Espressomatic Hey ended up needing to add more labels for other things, so i had to motivation to touch it up again. it now has * a better select field * ability to add defaults to be prefilled each time via the settings page * ability to provide ${CONTAINER_NAME} for auto replacement The only thing i didnt do was prefilling with existing labels based on container selections as i felt it might just ends up being more annoying than useful

@Espressomaticthanks for the feedback, yeah i kinda just got it working for a simple case, did what i needed it to. ive got to be honest the unraid under the hood isn't the nicest to play with, i spent alot longer trying to do the simple things that i lost alot of motivation to make it fancier But im sure ill get some time at some point to make it more magic like your thinking If i do * yeah maybe a config that defines a list of variables that can be referenced * list of defaults * prefill with existing containers for selection/modification

i created this plugin to solve this https://github.com/phyzical/docker.labelInjector

i have attempted to solve this with a plugin https://github.com/phyzical/docker.versions

@Zacaroniithanks for the links, yeah idk if ill bother with rma either. just know that if they start everyone will and they will do everything they can to delay and avoid. fingers crossed the next patch does the job,

@Squidsorry figured i would just hijack my own thread again, how does one go about publishing a plugin to CA? i came across this But it doesnt seem to say how i request an addition? do i just create a new plugin support thread and ping you to delegate the review? thanks!

quick update for future readers,t he latest bios update for my mobo almost removed these issues, looks like they apply similar bios changes to the above to keep it "less greedy"

well i feel silly 😃. Thanks for the reply, works as expected.

Hey! So ive finally gotten around to starting a plugin to support a changelog button. The idea will be to use opencontainer labels to attempt to pull changelog info. Anyway ive gotten a very basic plugin https://github.com/phyzical/docker.versions , i based it off of the https://github.com/scolcipitato/folder.view project . But when i try to copy it into the /usr/local/emhttp/plugins/ it breaks the existing plugins. if i try to install it in the plugin install via https://github.com/phyzical/docker.versions/blob/main/docker.versions.plg i get ``` plugin: installing: docker.versions.plg Executing hook script: pre_plugin_checks plugin: downloading: docker.versions.plg ... done Executing hook script: pre_plugin_checks plugin: XML file doesn't exist or xml parse error Executing hook script: post_plugin_checks ``` is the a doc or guide that explains the bare minimum as most ive stumbled across seem to gloss over alot of the nitty gritty details? thanks

Cool ui addition idea, i can see this being useful when people have system instability issues. did a run on 5 gb and seems to work as expected 👍