PhilBarker

Anyone having issues with fileflows since Unraid 7.0 upgrade? The container won't startup for me. It's very VERY slow but eventually just seems to hang here Only change I've made from before the upgrade to 7.0 was changing the docker settings from directory to overlay2 EDIT: after some playing around removing the PUID and PGID parameters it works fine

🤦‍♂️you sir are a legend yes of course I had - I ran it in background whilst watching syslog but never saw any output or success message so ran it again and it error'd - so thought it wasn't working (pro-tip, doing this late at night after a zfs resilver from a failed drive, probably not wise lol) when I checked today I saw the backup messages every 30 minutes in syslog, so I stopped docker, restarted, all working as it should thank you 🙂

I am I'm using this one which says it works for 6.12.13 - but it does not on mine

Can anyone tell me what I'm doing wrong with this? Running on 6.12.13 I get docker: Error: RAM-Disk Mod found incompatible files: 60ad47ed6cadcd544f92bd012d30a4f2 /etc/rc.d/rc.docker 99915e9e302b0df53cb3739951f4e33f /usr/local/emhttp/plugins/dynamix/scripts/monitor

Seeing as a few people have asked if this is stil relevant I thought I'd comment - yes, yes it is, VERY I built my plex server with 2 x 1tb SSD's, I believe they were 870 evo's Unfortunately they started throwing write errors in under a year. I know there was some weird firmware issues with them and a lot of noise about batches failing prematurely. So in the bin they went, I couldn't be bothered RMA'ing them, but I didn't have a lot of money to replace them so just went with Crucial BX500's Those are now 11months old and the first one has gone into a failing state with write errors🤦‍♂️ They are probably getting RMA'd because they're under a year old and have 80TBW, where they're advertised to cope with 300+ but anyway.... I did some digging because it feels like the cache pool with the 2 SSD's in are just constantly being written to. This guide is amazing, I setup the inotify and sat watching the file to see what was being written. Pihole was insanely noisy, constant pihole.log stuff, so I ended up pointing `/var/log` at `/tmp/docker/pihole/log` so that it would write directly to the Unraid tmp folder. As per @nitrodragon above, I then found it was constantly writing to the supervise status so pointed `/run` to `/tmp/docker/pihole/run` This left the files being written seeming to mostly be healthchecks and the unraid logs so I went through all my containers and set --health-interval=60m --log-driver syslog --log-opt syslog-address=udp://127.0.0.1:541 This dropped the writes massively, logs still work fine. I don't really care about healthchecks as I have uptimekuma monitoring all my essential services so once an hour is fine. At this point watching the writes it had dropped down to virtually nothing, just database activity in nextcloud which is kinda the only reason I used SSD's in the first place as DB's running on slow spinning sata drives is unusable. I then went on to replace the dying drive and this time bought 2 x 1tb western digital red drives. The ZFS pool drive replacement was a doddle, resilvered the new drive in 30 minutes. Then I swapped the other BX500 for a new WD red too. All back up and running, a massive drop in writes so hopefully these drives last longer. They have a much higher TBW and 5 year warranty. I don't know if it's due to the BX500's generally being utter shite or the drop in writes but the whole docker system seems 100x faster now. When I start the machine up it would take about 10 minutes for all my containers to come up, now it's 2 minutes. The UI and everything feels much snappier. I wish this guide was pinned somewhere or I'd seen it earlier in my 3 year Unraid journey, would have saved £400 in SSD's being thrashed to death writing pointless log files 😅

Thought I'd close this off. I had an electrician in yesterday so had to power off my server. When I started it back up, the array came online, dockers started, then rebooted. Next time array came up, started parity check, instantly rebooted. I decided to revisit itimpi's suggestion about PSU - wondering if I'd added so many hard drives that my PSU was now out of capacity Worked it all out to be about 560watts peak and I have a 650watt PSU (yeah, it's a bit close I'll get an upgrade soon) so should be fine Then it hit me..... it's on a UPS. A UPC 850va I did some googling and found it has a peak power output of 520w But when I moved into the new house I bought a Ubiquiti Dream Machine SE - and run wifi access points with PoE plus security cameras And I had that on the UPS too So basically the UPS was being asked for more power than it could output, and the server was restarting If anyone else finds this thread with similar issues - DEFINITELY check your PSU and general power supply

Welp.... 13 days with no issues, was sat watching a Plex film a day ago and it froze, went upstairs to see if the server had rebooted and it hadn't it was still powered on and disks clicking away. Couldn't SSH into it, web UI wouldn't load, couldn't ping it, and no video output on HDMI at all. Just completely hung up. Had to force the power off and reboot, then literally as it was starting up docker containers it restarted again for no reason, still no logs. I've updated Unraid to latest version now to see if maybe it fixes it, but I don't have high hopes

I hate leaving threads where I don't follow up with an answer... so I'll update this but not with anything useful! I took the server apart - re-seated all the memory, checked every cable connection, all the hard disk power and sata connectors - nothing seemed loose Put it back together, and it restarted within 2 hours of running then restarted again the next day So I shut it down and rebooted it with a memtest usb stick and let it sit all night running the full memtest suite - as I suspected 0 errors found. So I rebooted it back into unraid, and as of writing this, it's been up for 12 days without issue Just one of those things I guess, no idea what was causing it

Yeah I noticed the same - it seems github don't like direct linking to user images. Unfortunately at the time because it's not a widely used project it's the only logo I could find. Thanks for uploading

Thanks, I'll see if I can borrow one to swap out I've taken the whole thing apart now - checked every cable, took all the RAM out and re-seated it - will see if it happens again, it seems to be very random

I'll prefix this with saying yes I know it's not really an unraid issue, but I'm just hoping someone can point me in the right direction Have a small unraid home server, recently moved house, before the move it had uptime of months and never restarted on it's own. Since I've moved it's restarted 3 times completely randomly - there's nothing at all in syslog, just normal logs, then the startup logs, no errors or faults thrown. I can't help but think it's something to do with the move, but I've powered it down and been through reseating all the ram, checking every cable, reseated all the pcie cards etc... with nothing logged I really don't know where to start. I'll try running a long memory test overnight tonight, but are there any "common culprits" or any other tests I can do? Thanks

I thought as an experiment I'd try removing one of the apps stuck on update, removing the image, and then re-install This seemed to work except now Readarr I think updated but now updates are "not-available"

I'm having this issue also I noticed a week or so ago that a few containers permanently said "apply update" I applied the update, everything seemed to go fine, but then they would still say "apply update" it certainly seems to be updating images - and after the docker run command all completes succesfully UI still shows an update pending if I re-run the update, it pulls the same 5 updated images, reports success - and rinse and repeat this is growing now - it was originally readarr and calibre, now qbittorrentvpn and mediaelch too I've also tried installing the update from the apps page - which does the exact same thing, downloads 5 new images, reports success, still shows update required. My DNS has been set to cloudflare always, but this obviously isn't a DNS issue as it resolves, and isn't a bad checksum issue as there are updated images to pull.. Not sure if there's some other logs I could check somewhere?

So this might be completely unrelated - but I hate not following up on an issue I took a day to have a proper look at my server, there's been some other issues like randomly very slow copying files, even when rsyncing from disk to disk I discovered `top` was showing a massive IO wait.. all the time, causing high CPU usage After trial and error, closing VM's, shutting down docker containers one by one - I discovered this was caused by the file integrity plugin when I had ticked to automatically generate hash's on file creation/change. I turned that off and the IO wait instantly dropped from 60-80 to 0. The whole server feels tons faster now, disk speeds are a lot higher. I had presumed that was a good setting rather than having it churn through everything once a week on a schedule, but obviously not as it sets up a load of inotify tasks and grinds the disks to a halt. Since I've sorted that issue, and rebooted, I've had 3 days now without any problem with the USB stick or the VM's - so I'm not saying they're related but I wonder as there has been loads of people with this issue on the forum and none have actually found a cause for it (unless their USB was actually bad) maybe it's just a side effect of the server basically dying from some other reason.

Well.... this morning same issue with HASS VM - it's dead But looking at logs I didn't have the error about the USB pen and unlicensed Jan 7 06:06:33 Jarvis kernel: br0: port 2(vnet2) entered disabled state Jan 7 06:06:33 Jarvis kernel: device vnet2 left promiscuous mode Jan 7 06:06:33 Jarvis kernel: br0: port 2(vnet2) entered disabled state Jan 7 06:06:34 Jarvis usb_manager: Info: rc.usb_manager vm_action HASS stopped end - Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdd Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdf Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdc Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: nvme0n1 Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdi Jan 7 06:06:54 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sda Jan 7 06:07:02 Jarvis samba-dcerpcd[9932]: [2024/01/07 06:07:02.017455, 0] ../../lib/util/sys_popen.c:80(sys_popenv) Jan 7 06:07:02 Jarvis samba-dcerpcd[9932]: sys_popenv: fork failed: Resource temporarily unavailable Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth983191f) entered disabled state Jan 7 06:10:01 Jarvis kernel: veth8a82458: renamed from eth0 Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth983191f) entered disabled state Jan 7 06:10:01 Jarvis kernel: device veth983191f left promiscuous mode Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth983191f) entered disabled state Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth26a5504) entered blocking state Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth26a5504) entered disabled state Jan 7 06:10:01 Jarvis kernel: device veth26a5504 entered promiscuous mode Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth26a5504) entered blocking state Jan 7 06:10:01 Jarvis kernel: docker0: port 3(veth26a5504) entered forwarding state Jan 7 06:10:02 Jarvis kernel: eth0: renamed from veth849cba1 Jan 7 06:10:02 Jarvis kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth26a5504: link becomes ready Jan 7 07:00:01 Jarvis root: Restarting CA Auto Turbo Mode Jan 7 07:00:01 Jarvis kernel: mdcmd (56): set md_write_method 1 Jan 7 07:00:01 Jarvis kernel: Jan 7 07:16:58 Jarvis emhttpd: spinning down /dev/sda Jan 7 07:30:54 Jarvis kernel: usb 3-1: USB disconnect, device number 2 Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager usb_remove Broadcom_Corp_BCM20702A0_5CF370A921D6 /dev/bus/usb/003/002 003 002 Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager Device Match 003/002 vm: 003 002 Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager Removed 003/002 vm: nostate 003 002 Jan 7 07:30:54 Jarvis kernel: usb 3-1: new full-speed USB device number 5 using xhci_hcd Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager usb_add Broadcom_Corp_BCM20702A0_5CF370A921D6 /dev/bus/usb/003/005 003 005 Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM: chip id 63 Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM: features 0x07 Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM20702A Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM20702A1 (001.002.014) build 0000 Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM: firmware Patch file not found, tried: Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM: 'brcm/BCM20702A1-0b05-17cb.hcd' Jan 7 07:30:54 Jarvis kernel: Bluetooth: hci0: BCM: 'brcm/BCM-0b05-17cb.hcd' Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager Add device with more than one interface & connect serial . Jan 7 07:30:54 Jarvis usb_manager: Info: rc.usb_manager Autoconnect No Mapping found Broadcom_Corp_BCM20702A0_5CF370A921D6 /dev/bus/usb/003/005 003 005 port 3-1 Still looks like my bluetooth dongle randomly disconnected though The only other thing I'm finding that seems to be connected is usually a dead docker container. Today was Lidarr I looked at the rebuild-dndc logs and found this /usr/bin/docker run -d --name="binhex-lidarr" --net="container:binhex-qbittorrentvpn" -e UMASK="000" -e PUID="99" -e PGID="100" -e TZ="Europe/London" -v "/mnt/cache/appdata/data":"/data":rw -v "/mnt/user/Music/":"/media":rw -v "/mnt/short_cache/Downloads/Torrents/":"/downloads":rw -v "/mnt/user/appdata/binhex-lidarr":"/config":rw binhex/arch-lidarr 84c7e97f86ddcc9a8a3b6c8fc351e962e358f6138b986f72b93d54fab8e5e101 /usr/bin/docker: Error response from daemon: failed to start shim: start failed: runtime: failed to create new OS thread (have 7 already; errno=11) runtime: may need to increase max user processes (ulimit -u) fatal error: newosproc This occured at 06:05 and the HASS VM crashed 1 minute after

Yeah same here. My server has been up and running for maybe near a year now. I have used powertop for most of that but as you’ve confirmed the auto suspend is set as default anyway. Not had any issues until the last few weeks. Maybe this will solve it maybe it won’t. But if it does it doesn’t explain why it’s suddenly started happening

Hrmmmmmmmm sorry to spam this thread but.... Do you use powertop? I do and.. I checked /sys/module/usbcore/parameters and mine was 2. I'm going to try changing that to -1 - I'm wondering if powertop is making the usb devices suspend when it drops down to a certain c-state

Just remembered I have all syslogs backed up so checked them all Jan 1 06:04:34 Jarvis usb_manager: Info: rc.usb_manager vm_action HASS stopped end - Jan 1 06:05:00 Jarvis emhttpd: Unregistered - flash device error (ENOFLASH7) Jan 1 06:05:01 Jarvis emhttpd: Pro key detected, GUID: 0781-5571-5501-060921121721 FILE: /boot/config/Pro.key Jan 1 06:06:07 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did Jan 1 06:06:07 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did Jan 1 06:06:07 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did Jan 1 06:06:13 Jarvis emhttpd: Unregistered - flash device error (ENOFLASH7) Jan 1 06:06:14 Jarvis emhttpd: Pro key detected, GUID: 0781-5571-5501-060921121721 FILE: /boot/config/Pro.key Jan 1 06:10:31 Jarvis root: Mealie: Could not download icon https://cdn.statically.io/gh/KakersUK/docker-templates/mas Jan 1 06:48:15 Jarvis kernel: usb 7-1: USB disconnect, device number 6 Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager usb_remove Broadcom_Corp_BCM20702A0_5CF370A921D6 /dev/bus/usb Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager Device Match 007/006 vm: 007 006 Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager Removed 007/006 vm: nostate 007 006 Jan 1 06:48:15 Jarvis kernel: usb 7-1: new full-speed USB device number 7 using xhci_hcd Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager usb_add Broadcom_Corp_BCM20702A0_5CF370A921D6 /dev/bus/usb/007 Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM: chip id 63 Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM: features 0x07 Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM20702A Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM20702A1 (001.002.014) build 0000 Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM: firmware Patch file not found, tried: Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM: 'brcm/BCM20702A1-0b05-17cb.hcd' Jan 1 06:48:15 Jarvis kernel: Bluetooth: hci0: BCM: 'brcm/BCM-0b05-17cb.hcd' Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager Add device with more than one interface & connect serial . Jan 1 06:48:15 Jarvis usb_manager: Info: rc.usb_manager Autoconnect No Mapping found Broadcom_Corp_BCM20702A0_5CF370A9 almost the exact same timing when this happened last week but you can see that my bluetooth dongle disconnected, then reconnected - which is probably why HASS crashes as that device is passed through to the VM So it's almost like all the usb devices disconnect and reconnect. But I have absolutely no clue why

Yeah it's weird. I've had a bunch of bizarre issues on my server over Christmas... resolved all of them by swapping out my network card, finally ditched my pcie sata card for a proper HBA... now have nothing coming up in my syslog except every day I find those and one of my VM's is dead. Weirdly my other VM isn't affected (alpine linux) but the home assistant one crashes, and occasionally I find a few dead docker containers too - all die at the time the syslog shows the flash device errors. I've tried testing the usb stick in a PC and it seems fine, I can't see any errors about usb dropping or anything.

I'm having a very similar issue - only started a few weeks ago Nearly every day I get these errors Jan 6 06:06:40 Jarvis emhttpd: Unregistered - flash device error (ENOFLASH7) Jan 6 06:06:41 Jarvis emhttpd: Pro key detected, GUID: 0781-5571-5501-060921121721 FILE: /boot/config/Pro.key Jan 6 06:07:06 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdf Jan 6 06:07:06 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdc Jan 6 06:07:06 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: nvme0n1 Jan 6 06:07:06 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sdi Jan 6 06:07:06 Jarvis emhttpd: error: device_read_smart, 9579: Cannot allocate memory (12): device_spinup: stream did not open: sda Jan 6 06:07:16 Jarvis kernel: br0: port 2(vnet0) entered disabled state Jan 6 06:07:16 Jarvis kernel: device vnet0 left promiscuous mode Jan 6 06:07:16 Jarvis kernel: br0: port 2(vnet0) entered disabled state Jan 6 06:07:16 Jarvis usb_manager: Info: rc.usb_manager vm_action HASS stopped end - Jan 6 06:07:22 Jarvis emhttpd: Unregistered - flash device error (ENOFLASH7) Jan 6 06:07:23 Jarvis emhttpd: Pro key detected, GUID: 0781-5571-5501-060921121721 FILE: /boot/config/Pro.key Jan 6 06:08:00 Jarvis emhttpd: Unregistered - flash device error (ENOFLASH7) Jan 6 06:08:01 Jarvis emhttpd: Pro key detected, GUID: 0781-5571-5501-060921121721 FILE: /boot/config/Pro.key And my home assistant VM is crashed every morning I don't believe there's anything wrong with the USB pen, it's not very old and other thanit disconnecting around 6am it has no issues. I really don't know what's going on - it seems to fairly consistently happen just after 6am but I can't see anything else running/scheduled around that time

Just to close this off in case anyone else has the same issue in the future This 100% was my problem - the disk must have been formatted in my old server and transferred across without being wiped. It was the only disk with a 1MiB GPT, all the others are 4KiB. So when I formatted from exfs to zfs the size of the disk reported changed. The parity restore worked flawlessly and no lost data. The only other weird issue I had was the file integrity plugin started saying the hash's had changed for most of the files on disk3, again likely because of the GPT. I simply checked the files were ok, then removed the hash's and recalculated. All 4 of my other disks reformatted to ZFS without issues 😄

I'm not sure if the actual script itself supports a blank password, but you can try clicking the edit button next to the password field and changing "required" to no Then click save and you'll be able to continue with the field blank If that doesn't work you can try clicking the remove button next to the password fields, then they won't be passed to the container at all Let me know how you get on!

The only issue here is that folders on a 24tb array can't be moved to a 2tb cache and then back again. I'm going to have a play with https://github.com/SpaceinvaderOne/Unraid_ZFS_Auto_Dataset_Update to see if it can do regular folders on array disks. I believe it can

Thanks, I've kicked off a rebuild - fingers crossed! Your explanation probably makes sense, I had a native ubuntu server before moving to unraid, the disk might have been moved over from there and I forgot to zero it out first.

As per the instructions on the other thread I've tried starting the array with the disk removed. All files seem to be emulated ok, I can access them without issues. Attached new diagnostics with the disk now emulated. Does this mean I'm best just to add the disk back in and let it rebuild it from parity? jarvis-diagnostics-20231212-2129.zip