Zee-Unraid

I am having the same problem - anyone find any solutions?

Is this plugin still recommended? There have been quite a few recent negative posts about it; and I am wondering if I should give it a go; if it still works on other people's systems; or if there is an alternative please? Thank you

Great, thanks for confirming and all your support on the forums. I've ordered Corsair Vengeance LPX 32 GB (2x16 GB) DDR4 2400 MHz CL14 (CMK32GX4M2A2400C14) Best regards, Zee

I took the ram out, inspected, pins all look fine, gave them a blow. Re-inserted into the MoBo, with the clicks so that they went in correctly. Mem test after ~1min... 7k errors...

After 10min, I'm guessing that's enough to confirm it with this volume of errors. Do you have any guidance for me/ point me to a thread for considerations when purchasing replacement RAM at all? Or does it just need to work with my MoBo (I am thinking about the time I was unaware there were controller specificities for Unraid and don't want to make the same mistake again) I am using: Gigabyte Technology Co., Ltd. B250M-D2VX-SI-CF Intel® Core™ i7-7700K CPU Thanks

I'll just state, I am a novice and in no way suggesting the new Unraid version caused any issues - I am just highlighting that for my specific case I have had something weird go on in the last couple of days. Please be gentle with me. I wanted to share my findings and get some opinions; as I think I have found bad RAM. I recently replaced my cache drive after previous cache drive failure due to using in incorrect controller, that is now fixed and I am using a recommended controller... Potentially unrelated, in all fairness, but I didn't observe these sorts of errors before upgrade, and something strange happened the day after I upgraded after having a stable system for months. I haven't seen this before. I recently upgraded to Unraid 7.0.1. I noticed the following day that I came back to my machine in the morning and it was unresponsive to GUI, SSH, everything, I had to power cycle off manually (and thus lost all logs). This hasn't happened to me before. This morning I installed syslog to write to my cache in case it happened again, I'd set it up the day after the unresponsiveness but it didn't write correctly. However, overall I haven't observed any unresponsiveness since the initial episode (~2 days ago) However, upon using my system as normal today and attempting to install a new docker container (Immich by imagegenius), I started to get these failures when downloading the container: (I had previously installed the other postgres container from SpaceInvaderOne tutorial, which went fine and was running). Error: failed to register layer: archive/tar: invalid tar header I tried to install the docker again, and got this: Pulling image: ghcr.io/imagegenius/immich:latest IMAGE ID [2053444522]: Pulling from imagegenius/immich. IMAGE ID [114bfc582739]: Pulling fs layer. Downloading 100% of 31 MB. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [e1cde46db0e1]: Pulling fs layer. Downloading 100% of 6 KB. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [d403e9fdb1ff]: Pulling fs layer. Downloading 100% of 1 KB. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [fe7b05793469]: Pulling fs layer. Downloading 100% of 700 B. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [d04423d473c1]: Pulling fs layer. Downloading 100% of 427 B. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [ba37c0944114]: Pulling fs layer. Downloading 100% of 276 B. Download complete. Extracting. Pull complete. IMAGE ID [a3a7788d8bcf]: Pulling fs layer. Downloading 100% of 28 MB. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [cc0c773231d1]: Pulling fs layer. Downloading 100% of 4 KB. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [94dab31e49e0]: Pulling fs layer. Downloading 100% of 634 B. Verifying Checksum. Download complete. Extracting. Pull complete. IMAGE ID [8dfb42dbbd38]: Pulling fs layer. Downloading 100% of 475 MB. Verifying Checksum. Download complete. Extracting. IMAGE ID [0e4bb0a352c4]: Pulling fs layer. Downloading 100% of 385 MB. Verifying Checksum. Download complete. IMAGE ID [730e814ed456]: Pulling fs layer. Downloading 100% of 4 KB. Verifying Checksum. Download complete. TOTAL DATA PULLED: 919 MB Error: failed to register layer: read /var/lib/docker/tmp/GetImageBlob2883473593: input/output error I was seeing corruption from my Docker img in my logs: Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386097152 csum 0x5be13239 expected csum 0x5a5261a4 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 47, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386101248 csum 0x1605b97e expected csum 0x08a59cac mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 48, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386105344 csum 0x9f081d89 expected csum 0x60029bd5 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 49, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386109440 csum 0xf7335180 expected csum 0x1ecf0f34 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 50, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386113536 csum 0x09e00baf expected csum 0x352c45d0 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 51, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386117632 csum 0x295c1675 expected csum 0xaeaafcb7 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 52, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386121728 csum 0xccafe50e expected csum 0xa3a38d85 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 53, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386097152 csum 0x5be13239 expected csum 0x5a5261a4 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 54, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386097152 csum 0x5be13239 expected csum 0x5a5261a4 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 55, gen 0 Apr 23 12:45:24 ZOREO kernel: BTRFS warning (device loop2): csum failed root 5 ino 3652289 off 386097152 csum 0x5be13239 expected csum 0x5a5261a4 mirror 1 Apr 23 12:45:24 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 56, gen 0 I tried deleting my docker img and started to reinstall apps from CA Previous Apps section as my AppData backup and restore file would not import the previous AB backup file, stating "invalid source". So I had to do manually via Previous Applications. The data on my cache drive and app data is/ appears otherwise fine and preserved, just missing the containers. I added a couple of containers, (which operate just fine)... Then I started to see the same again in the logs, as below: Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33895 off 0 csum 0xd51a64e5 expected csum 0xb3c3997f mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 1, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33895 off 4096 csum 0x3959eaad expected csum 0xe31d428d mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 2, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33895 off 0 csum 0xd51a64e5 expected csum 0xb3c3997f mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 3, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33895 off 0 csum 0xd51a64e5 expected csum 0xb3c3997f mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 4, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33894 off 0 csum 0x6318f719 expected csum 0x0bcf6264 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 5, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33894 off 4096 csum 0xf828e7d9 expected csum 0xf3c70c26 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 6, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33894 off 0 csum 0xe4f2db55 expected csum 0x0bcf6264 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 7, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33894 off 0 csum 0xe4f2db55 expected csum 0x0bcf6264 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 8, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33893 off 0 csum 0x6498a8d9 expected csum 0xb6c1f1f7 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 9, gen 0 Apr 23 13:09:45 ZOREO kernel: BTRFS warning (device loop2): csum failed root 264 ino 33893 off 0 csum 0x6498a8d9 expected csum 0xb6c1f1f7 mirror 1 Apr 23 13:09:45 ZOREO kernel: BTRFS error (device loop2): bdev /dev/loop2 errs: wr 0, rd 0, flush 0, corrupt 10, gen 0 The SSD Cache drive looks fine itself after Smart Test: root@ZOREO:~# btrfs device stats /mnt/cache [/dev/mapper/sdf1].write_io_errs 0 [/dev/mapper/sdf1].read_io_errs 0 [/dev/mapper/sdf1].flush_io_errs 0 [/dev/mapper/sdf1].corruption_errs 21 [/dev/mapper/sdf1].generation_errs 0 root@ZOREO:~# smartctl -a /dev/sdf smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.6.78-Unraid] (local build) Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Samsung based SSDs Device Model: Samsung SSD 870 EVO 500GB Serial Number: S7EWNJ0WC28314V LU WWN Device Id: 5 002538 f33c0b619 Firmware Version: SVT02B6Q User Capacity: 500,107,862,016 bytes [500 GB] Sector Size: 512 bytes logical/physical Rotation Rate: Solid State Device Form Factor: 2.5 inches TRIM Command: Available, deterministic, zeroed Device is: In smartctl database 7.3/5660 ATA Version is: ACS-4 T13/BSR INCITS 529 revision 5 SATA Version is: SATA 3.3, 6.0 Gb/s (current: 6.0 Gb/s) Local Time is: Wed Apr 23 13:12:54 2025 BST SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 0) seconds. Offline data collection capabilities: (0x53) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. No Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 85) minutes. SCT capabilities: (0x003d) SCT Status supported. SCT Error Recovery Control supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 1 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 5 Reallocated_Sector_Ct 0x0033 100 100 010 Pre-fail Always - 0 9 Power_On_Hours 0x0032 099 099 000 Old_age Always - 4501 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 7 177 Wear_Leveling_Count 0x0013 100 100 000 Pre-fail Always - 0 179 Used_Rsvd_Blk_Cnt_Tot 0x0013 100 100 010 Pre-fail Always - 0 181 Program_Fail_Cnt_Total 0x0032 100 100 010 Old_age Always - 0 182 Erase_Fail_Count_Total 0x0032 100 100 010 Old_age Always - 0 183 Runtime_Bad_Block 0x0013 100 100 010 Pre-fail Always - 0 187 Uncorrectable_Error_Cnt 0x0032 100 100 000 Old_age Always - 0 190 Airflow_Temperature_Cel 0x0032 068 063 000 Old_age Always - 32 195 ECC_Error_Rate 0x001a 200 200 000 Old_age Always - 0 199 CRC_Error_Count 0x003e 100 100 000 Old_age Always - 0 235 POR_Recovery_Count 0x0012 099 099 000 Old_age Always - 1 241 Total_LBAs_Written 0x0032 099 099 000 Old_age Always - 901850932 252 Added_Bad_Flash_Blk_Ct 0x0032 100 100 000 Old_age Always - 1 SMART Error Log Version: 1 No Errors Logged SMART Self-test log structure revision number 1 Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 3 - # 2 Short offline Completed without error 00% 0 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing 256 0 65535 Read_scanning was never started Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. The above only provides legacy SMART information - try 'smartctl -x' for more root@ZOREO:~# The new CA memtest app is showing me failures across the board after just 1 loop. I stopped it after that. [23.04.2025 13:43:29 BST] /usr/bin/memtester 13G 5 ============================================================================================= memtester version 4.6.0 (64-bit) adapted for use on Unraid installations Copyright (C) 2001-2020 Charles Cazabon Copyright (C) 2024 desertwitch (modifications for Unraid) THIS PROGRAM IS PROVIDED AS IS AND WITHOUT ANY WARRANTIES It is licensed under GNU General Public License Version 2 pagesize is 4096 pagesizemask is 0xfffffffffffff000 want 13312MB (13958643712 bytes) got 13312MB (13958643712 bytes), trying mlock ...locked. Loop 1/5: Stuck Address : Testing... failed Random Value : Testing... failed Compare XOR : Testing... failed Compare SUB : Testing... failed Compare MUL : Testing... failed Compare DIV : Testing... failed Compare OR : Testing... failed Compare AND : Testing... failed Sequential Increment : Testing... failed Solid Bits : Testing... failed Block Sequential : Testing... failed Checkerboard : Testing... failed Bit Spread : Testing... failed Bit Flip : Testing... failed Walking Ones : Testing... failed Walking Zeroes : Testing... failed 8-bit Writes : Testing... failed 16-bit Writes : Testing... failed Loop 2/5: Stuck Address : Testing... failed Random Value : Testing... ============================================================================================= [23.04.2025 13:46:38 BST] The operation was interrupted and has not finished. [23.04.2025 13:46:38 BST] Code: SIGTERM - Program Interrupt. I am guessing I'll have to replace the RAM and/or MoBo? Is there advice or expert opinion on this before I throw my system out the window? Is there anything else I should do or just be done with it and purchase some new RAM and see? The array is currently offline and I think I'll keep it offline for now. Thank you in advance. zoreo-diagnostics-20250423-1358.zip

I've never ventured into that area, if it's relatively simple to do and you can point me to a guide of some sort; I'll give it a go for you.

I had this installed, which I believe was incompatible. https://www.amazon.co.uk/dp/B07QGRVGVM/ref=pe_27063361_487055811_TE_dp_1 I now have this, and no errors on my new cache: https://www.amazon.co.uk/dp/B08F56WKW7 (I'd installed the new cache using the previous one and started to see CRC errors immediately, so stopped it and purchased this)

Just to close my topic off: I found that I had a sata expansion card/ controller that wasn't compatible with Unraid, and could have been, at least partially responsible for the drive failure and corruption - as my cache drive was connected through this expansion card. From a "casual Unraid user" perspective, and considering that Unraid is designed to make complicated activities simpler - controller/ expansion card compatibility/ awareness, (that this is even a "thing"), really should be highlighted when users are adding drives through the Unraid UI. Or some sort of single pop up on installation/ acknowledgement. It might prevent people from having issues, reduce forum posts for help, and overall help the planet by preventing drive failures. It should be seriously considered from an environmental perspective, if nothing else. Finding it in a forum post (hardware compatibility), after the event, is pretty soul destroying and one of those eye-roll moments once I checked and could see my expansion card indeed was not compatible. Yes, it's my fault for not checking the forums; however please consider doing more on the Unraid side to at least bring this to people's attention somewhere between installation, drive addition in OS, and drive errors/ failure. E.g., when drive errors are first detected, perhaps a pop up/ alert asking "do you have an expansion card/ controller installed, check this post on compatibility". I'm not sure what's possible, but please if there is anything you can do on your side to prevent other users encountering issues and reducing environmental waste, please kindly consider it. Anyway, after I'd installed my new cache drive with the new controller; I found that I had some data from the array appdata, which wasn't visible to me on the array whilst the cache drive was offline, that reappeared with the new cache installed and it recovered most of my containers. I've still got some rebuilding and work to do, but at least I had some things top start from. Thank you regardless, @JorgeB, I very much appreciated your quick responses and no-nonsense help. You have hugely helped me here and for that I am very grateful. Please do consider/ discuss with the Unraid team if there can be anything done in this area. Wishing you all the best. Take care. Link to my specific resolution and learnings around sata expansion cards/ controllers:

Glad you found it useful. I personally haven't delved into c-states to be honest with you. Would be interested to hear of your findings in regards to controllers, (and thus if it's something I should be considering/ getting into)! Feel free to DM me to update me, if you feel it would not be relevant to the controller topic of this thread.

This thread has been extremely useful - I never even knew this was a thing with expansion cards/ controllers and Unraid, it definitely needs to be more prominent to users with a warning as they add drives, even when adding them for the first time!!! I suspect me using a incompatible expansion card/ controller was, at least partly, to blame for a cache drive failure (albeit I was messing around a bit). I was receiving CRC errors and warnings on my SSD cache which I had plugged into my controller which had a multiplier - but I didn't really comprehend or think it was serious until I tried to mass write to my cache one day and boom, cache drive dead. I purchased this ASM1166 MZHOU product from Amazon: https://www.amazon.co.uk/dp/B08F56WKW7 for £35. It has previously been mentioned and linked in this thread. Thank you all for commenting on it, it gave me enough confidence to purchase it! MZHOU PCIE SATA Card 6 Ports 4X PCIe SATA Expansion Card - 6 Gbps SATA 3.0 PCI Express Card - PCIe to SATA Controller Expansion Card with 6 SATA Cables and Low Profile Bracket I wanted to write and confirm that it already came installed with firmware 221118-003E-00, which at time of writing seems to be the latest FW for ASM1166. I checked it following the guide posted previously by @PhilBarker here (thank you!), which directed me to find the ROM checker and installer here (which contains outdated FW, but I used it to scan for it and check which FW mine had), the download you need is called "Fix compatibility issue and support Intel 600 series chipset motherboard" from the Sliverstone ECS06 site. Run this on your normal windows PC with the controller installed and you can scan for the expansion card. I had to plug it in and out a couple of times before it was recognised, (it eventually worked when I did this as my PC was on, a no-no I know). Plugged into my Unraid server Mobo, and works fine! Thank you all for your posts and help - I hope my post also contributes and helps people that come after me - it took me a long time to read this thread and consolidate my understanding of everything! @Gegga - I saw your post from May this year, FYI it looks like you flashed it with FW 11050000.ROM, which is a fair few years out of date just as a FYI. To you and everyone else who is searching for the FW file 221118-003E-00, or to check the latest conversation on these chips and FW; I found a thread here which seems to be alive and kicking of users uploading FW versions and commenting on them as they learn, (possibly posted very early on during this thread but can't remember): https://winraid.level1techs.com/t/latest-firmware-for-asm1064-1166-sata-controllers/98543/50. It appears you could get the latest rom file from the above winraid thread, and then replace the rom file to flash it with the FW of your choice with the tool from Silverstone. Just an observation for you, if you, or anyone in the future was interested. Thanks also to @JorgeB for his persistent replies, expertise and commitment to the community. A true Unraid legend. Rom checker/ Installer I used (note FW is out of date):ECS06 firmware for Intel 600series chipset.rar

Thank you for your support here. May I ask two questions, please: 1. Do you think it is possible to recover data from the cache? Unraid says the device is "healthy" and shows it has circa 150gb data from the Main screen, however using the terminal to inspect it is not possible/ does not show any data? What approaches are available to me to recover data from the device? 2. Is the device likely otherwise usable again in your experience, and if so, what methods should I undertake to attempt to repair it? Many thanks!

Thanks very much. Please see attached new diagnostics. I've used a new SATA cable fresh from a wrapped packet; but had to use the power cable from the unassigned CCTV drive as I'm a little short on them in my setup. Below, I also post the latest result from my last parity check which completed after I posted my previous diagnostics, (and before the above cable swap out), if that's of importance/ helps: 2024 Oct 5 23:39:20|31965|125161488|0|0|check P|3907018532 Many thanks again! zoreo-diagnostics-20241006-1705.zip

Thanks so much for taking the time to review it. May I kindly ask, what do you mean replace both cables, and why? Thanks again, Zee

Hello, thanks for viewing. Hopefully you can impart some wisdom on me and I can learn from my mistakes. My diagnostics export is attached at the bottom of the post. It's fair to point out that, compared to most users on this forum, I'm likely a bit of a novice when it comes to both Unraid, coding/ programming etc., so would welcome clear instructions if you need me to check anything or carry out remedial tasks or require information. I would very much appreciate your patience whilst I am learning and navigating. I have unfortunately rebooted my system since the critical failure occurred, (which I now know not to do before taking a download of the syslog!), however I can tell you what I was doing and why - along with sharing the current status of the system. I also have various code snippets along the way, because I was using ChatGPT to (rightly or wrongly), advise and counsel as I went both before and after the issue occurred, however I believe it was my human mistake that caused the failure/ fallout (steps 8-11). What happened: I was installing a Local LLM model on my machine in a Docker container, the first one worked fine but I was not happy with the LLM outputs. I wanted to try a different LLM and it didn't quite work/ go to plan; I think the container ran out of space during download/ installation and it failed, I'd previously checked the capacity of my system and it seemed fine, but something fell over and the install failed. At this point, my other containers and system was still in tact. No worries. This morning I was attempting to assess the size limits and understand what was going on with the container and how I could fix it. I ran this command to review what is taking up space: root@ZOREO:~# docker system df TYPE TOTAL ACTIVE SIZE RECLAIMABLE Images 21 7 14.73GB 7.569GB (51%) Containers 7 3 6.491GB 23.82MB (0%) Local Volumes 5 1 0B 0B Build Cache 0 0 0B 0B root@ZOREO:~# docker logs 714aab13613e Error response from daemon: configured logging driver does not support reading root@ZOREO:~# docker volume ls DRIVER VOLUME NAME local 10ce6bb36f5e22542789ef6fb46c6d1a326c8679f9cdf8711eb102b83ea9aec5 local 25a4529cd7273688e4bce4b7d8770bd6b97a69c0f959c7c6e5ae631098c2237d local 78bedba53481c28cfec112f0306eaec7419b47e1cec0c35e0c2ee2aa38513528 local a46d302e5ccd00dc10fee30b986f71851fcd270bb918282abd93c2c0090928f2 local eeedfee9170ad5c678f3148d0d368b89f623afe75ede7269f104032a54381a3b I also then ran the following command line, to assess space utilisation: du -sh /var/lib/docker/* What I didn't realise at this point, was that step 6, above, then started to consume 100% of my CPU usage; and at the same time on my other screen, I'd noticed a number of orphaned images appear in the Docker menu page, so I started clicking them and selecting "remove", because I was tidying up some old of the old images and things I had been playing around with yesterday. I think this is what caused my system to go into meltdown. I hadn't realised the terminal had not finished running the du command and this is possibly what caused the following hanging. Whilst the system was hanging, I could access another terminal and putty to review the system utilisation and I attempted to kill the tasks related to the du command to try and bring the system back responsive. It didn't work after 5min or so. I tried typing reboot as a last resort in the terminal, nothing happened after 20mins. By this point I had 2 zombie processes running. I decided I had no other option that to hard press the power button to reset the system and attempt to bring the system back to life. Which worked. I can login to Unraid fine, however my Docker service seems to be fried and unresponsive. I've tried to reinstall it, but I am concerned the issue could be my cache drive or I might need to either reformat it or reconfigure my cache, which is a little out of my comfort zone and hence why I am here before I potentially make further mistakes. I think I may have lost my previous containers; the only two active at the time were Shinobi and NTP, both related to local home CCTV, (which has been working mostly fine). These are the only ones I really care about recovering. When I visit the Docker page from the Unraid menu, I see the error: "Docker Service failed to start." I think I have installed Docker again properly, but for some reason it's not fully working and I would like help to try and reload my previous containers please, assuming they won't reappear automatically in the Docker menu. I think I also took a backup of my Docker data before I attempted the re-install, see below, but I'm not sure how I can recover/ reintegrate it - or frankly if it's still there at all now! Any help or assistance very much appreciated! Thank you very much! Backup command: root@ZOREO:~# sudo tar -czvf docker_backup.tar.gz /var/lib/docker tar: Removing leading /' from member names /var/lib/docker/ /var/lib/docker/runtimes/ /var/lib/docker/tmp/ /var/lib/docker/buildkit/ /var/lib/docker/buildkit/history.db /var/lib/docker/buildkit/cache.db /var/lib/docker/buildkit/executor/ /var/lib/docker/buildkit/metadata_v2.db /var/lib/docker/buildkit/snapshots.db /var/lib/docker/buildkit/containerdmeta.db /var/lib/docker/buildkit/content/ /var/lib/docker/buildkit/content/ingest/ /var/lib/docker/swarm/ /var/lib/docker/network/ /var/lib/docker/network/files/ /var/lib/docker/network/files/local-kv.db /var/lib/docker/image/ /var/lib/docker/image/overlay2/ /var/lib/docker/image/overlay2/repositories.json /var/lib/docker/image/overlay2/distribution/ /var/lib/docker/image/overlay2/distribution/diffid-by-digest/ /var/lib/docker/image/overlay2/distribution/diffid-by-digest/sha256/ /var/lib/docker/image/overlay2/distribution/diffid-by-digest/sha256/c1ec31eb59444d78df06a974d155e597c894ab4cda84f08294145e845394988e /var/lib/docker/image/overlay2/distribution/v2metadata-by-diffid/ /var/lib/docker/image/overlay2/distribution/v2metadata-by-diffid/sha256/ /var/lib/docker/image/overlay2/distribution/v2metadata-by-diffid/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e /var/lib/docker/image/overlay2/imagedb/ /var/lib/docker/image/overlay2/imagedb/metadata/ /var/lib/docker/image/overlay2/imagedb/metadata/sha256/ /var/lib/docker/image/overlay2/imagedb/content/ /var/lib/docker/image/overlay2/imagedb/content/sha256/ /var/lib/docker/image/overlay2/imagedb/content/sha256/d2c94e258dcb3c5ac2798d32e1249e42ef01cba4841c2234249495f87264ac5a /var/lib/docker/image/overlay2/layerdb/ /var/lib/docker/image/overlay2/layerdb/mounts/ /var/lib/docker/image/overlay2/layerdb/mounts/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/ /var/lib/docker/image/overlay2/layerdb/mounts/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/parent /var/lib/docker/image/overlay2/layerdb/mounts/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/init-id /var/lib/docker/image/overlay2/layerdb/mounts/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/mount-id /var/lib/docker/image/overlay2/layerdb/sha256/ /var/lib/docker/image/overlay2/layerdb/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e/ /var/lib/docker/image/overlay2/layerdb/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e/cache-id /var/lib/docker/image/overlay2/layerdb/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e/size /var/lib/docker/image/overlay2/layerdb/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e/diff /var/lib/docker/image/overlay2/layerdb/sha256/ac28800ec8bb38d5c35b49d45a6ac4777544941199075dff8c4eb63e093aa81e/tar-split.json.gz /var/lib/docker/image/overlay2/layerdb/tmp/ /var/lib/docker/overlay2/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/lower /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/work/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/work/work/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/link /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1/diff/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/committed /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/lower /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/work/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/work/work/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/link /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/.dockerenv /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/sys/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/proc/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/etc/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/etc/mtab /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/etc/hosts /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/etc/resolv.conf /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/etc/hostname /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/dev/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/dev/console /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/dev/shm/ /var/lib/docker/overlay2/392c87c035536224bfb044458f7299be153bf7a7af62beea2b14a54890fb57b1-init/diff/dev/pts/ /var/lib/docker/overlay2/be927523c8e3f437c5ff47ed9c8d5bae9a8d1f0f34ff9a02d61184f0326a4e70/ /var/lib/docker/overlay2/be927523c8e3f437c5ff47ed9c8d5bae9a8d1f0f34ff9a02d61184f0326a4e70/committed /var/lib/docker/overlay2/be927523c8e3f437c5ff47ed9c8d5bae9a8d1f0f34ff9a02d61184f0326a4e70/link /var/lib/docker/overlay2/be927523c8e3f437c5ff47ed9c8d5bae9a8d1f0f34ff9a02d61184f0326a4e70/diff/ /var/lib/docker/overlay2/be927523c8e3f437c5ff47ed9c8d5bae9a8d1f0f34ff9a02d61184f0326a4e70/diff/hello /var/lib/docker/overlay2/l/ /var/lib/docker/overlay2/l/QIOQQEL7TZ2DNGOVNYURQYIYMR /var/lib/docker/overlay2/l/SKLEOGYY2B4J6Z7QXWEDHIH7OR /var/lib/docker/overlay2/l/6G43L2J7RRIMNMGB66EZSJOCAC /var/lib/docker/engine-id /var/lib/docker/volumes/ /var/lib/docker/volumes/backingFsBlockDev /var/lib/docker/volumes/metadata.db /var/lib/docker/plugins/ /var/lib/docker/plugins/storage/ /var/lib/docker/plugins/storage/ingest/ /var/lib/docker/plugins/tmp/ /var/lib/docker/containers/ /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/ /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/config.v2.json /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/hostconfig.json /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b-json.log /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/mounts/ /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/hostname /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/resolv.conf.hash /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/resolv.conf /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/hosts /var/lib/docker/containers/d8d13d11f6e09760ce668b362257237a163ee89cee4491a013507153ae23c40b/checkpoints/ /var/lib/docker/containerd/ /var/lib/docker/containerd/daemon/ /var/lib/docker/containerd/daemon/io.containerd.runtime.v1.linux/ /var/lib/docker/containerd/daemon/io.containerd.runtime.v2.task/ /var/lib/docker/containerd/daemon/io.containerd.runtime.v2.task/moby/ /var/lib/docker/containerd/daemon/io.containerd.metadata.v1.bolt/ /var/lib/docker/containerd/daemon/io.containerd.metadata.v1.bolt/meta.db /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.overlayfs/ /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.overlayfs/snapshots/ /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.native/ /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.native/snapshots/ /var/lib/docker/containerd/daemon/io.containerd.snapshotter.v1.blockfile/ /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/ /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/blobs/ /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/blobs/sha256/ /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/blobs/sha256/e2fc4e5012d16e7fe466f5291c476431beaa1f9b90a5c2125b493ed28e2aba57 /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/blobs/sha256/91fb4b041da273d5a3273b6d587d62d518300a6ad268b28628f74997b93171b2 /var/lib/docker/containerd/daemon/io.containerd.content.v1.content/ingest/ /var/lib/docker/containerd/daemon/tmpmounts/ root@ZOREO:~# Current Docker info: root@ZOREO:~# /etc/rc.d/rc.docker start /usr/bin/dockerd is already running root@ZOREO:~# ls /var/log/ btmp dmesg libvirt/ messages packages@ plugins/ removed_scripts@ scripts@ spooler unbalanced.log wtmp cron faillog maillog nfsd/ phplog pwfail/ removed_uninstall_scripts@ secure swtpm/ vfio-pci debug lastlog mcelog nginx/ pkgtools/ removed_packages@ samba/ setup@ syslog vfio-pci-errors root@ZOREO:~# ls -la /var/lib/docker total 4 drwx--x--- 13 root root 280 Oct 5 14:21 ./ drwxr-xr-x 15 root root 300 Apr 3 2024 ../ drwx--x--x 4 root root 180 Oct 5 14:21 buildkit/ drwx--x--x 3 root root 60 Oct 5 14:21 containerd/ drwx--x--- 2 root root 40 Oct 5 14:21 containers/ -rw------- 1 root root 36 Oct 5 14:21 engine-id drwx------ 3 root root 60 Oct 5 14:21 image/ drwxr-x--- 3 root root 60 Oct 5 14:21 network/ drwx--x--- 3 root root 60 Oct 5 14:21 overlay2/ drwx------ 4 root root 80 Oct 5 14:21 plugins/ drwx------ 2 root root 40 Oct 5 14:21 runtimes/ drwx------ 2 root root 40 Oct 5 14:21 swarm/ drwx------ 2 root root 40 Oct 5 16:01 tmp/ drwx-----x 2 root root 80 Oct 5 14:21 volumes/ root@ZOREO:~# docker --version Docker version 24.0.9, build 2936816 root@ZOREO:~# ls -la /var/lib/docker/containers total 0 drwx--x--- 2 root root 40 Oct 5 14:21 ./ drwx--x--- 13 root root 280 Oct 5 14:21 ../ root@ZOREO:~# zoreo-diagnostics-20241005-1603.zip