adfrost81 Posted May 30, 2023 Share Posted May 30, 2023 (edited) So my docker service started having all sorts of problems and that lead me down the path of deleting the docker.img. Had some issues getting it deleted, but I finally succeeded. Every procedure list I've seen says to simply start the service after deleting it and it will recreate the image from scratch. Mine is not doing that. It will not start the service and will not create the image. Any ideas on what I could be doing wrong here? Seems like a pretty simple procedure. Docker settings: The appdata share is set to prefer the cache, so it should be there. I can't tell in the GUI, but the docker directory is empty when I check in the command line. I don't know if this is helpful, but here's what the system log shows: Edited May 30, 2023 by adfrost81 Added screenshot of system log. Quote Link to comment
JorgeB Posted May 30, 2023 Share Posted May 30, 2023 Please post the diagnostics. Quote Link to comment
adfrost81 Posted May 30, 2023 Author Share Posted May 30, 2023 (edited) @JorgeB See attached. trantor-diagnostics-20230530-1821.zip Edited May 30, 2023 by adfrost81 Quote Link to comment
JorgeB Posted May 31, 2023 Share Posted May 31, 2023 May 29 19:43:36 Trantor kernel: ata1: COMRESET failed (errno=-16) May 29 19:43:36 Trantor kernel: ata1: limiting SATA link speed to 1.5 Gbps May 29 19:43:36 Trantor kernel: ata1: hard resetting link May 29 19:43:41 Trantor kernel: ata1: COMRESET failed (errno=-16) May 29 19:43:41 Trantor kernel: ata1: reset failed, giving up May 29 19:43:41 Trantor kernel: ata1.00: disable device May 29 19:43:41 Trantor kernel: ata1: EH complete Cache1 dropped offline, check/replace cables and post new diags after array start. Quote Link to comment
adfrost81 Posted May 31, 2023 Author Share Posted May 31, 2023 @JorgeB Alright, I reseated the cable, but I notice this error comes up when I start the array. Would this be related to having added another cache drive (after the issue with Docker started). Seems like the drive is started... Docker service still will not start. Attached updated Diagnostics file. trantor-diagnostics-20230531-0719.zip Quote Link to comment
JorgeB Posted May 31, 2023 Share Posted May 31, 2023 It dropped again: May 31 05:13:51 Trantor kernel: ata1: link is slow to respond, please be patient (ready=0) May 31 05:14:20 Trantor kernel: ata1: COMRESET failed (errno=-16) May 31 05:14:20 Trantor kernel: ata1: limiting SATA link speed to 3.0 Gbps May 31 05:14:20 Trantor kernel: ata1: hard resetting link May 31 05:14:25 Trantor kernel: ata1: COMRESET failed (errno=-16) May 31 05:14:25 Trantor kernel: ata1: reset failed, giving up May 31 05:14:25 Trantor kernel: ata1.00: disable device Quote Link to comment
adfrost81 Posted May 31, 2023 Author Share Posted May 31, 2023 Maybe a faulty drive? I don’t see any indications (other than those log entries and that notification) that there is anything wrong with it. Quote Link to comment
JorgeB Posted May 31, 2023 Share Posted May 31, 2023 If cables were already replaced try this: https://forums.unraid.net/topic/134954-warning-crucial-mx500-ssds-world-of-pain-stay-away-from-these/?do=findComment&comment=1255816 Quote Link to comment
adfrost81 Posted June 1, 2023 Author Share Posted June 1, 2023 (edited) Read through that post and decided to remove the drive from the system to see if I could check/update the firmware on my PC. That was not successful...even with WinBtrfs installed, I could not get the drive to show up in Windows. I put it back in the system, but it wouldn't detect in Unraid either. If it wasn't borked before, it seems like it is now, as it is saying it will overwrite whatever was on there if I re-add it as a cache drive. Before removing it from the system, I tried using the Mover to transfer stuff off, but it was nothing but errors. I have not transferred anything new to the array in a couple weeks...what is the chance that I will actually lose data? Wouldn't everything be copied over to the array at this point? If I try to Mount the drive as an unassigned device, I get errors. trantor-diagnostics-20230531-2132.zip Edited June 1, 2023 by adfrost81 Quote Link to comment
JorgeB Posted June 1, 2023 Share Posted June 1, 2023 Post the output of btrfs fi show Quote Link to comment
adfrost81 Posted June 1, 2023 Author Share Posted June 1, 2023 When I run that command in the terminal, it does nothing. Quote Link to comment
JorgeB Posted June 2, 2023 Share Posted June 2, 2023 Not a good sign but reboot and post new diags after array start. Quote Link to comment
adfrost81 Posted June 2, 2023 Author Share Posted June 2, 2023 Alright, but that will wipe the drive according to the UI, so if there was any data on it, it’ll be gone for sure… Quote Link to comment
JorgeB Posted June 2, 2023 Share Posted June 2, 2023 There's shouldn't be a "data will be deleted" warning after a reboot, but if there is unassign the device before array start. Quote Link to comment
adfrost81 Posted June 3, 2023 Author Share Posted June 3, 2023 After reboot, it's still showing the drive will be overwritten if I start the array. However, the command worked this time. Here are the results: root@Trantor:~# btrfs fi show warning, device 2 is missing Label: none uuid: a8311295-b025-4362-b378-ea013fc3a3df Total devices 2 FS bytes used 270.68GiB devid 1 size 465.76GiB used 296.79GiB path /dev/sdb1 *** Some devices missing Quote Link to comment
JorgeB Posted June 3, 2023 Share Posted June 3, 2023 Assuming the other pool device is still sdc (adjust the command if it isn't) post the output of: btrfs-select-super -s 1 /dev/sdc1 then again btrfs fi show Quote Link to comment
adfrost81 Posted June 3, 2023 Author Share Posted June 3, 2023 (edited) Yes, I had added a 1TB SSD that I had unplugged for troubleshooting. That is now plugged in and here is the result: root@Trantor:~# btrfs fi show Label: none uuid: a8311295-b025-4362-b378-ea013fc3a3df Total devices 2 FS bytes used 270.68GiB devid 1 size 465.76GiB used 296.79GiB path /dev/sdb1 devid 2 size 931.51GiB used 17.03GiB path /dev/sdc1 root@Trantor:~# btrfs-select-super -s 1 /dev/sdb1 using SB copy 1, bytenr 67108864 I have not started the array, as it is is showing both drives will be overwritten. Not a big deal for the 1TB, since it is empty. It does look like data is still on the original 500GB cache (sdb1), but It seems inaccessible at this point. But was that data also transferred to the array? I'm not clear on how cache works...does it store a copy, or the ONLY copy until it is transferred? Edited June 3, 2023 by adfrost81 Quote Link to comment
itimpi Posted June 3, 2023 Share Posted June 3, 2023 14 minutes ago, adfrost81 said: or the ONLY copy until it is transferred? This Quote Link to comment
adfrost81 Posted June 3, 2023 Author Share Posted June 3, 2023 1 hour ago, itimpi said: This I'm really confused on how there is any data on that drive then. The mover is scheduled to run daily, and I had not transferred anything new onto the server even this week, so it should've been empty. Unless the Mover wasn't working or something. I'm just wary of what could be on that drive. It's probably mostly replaceable media, but... Quote Link to comment
itimpi Posted June 3, 2023 Share Posted June 3, 2023 1 hour ago, adfrost81 said: I'm really confused on how there is any data on that drive then. The mover is scheduled to run daily, and I had not transferred anything new onto the server even this week, so it should've been empty. Unless the Mover wasn't working or something. I'm just wary of what could be on that drive. It's probably mostly replaceable media, but... It depends on the Share settings whether items get moved to the array. This only happens for shares with Use Cache=Yes set. Shares such as ‘appdata’ and ‘system’ are typically set with Use Cache=Prefer which keeps them on the cache pool. Shares to Use Cache=Only or Use Cache=No are ignored for moving files after initial placement. Quote Link to comment
adfrost81 Posted June 3, 2023 Author Share Posted June 3, 2023 6 hours ago, itimpi said: It depends on the Share settings whether items get moved to the array. This only happens for shares with Use Cache=Yes set. Shares such as ‘appdata’ and ‘system’ are typically set with Use Cache=Prefer which keeps them on the cache pool. Shares to Use Cache=Only or Use Cache=No are ignored for moving files after initial placement. IIRC, appdata and system were set to Use Cache=prefer setting. I don't think I messed with the defaults. I guess all I can do at this point is start the array, wipe the drives, and hope I didn't lose too much. Quote Link to comment
JorgeB Posted June 4, 2023 Share Posted June 4, 2023 Unassign both pool devices, start array, stop array, re-assign both pool devices, start array, post new diags. Quote Link to comment
adfrost81 Posted June 6, 2023 Author Share Posted June 6, 2023 On 6/4/2023 at 5:14 AM, JorgeB said: Unassign both pool devices, start array, stop array, re-assign both pool devices, start array, post new diags. After a reboot, I was able to get the btrfs-select-super -s 1 /dev/sdb1 command ran, and was then able to mount the drive as an unassigned device. Then I was able to run a scrub. Here are the results: FS: btrfs Executing file system scrub: /sbin/btrfs scrub start -B -R -d -r /dev/sdb1 2>&1 Scrub device /dev/sdb1 (id 1) canceled Scrub started: Mon Jun 5 17:58:41 2023 Status: aborted Duration: 0:01:38 data_extents_scrubbed: 0 tree_extents_scrubbed: 125 data_bytes_scrubbed: 0 tree_bytes_scrubbed: 2048000 read_errors: 1 csum_errors: 0 verify_errors: 0 no_csum: 0 csum_discards: 0 super_errors: 0 malloc_errors: 0 uncorrectable_errors: 0 unverified_errors: 0 corrected_errors: 0 last_physical: 5242880 File system corruption detected! I was able to access the share via the console: root@Trantor:~# cd /mnt/disks/1529101AA26A root@Trantor:/mnt/disks/1529101AA26A# ls -l /bin/ls: cannot access 'video': Input/output error total 0 drwxrwxrwx 1 nobody users 118 Feb 1 2022 appdata/ drwxrwxrwx 1 nobody users 0 Jan 20 2022 domains/ drwxrwxrwx 1 nobody users 26 Jan 20 2022 system/ d????????? ? ? ? ? ? video/ root@Trantor:/mnt/disks/1529101AA26A# I started the array and checked the video folder on the array...everything appears to be present that was last transferred, so at this point, I'm fine with abandoning this drive. I have a couple new drives I've purchased and will use those for cache going forward. I'm fine with recreating the appdata from scratch. Thanks for all the help @JorgeB @itimpi Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.