daan_SVK

I appreciate the time you took to respond but it was indeed incomplete image on the USB stick. this was a new install in new environment so I was suspecting something wrong with the network, I never had issues with the USB creator before. I will stick with the manual USB creation, is like we used to do it in the 90's anyway.

manual install indeed resolved. I wish there was a way to tell the USB creator failed to write the full image to the USB stick. All it says is "Writing done!" which really indicates it was done successfully.

I used the USB creator from the website, multiple USB keys as well. Should I just extract it manually? Doesn't the creator check the USB integrity once it writes the image?

tower-diagnostics-20190101-1426.zip here is the zip, thank for looking at it.

hi there, I'm just looking for some ideas on how to troubleshoot this further. I am trying to test a new build, it's a Lenovo P620 workstation. I imaged a new USB key and booted it up, I get no webgui, connection refused. - I can ping the server by hostname and IP - router shows the server by hostname as a connected device - I get no local gui because the server has a P2000 and it needs a driver first - I can not ssh into the server as the root password hasn't been set in the webgui first - I tried different USB sticks and different USB ports, it's always the same the server runs headless, is there anything else I can try?

I will replace the cable, it's just weird the server started having all these odd issues all of a sudden.

Sure, please see attached tower-diagnostics-20230207-1932.zip

so the disk rebuild failed with read errors again on the same drive so I replaced it and the replacement drive is rebuilding now, however I now see this in the log: Feb 7 17:51:28 Tower kernel: ata9.00: exception Emask 0x10 SAct 0x0 SErr 0x280100 action 0x6 frozen Feb 7 17:51:28 Tower kernel: ata9.00: irq_stat 0x08000000, interface fatal error Feb 7 17:51:28 Tower kernel: ata9: SError: { UnrecovData 10B8B BadCRC } Feb 7 17:51:28 Tower kernel: ata9.00: failed command: READ DMA EXT Feb 7 17:51:28 Tower kernel: ata9.00: cmd 25/00:40:68:1e:da/00:05:4b:00:00/e0 tag 4 dma 688128 in Feb 7 17:51:28 Tower kernel: res 50/00:00:67:1e:da/00:00:4b:00:00/e0 Emask 0x10 (ATA bus error) Feb 7 17:51:28 Tower kernel: ata9.00: status: { DRDY } Feb 7 17:51:28 Tower kernel: ata9: hard resetting link Feb 7 17:51:28 Tower kernel: ata9: SATA link up 6.0 Gbps (SStatus 133 SControl 300) Feb 7 17:51:28 Tower kernel: ata9.00: configured for UDMA/133 Feb 7 17:51:28 Tower kernel: ata9: EH complete and also my parity drive UDMA CRC error count just went from 0 to 1. I was originally thinking to replace the Sata cable to the disabled drive but now with the CRC error on the parity drive I'm wondering if I should just abandon the motherboard controller and move all the drives onto an LSI card.

I replaced the RAM and disabled C-states, I can't believe they were enabled. The drive is rebuilding now onto itself, I will report back. thanks again.

I did as you suggested, rebooted and the btrfs errors cleared. I was hoping that was the end of it but the server locked up with a Kernel Panic two days later. Rebooted with a successful parity check, the server ran OK for another two or three days. Last night it locked up again with Kernel Panic. After it was rebooted, a disk was disabled during the parity check which never happened before. I have a spare disk that I can replace the faulty one, if it is indeed faulty. However, I can not stop the array as the server is reporting that the parity check is running. It does not appear so as all the disks are spun down. Pressing the Cancel or Resume Parity check button does not re-enable the Stop array button so I'm not sure how to proceed. the latest diagnostics is below, what's my best course of action here? thanks in advance! tower-diagnostics-20230205-1050.zip

sure, please see attached. the pool was rebalanced and scrubbed after the docker image was recreated. tower-diagnostics-20230127-1717.zip

I deleted and re-created the docker container, reinstalled all my dockers, but immediately saw more btrfs errors in the log: Jan 27 15:22:40 Tower kernel: BTRFS error (device loop2: state EA): parent transid verify failed on 335167488 wanted 2298130 found 2298088 Jan 27 15:22:40 Tower kernel: BTRFS error (device loop2: state EA): parent transid verify failed on 335167488 wanted 2298130 found 2298088 Jan 27 15:22:40 Tower kernel: BTRFS error (device loop2: state EA): parent transid verify failed on 335167488 wanted 2298130 found 2298088 Jan 27 15:22:40 Tower kernel: BTRFS error (device loop2: state EA): parent transid verify failed on 335167488 wanted 2298130 found 2298088 ran scrub on the cache pool, no errors reported. Are the errors from within the docker container img? How do I resolve this for good? I'm rebalancing the pool now as I saw a thread where the full FS allocation caused the same error on cache.

just to be clear, are you saying recreating the docker image is a better approach than reformatting the cache pool? I'd like to address the possible cause of the corrupted image as well.

Thanks for the reply, Yes, I am aware, but this corruption has been on the cache pool for some time without a changed count. I will run another memtest but I still need to deal with the read only docker image.

hi guys, I keep loosing the ability to write to my docker image file. After seeing the below in my logs, I'm suspecting the cache or the image file is corrupted and was hoping to get some guidance on what should I do next. Jan 25 17:29:04 Tower kernel: ---[ end trace 0000000000000000 ]--- Jan 25 17:29:04 Tower kernel: BTRFS: error (device loop2: state A) in __btrfs_free_extent:3079: errno=-2 No such entry Jan 25 17:29:04 Tower kernel: BTRFS info (device loop2: state EA): forced readonly Jan 25 17:29:04 Tower kernel: BTRFS: error (device loop2: state EA) in btrfs_run_delayed_refs:2157: errno=-2 No such entry Jan 25 17:29:14 Tower flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update Recreate the docker.img or nuke the whole cache pool and reformat it? thanks in advance for reading, tower-diagnostics-20230125-1719.zip