July 21, 20241 yr Hello, I recently got an error that my drive was failing. I had to manually stop the mover, and after the system went unresponsive for 10+ minutes I made the decision to do an unclean shutdown. When I booted back up I deselected the disabled drive in parity 2 slot, started the array, stopped array, selected an unassigned disk already in my machine, started array, and then started the parity rebuild. I am about 30 minutes into that rebuild and the parity 2 drive has failed again. I've attached my errors below, and attached my diagnostics. Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797856 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797864 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797872 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797880 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797888 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797896 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797904 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223797912 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798352 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798360 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798368 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798376 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798384 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798392 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798400 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798408 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798416 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798424 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798432 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798440 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798448 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798456 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798464 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798472 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798632 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798640 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798648 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798656 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798664 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798672 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798680 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798688 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798696 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798704 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798712 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798720 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798728 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798736 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798744 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798752 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798760 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798768 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798776 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798784 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798792 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798800 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798808 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798816 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798824 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798832 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798840 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798848 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798856 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798864 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798872 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798880 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798888 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798896 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798904 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798912 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798920 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798928 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=1223798936 Jul 20 17:47:43 Skynet kernel: md: disk29 write error, sector=8593780232 Jul 20 17:47:44 Skynet flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update Jul 20 17:47:44 Skynet kernel: mpt2sas_cm0: handle(0x18) sas_address(0x5000c500a6c8fc21) port_type(0x1) Jul 20 17:47:44 Skynet kernel: scsi 8:0:25:0: Direct-Access SEAGATE ST10000NM0226 KTB5 PQ: 0 ANSI: 6 Jul 20 17:47:44 Skynet kernel: scsi 8:0:25:0: SSP: handle(0x0018), sas_addr(0x5000c500a6c8fc21), phy(5), device_name(0x5000c500a6c8fc20) Jul 20 17:47:44 Skynet kernel: scsi 8:0:25:0: enclosure logical id (0x5001438022e55665), slot(38) Jul 20 17:47:44 Skynet kernel: scsi 8:0:25:0: qdepth(254), tagged(1), scsi_level(7), cmd_que(1) Jul 20 17:47:44 Skynet kernel: scsi 8:0:25:0: Power-on or device reset occurred Jul 20 17:47:44 Skynet kernel: sd 8:0:25:0: Attached scsi generic sg18 type 0 Jul 20 17:47:44 Skynet kernel: end_device-8:1:13: add: handle(0x0018), sas_addr(0x5000c500a6c8fc21) Jul 20 17:47:44 Skynet kernel: sd 8:0:25:0: [sdac] Spinning up disk... Jul 20 17:48:01 Skynet kernel: ...............ready Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] 2441609216 4096-byte logical blocks: (10.0 TB/9.10 TiB) Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] Write Protect is off Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] Mode Sense: db 00 10 08 Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] Write cache: disabled, read cache: enabled, supports DPO and FUA Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] Preferred minimum I/O size 4096 bytes Jul 20 17:48:01 Skynet kernel: sdac: sdac1 Jul 20 17:48:01 Skynet kernel: sd 8:0:25:0: [sdac] Attached SCSI disk Jul 20 17:48:02 Skynet root: file: /mnt/mediacache/TV/Rugrats/Season 2/Rugrats (1991) - S02E40-E41 - Feeding Hubert and Spike The Wonder Dog [WEBDL-1080p].mkv Jul 20 17:48:02 Skynet unassigned.devices: Partition '/dev/sdac1' does not have a file system and cannot be mounted. Jul 20 17:49:21 Skynet root: file: /mnt/mediacache/TV/Rugrats/Season 2/Rugrats (1991) - S02E39 - My Friend Barney [WEBDL-1080p].mkv Jul 20 17:49:25 Skynet emhttpd: error: hotplug_devices, 1709: No such file or directory (2): tagged device ST10000NM0226_ZA290T3P0000C905MDCF_35000c500a6c8fc23 was (sdr) is now (sdac) Jul 20 17:49:25 Skynet emhttpd: read SMART /dev/sdac Jul 20 17:49:25 Skynet kernel: emhttpd[11286]: segfault at 67c ip 0000000000418359 sp 00007ffcc60002c0 error 4 in emhttpd[404000+1a4000] likely on CPU 16 (core 4, socket 0) Jul 20 17:49:25 Skynet kernel: Code: c0 75 15 48 c7 45 f8 75 ce 5a 00 48 c7 45 f0 a5 bb 5a 00 e9 38 01 00 00 8b 45 ec 89 c7 e8 2d 95 ff ff 48 89 45 d8 48 8b 45 d8 <8b> 80 7c 06 00 00 85 c0 0f 94 c0 0f b6 c0 89 45 d4 48 8b 45 e0 be Jul 20 17:49:44 Skynet flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update skynet-diagnostics-20240720-1917.zip
July 21, 20241 yr Author More errors on the rebuild, two different disks. This is really odd and doesn't make any sense why all these drives would be having issues all of a sudden? Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674512 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674520 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674528 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674536 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674544 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674552 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674560 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674568 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674576 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674584 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674592 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674600 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674608 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674616 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674624 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674632 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674640 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674648 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674656 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674664 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674672 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674680 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674688 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674696 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674704 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674712 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674720 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674728 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674736 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674744 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674752 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674760 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674768 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674776 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674784 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674792 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674800 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674808 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674816 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674824 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674832 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674840 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674848 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674856 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674864 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674872 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674880 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674888 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674896 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674904 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674912 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674920 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674928 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674936 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674944 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674952 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674960 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674968 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674976 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674984 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000674992 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675000 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675008 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675016 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675024 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675032 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675040 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675048 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675056 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675064 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675072 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675080 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675088 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675096 Jul 20 21:18:09 Skynet kernel: md: disk14 read error, sector=1000675104 Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: mpt2sas_cm0: log_info(0x31120101): originator(PL), code(0x12), sub_code(0x0101) Jul 20 21:18:09 Skynet kernel: sd 8:0:20:0: device_block, handle(0x001f) Jul 20 21:18:10 Skynet kernel: sd 8:0:20:0: device_unblock and setting to running, handle(0x001f) Jul 20 21:18:10 Skynet kernel: sd 8:0:20:0: Power-on or device reset occurred Jul 20 21:19:24 Skynet flash_backup: adding task: /usr/local/emhttp/plugins/dynamix.my.servers/scripts/UpdateFlashBackup update
July 21, 20241 yr Author 3 hours ago, JorgeB said: Looks like a power/connection issue, for both disks. That’s what I was thinking as the smart data came back good. I’m 33% through the parity rebuild, should I let it continue or should I stop had check wires? I have no way to know if it’s power or breakout data cables causing the issue. I have no errors since this initial incident though.
July 21, 20241 yr Community Expert Check the syslog, if new errors are still being logged I would cancel, if not see if it can finish.
July 21, 20241 yr Author No read / write errors since last night, But I noticed these two kernel trap errors pop up. Any relevance?
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.