April 25, 20224 yr I was doing some configuration on my firewall which is running in a VM on my Unraid server. I noticed how some threads where 100% load and overall load was over 60% which was not normal for my 16 core Xeon. I waited, the CPU load was going down at some point but increased again. I restarted the server. After the server back I noticed the array troubeling to start up. partiy 2 and disk 4 where disabled (from 6 HDDs and I have 2 cache SSDs). I don't think the Firewall VM had anything to do with it. I will post logs and stuff I collected during this situation below. I now get errors when starting the array or rebuilding it. I don't thing this is due to bad drives or corruped data from my point of view and more being errors with my Adaptec Raid Controller somehow as first guess. I need my server back and up running, save as soon as possible. So if anyone can have a look at the logs and diagnostics, many thanks. Edited April 26, 20224 yr by aronmal
April 25, 20224 yr Author Apr 25 18:32:57 Tower kernel: ? md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? rmw5_write_data+0x17d/0x17d [md_mod] Apr 25 18:32:57 Tower kernel: md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? init_wait_entry+0x29/0x29 Apr 25 18:32:57 Tower kernel: ? md_seq_show+0x6c8/0x6c8 [md_mod] Apr 25 18:32:57 Tower kernel: kthread+0xde/0xe3 Apr 25 18:32:57 Tower kernel: ? set_kthread_struct+0x32/0x32 Apr 25 18:32:57 Tower kernel: ret_from_fork+0x22/0x30 Apr 25 18:32:57 Tower kernel: </TASK> Apr 25 18:32:57 Tower kernel: ---[ end trace 7c70e023e1918754 ]--- Apr 25 18:32:57 Tower kernel: DMAR: ERROR: DMA PTE for vPFN 0xac679 already set (to 160047801 not 1a7026803) Apr 25 18:32:57 Tower kernel: ------------[ cut here ]------------ Apr 25 18:32:57 Tower kernel: WARNING: CPU: 9 PID: 12007 at drivers/iommu/intel/iommu.c:2387 __domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Modules linked in: xfs md_mod nvidia(PO) m88rs6000t m88ds3103 i2c_mux regmap_i2c smipcie dvb_core mc ip6table_filter ip6_tables iptable_filter ip_tables x_tables ipmi_ssif ast drm_vram_helper drm_ttm_helper ttm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd drm rapl intel_cstate intel_uncore igb aacraid backlight i2c_i801 agpgart acpi_ipmi i2c_algo_bit syscopyarea i2c_smbus sysfillrect ahci sysimgblt i2c_core input_leds fb_sys_fops led_class libahci intel_pch_thermal wmi ipmi_si acpi_power_meter acpi_pad button Apr 25 18:32:57 Tower kernel: CPU: 9 PID: 12007 Comm: unraidd0 Tainted: P W O 5.15.27-Unraid #1 Apr 25 18:32:57 Tower kernel: Hardware name: bluechip bluechip SERVERline/X11SPL-F, BIOS 3.5 05/19/2021 Apr 25 18:32:57 Tower kernel: RIP: 0010:__domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Code: 2b 48 8b 4c 24 08 48 89 c2 4c 89 e6 48 c7 c7 d4 c7 ef 81 e8 b7 95 2c 00 8b 05 de 3c c3 00 85 c0 74 08 ff c8 89 05 d2 3c c3 00 <0f> 0b 8b 74 24 38 b8 34 00 00 00 8d 0c f6 83 e9 09 39 c1 0f 4f c8 Apr 25 18:32:57 Tower kernel: RSP: 0018:ffffc900038b7810 EFLAGS: 00010246 Apr 25 18:32:57 Tower kernel: RAX: 0000000000000000 RBX: ffff88819f1d33c8 RCX: 0000000000000000 Apr 25 18:32:57 Tower kernel: RDX: 0000000000000000 RSI: ffff88903fc5c550 RDI: ffff88903fc5c550 Apr 25 18:32:57 Tower kernel: RBP: ffff88810e7b8000 R08: ffff88907ffa70a8 R09: 0000000000000001 Apr 25 18:32:57 Tower kernel: R10: 00000000ffffffff R11: ffff888154cf9ab0 R12: 00000000000ac679 Apr 25 18:32:57 Tower kernel: R13: ffff88819f1d33c8 R14: 0000000000001000 R15: 00000000ac679000 Apr 25 18:32:57 Tower kernel: FS: 0000000000000000(0000) GS:ffff88903fc40000(0000) knlGS:0000000000000000 Apr 25 18:32:57 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 25 18:32:57 Tower kernel: CR2: 000014c46c267ff8 CR3: 000000000200a003 CR4: 00000000007706e0 Apr 25 18:32:57 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Apr 25 18:32:57 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Apr 25 18:32:57 Tower kernel: PKRU: 55555554 Apr 25 18:32:57 Tower kernel: Call Trace: Apr 25 18:32:57 Tower kernel: <TASK> Apr 25 18:32:57 Tower kernel: intel_iommu_map_pages+0xf3/0x102 Apr 25 18:32:57 Tower kernel: __iommu_map+0x138/0x211 Apr 25 18:32:57 Tower kernel: __iommu_map_sg+0x8c/0x110 Apr 25 18:32:57 Tower kernel: iommu_dma_map_sg+0x245/0x3e3 Apr 25 18:32:57 Tower kernel: __dma_map_sg_attrs+0x63/0x95 Apr 25 18:32:57 Tower kernel: dma_map_sg_attrs+0xa/0x12 Apr 25 18:32:57 Tower kernel: scsi_dma_map+0x34/0x3e Apr 25 18:32:57 Tower kernel: aac_build_sg+0x26/0xb8 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_32+0x23/0xa0 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_cmd+0x467/0xf94 [aacraid] Apr 25 18:32:57 Tower kernel: ? sd_init_command+0x572/0x934 Apr 25 18:32:57 Tower kernel: ? wbt_issue+0x16/0x3c Apr 25 18:32:57 Tower kernel: aac_queuecommand+0x17/0x21 [aacraid] Apr 25 18:32:57 Tower kernel: scsi_queue_rq+0x57d/0x6db Apr 25 18:32:57 Tower kernel: blk_mq_dispatch_rq_list+0x2a7/0x4da Apr 25 18:32:57 Tower kernel: __blk_mq_do_dispatch_sched+0x23d/0x281 Apr 25 18:32:57 Tower kernel: __blk_mq_sched_dispatch_requests+0xd5/0x129 Apr 25 18:32:57 Tower kernel: blk_mq_sched_dispatch_requests+0x2f/0x52 Apr 25 18:32:57 Tower kernel: __blk_mq_run_hw_queue+0x50/0x76 Apr 25 18:32:57 Tower kernel: __blk_mq_delay_run_hw_queue+0x4d/0x108 Apr 25 18:32:57 Tower kernel: blk_mq_sched_insert_requests+0xa2/0xd9 Apr 25 18:32:57 Tower kernel: blk_mq_flush_plug_list+0xfb/0x12c Apr 25 18:32:57 Tower kernel: blk_finish_plug+0x1f/0x2c Apr 25 18:32:57 Tower kernel: unraidd+0x11ed/0x1237 [md_mod] Apr 25 18:32:57 Tower kernel: ? md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? rmw5_write_data+0x17d/0x17d [md_mod] Apr 25 18:32:57 Tower kernel: md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? init_wait_entry+0x29/0x29 Apr 25 18:32:57 Tower kernel: ? md_seq_show+0x6c8/0x6c8 [md_mod] Apr 25 18:32:57 Tower kernel: kthread+0xde/0xe3 Apr 25 18:32:57 Tower kernel: ? set_kthread_struct+0x32/0x32 Apr 25 18:32:57 Tower kernel: ret_from_fork+0x22/0x30 Apr 25 18:32:57 Tower kernel: </TASK> Apr 25 18:32:57 Tower kernel: ---[ end trace 7c70e023e1918755 ]--- Apr 25 18:32:57 Tower kernel: DMAR: ERROR: DMA PTE for vPFN 0xac67a already set (to 10195c801 not 13698d803) Apr 25 18:32:57 Tower kernel: ------------[ cut here ]------------ Apr 25 18:32:57 Tower kernel: WARNING: CPU: 9 PID: 12007 at drivers/iommu/intel/iommu.c:2387 __domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Modules linked in: xfs md_mod nvidia(PO) m88rs6000t m88ds3103 i2c_mux regmap_i2c smipcie dvb_core mc ip6table_filter ip6_tables iptable_filter ip_tables x_tables ipmi_ssif ast drm_vram_helper drm_ttm_helper ttm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd drm rapl intel_cstate intel_uncore igb aacraid backlight i2c_i801 agpgart acpi_ipmi i2c_algo_bit syscopyarea i2c_smbus sysfillrect ahci sysimgblt i2c_core input_leds fb_sys_fops led_class libahci intel_pch_thermal wmi ipmi_si acpi_power_meter acpi_pad button Apr 25 18:32:57 Tower kernel: CPU: 9 PID: 12007 Comm: unraidd0 Tainted: P W O 5.15.27-Unraid #1 Apr 25 18:32:57 Tower kernel: Hardware name: bluechip bluechip SERVERline/X11SPL-F, BIOS 3.5 05/19/2021 Apr 25 18:32:57 Tower kernel: RIP: 0010:__domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Code: 2b 48 8b 4c 24 08 48 89 c2 4c 89 e6 48 c7 c7 d4 c7 ef 81 e8 b7 95 2c 00 8b 05 de 3c c3 00 85 c0 74 08 ff c8 89 05 d2 3c c3 00 <0f> 0b 8b 74 24 38 b8 34 00 00 00 8d 0c f6 83 e9 09 39 c1 0f 4f c8 Apr 25 18:32:57 Tower kernel: RSP: 0018:ffffc900038b7810 EFLAGS: 00010246 Apr 25 18:32:57 Tower kernel: RAX: 0000000000000000 RBX: ffff88819f1d33d0 RCX: 0000000000000003 Apr 25 18:32:57 Tower kernel: RDX: 0000000000000000 RSI: ffffc900038b76a0 RDI: 0000000000000001 Apr 25 18:32:57 Tower kernel: RBP: ffff88810e7b8000 R08: ffff88907ffa70a8 R09: 0000000000000001 Apr 25 18:32:57 Tower kernel: R10: 00000000ffffffff R11: ffff888154cf94f0 R12: 00000000000ac67a Apr 25 18:32:57 Tower kernel: R13: ffff88819f1d33d0 R14: 0000000000001000 R15: 00000000ac67a000 Apr 25 18:32:57 Tower kernel: FS: 0000000000000000(0000) GS:ffff88903fc40000(0000) knlGS:0000000000000000 Apr 25 18:32:57 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 25 18:32:57 Tower kernel: CR2: 000014c46c267ff8 CR3: 000000000200a003 CR4: 00000000007706e0 Apr 25 18:32:57 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Apr 25 18:32:57 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Apr 25 18:32:57 Tower kernel: PKRU: 55555554 Apr 25 18:32:57 Tower kernel: Call Trace: Apr 25 18:32:57 Tower kernel: <TASK> Apr 25 18:32:57 Tower kernel: intel_iommu_map_pages+0xf3/0x102 Apr 25 18:32:57 Tower kernel: __iommu_map+0x138/0x211 Apr 25 18:32:57 Tower kernel: __iommu_map_sg+0x8c/0x110 Apr 25 18:32:57 Tower kernel: iommu_dma_map_sg+0x245/0x3e3 Apr 25 18:32:57 Tower kernel: __dma_map_sg_attrs+0x63/0x95 Apr 25 18:32:57 Tower kernel: dma_map_sg_attrs+0xa/0x12 Apr 25 18:32:57 Tower kernel: scsi_dma_map+0x34/0x3e Apr 25 18:32:57 Tower kernel: aac_build_sg+0x26/0xb8 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_32+0x23/0xa0 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_cmd+0x467/0xf94 [aacraid] Apr 25 18:32:57 Tower kernel: ? sd_init_command+0x572/0x934 Apr 25 18:32:57 Tower kernel: ? wbt_issue+0x16/0x3c Apr 25 18:32:57 Tower kernel: aac_queuecommand+0x17/0x21 [aacraid] Apr 25 18:32:57 Tower kernel: scsi_queue_rq+0x57d/0x6db Apr 25 18:32:57 Tower kernel: blk_mq_dispatch_rq_list+0x2a7/0x4da Apr 25 18:32:57 Tower kernel: __blk_mq_do_dispatch_sched+0x23d/0x281 Apr 25 18:32:57 Tower kernel: __blk_mq_sched_dispatch_requests+0xd5/0x129 Apr 25 18:32:57 Tower kernel: blk_mq_sched_dispatch_requests+0x2f/0x52 Apr 25 18:32:57 Tower kernel: __blk_mq_run_hw_queue+0x50/0x76 Apr 25 18:32:57 Tower kernel: __blk_mq_delay_run_hw_queue+0x4d/0x108 Apr 25 18:32:57 Tower kernel: blk_mq_sched_insert_requests+0xa2/0xd9 Apr 25 18:32:57 Tower kernel: blk_mq_flush_plug_list+0xfb/0x12c Apr 25 18:32:57 Tower kernel: blk_finish_plug+0x1f/0x2c Apr 25 18:32:57 Tower kernel: unraidd+0x11ed/0x1237 [md_mod] Apr 25 18:32:57 Tower kernel: ? md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? rmw5_write_data+0x17d/0x17d [md_mod] Apr 25 18:32:57 Tower kernel: md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? init_wait_entry+0x29/0x29 Apr 25 18:32:57 Tower kernel: ? md_seq_show+0x6c8/0x6c8 [md_mod] Apr 25 18:32:57 Tower kernel: kthread+0xde/0xe3 Apr 25 18:32:57 Tower kernel: ? set_kthread_struct+0x32/0x32 Apr 25 18:32:57 Tower kernel: ret_from_fork+0x22/0x30 Apr 25 18:32:57 Tower kernel: </TASK> Apr 25 18:32:57 Tower kernel: ---[ end trace 7c70e023e1918756 ]--- Apr 25 18:32:57 Tower kernel: DMAR: ERROR: DMA PTE for vPFN 0xac67b already set (to 19f096801 not 156fae803) Apr 25 18:32:57 Tower kernel: ------------[ cut here ]------------ Apr 25 18:32:57 Tower kernel: WARNING: CPU: 9 PID: 12007 at drivers/iommu/intel/iommu.c:2387 __domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Modules linked in: xfs md_mod nvidia(PO) m88rs6000t m88ds3103 i2c_mux regmap_i2c smipcie dvb_core mc ip6table_filter ip6_tables iptable_filter ip_tables x_tables ipmi_ssif ast drm_vram_helper drm_ttm_helper ttm x86_pkg_temp_thermal intel_powerclamp coretemp kvm_intel kvm drm_kms_helper crct10dif_pclmul crc32_pclmul crc32c_intel ghash_clmulni_intel aesni_intel crypto_simd cryptd drm rapl intel_cstate intel_uncore igb aacraid backlight i2c_i801 agpgart acpi_ipmi i2c_algo_bit syscopyarea i2c_smbus sysfillrect ahci sysimgblt i2c_core input_leds fb_sys_fops led_class libahci intel_pch_thermal wmi ipmi_si acpi_power_meter acpi_pad button Apr 25 18:32:57 Tower kernel: CPU: 9 PID: 12007 Comm: unraidd0 Tainted: P W O 5.15.27-Unraid #1 Apr 25 18:32:57 Tower kernel: Hardware name: bluechip bluechip SERVERline/X11SPL-F, BIOS 3.5 05/19/2021 Apr 25 18:32:57 Tower kernel: RIP: 0010:__domain_mapping+0x2e5/0x390 Apr 25 18:32:57 Tower kernel: Code: 2b 48 8b 4c 24 08 48 89 c2 4c 89 e6 48 c7 c7 d4 c7 ef 81 e8 b7 95 2c 00 8b 05 de 3c c3 00 85 c0 74 08 ff c8 89 05 d2 3c c3 00 <0f> 0b 8b 74 24 38 b8 34 00 00 00 8d 0c f6 83 e9 09 39 c1 0f 4f c8 Apr 25 18:32:57 Tower kernel: RSP: 0018:ffffc900038b7810 EFLAGS: 00010246 Apr 25 18:32:57 Tower kernel: RAX: 0000000000000000 RBX: ffff88819f1d33d8 RCX: 0000000000000000 Apr 25 18:32:57 Tower kernel: RDX: 0000000000000000 RSI: ffff88903fc5c550 RDI: ffff88903fc5c550 Apr 25 18:32:57 Tower kernel: RBP: ffff88810e7b8000 R08: ffff88907ffa70a8 R09: 0000000000000001 Apr 25 18:32:57 Tower kernel: R10: 00000000ffffffff R11: ffff888154cf8910 R12: 00000000000ac67b Apr 25 18:32:57 Tower kernel: R13: ffff88819f1d33d8 R14: 0000000000001000 R15: 00000000ac67b000 Apr 25 18:32:57 Tower kernel: FS: 0000000000000000(0000) GS:ffff88903fc40000(0000) knlGS:0000000000000000 Apr 25 18:32:57 Tower kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 Apr 25 18:32:57 Tower kernel: CR2: 000014c46c267ff8 CR3: 000000000200a003 CR4: 00000000007706e0 Apr 25 18:32:57 Tower kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Apr 25 18:32:57 Tower kernel: DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 Apr 25 18:32:57 Tower kernel: PKRU: 55555554 Apr 25 18:32:57 Tower kernel: Call Trace: Apr 25 18:32:57 Tower kernel: <TASK> Apr 25 18:32:57 Tower kernel: intel_iommu_map_pages+0xf3/0x102 Apr 25 18:32:57 Tower kernel: __iommu_map+0x138/0x211 Apr 25 18:32:57 Tower kernel: __iommu_map_sg+0x8c/0x110 Apr 25 18:32:57 Tower kernel: iommu_dma_map_sg+0x245/0x3e3 Apr 25 18:32:57 Tower kernel: __dma_map_sg_attrs+0x63/0x95 Apr 25 18:32:57 Tower kernel: dma_map_sg_attrs+0xa/0x12 Apr 25 18:32:57 Tower kernel: scsi_dma_map+0x34/0x3e Apr 25 18:32:57 Tower kernel: aac_build_sg+0x26/0xb8 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_32+0x23/0xa0 [aacraid] Apr 25 18:32:57 Tower kernel: aac_scsi_cmd+0x467/0xf94 [aacraid] Apr 25 18:32:57 Tower kernel: ? sd_init_command+0x572/0x934 Apr 25 18:32:57 Tower kernel: ? wbt_issue+0x16/0x3c Apr 25 18:32:57 Tower kernel: aac_queuecommand+0x17/0x21 [aacraid] Apr 25 18:32:57 Tower kernel: scsi_queue_rq+0x57d/0x6db Apr 25 18:32:57 Tower kernel: blk_mq_dispatch_rq_list+0x2a7/0x4da Apr 25 18:32:57 Tower kernel: __blk_mq_do_dispatch_sched+0x23d/0x281 Apr 25 18:32:57 Tower kernel: __blk_mq_sched_dispatch_requests+0xd5/0x129 Apr 25 18:32:57 Tower kernel: blk_mq_sched_dispatch_requests+0x2f/0x52 Apr 25 18:32:57 Tower kernel: __blk_mq_run_hw_queue+0x50/0x76 Apr 25 18:32:57 Tower kernel: __blk_mq_delay_run_hw_queue+0x4d/0x108 Apr 25 18:32:57 Tower kernel: blk_mq_sched_insert_requests+0xa2/0xd9 Apr 25 18:32:57 Tower kernel: blk_mq_flush_plug_list+0xfb/0x12c Apr 25 18:32:57 Tower kernel: blk_finish_plug+0x1f/0x2c Apr 25 18:32:57 Tower kernel: unraidd+0x11ed/0x1237 [md_mod] Apr 25 18:32:57 Tower kernel: ? md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? rmw5_write_data+0x17d/0x17d [md_mod] Apr 25 18:32:57 Tower kernel: md_thread+0x103/0x12a [md_mod] Apr 25 18:32:57 Tower kernel: ? init_wait_entry+0x29/0x29 Apr 25 18:32:57 Tower kernel: ? md_seq_show+0x6c8/0x6c8 [md_mod] Apr 25 18:32:57 Tower kernel: kthread+0xde/0xe3 Apr 25 18:32:57 Tower kernel: ? set_kthread_struct+0x32/0x32 Apr 25 18:32:57 Tower kernel: ret_from_fork+0x22/0x30 Apr 25 18:32:57 Tower kernel: </TASK> Apr 25 18:32:57 Tower kernel: ---[ end trace 7c70e023e1918757 ]--- Apr 25 18:32:57 Tower kernel: DMAR: DRHD: handling fault status reg 2 Apr 25 18:32:57 Tower kernel: DMAR: [DMA Write NO_PASID] Request device [66:00.0] fault addr 0xac670000 [fault reason 0x05] PTE Write access is not set Apr 25 18:32:57 Tower kernel: DMAR: [DMA Write NO_PASID] Request device [66:00.0] fault addr 0xac671000 [fault reason 0x05] PTE Write access is not set Apr 25 18:32:57 Tower kernel: DMAR: DRHD: handling fault status reg 200 Apr 25 18:32:57 Tower kernel: DMAR: DRHD: handling fault status reg 202 Apr 25 18:32:57 Tower kernel: DMAR: [DMA Write NO_PASID] Request device [66:00.0] fault addr 0xac672000 [fault reason 0x05] PTE Write access is not set Apr 25 18:32:57 Tower kernel: DMAR: [DMA Write NO_PASID] Request device [66:00.0] fault addr 0xac673000 [fault reason 0x05] PTE Write access is not set Apr 25 18:32:57 Tower kernel: DMAR: [DMA Write NO_PASID] Request device [66:00.0] fault addr 0xac673000 [fault reason 0x05] PTE Write access is not set Apr 25 18:33:07 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:07 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:24 Tower sshd[16066]: Connection from 10.1.0.193 port 35322 on 10.1.0.15 port 22 rdomain "" Apr 25 18:33:24 Tower sshd[16066]: Failed publickey for root from 10.1.0.193 port 35322 ssh2: RSA SHA256:60UN6V0VaNRuoWiAuF7EhnvmNT4BgJVi7Px/pyOTtoA Apr 25 18:33:24 Tower sshd[16066]: Postponed keyboard-interactive for root from 10.1.0.193 port 35322 ssh2 [preauth] Apr 25 18:33:27 Tower sshd[16066]: Postponed keyboard-interactive/pam for root from 10.1.0.193 port 35322 ssh2 [preauth] Apr 25 18:33:27 Tower sshd[16066]: Accepted keyboard-interactive/pam for root from 10.1.0.193 port 35322 ssh2 Apr 25 18:33:27 Tower sshd[16066]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0) Apr 25 18:33:27 Tower elogind-daemon[1456]: New session c1 of user root. Apr 25 18:33:27 Tower sshd[16066]: Starting session: shell on pts/0 for root from 10.1.0.193 port 35322 id 0 Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:33:37 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:33:37 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-14 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 18:33:37 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 18:33:49 Tower kernel: mdcmd (37): nocheck Pause Apr 25 18:34:47 Tower kernel: aacraid 0000:66:00.0: IOP reset failed Apr 25 18:34:47 Tower kernel: aacraid 0000:66:00.0: ARC Reset attempt failed Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#391 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#391 CDB: opcode=0x2a 2a 00 00 20 3a 20 00 00 20 00 Apr 25 18:34:47 Tower kernel: print_req_error: 58 callbacks suppressed Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2112032 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: btrfs_dev_stat_print_on_error: 83 callbacks suppressed Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 151, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#392 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#392 CDB: opcode=0x2a 2a 00 00 20 3a 60 00 00 40 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2112096 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 152, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 153, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#393 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#393 CDB: opcode=0x2a 2a 00 00 20 39 a0 00 00 60 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2111904 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 154, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#394 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#394 CDB: opcode=0x2a 2a 00 00 20 3c 00 00 01 60 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2112512 op 0x1:(WRITE) flags 0x1800 phys_seg 44 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 155, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 156, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 157, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#395 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#395 CDB: opcode=0x2a 2a 00 00 20 37 a0 00 00 40 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2111392 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 158, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#396 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#396 CDB: opcode=0x2a 2a 00 00 20 38 00 00 00 20 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2111488 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 159, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#397 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#397 CDB: opcode=0x2a 2a 00 00 20 3b c0 00 00 20 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2112448 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 160, rd 14, flush 6, corrupt 0, gen 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#398 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#398 CDB: opcode=0x2a 2a 00 00 20 3e 00 00 00 20 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2113024 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#399 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#399 CDB: opcode=0x2a 2a 00 00 20 3e 40 00 00 20 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2113088 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#400 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=185s Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: [sdi] tag#400 CDB: opcode=0x2a 2a 00 00 20 3f 40 00 00 20 00 Apr 25 18:34:47 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2113344 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:34:47 Tower kernel: sd 1:1:7:0: rejecting I/O to offline device Apr 25 18:34:47 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 18:34:47 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 18:34:47 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 18:03:42 Tower emhttpd: shcmd (123): mkdir -p /mnt/cache Apr 25 18:03:42 Tower emhttpd: /mnt/cache uuid: dfc0c8b4-bb27-44d2-b15e-006bbee148b4 Apr 25 18:03:42 Tower emhttpd: /mnt/cache found: 2 Apr 25 18:03:42 Tower emhttpd: /mnt/cache extra: 0 Apr 25 18:03:42 Tower emhttpd: /mnt/cache missing: 0 Apr 25 18:03:43 Tower emhttpd: /mnt/cache Label: none uuid: dfc0c8b4-bb27-44d2-b15e-006bbee148b4 Apr 25 18:03:43 Tower emhttpd: /mnt/cache Total devices 2 FS bytes used 1.17TiB Apr 25 18:03:43 Tower emhttpd: /mnt/cache devid 1 size 1.82TiB used 1.37TiB path /dev/sdh1 Apr 25 18:03:43 Tower emhttpd: /mnt/cache devid 2 size 1.82TiB used 1.37TiB path /dev/sdi1 Apr 25 18:03:43 Tower emhttpd: shcmd (124): mount -t btrfs -o noatime,space_cache=v2,discard=async -U dfc0c8b4-bb27-44d2-b15e-006bbee148b4 /mnt/cache Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): flagging fs with big metadata feature Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): turning on async discard Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): using free space tree Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): has skinny extents Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): bdev /dev/sdh1 errs: wr 0, rd 0, flush 0, corrupt 16, gen 0 Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): bdev /dev/sdi1 errs: wr 71, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:03:43 Tower sshd[9267]: Connection from 10.1.0.193 port 35320 on 10.1.0.15 port 22 rdomain "" Apr 25 18:03:43 Tower sshd[9267]: Failed publickey for root from 10.1.0.193 port 35320 ssh2: RSA SHA256:60UN6V0VaNRuoWiAuF7EhnvmNT4BgJVi7Px/pyOTtoA Apr 25 18:03:43 Tower sshd[9267]: Postponed keyboard-interactive for root from 10.1.0.193 port 35320 ssh2 [preauth] Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): enabling ssd optimizations Apr 25 18:03:43 Tower kernel: BTRFS info (device sdh1): checking UUID tree Apr 25 18:03:44 Tower emhttpd: shcmd (125): sync Apr 25 18:03:44 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-1 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-0 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 18:03:44 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 18:03:47 Tower sshd[9267]: Postponed keyboard-interactive/pam for root from 10.1.0.193 port 35320 ssh2 [preauth] Apr 25 18:03:47 Tower sshd[9267]: Accepted keyboard-interactive/pam for root from 10.1.0.193 port 35320 ssh2 Apr 25 18:03:47 Tower sshd[9267]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0) Apr 25 18:03:47 Tower elogind-daemon[1457]: New session c1 of user root. Apr 25 18:03:47 Tower sshd[9267]: Starting session: shell on pts/0 for root from 10.1.0.193 port 35320 id 0 Apr 25 18:04:20 Tower kernel: aacraid 0000:66:00.0: IOP reset succeeded Apr 25 18:04:20 Tower kernel: aacraid: Comm Interface type2 enabled Apr 25 18:04:29 Tower kernel: aacraid 0000:66:00.0: Scheduling bus rescan Apr 25 18:05:09 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:09 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host adapter abort request. Apr 25 18:05:40 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 18:05:40 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-24 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 18:05:40 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 18:06:38 Tower nginx: 2022/04/25 18:06:38 [error] 5206#5206: *597 upstream timed out (110: Connection timed out) while reading upstream, client: 10.1.0.193, server: , request: "POST /update.htm HTTP/1.1", upstream: "http://unix:/var/run/emhttpd.socket:/update.htm", host: "10.1.0.15", referrer: "http://10.1.0.15/Main" Apr 25 18:06:50 Tower kernel: aacraid 0000:66:00.0: IOP reset failed Apr 25 18:06:50 Tower kernel: aacraid 0000:66:00.0: ARC Reset attempt failed Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: Device offlined - not ready after error recovery Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#326 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#326 CDB: opcode=0x2a 2a 00 00 20 10 e0 00 01 80 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2101472 op 0x1:(WRITE) flags 0x1800 phys_seg 47 prio class 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 72, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 73, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 74, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 75, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#328 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#328 CDB: opcode=0x2a 2a 00 00 20 12 80 00 02 00 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2101888 op 0x1:(WRITE) flags 0x1800 phys_seg 64 prio class 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 76, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 77, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 78, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 79, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#342 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#342 CDB: opcode=0x2a 2a 00 00 20 14 a0 00 00 40 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2102432 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 80, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#343 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#343 CDB: opcode=0x2a 2a 00 00 20 15 40 00 00 20 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2102592 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:06:50 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 81, rd 0, flush 6, corrupt 0, gen 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#344 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#344 CDB: opcode=0x2a 2a 00 00 20 15 a0 00 00 20 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2102688 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#345 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#345 CDB: opcode=0x2a 2a 00 00 20 15 e0 00 00 80 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2102752 op 0x1:(WRITE) flags 0x1800 phys_seg 16 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#346 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#346 CDB: opcode=0x2a 2a 00 00 20 16 80 00 00 20 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2102912 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#347 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#347 CDB: opcode=0x2a 2a 00 00 20 17 a0 00 00 20 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2103200 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#348 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#348 CDB: opcode=0x2a 2a 00 00 20 17 e0 00 00 40 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2103264 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#349 UNKNOWN(0x2003) Result: hostbyte=0x03 driverbyte=DRIVER_OK cmd_age=130s Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: [sdi] tag#349 CDB: opcode=0x2a 2a 00 00 20 18 40 00 00 40 00 Apr 25 18:06:50 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2103360 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 18:06:50 Tower kernel: sd 1:1:7:0: rejecting I/O to offline device Apr 25 18:06:58 Tower ntpd[1484]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized Apr 25 18:07:50 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: Adapter health - -3 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-0 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-18 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 18:07:50 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#809 device offline or changed Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#823 device offline or changed Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#824 device offline or changed Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#770 device offline or changed Apr 25 17:27:12 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#771 device offline or changed Apr 25 17:27:12 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 17:27:12 Tower kernel: sd 1:1:7:0: [sdi] tag#772 device offline or changed Apr 25 17:27:12 Tower kernel: BTRFS warning (device sdh1): lost page write due to IO error on /dev/sdi1 (-5) Apr 25 17:27:12 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:27:12 Tower kernel: BTRFS info (device sdh1): checking UUID tree Apr 25 17:27:14 Tower emhttpd: shcmd (46): sync Apr 25 17:27:17 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-1 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-0 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:27:17 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:27:45 Tower webGUI: Successful login user root from 10.1.0.193 Apr 25 17:27:51 Tower kernel: aacraid 0000:66:00.0: IOP reset succeeded Apr 25 17:27:51 Tower kernel: aacraid: Comm Interface type2 enabled Apr 25 17:28:00 Tower kernel: aacraid 0000:66:00.0: Scheduling bus rescan Apr 25 17:28:41 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:28:41 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:29:11 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:29:11 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:29:11 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:29:11 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:29:11 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:29:11 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:29:11 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:29:11 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:29:11 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-3 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:29:11 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:29:45 Tower kernel: aacraid 0000:66:00.0: IOP reset succeeded Apr 25 17:29:45 Tower kernel: aacraid: Comm Interface type2 enabled Apr 25 17:29:54 Tower kernel: aacraid 0000:66:00.0: Scheduling bus rescan Apr 25 17:30:02 Tower shutdown[9779]: shutting down for system reboot Apr 25 17:30:04 Tower kernel: sdi: detected capacity change from 3907029168 to 0 Apr 25 17:30:04 Tower kernel: sd 1:1:7:0: [sdi] tag#404 access beyond end of device Apr 25 17:30:04 Tower kernel: print_req_error: 333 callbacks suppressed Apr 25 17:30:04 Tower kernel: blk_update_request: I/O error, dev sdi, sector 7454656 op 0x0:(READ) flags 0x1000 phys_seg 4 prio class 0 Apr 25 17:30:04 Tower kernel: btrfs_dev_stat_print_on_error: 500 callbacks suppressed Apr 25 17:30:04 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 582, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:04 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-1 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-0 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:30:04 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:30:38 Tower kernel: aacraid 0000:66:00.0: IOP reset succeeded Apr 25 17:30:38 Tower kernel: aacraid: Comm Interface type2 enabled Apr 25 17:30:47 Tower kernel: aacraid 0000:66:00.0: Scheduling bus rescan Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#915 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282450784 op 0x1:(WRITE) flags 0x1800 phys_seg 4 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 583, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#916 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282451136 op 0x1:(WRITE) flags 0x1800 phys_seg 11 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 584, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 585, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#917 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282451328 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 586, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#918 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282451616 op 0x1:(WRITE) flags 0x1800 phys_seg 16 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 587, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 588, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#919 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282451968 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 589, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#920 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282452608 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 590, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#921 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282453728 op 0x1:(WRITE) flags 0x1800 phys_seg 12 prio class 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 591, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: BTRFS error (device sdh1): bdev /dev/sdi1 errs: wr 592, rd 1, flush 6, corrupt 0, gen 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#922 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282454080 op 0x1:(WRITE) flags 0x1800 phys_seg 20 prio class 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#923 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 2282455840 op 0x1:(WRITE) flags 0x1800 phys_seg 8 prio class 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#924 access beyond end of device Apr 25 17:30:57 Tower kernel: blk_update_request: I/O error, dev sdi, sector 7454656 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0 Apr 25 17:30:57 Tower kernel: sd 1:1:7:0: [sdi] tag#925 access beyond end of device Apr 25 17:30:57 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-1 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-0 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:30:57 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:31:16 Tower ntpd[1484]: kernel reports TIME_ERROR: 0x41: Clock Unsynchronized Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#1000 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#48 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#49 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#51 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#52 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#911 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#912 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#914 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#56 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#57 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#59 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#0 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#917 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#918 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#920 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#2 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#4 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#6 device offline or changed Apr 25 17:00:45 Tower kernel: BTRFS error (device sdh1): error writing primary super block to device 2 Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#12 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#16 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#13 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#14 device offline or changed Apr 25 17:00:45 Tower kernel: sd 1:1:7:0: [sdi] tag#15 device offline or changed [...] Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#1011 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#256 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#1012 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#1013 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#1014 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#1015 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#962 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#963 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#990 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#991 device offline or changed Apr 25 17:03:39 Tower kernel: sd 1:1:7:0: [sdi] tag#992 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#39 device offline or changed Apr 25 17:03:41 Tower kernel: btrfs_printk: 3143 callbacks suppressed Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9a0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9b8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9c0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9c8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9d8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9e0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9e8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9d0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9f0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fa9f8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa08 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa00 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa10 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa18 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa20 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa28 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa30 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa40 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa48 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa38 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa50 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa58 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa60 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa68 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa70 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa78 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa80 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa88 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa90 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faa98 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faaa0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faaa8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faab0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faab8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faac0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faac8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faad0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faad8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faaf0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faae8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faae0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faaf8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab00 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab08 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab10 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab18 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab20 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab28 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab30 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab38 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab40 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab48 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab50 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab58 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab60 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab70 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab68 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab78 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab80 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab88 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab90 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fab98 len 0 err no 10 Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#40 device offline or changed Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faba0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160faba8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabb0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#41 device offline or changed Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabb8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabc0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabc8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabd0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabd8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabe0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabe8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabf0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fabf8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac08 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac00 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac10 len 0 err no 10 Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#42 device offline or changed Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac18 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac20 len 0 err no 10 Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#43 device offline or changed Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac28 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac30 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac40 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac38 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac48 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x160fac50 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3ee8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3ef0 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f00 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3ef8 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f08 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f10 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f18 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f20 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f30 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f28 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f38 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f40 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f48 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f50 len 0 err no 10 Apr 25 17:03:41 Tower kernel: BTRFS warning (device sdh1): direct IO failed ino 2102925 rw 0,0 sector 0x1a1c3f58 len 0 err no 10 Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#44 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#45 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#47 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#48 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#277 device offline or changed [...] Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#319 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#256 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#257 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#258 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#259 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#260 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#261 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#262 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#263 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#264 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#265 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#266 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#267 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#268 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#269 device offline or changed Apr 25 17:03:41 Tower kernel: sd 1:1:7:0: [sdi] tag#270 device offline or changed Apr 25 17:04:08 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:08 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:04:39 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:04:39 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-88 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:04:39 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Apr 25 17:04:50 Tower webGUI: Successful login user root from 10.1.0.193 Apr 25 17:05:12 Tower kernel: aacraid 0000:66:00.0: IOP reset succeeded Apr 25 17:05:12 Tower kernel: aacraid: Comm Interface type2 enabled Apr 25 17:05:21 Tower kernel: aacraid 0000:66:00.0: Scheduling bus rescan Apr 25 17:05:56 Tower sshd[26550]: Connection from 10.1.0.193 port 35310 on 10.1.0.15 port 22 rdomain "" Apr 25 17:05:56 Tower sshd[26550]: Failed publickey for root from 10.1.0.193 port 35310 ssh2: RSA SHA256:60UN6V0VaNRuoWiAuF7EhnvmNT4BgJVi7Px/pyOTtoA Apr 25 17:05:56 Tower sshd[26550]: Postponed keyboard-interactive for root from 10.1.0.193 port 35310 ssh2 [preauth] Apr 25 17:05:59 Tower sshd[26550]: Postponed keyboard-interactive/pam for root from 10.1.0.193 port 35310 ssh2 [preauth] Apr 25 17:05:59 Tower sshd[26550]: Accepted keyboard-interactive/pam for root from 10.1.0.193 port 35310 ssh2 Apr 25 17:05:59 Tower sshd[26550]: pam_unix(sshd:session): session opened for user root(uid=0) by (uid=0) Apr 25 17:05:59 Tower elogind-daemon[1452]: New session c2 of user root. Apr 25 17:05:59 Tower sshd[26550]: Starting session: shell on pts/1 for root from 10.1.0.193 port 35310 id 0 Apr 25 17:06:02 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:06:02 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:06:32 Tower kernel: aacraid: Host adapter abort request. Apr 25 17:06:32 Tower kernel: aacraid: Outstanding commands on (1,1,7,0): Apr 25 17:06:32 Tower kernel: aacraid: Host bus reset request. SCSI hang ? Apr 25 17:06:32 Tower kernel: sdi: detected capacity change from 3907029168 to 0 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: midlevel-0 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: lowlevel-0 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: error handler-88 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: firmware-0 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: outstanding cmd: kernel-0 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: Controller reset type is 3 Apr 25 17:06:32 Tower kernel: aacraid 0000:66:00.0: Issuing IOP reset Edited April 25, 20224 yr by aronmal
April 26, 20224 yr Community Expert 14 hours ago, aronmal said: my Adaptec Raid Controller somehow as first guess. It would also be my main suspect.
May 4, 20233 yr I have been having the exact same issue. 2 drives not showing up and the same errors in the logs. You can see my post here. Did you ever fix your issue?
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.