Run xfs_repair again without -n or nothing will be done.

It says:
xfs_repair: libxfs_device_zero write failed: Input/output error

disk2 checkfs_log.txt

Post new diags after running xfs_repair.

Here it is. It's after power shutdown and fresh start, not immediately after xfs_repair - is that OK?

Now both Disk2 and Disk3 show "Unmountable: wrong or no filesystem..."

linertower-diagnostics-20230424-2052.zip

Edited April 24, 20233 yr by mamaraci

3 minutes ago, mamaraci said:

not immediately after xfs_repair - is that OK?

A reboot clears the log, did you run xfs_repair again?

2 minutes ago, JorgeB said:

A reboot clears the log, did you run xfs_repair again?

Oh. 

Ok, now I'm running xfs_repair(s) with -v only for the corrections on my both disks 2 and 3 - since there are both already "Unmountable", then in this session, before reboot I'm making the fresh diags, and post it here. 
Right? Sorry, feeling realy dumb... 

So, I've run xfs_repair(s) with modify flags, both on Disks 2 and 3, and extracted the diags immediately after that. Hope it would help... Thanks in advance!

 

linertower-diagnostics-20230425-1021.zip

Edited April 25, 20233 yr by mamaraci

1 hour ago, mamaraci said:

So, I've run xfs_repair(s) with modify flags, both on Disks 2 and 3, and extracted the diags immediately after that. Hope it would help... Thanks in advance!

 

linertower-diagnostics-20230425-1021.zip 206.91 kB · 2 downloads

You should restart the array in normal mode and then get new diagnostics so we can see if the problem drives now mount OK.

Apr 24 21:19:52 LinerTower kernel: ata5.00: failed command: READ FPDMA QUEUED
Apr 24 21:19:52 LinerTower kernel: ata5.00: cmd 60/80:80:18:df:e0/00:00:e8:00:00/40 tag 16 ncq dma 65536 in
Apr 24 21:19:52 LinerTower kernel:         res 41/40:00:18:df:e0/00:00:e8:00:00/40 Emask 0x409 (media error) <F>
Apr 24 21:19:52 LinerTower kernel: ata5.00: status: { DRDY ERR }
Apr 24 21:19:52 LinerTower kernel: ata5.00: error: { UNC }

 

Disk2 appears to be failing, and since there's already a disabled disk it would be a problem, run an extended SMART test on it and post new diags.

Ok, I've run again extended tests for both Disk2 and Disk3. For Disk3 it doesn't even launch, showing "completed" just in a couple of seconds after launch.

 

On the Disk2 it worked all night, it was 90% in the morning today, and after that it's stopped at 90% with "Interrupted (host reset)" and after the relaunch with the "Aborted by host" error.

 

Actually, I'm already OK with the data loss that I would have in an amount that is visible now - some folders are gone, and the rest I can see and use. 

Luckily, those that are already lost are the most sensitive and, thus, backed-up to Amazon Glacier in full.

 

Of course I would feel the pain downloading it back from Glacier (I've tried already, and Amazon would obviously force me to buy "Provisioned capacity unit" to download it... 

 

So, probably should I already go for the disks swap?.. and the question is the same: the proper order, not to ruin things further? Disk2 or Disk3 should be replaced first?  

And, second question should I backup the rest of the files (that visible now), since during these disks swaps something can be lost additionally vs what I've already lost? 

 

Thanks for your attention, really appreciate your support! 

linertower-diagnostics-20230426-0754.zip

 

PS. Also decided to change Power supply before disks swapping - just in case, probably, current one is behaving poorly already... Ordered a new one. 

Edited April 26, 20233 yr by mamaraci

If you are OK with loosing all data on disk2 and already saved anything important from the emulated disk3, you can do a new config with a new disk2 and the remaining disks and then re-sync parity.

Thanks, @JorgeB!

So, my way would be:

1) save everything from Disk 3 - or check at least that it's backed-up - 1st priority

2) replace the power supply

3) change the disk 2, rebuilt

4) change the disk 3, rebuilt

 

You cannot rebuild disk2 with single parity and disk3 disabled.

1 hour ago, JorgeB said:

You cannot rebuild disk2 with single parity and disk3 disabled.

ok clear, so would start with disk 3 and and disk 2 as a new.

2 hours ago, JorgeB said:

new config with a new disk2 and the remaining disks and then re-sync parity.

https://wiki.unraid.net/Manual/Tools#New_Config

Thanks guys for your support! I've replaced Power Supply (no changes after that), then replaced Disk2 with a new one. It started clicking ike a hell (one of 3 Ironwolfs bought for the replacement creates clicky sounds - returned it), so I've stopped, inserted another one. Recreated parity after New Config, and currently moving with Unbalance everything from Disk 3 to new Disk 2 inside the array.

Then the plan is to replace Disk 3 with a new disk, then get rid of Disk 4 at all. 

 

The question I have now: on disk 3 there is (looks like non-corrupted) a folder, that is just missing in the Shares. 

So, I have the Share "videos", and it's not accessible anymore by SMB, it's not visible in/mnt/user/, but it exisits on /mnt/Disk3/ directly, with its contents inside. What's the proper way of restoring it to become a normal share? 

Please post the diagnostics.

After moving the content from Disk3 to new replaced Disk2 share that I was missing returned now and visible & accessible. Thanks a lot!

linertower-diagnostics-20230503-0738.zip

So everything is OK with the shares now?

 

You need to scrub your pool, lots of corruption detected, also a good idea to run memtest.

18 minutes ago, JorgeB said:

So everything is OK with the shares now?

You need to scrub your pool, lots of corruption detected, also a good idea to run memtest.

Thanks for your support! Yes, shares look OK now.

As for the cache /pool, i've run the checks, it shows the errors, but I think it doesn't correct anything - would look more carefuly into this. Also would search for the memtest as well. Thanks again! 

1 hour ago, mamaraci said:

but I think it doesn't correct anything

It can't correct single it's a single device pool, run memtest when possible.