Had to review the thread a little

On 7/29/2022 at 5:41 PM, trurl said:

Missing/emulated disk8 is mounted as XFS and has 1.3T data

On 7/29/2022 at 5:52 PM, trurl said:

Disk2 has 6 reallocated and 14 pending. Could be worse I guess.

 

You can see how much of each disk is used by each user share by clicking Compute... on the User Shares page, or Compute All button at bottom.

On 7/29/2022 at 5:52 PM, trurl said:

prioritize backups of anything important and irreplaceable before anything else

Can you copy that data to somewhere off the array?

 

2 minutes ago, sabarirjpm said:

No got it as used drives. 3 Nos. SMART says healthy on this. 

Does look OK but hasn't had extended self-test

Current parity looks OK so parity copy part of the parity swap should be fine.

 

The reason I am asking about backups is because I am concerned disk2 might be a problem while rebuilding disk8 to the (will be) former parity disk.

 

Disk2 is currently involved in emulating disk8 anyway whether used for copying emulated disk8 data or for disk8 rebuild, but might not be involved as much for just copying the data from emulated disk8 as it would be emulating the full disk8 for a rebuild.

  

On 7/29/2022 at 5:52 PM, trurl said:

You can see how much of each disk is used by each user share by clicking Compute... on the User Shares page, or Compute All button at bottom

 

Here is what I see for your shares in your anonymized diagnostics, but it doesn't really tell me the full name of the shares or how much is on each disk for each share.

Spoiler

4-------s                         shareUseCache="no"      Exists on disk6, disk8, disk9
B--------------d                  shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7
C--------s                        shareUseCache="no"      Exists on disk3
d--a                              shareUseCache="no"      Exists on disk5
D------------------m              shareUseCache="no"      Exists on disk2, disk5, disk6, disk9
G-----------h                     shareUseCache="no"      Exists on disk1, disk2, disk3
H---------------h                 shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7, disk8, disk9
H-------------i                   shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7, disk8, disk9
H-----------------m               shareUseCache="no"      Exists on disk2, disk5, disk6, disk7, disk8, disk9
H--------------u                  shareUseCache="no"      Exists on disk4, disk5, disk6, disk8
H------c                          shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk7, disk9
K---------s                       shareUseCache="no"      Exists on disk5
l--------d                        shareUseCache="no"      Exists on disk4
M----------l                      shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7, disk8, disk9
M---------l                       shareUseCache="no"      Exists on disk2, disk3, disk4
M---------------d                 shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk8, disk9
P----s                            shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk7, disk9
R----------s                      shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7, disk8
R----s                            shareUseCache="no"      Exists on disk2, disk3, disk4
S------e                          shareUseCache="no"      Exists on disk1, disk2, disk3, disk4
T---------------s                 shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk7
T------s                          shareUseCache="no"      Exists on disk1, disk2, disk3, disk4, disk5, disk6, disk7, disk8, disk9
V---------s                       shareUseCache="no"      Exists on disk4

 

 

 

So you would like to backup the data in Disk 8 or in Disk 2 to begin with or both?

Is all of it important and irreplaceable? 

 

Where would you copy it? 

Started taking the backups of Disk 8 currently and while it is in progress, see read errors in Disk 2. Attached the diagnostics. With this now, I am worried what would happen during the parity swap. Any suggestions? 

 

 

tower-diagnostics-20220803-1549.zip

It has to read all the other disks to emulate the missing disk from the parity calculation, any problem reading all the other disks can cause problems with the emulation. The emulated data is what is being copied, and what would be rebuilt during parity swap.

 

Where are you copying the data to?

 

SMART report for disk2 seems to be getting worse.

 

How much left to be copied?

Am using one of the 12TBs(formatted as NTFS and added to a Windows PC) that I have and moving the data over the network. Another 120 GB pending to be copied across for disk 8 which is emulated and another 2TB of disk2 to be backed up.

 

Thought of completing the emulated drive, do a parity swap and then replace the data drive disk 2. Now I think I need to backup the disk 2 before doing a parity swap. Correct me if I am wrong

 

19 minutes ago, sabarirjpm said:

backup the disk 2 before doing a parity swap

Yes, and it is going to have to be replaced after you rebuild disk8.

Will take the disk 2 backup. After the parity swap and rebuilding Disk8, while replacing the disk 2, I would like it to be in XFS how do I get it. According to the wiki, it will again be reiserfs. Eventually, would like every drive moved to xfs 

Parity doesn't know anything about filesystems, it is all just bits, a good rebuild can only result in those same bits, which represent that same filesystem and its files.

 

You have to copy the data from reiserfs disks so you can reformat them as XFS

Backing up the Disk 2 did not work as it threw continuous read errors and had to cancel. Proceeded without backup as it had mostly movies. Parity Swap, Disk 8 Rebuild done. It again had read errors while rebuilding Disk8. Swapped Disk2 with a 4TB drive and rebuilding is in progress. Attached are the logs after the rebuild of Disk8. Did I loose any data?

tower-diagnostics-20220805-0617.zip

Attached is the log after the Disk2 replacement with a 4TB. 

tower-diagnostics-20220805-1734.zip

5 hours ago, sabarirjpm said:

Did I loose any data?

Looks like all disks mounted so probably not.

Thanks much. Have the following to-dos.
1. Move the data from newly added 4TB on to other disks using Unbalance plugin and convert the filesystem to xfs
2. Copy over the data/ backups available in the temporary 12TB drive that I had off the array and move it to unRAID
3. Use shrink-array to consolidate few disks
4. Convert the rest to XFS
4. Use one of the left over 2TBs after the consolidation as a cache disk..

Will keep posted on the progress. Thanks all along for getting my array back on shape

Edited August 6, 20223 yr by sabarirjpm

Over the last week, did the following. 

1. 4TB data is moved across using unbalance plugin. Was very slow but waited for it to complete. 

2. Copied over the data from the 12 TB drive which was off the array to unRAID and added the 12 TB to the array. Again a time consuming process which involved the preclearing of the 12 TB drive. 

3. Used the Clear an unRAID array data drive user script as mentioned in the shrink array procedure and cleared of one 2 TB drive. (Used the second option

The "Clear Drive Then Remove Drive" Method)

4. Stopped the array after the dd command is completed (Could not see the progress in the UI. Had to ssh into tower and confirm that the dd command is no longer running.)

5. Unassigned the drive which was cleared

6. Made the unassigned drive as the cache drive. Started the array by confirming that the parity is valid

Now I see that when the array is started, The numbering of the disks starts from Disk2(Disk1 was unassigned from the array and made as a cache). Is that expected? I haven't yet restarted the array yet. tower-diagnostics-20220815-0144.zip

Attached are the diags. Please let me know if the steps followed are correct

tower-diagnostics-20220815-0144.zip

3 hours ago, sabarirjpm said:

The numbering of the disks starts from Disk2(Disk1 was unassigned from the array and made as a cache). Is that expected?

If there's no disk1 assigned it's expected.

I am consolidating the old 2 tbs to the newer 12 TBs. So atleast 5 of them will be removed. Will the disk be numbered disk2, disk 9, disk 10, disk 11? 

The disks will be numbered according to the slots they are assigned, when you do a new config you can reorder the disks, if using parity1 only it will remain valid, if you have parity2 that one will need re-syncing.

Ok now after repeating shrink array for 5 of my 2TB hard drives, removed them and ran the parity sync just to confirm that everything is fine and I see lot of sync errors that are corrected. The parity check is still in progress with another 10 hours pending with approximately 18 hours elapsed. Does that mean that somewhere while using the shrink array option I did something wrong? Attached are the diagnostics. Is my data safe?

tower-diagnostics-20220821-0445.zip

Just to add, I shuffled the connectors when I removed the old drives and ordered them on the remaining drives

If you removed array disks you need to do a parity sync, not parity check.

@jorgeb

Thanks for the reply. I do not understand. I have only one parity and followed the shrink array procedure by removing one disk at a time and every time I remove a drive I start the array saying that the parity is already valid. The parity should have remained valid as the zeroing of each of the drives would have written the same to the parity. After all the disks are removed from the array one by one, I did a parity check to check if everything is fine. That's when I noticed the above. So I assume that somewhere during the above process, I could have done something wrong. Correct me if I am wrong

There are 2 shrink array procedures. Are you saying each time you followed the procedure that includes running the script to clear the drive while it is in the array?

 

I never bother with that, especially in the case of removing multiple disks. You can just remove them all at once and rebuild parity.

 

Might as well let it correct all the errors and then you can do a noncorrecting parity check when finished to confirm.

 

Do you have any errors showing in the Errors column in Array Devices?

Event: Parity Check Tuning
Subject: [TOWER]  finished (187721164 errors)
Description: Elapsed Time 22 hr, 18 min, 38 sec, Runtime 22 hr, 18 min, 38 sec, Increments 1, Average Speed 149.4MB/s
Importance: normal

 

This is what happened.. I did follow the second instruction where every disk was zeroed out and removed 

 

I don't remember seeing any errors in any of the drives. Have restarted the check again now and will let you know how it goes