[SOLVED] Unraid 6.7.0 – Data not written to new disk when rebuilding array.

[NOR_NAS]

The server is configured with one 4TB parity disk and two 4TB and three 2 TB data disks for a total of 14 TB data storage. I also have a 500 GB SSD Cache. The server runs several dockers and VMs.

 

I found the server running with one of the 2TB data disks (disk 5) disabled and decided to exchange this for a 4TB.

 

I followed the description I found on https://wiki.unraid.net/Replacing_a_Data_Drive for replacing the disk. 

 

The procedure ran without any apparent error messages and I received two positive event notification on mail:

#1

Event: Unraid Disk 5 message
Subject: Notice [NASFERATU] - Disk 5 returned to normal operation
Description: WDC_WD40EFRX-68WT0N0_WD-WCC4E1SVL51X (sdj)
Importance: normal

 

#2

Event: Unraid Parity sync / Data rebuild
Subject: Notice [NASFERATU] - Parity sync / Data rebuild finished (0 errors)
Description: Duration: 12 hours, 20 minutes, 5 seconds. Average speed: 90.1 MB/s
Importance: normal

 

On closer inspection I noticed that the array capacity was still at 14TB. In the main window in Unraid the disk details for disk 5 showed:

·      Green light for the disk on the far left

·      The correct serial number and capacity of the new disk on the left side

·      The capacity and utilisation numbers from the old disk on the right-hand side

 

When accessing the disk directly using the folder icon on the right-hand side, Unraid tells me the disk has no files or folders.

 

I tried to rebuild the array twice. First with an unformatted disk that was not precleared (couldn’t remember where I stored the precleared one at first ) and later with a disk that was precleared and formatted with XFS. Both tries gave the same result.

 

I keep off-site backups of all my data so one solution would be to set up the array from scratch. This is however a lengthy process and, since rebuilding an array is a vital function in Unraid, I would like to be able to run this procedure successfully.

 

I would appreciate any advice on how to move forward with rebuilding the array

Edited December 3, 2019 by NOR_NAS
Problem solved

[JorgeB]

Did you at any point format the disk using the GUI?

 

 

[itimpi]

When a disk is disabled then if run properly the data that is on the rebuilt disk will be the same as you can see on the emulated drive (while the disk is disabled) before doing the rebuild.   A mistake that many people make is to at some time to do a format from the GUI thinking this is a good idea not realising that effectively wipes the data from the emulated drive and updates parity accordingly so you end up rebuilding a disk with no files on it.

 

The 6.8.0 RC series has had a more prominent alert added to try and reduce the instance of such mistakes - we do not yet know how effective that will prove to be.

[trurl]

As suggested, probably you formatted the disk when it was in the array so you rebuilt a formatted disk. 

 

Format means "write an empty filesystem to this disk". That is what it has always meant in every operating system you have ever used. Unraid treats this write operation just as it does any other, by updating parity. So after formatting a disk in the array, parity agrees the disk has an empty filesystem, and rebuilding that formatted disk results in an empty filesystem. 

 

Format is never part of the rebuild process.

[NOR_NAS]

Thanks for getting back to me so quickly.

 

The formatting suggestion might look like the obvious answer, but I doubt that this is the (only) problem.

 

The original Disk 5 still contains all the files. I am able to access them from a different computer. The first time I rebuilt the array was with a new disk directly from the shop. The disk was neither precleard nor formatted. The second time I rebuilt was with a precleared disk. I formatted the disk with xfs as an unassigned device before I assigned it to the array.

 

If the disk was formatted the available capacity should be reported as 4 TB with 4 TB free.  Now Unraid tells me that the 4 TB disk is 2 TB and 1.75 TB is used. 

 

When viewing the content of the disk from the Unraid GUI (folder icon on the right-hand side) there are no files.

 

I turned the server off after the rebuild to minimize disk usage, so the diagnostic data attached is from a freshly booted server. I would appreciate if anyone could take a look at them to see if there are any other clues to what have happened.  

 

 

Edited December 3, 2019 by NOR_NAS
Problem solved

[Squid]

Run file system checks on disk 5

Nov 28 21:14:33 NASferatu emhttpd: shcmd (65): xfs_growfs /mnt/disk5
Nov 28 21:14:33 NASferatu kernel: XFS (md5): Metadata CRC error detected at xfs_allocbt_read_verify+0xd/0x3a [xfs], xfs_allocbt block 0xaea86690 
Nov 28 21:14:33 NASferatu kernel: XFS (md5): Unmount and run xfs_repair

 

[NOR_NAS]

Thank you for getting back to me Squid.

 

I´m on the road for a couple of days, but will try running the file system check as soon as I return home and report the result.

[NOR_NAS]

Looks like a file system check did the trick.

 

Thank you Squid. You saved me from restoring files for the next days.