Help: Disk shows "unformatted"

[Roadini]

Hello,

 

I'm using Unraid 6.0-beta12.

 

I was copying large amounts of data to the hard drives for days. I noticed this morning, if I delete files, they reappear after I refresh the share. So I restarted the unraid Server. After the restart one disk (5) is shown as unformatted. Stop Array Operation from the Webgui doesn't work. Array can't be stopped anymore. Smart Status of all disks show that everything should be fine.

 

Please help to recover my Server.

 

Thanks in advance.

 

Roadini

Unraid_log.txt

Unraid_System_log.txt

[Roadini]

Screenshot from unraid web gui

[itimpi]

A disk suddenly coming yp as unformatted tends to indicate that some sort of file system corruption has occurred.

 

The normal sequence to recover this is:

- Stop the array and restart it in Maintenance mode

- From a console/telnet session run a command of the form:

reiserfsck --check /dev/md5

This command will run for some time, so if using telnet do not close the telnet window while it is running (or run the command under screen).

 

The results of this command will confirm if corruption exists, and also suggest recovery action if it is found.  It is probably a good idea to check back here with the results before taking any recovery action suggested by reiserfsck

[Roadini]

I would love to, but I can't stop Array (see above). Furthermore other disks of array look "strange" (see picture of second post).

[trurl]

Can't stop Array (see above).

Reboot in SAFE mode.

[Roadini]

Can't stop Array (see above).

Reboot in SAFE mode.

 

How do you do that? Only know that Option from the gui. But to get that you have to stop the Array. Or is there another Option like doing a hard reboot and then some menue upon reboot? Sorry for such a beginner question. Can't find anything in the Wiki. Unraidserver doesn't run headless.

 

The console says, device or ressource is busy.

[dgaschk]

First, try to get a new syslog and attach it.

 

Hold the power button down until the system shuts-down then reboot or press the reset button. Select SAFE-mode from the boot menu.

 

Or see console commands in my sig for CLI shutdown procedure.

[Roadini]

Thanks found everything like you said.

 

Didn't want to do a hard restart until i was sure it won't hurt anything an I know what I m supposed to do. ; )

 

Seems like one of my AOC-SASLP-MV8 has a Problem. Tells me on all 8 Ports of that one Adapter that PD Disk down. Other Adapter doesn't bring up any failures. I'll wait for the check error log and than look into the Adapter.

[Roadini]

Well Adapter works fine. Shut down the Server and made sure both Cards are seated fine. No Pd Disk Down Errors anymore while starting up.

 

One more trival question: After activating the checkdisk command it won't show the progress or result of the checkdisk. What do i do wrong? tells me log will be in stdout. Where do i find that log?

 

After bootup console Shows (by itself):

 

"REISERFS error (device md5): vs-5150 search_by_key: invalid Format found in block 26745501.

rmdir: failed to remove '/mnt/disk5' : Device or ressource busy"

Unraid_Save_Mode.txt

[itimpi]

One more trival question: After activating the checkdisk command it won't show the progress or result of the checkdisk. What do i do wrong? tells me log will be in stdout. Where do i find that log?

stdout just means the screen unless you have redirected it to a file.

 

The reiserfsck command should certainlyshow progress on-screen while it i running.  Maybe you are just not waiting long enough as with a 3TB disk it is likely to take a few hours to complete.

 

After bootup console Shows (by itself): REISERFS error (device md5): vs-5150 search_by_key: invalid Format found in block 26745501.

That confirms that there appears to be some sort of file system corruption.  Why it occurred in the first place is not at all clear, but anything that causes a write to fail can lead to this.

[Roadini]

Thanks for the quick reply.

 

I waited all night (about 8 hours). Nothing showed up.

 

Update: Just rebooted again => Started in Safe Mode again => did a reiserfsck --check /dev/md5 again => Acknowleged with Yes again

 

Now ist working and showing Progress. I ll wait for the check to finish and will post the results.

[itimpi]

I waited all night (about 8 hours). Nothing showed up.

The only thing that I can think would cause that is that the system 'hung' forsome reason.

 

Update: Just rebooted again => Started in Safe Mode again => did a reiserfsck --check /dev/md5 again => Acknowleged with Yes again

 

Now ist working and showing Progress. I ll wait for the check to finish and will post the results.

Well at least you now know what to expect.  At some time I would guess the online documentation could do with some screen shots to show what you should expect to see.

[Roadini]

Output of reiserfsck (14 min):

 

1 of 97-bad_leaf: block 465044001, items 7 and 8: the wrong order of items: [3 8 0x1 DIR (3)], [3 9 0x1 IND (1)]

/ 9 (of 95 //75 (of 121 |block 26698894: The level of the node (22567) is not correct, (1) expected the problem in the internal node occured (26698894), whole subtree is skipped

finished

Comparing bitmaps..vpf-10640: The on-disk and the correct bitmap differs. Bad nodes were found, semantic pass skipped

2 found corruptions can be fixed only when running with --rebuild-tree

 

What would be the next step? Should I do a "reiserfsck --rebuild-tree /dev/md5"?

[itimpi]

It looks like corruption was indeed found.

 

You can run with the --rebuild-tree option in place of the --check option and it should get the disk back to a consistent state.  This is what I would recommend at this stage as normally it is all that is required.  It will run for a similar length of time to the --check option.

 

There is an additional option --scan-whole-partition that can be used if any files seem to have been lost.  This reads every block on the disk looking for what could be file entries and then re-instates any files that it finds.  The downsides of using this option are:

• It takes a very long time to run as every sector on the disk needs to be read
  
• It can recover spurious files that it places in the lost+found folder and often the filenames are cryptic so it can be hard to work out what they are.
  

I would recommed against using this option unless you really think you need it.  The only time it is normally recomended is if the user has done something that could have caused major file system corruption.  However the results can sometime be miraculous in what is recovered.  This is one big advantage of using reiserfs against xfs or btrfs in that the recovery tool seems to be much better at recovering data after major corruption has occurred.