OK, I FAILED at the 1st direction to "capture your complete syslog, BEFORE YOU REBOOT!"

 

I did download the Diagnostics.zip, which is attached (and I believe includes the SMART reports for my drives).

 

Here's what I recall happening:
Rather than the default "Tower" I'd named my box "unraid."
Over a year ago Disk 3 went bad - replaced it with a larger (3TB drive). Don't recall if I pre-cleared it first.  It formatted at 2.2 TB instead of 3 TB.  Didn't think much of it.

A few months later Disk 12 went bad - replaced it with a larger (5TB drive). Don't think I pre-cleared it first.  It formatted at 2.2 TB instead of 5 TB. Thought the lack of pre-clear might have been why.  Figured I'd look into it sometime, and then just kind of ignored it, as it was working fine, and I wasn't low on space.

A few weeks ago Disk 4 went bad. I ignored it for a while, then decided to replace it with a larger (4TB drive).  Didn't pre-clear it.  I hot swapped it and the system wouldn't see it.  Decided to shutdown, but first moved everything from the emulated Disk 4 to empty space on Disk 15 using MC through a Putty window on my Win10 laptop.  Then when Disk 4 was empty saw that unRAID v. 6.3.5 was out, so did the update through the browser (chrome) to IP address on my network.  
Then last week I tried the shutdown commands but nothing would work, and I ended up just powering down with the power button.
I could never get it to re-boot after that.  Or so I thought, as any IP address in my chrome browser (I tried IE too) just timed out.
A few days into it I realized I could still access the IP address through a putty SSH session, and use MC, but the user directories under /MNT/ were gone. In other words, it looked to me like the array wasn't launching.
I'd read somewhere that the start up filed edited in windows with a M^ at the end of the line might cause issues, so I edited go and other config files in MC to delete that when present.  No joy.
Yesterday I realized that I could access the unRAID GUI through my chrome browser at //unraid:8080 rather than the IP address, but it was VERY laggy and would time out if I tried to change tabs.
Thinking it might be the flash, I pulled it,  plugged it into my Win10 laptop, backed the whole thing up to my laptops HD.  Then I DL'ed the 6.3.5 .zip file, and copied over go and network.cfg and a couple others.  Then ran the make bootable bat, and put the USB back in the unRAID box.  Again, I could Putty/SSH to the IP, and access the unRAID GUI through my chrome browser at //unraid:8080, but still VERY laggy and would still time out if I tried to change tabs.
Finally today I pulled the USB again, moved everything to another dir on my Win10 laptop, and copied the new install of the 6.3.5 .zip file contents.  I added preclear_bjp.sh (with it's txt and readvz[64 bit]), my pro.key in /config (as well as /shares and /ssh directories and network, share, smart and super files back from my laptop back up)  I also added /unMENU and /packages directories back from my laptop back up.  I edited the go to add the /boot/unMENU/uu to go and ran the make bootable bat, and put the USB back in the unRAID box.
Now I can access the IP through my chrome browser, as well as at //tower:8080 and no real lag to click tabs.  I can also putty/ssh to the IP.  But now when I use Putty to run MC there is NOTHING under /MNT/ 
Before there was CACHE and DISKS 1-15, just no USER where my shares were.  
In unMENU /main it shows 3 bad disks.  Disk 4, which it lets me assign the replacement 4TB drive to now, as well as Disks 3 & 12, the prior replacements that formatted at 2.2 TB instead of 3 & 5 respectively.

Like this:

 

Disk 3    ST3000DM001-1ER166_Z501GCAV - 3 TB (sdg) 
Wrong        ST3000DM001-1ER166_Z501GCAV - 2.20 TB    ALL DATA ON THIS DISK WILL BE ERASED WHEN ARRAY IS STARTED

 

Disk 4    WDC_WD40EZRZ-00WN9B0_WD-WCC4E1EZKZ9U - 4 TB (sdf)    

 

Disk 12    ST5000DM000-1FK178_W4J1FL5S - 5 TB (sdh)
Wrong     ST5000DM000-1FK178_W4J1FL5S - 2.20 TB    ALL DATA ON THIS DISK WILL BE ERASED WHEN ARRAY IS STARTED

 

Array Operation - Stopped. Invalid configuration -     Too many wrong and/or missing disks!

 

Oh, and I moved my drive trays around tonight, so that they are now organized in descending order of size in my bays, left to right and top to bottom.  I don't think this should make any difference, since I understand that v6 uses disk ID rather than placement on controller cards/bays, but thought I'd mention it just in case.

 

Is there a way to force unRAID to start the array using Disks 3 & 12, with the mismatched size, without loosing the data on those drives?  It's the same disks, same, make, same model, same serial #.  

Help!

 

Oh yeah, how do I rename Tower to unraid while I'm at it?

tower-diagnostics-20170604-0013.zip

Edited June 7, 20179 yr by MDEnce
solved

To properly rebuild disk4 you need the other disks as they were, you have an older LSI controller that's limited to 2.2TB, this is why disks 3 and 13 are that size, just connect both disks on the LSI, leave the new 4TB where it is and will be able to rebuild disk4, after that you can expand the other disks buy rebuilding one at a time on another controller.

Thx JB, but when I moved Disks 3 & 12 to other slots, the unRAID box reverts back to behaving like it did before I re-did the usb flash drive.  I can access the IP address through a putty SSH session, and use MC, but the user directories under /MNT/ are gone, and I can't get it to load in a chrome browser by either name or IP address.  Even tried pulling the replacement Disk 4 too.  Still nada. Thoughts?

Type diagnostics on the console/SSH and upload the zip.

Quote

 

OK, here you go.  

Sorry for the delay.

It was a PITA since my USB flash is in the inside of the box.  Had to tear it down to pull it and 

put it in my laptop

unraid-diagnostics-20170605-2029.zip

You need to check filesystem on disk7 (md7) after starting the array in maintenance mode:

 

https://wiki.lime-technology.com/Check_Disk_Filesystems#Drives_formatted_with_XFS

Does that mean I'll need to connect a monitor and kb to the unraid box (I run headless) or there some ssh command to restart in maintenance mode?

Disable array auto-start by editing disk.cfg on your flash drive (config/disk.cfg) and changing startArray="yes" to "no".

Then start the array in maintenance mode using the webGUI and check filesystem on disk7

xfs_repair -v /dev/md7
Phase 1 - find and verify superblock...
        - block cache size set to 709856 entries
Phase 2 - using internal log
        - zero log...
zero_log: head block 463014 tail block 462128
ERROR: The filesystem has valuable metadata changes in a log which needs to
be replayed.  Mount the filesystem to replay the log, and unmount it before
re-running xfs_repair.  If you are unable to mount the filesystem, then use
the -L option to destroy the log and attempt a repair.
Note that destroying the log may cause corruption -- please attempt a mount
of the filesystem before doing this.

 

not sure what this is telling me.  Do I just stop the maintenance mode in the GUI, and then what?
 

use -L, i.e.

 

xfs_repair -vL /dev/md7 

 

OK, but now it is telling me "Unmountable disk present:

Disk 7 • WDC_WD30EFRX-68EUZN0_WD-WCC4N6ZVE5A3 (sdj)"

 

I'm letting parity re-build my replaced Disk 4, but when that finishes, am I going to have to format Disk 7, and either lose what's on it, or have to  let parity rebuild it too (I didn't think unRAID could rebuild 2 failed disks without a 2nd parity drive), or is there something else I should do?

 

Don't format disk7, parity can't fix a corrupt filesystem.

 

Did you ran xfs_repair like I posted above?

I did NOT reformat.

I DID run xfs_repair.  First with -v (since I'd deleted the log with the -vL).  Then I ran -vL.  Both stopped after phase 2.  Then I ran with -v again, and it went through to phase 8.  Then I shut down the array, and when I restarted it SAID Disk7 was back.

Not sure I trust it.  The wiki on re-doing a drive says I should move the data, re-format as ReiserFS, then reformat back toXFS.  I'm moving data now.  Should I do the rest, or is that a waste of time?

BTW, I really appreciate the assistance.

If the disk was very corrupt it can be a good idea to re-format it to avoid repeating issues in the future, so continue, move all data off it, format to reiser, re-format to xfs and move data back.

Will do.

Thanks again.