Phew, i'm a bit stressed...my girlfriend arrived at Berlin airport from San Francisco last night, where she checked in my carefully wrapped up unRAID server with 15 disks (32TB)...first thing that went wrong: luggage was lost...this morning we got the call and suitcases (hers and mine).

 

Brought the system to my office here, and right off the bat the parity disk couldn't be found (all other disks showed up fine)...opened up the case to check connections and found one SATA connection at the drive cage had come loose...made sure it's in tightly, brought the machine back up, parity disk now showed as present, all other disks as green, but parity showed as having to be re-calculated...so i started a parity check and when i returned an hour later, the parity check had stopped, with an error.

 

Unformatted disk present:

 

Disk 13 • WDC_WD30EFRX-68AX9N0_WD-WMC1T0056148 (sdh)

 

the disk was definitely present and green when i started the parity check, but now this.

when i look at the Array Devices panel i also see that missing disk 13 shows 14 errors and Parity shows 2 errors.

 

needless to say, i am now a bit freaked out about what to do next...or rather, i'm afraid of doing *anything* before i haven't consulted the forum, as i always did all these many years.

 

what is the safest way to proceed now?

i attached the screen-caps of Array Devices and Array Operation and the newest Syslog.

i hope somebody can give me a pointer as to which way to go next!

syslog-20150126-064009.txt.zip

Hopefully this is just another loose connector or cable.

 

But unfortunately you've already lost any possibility of re-building the disk, since you did the new parity sync.

 

So I'd do one of two things:

 

(1)  Reseat the cables for disk #13, and if it's in a hot-swap cage, remove it and re-seat it.  [Not a bad idea to do this for ALL drives since the system was shipped]    Then do a New Config and assign ONLY your data drives to the configuration ... and see if it's now shown okay.    If so, confirm that you can "see" all of the data on your array okay -- and then assign the parity disk and let it do a parity sync.    When the parity sync completes, be sure to do a parity check to confirm all was okay.

 

If that doesn't resolve the disk #13 issue, then I'd do this:

 

(2)  Do a New Config WITHOUT including the problem disk and let the system do a parity sync.  You may still want to do the reseating of drives/cables and configure without parity first ... just to confirm all is otherwise okay.    Then I'd connect the problem drive to a PC using the free LinuxReader [ http://www.diskinternals.com/linux-reader/ ] to access it, and copy any data you can recover to your newly configured array.

 

Not sure how you packed your system; but for future reference, I always remove all drives and pack and ship those separately from the system.  Puts much less stress on the case and better protection for the drives.

 

 

Note that if neither #1 or #2 works, you've simply lost the data on drive #13.    Your mistake was in starting the parity calculation after it had initially been "missing".    You COULD have done a "New Config" at that point (after reseating the cables) and used the "Trust Parity" option to force UnRAID to use the existing parity drive.    [i assume you did a parity check BEFORE packing the system for transit to confirm all was well before disassembling it.]

 

Parity should still be good. The overwrite should have been identical to the existing data. You may be able to select parity is good and then rebuild disk13. If step 1 fails, I'll post a link.

yeah, i should have listened to my gut when it said to better consult the forum before doing *anything* when i first saw the Parity "missing" alert...well, hindsight yada yada.

 

yes, i did do a full-parity check before i packaged the system, and you are - of course - correct about taking the disks out before shipping, next time, but i wasn't around to package it and the person who did was a good packager, but not someone i would have felt comfortable asking to do what it would have taken to separate the array from the server...but your points are well taken, thank you very much!

 

i'll have to re-read all of this tomorrow, with a fresh mind, as i got way too little sleep last night and had too much stress over all of this today, to be at my best at the moment...and it is getting late here in Berlin.

 

but let me ask you these things (and please point me to the Wiki if it's been answered there):

 

> for a new config i'll have to use the New Config plugin in Upgrade Utils? this will clear the disk.cfg file so that i can start from scratch? i would need to note down the serial numbers before i do this, so that i can re-assign them in the correct order, corresponding to the physical disk slots? does the New Config also clear the write errors that show on Disk 13 and Parity?

 

> should it come down to Drive 13 not being recoverable by means of unRAID, are you (or anyone) aware of a Mac tool that can read the disk format (what is the format again, ReiserFS)?

 

in closing, thanks so much for taking the time for such a detailed set of instructions...had i even known that a "trust parity" option exists, then maybe a flag would have gone up when i saw the "Parity "missing" alert, since my first reaction was "hmm, well, i know the parity was fine"...but maybe next time, should there ever be a next time...or if someone should be in a similar bind...where is this option found?

 

thanks!! and i'll report back when i made any progress.

Hi again dgaschk!

 

how do i manually select parity to be good to override the system's refusal to even let me select a Parity disk right now...the array if offline and the drop-down next to Parity won't show me any available disks to select...is this the point at which i'll first have to do the "New Config" command before i can do anything else? i'm off premises now, so i can only remote desktop to the machine that has the WebGUI to unRAID on it...guess a remote controlled "New Config" might not be a good idea, being that i'd have to get all the serials of the physical locations noted down before configuring a new array, no? although i could probably just do so from a screen cap, or am i treading onto even thinner ice? i'd be very confident that i checked all connections very thoroughly when i re-connected the one that went to the Parity drive...i went through all of them *twice* before closing the case again....the fact that Disk 13 is shown as missing is bc of the write errors, i'd think...something that would be cleared (at least temporarily) when doing a New Config, i take it? but why is Parity not showing up at all right now?, not even in the drop-down? also disabled bc of the 2 write errors it had, maybe?

 

well, still feeling (and probably acting) like a total noob, even after all these years, when things go *this* wrong!

You can't manually force parity to be good with the current config;  but you CAN check the "Trust Parity" box if you do a New Config.

 

You don't have to assign all of the drives to the same slots they were in -- doesn't hurt to do that, but it's not mandatory.  The only thing you DO want to be sure of is that you use the same parity disk.

 

HOWEVER ... Dgaschk is correct that the parity disk MAY be okay -- it depends on just what you did and how the system aborted.

 

You originally said "...  parity showed as having to be re-calculated ...", but then you said you "... started a parity check ..."      So did you start a new Parity Sync ? ... or was it a Parity Check ??

 

In any event, the FIRST thing I'd do, if you have a PC you can use, is attach disk13 to it, install LinuxReader, and see if you can in fact read the data from that drive.    Do NOT do ANYTHING that might write to the drive ... but it's safe to try and read it.    If you can in fact read the data okay; then I'd do what I suggested earlier -- i.e. just rebuild your array with a New Config and regenerate parity; then copy the data from Disk13 back to the array.    If necessary (to gain space) you may need to add another drive to the system before copying the data ... but that's the safest way to do this IF the data on #13 is readable.

 

If the data on #13 is not readable, then there MAY be a way to force a rebuild.  Do you by any chance have a backup of the flash drive from BEFORE the system was shipped?

 

... Note that if you don't have a backup of the Flash drive, it may not be possible to force the rebuild.  The problem is the system doesn't think there's a "missing" drive, but that one of the drives isn't formatted.

 

What MAY work in that situation is to do a New Config with Trust Parity (including ALL of the drives);  then unassigning drive #13 and restarting the array -- so it shows a "missing" drive;  then stopping the array, re-assigning drive #13; and then starting the array again and letting it rebuild #13 onto itself.

 

IF parity is indeed still good, then this MAY result in a recovery of #13.    But before doing this, I'd certainly true LinuxReader on a PC to see if there is directly readable data you can recover, because once you do a rebuild, it's too late to try that.

 

a lot of "if's", but i can say for sure that i am not aware of having made a backup of the flash drive before the system shipped.

 

as far as the exact state of the parity sync vs rebuild i also can't say for sure...i faintly remember it saying something about parity not being valid due to un un-clean shut down, maybe? and whatever the suggested action with a "start" button was, i did start...i feel pretty stupid now that i didn't act more carefully and took so many risks, albeit unknowingly, but i should have known better after 10 or so years with unRAID.

 

well, be that as it may, tomorrow i better take out Disk 13, try to find a Mac utility or Fuse extension that will allow me to do what Linux Reader on the PC would do (maybe something from this list: http://alternativeto.net/software/diskinternals-linux-reader/?platform=mac) and start there...i've been running WD Green drives all these years but started to replace the failing ones with the Red (EFRX) these past 3 years, and had just replaced the parity with a new 4TB Red drive in preparation of a new round of drive upgrades to 4TB...Drive 13 was one of the 3TB EFRX drives, also fairly new, so it would be disappointing to see two of them fail like this, but not all *that* surprising when looking at the track record of WD drives in general (from what i've experienced and read online).

 

anyway, if i can't find anything as good as Linux Reader looks, maybe i can get it to work through my Bootcamp Windows 7 partition.

here's crossing my fingers and all my toes that Drive 13 will have most of its data still readable and that i can correctly follow all of these instructions...since i've never done a New Config, i'm more than a little nervous to mess something up even more.

 

in closing, for the night, a sad little tale of backing up...

 

*if* i did lose critical parts of Drive 13, the tragic thing about this would be that i spent the past 8 months (!!) uploading my most critical 16TB of production data (incl. everything on Drive 13) to a cloud storage service called Bitcasa, that i used since the days when they still offered unlimited storage for $100/year and had been grandfathered in when they went to $1000/year from one day to the next, being promised that we early adopters would never ever have to pay more than $100/year for as long as Bitcasa was in business...then, in October, after 8 gruesome months of wrestling their ridiculously buggy client software into uploading the last Terabyte of my most valuable productions, we were all informed that due to a new client software update that fixed most if not all of the problems we had all been wrestling with for so long, they decided to "simplify" their various plans into one 10TB plan (no more infinite) and we *all* (early adopters like me included) had to chose to either "upgrade" to the new software and 10TB plan, or get our data off their server within 3 weeks time...due to the slow download times, a task that was simply impossible to do...as was paying $1000/year...so, long story short, i lost my 16TB backup that i labored over for so long in a moment...for a short period of time it gave me a lot of joy to think that all my data was backed up in the Cloud, but this really left me feeling like i had been kicked in the teeth.

 

bitcasa are a bunch of criminals, as was confirmed by one of their tech support staff who wrote me, from his personal email, after this happened, to tell me how pre-meditated all of this was.

 

so, with that said, i think i deserve a break here, and will do all i can tomorrow to do my part to not mess things up any more than i already have.

thanks, as always, gentlemen, for so generously sharing you knowledge...the unRAID community is amazing.

Unfortunately, there don't seem to be any Reiserfs IFS packages like LinuxReader that will let you read Reiserfs data on a Mac.    It's been discussed before here:  http://lime-technology.com/forum/index.php?topic=27363.0

 

However, there are a couple alternatives:

 

(1)  The simplest approach is to load a copy of UnRAID Basic (the free version) on a different USB Flash drive and boot to your UnRAID system with it.  Assign ONLY the troublesome disk (#13) as a data drive -- with no parity -- and then see if the disk is shown okay [Do NOT Start the array if it shows the disk is unformatted].  If it IS shown okay, then you should be able to access it over your network, and you can then copy the data from it to another storage location ... clearly this may require you pick up an external drive with enough storage space if your Mac doesn't have enough free space.

 

(2)  If #1 doesn't work; then you can try using LinuxReader in your Windows 7 VM on the Mac.

 

(3)  If neither of those work; I'd try using the Reiserfsck utility to do a scan and recovery of the files.    You could either boot to your original flash drive; or the one you created for #1 ... you're not going to Start the UnRAID software here;  you simply want access to the Linux command line to run Reiserfsck.    Be CERTAIN you do this for the correct drive -- if you want to eliminate any possibility of error you could physically unplug the SATA cables from all other drives.  Reiserfsck is VERY powerful ... so be sure to read the details of the various options.    This is a good overview of using it:  http://lime-technology.com/wiki/index.php/Check_Disk_Filesystems

 

 

ok, back at my system.

 

first things first...before i did anything, i took another screen-cap of what the array looks like in my WebGUI (screen attached).  then i did a clean shutdown, triple checked all SATA cable connections and re-seated drive cages 13 (Drive 13) and 15 (Parity) and booted it back up...

after the server came back up i was quite surprised and encouraged that i got what looks to be a rather promising array state (see second screen-cap, and for good measure i *wanted* to attach another syslog, but right now i don't seem to be able to see any of the shares in my finder...i thought tha Flash share always showed up, even if the array wasn't started, but right now i don't seem to be able to connect to the SMB or AFP shares of my unRAID and am afraid of doing anything that could compromise my current state).

 

having learned *something* from my actions yesterday, i will *not* do *anything* until i get further instructions from the seasoned forum members here...please let me know what step to take next...i hope this is the break i was hoping for!

dammit!! because the drives started to show severe overheating, i closed the case up again and put it upright, and when i looked at the WebGUI again i saw that the Parity drive now disappeared again (see attached)...i didn't think that i could hurt anything by putting the server back into a position where cooling would work properly, knowing that it might be many hours before i get further instructions.

 

did i mess up again?!

And now Drive 14 is also showing as "missing" (see attached), and temperatures *still* hover in the high 40's (celsius) with Drive 7 at 51...good grief!

 

what would be the safest way to get a syslog off my flash drive to share with you? i'm afraid to put the machine on its side to open it up and get the flash drive from the motherboard, into which it is plugged.

 

my gut says "clean shutdown, check all cabling again, re-seat drive cages again and boot up again", but i'm not going to listen to my gut until i get your input.

 

damn, this is stressful!

 

PS: i ended up taking the Flash drive out and putting it on my Mac...the latest syslog has ballooned to a mighty 139MB, but here is a link to a compressed 5.9MB version on my dropbox: https://www.dropbox.com/s/4343m98mvu41l93/syslog-20150126-193319.txt.zip?dl=0

If this high temperature condition is a new thing, You really need to figure out why you have this problem.  Are all the fans (case and CPU)  running?  Is there a lot of dust and dirt inside the case.  (I assume that your power supply is rated for 100 -240 volt input or you have reset the switch to to 240 volts.)  High temperatures can cause electronic devices to become flaky.

not totally new, but it hasn't been quite this bad...i'm trying to make it back to a protected and complete array, then i am evaluating how i might be able to back up the entire data pool and tweak this case to be within proper specs again.

 

and yes, it is a 110-240 supply, auto-switching with all fans running and a mostly dust-free interior...i'm still cleaning it up more, but i think the high temps are mostly due to sub-optimal drive cages and fans, which will need an upgrade soon (maybe an entirely new case with less but bigger drives).

 

but first i need to get this array to be up and running, and ideally protected again, so that i can offload/backup. 

not totally new, but it hasn't been quite this bad...i'm trying to make it back to a protected and complete array, then i am evaluating how i might be able to back up the entire data pool and tweak this case to be within proper specs again.

 

and yes, it is a 110-240 supply, auto-switching with all fans running and a mostly dust-free interior...i'm still cleaning it up more, but i think the high temps are mostly due to sub-optimal drive cages and fans, which will need an upgrade soon (maybe an entirely new case with less but bigger drives).

 

but first i need to get this array to be up and running, and ideally protected again, so that i can offload/backup.

 

OK.  Let's take a minute and look at the situation.  As I reviewed your screen captures, it appears that the 'defective' drives keep changing and/or then appear and disappear.  Drives don't just change from good to bad and back again in a normal situation.  Since you have fifteen drives in your system, one possibility is a power supply issue.  Remember, you went from 120 volts to 240 volts and that probably changed the conversion efficiency of the supply.  An another thing, does the power supply have a single 12V rail or is a dual rail supply?  (Single rail supplies are recommended for servers with large numbers of drives!)  Flaky 12V rails have caused problems that look like this for people in the past.

 

I would also bet that ALL of your drives are good and the data is there on each drive.  You might want to consider rebuilding your physical system and simply use those drives in the new case and drive enclosures.  (I, personally, would recommend using front loading drive enclosures so you can change drives without having to open up the case!  This prevents problems that have occurred when moving cables to change one drive loosens a cable to another drive.) 

your points are well taken and i do agree that a partial or maybe even full hardware upgrade may be in order, although the current setup is fairly new (maybe 3 years or 4)...the 5-in-3 front-loading drive cages are of high quality and the same ones used in one of Tom's more recent turn-key solutions ($120 per cage)...i think the shortcoming is that when i first configured the system i had to live with it in my one-room studio, so that i may have went overboard when replacing the cages' fans with a decisively more quiet variety that may simply not be cooling the drives properly...but the current temps are atypically high and may have more to do with the dust that has accumulated in my last space.

 

i am pretty sure that i purchased one of the single-rail PSU's that were recommended by Tom or one the forum members in the hardware forum (i spent weeks obsessing over ever component's make and model)...the case is one of those beautiful quiet aluminum models (an "Anker", maybe?) that allowed all 3 drive cages to be mounted in the front as an inexpensive 4U 15-drive upright or side-ways server and were difficult to get at the time but i was lucky to snag one while stock was replenished at NewEgg...but after 3 trips across the ocean it has a big dent in the top panel that is a bit unsightly if insignificant, really...the most immediate improvement would probably be a more modern mobo and CPU with a wider memory-bus, but the dual-core 2GHz with 4GB of RAM have so far been barely stressed by how i have used it, so even on that end, it may still be ok for a while...maybe when i get into unRAID OS 6 with Dockers and VM's, but with all the other stuff (i.e. paying projects) i need to apply my time to i wonder whether it's *really* where my money and time should go right now.

 

the main problem right now is to get the array to come back up so that i can backup the most important data, and maybe i'll have to put some faster fans in the drive cages *before* doing so to keep it all within spec while backing up, but maybe someone here can tell me whether it would be safe to *first* bring the array back online somehow...and if you think it might be, then is it safe for me to shut it down again, re-seat all the cables and drives yet again, boot it back up, and *if* i get an ideal-case scenario like this morning again, where all drives show up and it only asks for being brought online, causing a parity-sync, to do that? or would it be better to do the recommended New Config and tell it to accept the parity as valid, so it would not thrash through 4TB of parity, which is typically a 18hrs or so affair, during which other things could go wrong again?

 

my question , i guess, is: what is my best chance of getting this thing backed up before doing major hardware changes, knowing what these last screen-caps and syslog say about my system?

 

thanks for your patience and excuse my rather frazzled questions.

Since each data drive is completely independent in unRAID, it is not really necessary to get all of it back up at once just to make backups of one or more of the disks. If some of it is really, really important, and you know which disks are involved, you could just boot up the free version of unRAID from another flash drive and just put in a few disks to copy from at a time. Just leave the parity drive out so it doesn't get changed, and don't write anything to any of the data drives. And don't let unRAID format anything!

A few thoughts ...

 

=>  One of your fundamental problems is insecure connections between the drives and the controller.    You've re-seated all the SATA connections, so this may be in the internal connectors in the drive cages.    The system was shipped with drives mounted, which would cause a fair amount of vibration and stress on those connectors, so they may be less secure now and subject to lost connections with very tiny amounts of vibration.  This would explain your weird "missing drive" scenarios -- parity, disk #14, etc.

 

=>  Another issue is the drive temps.  Is the case closed, so you're getting the normal airflow through the cages?  If not, that's probably the issue -- an open side lets air come in through the side to satisfy the suction created by rear fans on the drive cages instead of through the front and across the drives.    If that's not the case, then it may indeed be related to a change in the power output due to the input voltage difference, but that seems pretty unlikely, as regulators aren't effected by the input voltage [if the base voltage was slightly higher, the regulators would run warmer, but the DC output voltages wouldn't change].

 

=>  It does seem pretty likely that your drives themselves are probably okay, with the data intact.  Since you don't have backups, I'd be VERY careful about doing anything that might change that.

 

=>  A fairly simple (if a bit labor intensive) way to confirm that the data on the drives is okay is to do the following:

 

    -- Disconnect ALL of the drives (both power and SATA cables).    Make a list of all of your drives and their serial numbers (you've already got this list in the graphics you've posted here).

 

    -- Now, 2 drives at a time, connect the two drives directly to SATA ports (not through the cages);  boot to a new USB flash drive configured with v5.0.6 Basic (the free version);  and assign those two drives as data drives (No parity drive).    If they show as unformatted, do NOT let the array format the drive -- but note which drive(s) have this problem.    If they "look" okay when assigned to the array, browse the disks a bit from your client (Mac I presume) and satisfy yourself that the drives are okay.    Keep notes r.e. which drives you've checked and look good or bad.    Now repeat this for the next 2 drives -- doing a "New Config" each time -- until you've cycled through all of the data drives.

 

    -- When you've finished checking them all, if there are any disks that weren't readable, I'd connect them (one at a time); boot to UnRAID (the same v5.0.6 version you used for the tests); confirm one more time that the drive isn't readable -- just to be CERTAIN it wasn't a loose SATA connector on the drive [Do  "New Config" and assign it as the only data drive]; and then, from the Linux command line, do the Reiserfsck checks I outlined earlier.

 

This process will eliminate the cage connections as a possible issue (since they won't be used);  will ensure you don't have an insufficient power problem (not likely anyway, but nevertheless you'll never be powering more than 2 drives at a time);  and will ensure you don't have some strange issue that was resolved in the last couple releases of v5 (since you'll be running the final version).

 

When you're done, you'll know which drives are readable, and which -- if any -- aren't.  You won't, of course, have anything backed up.

 

As for backing everything up -- that depends on what your plans are for moving forward.  If you plan to have a dedicated backup server, I'd build it now ... a small mini-ITX box with 6 6TB drives would give you 30TB of backup space (more than your current 22TB of data); or you could use a larger system with a higher capacity if you have a larger capacity target.    Then you could repeat the 2-drives-at-a-time process above and simply copy all of the data to your backup server.    If you plan to just use external drives [either stand-alone or bare drives mounted through a USB3 or eSATA dock], you could go ahead and buy those and, again repeating the 2-drives-at-a-time process, copy the data to the external drives.    Seagate has 8TB Backup Plus drives with USB v3 connections ... 3 of those would be enough to backup all of your data.  Or you could buy the bare drives (~ $250 for 8TB) and use a USBv3 or eSATA dock to connect them ... storing the drives in one of WiebeTech's DriveBox units (an excellent way to store drives).  [ http://www.amazon.com/DriveBox-3851-0000-11-Hard-Disk-Case/dp/B004UALLPE ]

 

 

understood...but if i want to use this unRAUD system for a while longer, in the same way that i have used it up to the point of having it packed for shipping, then how would i best go about bringing it back online right now? assuming nothing intrinsic is broken with any of the disks and that i can get the temperatures under control, there should be no reason to at least try to get it back online, parity drive and all, no?

 

so maybe i should re-phrase my question to: *if* i get an ideal-case scenario like this morning again, where all drives show up and it only asks for being brought online, causing a parity-sync, what is my best chance of getting this thing to be fully operational again at this point?

 

i suppose if i did get to the point again where doing a parity-sync of the complete array could be started, and it stopped again bc of one of the disks (other than Disk 13) showing errors, *then* Frank1940's suspicion of a funky PSU would become the next likely candidate for replacement? if it's Disk 13 again, then maybe start with  garycase's suggestion of focusing on recovering Drive 13?

 

hope i am not starting to sound too stubborn, but i am reluctant to give up on the idea of bringing this system back to life again, like so many times before...the reason why i've stayed with unRAID for this long is bc even though things have looked dire so many times, there was always a way out again and things went back to normal for years after that.

Before writing to parity (or any other disk) it might be worthwhile to get a smart test report for each drive.

posted this after you posted yours, garycase.

 

your thoughts, as always, are about as reasonable as it gets and should really be taken at face-value by me.

 

as much as i'd like to get my baby back up and running again, as so many times before after major trouble reared its head, the danger of messing something up is too great.

 

the first and foremost goal should be to make a backup, 'nough said.

 

what would the danger be of trying to to a clean shutdown, checking all connections again, rebooting the machine and bringing the array back online *if* all disks show up green? i should first remove the parity drive, or un-assign it in the WebGUI, so it won't attempt to do a Parity-Sync operation? would that be ok? this would simply be for the purpose of pulling the data off the array within the directory structure i have it in...if i pulled the data off drive by drive, there would be a pretty big mess of duplicate/triplicate/etc. directories that all contain parts of the whole (due to the way unRAID distributes content across multiple disks)...would that be ok, since i wouldn't be writing anything to the array? it's just that i cringe at the alternative, which would probably take me months of tedious re-combining of directory structures, if you follow what i mean.

 

and since you just made this comment about SMART reports, trurl, would it be ok to do those through the WebGUI after rebooting my system successfully with all disks being green?

understood...but if i want to use this unRAUD system for a while longer, in the same way that i have used it up to the point of having it packed for shipping, then how would i best go about bringing it back online right now? assuming nothing intrinsic is broken with any of the disks and that i can get the temperatures under control, there should be no reason to at least try to get it back online, parity drive and all, no?

 

Conceptually that's certainly true.    But the first step is to be CERTAIN all of the drives are okay;  and to do that the surest approach is to eliminate the potentially insecure connections in the cages and test each of the drives.    The 2-drive-at-a-time test I suggested will take a few hours; but will also give you the best confidence in the integrity of the drives.    I'd take the time to do that if you want to be certain your data is all intact.

 

Assuming that shows that all of the drives are okay, I think that will also confirm that the fundamental issue is the connectors in the drive cages.  There are a couple of approaches you can take to try and resolve that.  The first thing I'd try is to simply clean all of the contacts -- remove all drives; then carefully clean both the power and data connectors with a contact cleaner [ http://www.frozencpu.com/products/971/blo-03/Blow_Off_Contact_Cleaner.html?gclid=CMr_nf6BtcMCFQqPaQodIQoASg ]; and then reseat the drives.    If that doesn't resolve it, then you likely have a microscopic break on one (or more) of the cages' circuit boards.    If that's the case, you simply need to either replace the cage.

 

One other thing:  If all of the drives that have had this "disappearing" issue (e.g. parity, #13, and #14) happen to be in the same cage, then you already know where the problem is -- so just replace that cage 

 

great suggestions, garycase!

 

looks like i've got many hours of troubleshooting ahead of me, but it is 25 years of my work on that array, so what's another few days of mending...at least there is hope that the data is ok.

... what would the danger be of trying to to a clean shutdown, checking all connections again, rebooting the machine and bringing the array back online *if* all disks show up green? i should first remove the parity drive, or un-assign it in the WebGUI, so it won't attempt to do a Parity-Sync operation? would that be ok? this would simply be for the purpose of pulling the data off the array within the directory structure i have it in...if i pulled the data off drive by drive, there would be a pretty big mess of duplicate/triplicate/etc. directories that all contain parts of the whole (due to the way unRAID distributes content across multiple disks)...would that be ok, since i wouldn't be writing anything to the array? it's just that i cringe at the alternative, which would probably take me months of tedious re-combining of directory structures, if you follow what i mean.

 

IF all disks show green (except parity) you COULD do a "New Config" (with your Pro key) and assign all of the data drives, but no parity drive ... and then copy your data off.  However, if (as I suspect) the issue is the contacts in the drive cages, there would be a risk of losing a drive during this copy process (drive vibrations are likely to cause a contact issue, as they apparently did already).    Note that copying the data off a couple drives at a time won't cause quite the "mess" you implied ["... tedious re-combining of directory structures"] => the structure will still be intact, so if you copied all of your backup disks back to the array, everything will be exactly as it was.

 

If you DO want to try that, I'd at least clean all of the contacts first (as I outlined above).

 

As for SMART reports on the disks -- sure, you can simply do those through UnMenu.

 

 

... but it is 25 years of my work on that array ...

 

And no backups !! ??

 

Definitely time to rectify that !!    I do think there's an excellent chance that you'll be able to recover most, if not all, of your data.