Woke up this morning, and nothing seemed to be working right. Sabnzb had errors writing last night; couldn't ping PlexPy; shares seemed to be missing (again). Took array offline, and rebooted. Now Docker is missing from the UI, no Docker apps I've got installed show up in the Apps tab, and drilling down into the cache drive seems to indicate a bunch of stuff missing. Still no shares, either.

 

I'm assuming the cache drive is pooched. (I'm also assuming my data drives are fine, so I'm not panicking or frantic; just slightly annoyed and resigned.)

 

How do I safely get back up and running? I'm assuming I'll have to replace the cache drive (got any recommendations for a 240GB or larger SSD)? What files are needed for diagnoses? What's the order of operations to restore what I had without inadvertently screwing up my precious, precious data?

What files are needed for diagnoses?

Tools - Diagnostics

Thanks... diagnostic file attached!

shinagawa-diagnostics-20161211-0917.zip

Absolute ton of errors relating to the cache drive.  First thing to do is reseat all the cabling to the cache and try it all over again.

Looks more like the SSD needs to be replaced, but like Squid mentioned try replacing the cables first.

Done. Changed SATA cable, but used same SATA port on mobo. Re-seated power cable. No change that I can see (e.g. Dockers tab and shares still missing). Diagnostic file attached. Is heat ever an issue? I'm wondering if I've contributed to the problems with this drive through near-continuous downloads/unpacks for many hours at a time.

 

On the way down, after stopping the array, "Cache Devices" showed nothing (again: this happened before a re-boot this morning, too), with nothing to choose in the pulldown. Sounds ominous. On the way back up, before starting the array, cache drive was back (again: same deal this morning). I don't think I trust that cache drive anymore, so I'm leaning toward a new SSD, too.

 

As a bit of an aside, what do I optimize for in an SSD? Capacity, or speed, or...? Of course I know how to find generic SSD info and opinion, but since the application I'm going to be using it in is unRAID with Dockers, I'm more interested in the advice and opinion of people who use unRAID. I don't think I had any issues with 240GB (largest files I'd download are about 50GB; usual in the 10-15GB range), and I'd have thought even 6Gb SATA would be more than ample to juggle Sonarr, sabnzb, Plex, etc.

 

I've been having what I thought might be NIC issues (mobo onboard) for some time—slow to update or frozen WebUIs for various Docker apps, download speed dropping to zero, that sort of thing—but perhaps that could all be cache drive-related, since that's where the apps reside? In any case, today's more catastrophic failure isn't the only weirdness I've had in the last while.

 

You'll perhaps also recall that the forum helped me to install Community Applications (CA) recently, and to delete and restore my Docker file when stuff wasn't working right. That was on the same cache drive, and around that same time I'd also found shares not listed in the unRAID GUI, even though I could still get to them from Windows Explorer by typing in the share name directly. So whatever's happening now has perhaps been showing symptoms for a while.

 

shinagawa-diagnostics-20161211-1019.zip

OK, I've now got a new Samsung 850 EVO 250GB m.2 SSD installed. How do I assign drives so that I can recover shares and Dockers (or as much as possible, at least)? New SSD as cache drive, and failing drive as unassigned?

 

If that's right, then what do I do next to get shares back? Presumably I need to do that before using CA to reinstall Dockers?

 

I've tried searching for threads on how to recover from a cache drive failure, but nothing I've seen is enough like my situation for me to know for sure how to proceed.

If the failing SSD still works try the replace cache procedure :

 

https://lime-technology.com/wiki/index.php/Replace_A_Cache_Drive

 

I found the "replace a cache drive" instructions on my own, but I can't get any further than Step 3, because the array is already offline. Hasn't anybody ever had a cache drive go flaky before?! It seems sort of nutty to me that there's almost no good information to read about how to solve this sort of problem.

 

I don't want to start the array until I understand what's going to happen. Last time I did that (accidentally!) after changing out power cables, I ended up having to take a day or so to re-write a parity drive that was perfectly good (it just didn't have power when I put the array back online at first; when I re-connected the power, unRAID said I had to rebuild the parity).

 

I have installed the new SSD. The old one's still hooked up, but I guess it's unassigned? It's in the list, but I'm not assigning it to anything. All drives are using btrfs.

 

I already know the shares are missing. I want to know how to get the shares back (surely that information is backed up someplace?).

 

Pretty sure I have correctly re-installed Dockers once, but I'm also pretty sure I'd better not try that until shares are restored.

 

A nice explanation like I'm 5 (bonus: "and stupid") would be cool. I don't want to accidentally do something irreversible and b0rk the 40 TB of pooled drive space I've got.

 

 

You don't have to do anything to restore the data shares, shares are all top level folders in all disks + cache, you only need to worry about cache only shares or shares that could still have some data on the old cache.

 

If the old cache still works doing the procedure I liked above will move them to the array, if it doesn't you'll have to restore them from backups.

I'm sure this will all seem very straightforward to me in hindsight, but at the moment I still have questions:

 

1. I don't recall making any cache-only shares specifically. But maybe I did? I had an 'appdata' directory/share that surely must have been cache-only. I don't remember creating it manually (although maybe I did). If I restore Dockers without explicitly building an 'appdata' cache-only share, will the first restored/reinstalled Docker app create it?

 

2. How can I "restore [cache-only shares formerly present on the flaky SSD] from backups" if/when restoring from the old cache doesn't work? Where do I find the backups? (On the boot device, I'm guessing?)

1. I don't recall making any cache-only shares specifically. But maybe I did? I had an 'appdata' directory/share that surely must have been cache-only. I don't remember creating it manually (although maybe I did). If I restore Dockers without explicitly building an 'appdata' cache-only share, will the first restored/reinstalled Docker app create it?

 

Appdata folder location is specified in the docker settings, you'll need to reconfigure all dockers without the old appdata.

 

 

2. How can I "restore [cache-only shares formerly present on the flaky SSD] from backups" if/when restoring from the old cache doesn't work? Where do I find the backups? (On the boot device, I'm guessing?)

 

The share settings are in the flash drive, if you create a share with the same name it will have the old settings, but it's up to you to backup those shares data if there was any.

it's up to you to backup those shares data if there was any.

Well, that's not good news. That is something I didn't know. I have 2 x 8 TB of parity information, and still I have to do some sort of manual backup?!? I thought unRAID was supposed to be *better* than stupid WHS...!

 

Based on what I think I know now, here's what I'm planning:

1. Start the array with the old, flaky cache drive, and work through the procedure to move anything I can off the old drive.

2. Assign the new SSD as the cache, and see about getting the appdata folder back, and populated (if possible).

3. Using CA, restore previous Docker apps (and hopefully not losing much data in the process).

4. Bug the forum for an explanation of what and how to backup stuff that parity doesn't cover (after I search and read a bunch, first, of course).

 

Thanks very much for your help. Hope I do it justice as I get re-building!

Well, that's not good news. That is something I didn't know. I have 2 x 8 TB of parity information, and still I have to do some sort of manual backup?!? I thought unRAID was supposed to be *better* than stupid WHS...!

 

I was talking about the cache only shares, but like most will tell unRAID by itself is not a backup, it protects you in the event of a hard disk failure, but say you get infected with ramsomware, all your shares can get encrypted.

No, I understood that you were talking about the cache shares. My comments still stand: it's shocking that those aren't automatically protected by parity info! It's idiotic that some of my 16 TB of parity isn't automatically used to protect against a failure of 240 GB of cache drive! What's the point of having a server with dual parity, if it can't protect against a cache drive failure without the user having to post dumb questions like mine in the forum?!

 

I also understand that unRAID parity isn't a backup solution, and that it's still vulnerable to ransomware, or fire, or theft. If I didn't have to do dumb things like spend a week figuring out how one goes about recovering from a cache drive failure, maybe by now I'd have built a second, backup server....

 

**Well, that went better than expected: after firing up the old SSD and not being able to get Mover to do anything, and not being able to see any user shares, I resigned myself to some painful period of time (hours?) setting everything back up. Imagine my pleasant surprise, then, when I started the array with the new SSD, immediately had shares again, and was then able to restore the configurations of all my previous Docker apps through Community Applications and the backup function there! I guess I was partly wrong about the cache drive stuff not being backed up. Still have to set up Plex and Sonarr (et al) from scratch within the Web UIs.

No, I understood that you were talking about the cache shares. My comments still stand: it's shocking that those aren't automatically protected by parity info! It's idiotic that some of my 16 TB of parity isn't automatically used to protect against a failure of 240 GB of cache drive! What's the point of having a server with dual parity, if it can't protect against a cache drive failure without the user having to post dumb questions like mine in the forum?!

 

 

On the other hand it could be called shocking that this was no clear to you... The whole reason for the appearance of the cache drive was to have it -not- be parity protected, this is why it is faster and meant for temporary storage only..

 

If you use it for appdata I would advise to either use a plugin backup solution like CA and/or use a btrfs drive pool for your cache drive..

Fair enough. I did think about that after I posted: what I really want(ed) is a daily cache drive snapshot, say, so a drive failure would be easily recoverable. That's clearly not the job of parity.

 

My cache drive is BTRFS, so I guess I'm (happily, accidentally) one step closer to having a cache drive pool for redundancy. And I think I know just enough about Community Applications that I'll be able to sort out backups. I already have CA set up so I can restore Docker apps and their settings. But the actual operating data from said Dockers (e.g. Plex library, Sonarr download client settings, etc.), not so much.

 

I also wish all this stuff was better explained, and kept up to date. unRAID is super-powerful and flexible, and this forum is fantastically helpful, but I wish it were easier to start out and be walked through many of the options so that a new user could make informed decisions and end up with a system that the user feels confident she or he understands well. In my case, I thought I mostly understood what I had and how it worked, but the way I'd set up unRAID meant reality was actually something less than that.

 

I also wish I'd bought an SSD that didn't die within about six months, so maybe I'd have had time to read the forums voraciously for a bit longer, so I'd have had a chance to understand better where and how I wasn't protected....

 

 

I already have CA set up so I can restore Docker apps and their settings. But the actual operating data from said Dockers (e.g. Plex library, Sonarr download client settings, etc.), not so much.

 

I also wish all this stuff was better explained, and kept up to date.

Check out the manual for CA (Settings - Community Applications Section - Manual)  Also, most if not all pages / plugins have help enabled by pressing "Help" on the GUI