Docker Failed to Start - New Drive Needed?


Recommended Posts

I had an I/O error on transmission.  I tried to restart that docker and then the whole Docker now fails.  I'm guessing I probably need to replace my SSDs?  I just thought even thought they were showing bad SMART stats, I didn't think I could wear the SSD out before filling up 100TB.  Usually they can do like 200TB right?

 

Just want to confirm before I place the order, I had already been looking to replace them just in case.

 

tower-diagnostics-20210510-2057.zip

Link to comment
On 5/11/2021 at 4:16 AM, JorgeB said:

Syslog is showing a problem with cache2, looks more like a connection problem, replace cables and post new diags after array start.

I restarted, it's asking me to wipe the drive?  That doesn't sound good.

image.thumb.png.5a913f28b53d621c6624fa92abb71931.png

 

I'm almost positive I haven't written more than 200TB across both drives.  They've been definitely used hard, so if it really is broken, no problem.  I've probably written 1.5-1.8x my current size since I replace data and it has to go through the SSD. So maximum, I think I've written 130-150TB since I only have 80TB of data.

tower-diagnostics-20210512-1731.zip

Edited by tential
Link to comment

Try this, if Docker/VM services are using the cache pool disable them, unassign all cache devices, start array to make Unraid "forget" current cache config, stop array, reassign all cache devices (there can't be an "All existing data on this device will be OVERWRITTEN when array is Started" warning for any cache device), re-enable Docker/VMs if needed, start array, post new diags after array start.

Link to comment

You should disable docker until you get cache fixed. Your System and Appdata shares have been recreated on disk16 since cache wasn't available.

 

You will want to delete those or move them back to cache before enabling docker again. Probably deleting would be more appropriate since you will want to get your appdata from backup when you start over with docker.

Link to comment
7 hours ago, trurl said:

The more likely reason for your filesystem corruption isn't bad disks. Have you ever filled cache?

Yes, I've filled it previously, multiple times.  I've run out of space a bunch previously.  I haven't run out of space recently though.  I probably dipped below 20 free GB when this happened, but I try not to go below 50-20GB remaining now.

 

I disabled docker, but it wasn't running/saying no dockers were installed.

 

What step should I take next to reenable cache?

I misunderstood your post and thought the data was safe on disk 16.  I'm guessing filing the drive wasn't too good for it?

Edited by tential
Link to comment

trurl says the data is on disk 16 already? (Did not know this was something that even happened.  Would love an explanation of how/why it backed it up like this?  Not complaining, cool! Just want to learn lol).

 

So it's ok for me to reformat, and just move the data from disk16, back to my cache, then restart my docker?

Link to comment
3 hours ago, JorgeB said:

trurl mentioned that since cache wasn't available new docker/VM images were created on the array, but those are new, they won't have any of your data.

If there is no easy way to reenable the pool so that it works again, or easy way to recover the data, and I have to delete the data, so be it.  Just didn't want to jump to that conclusion.  And didn't want to keep using the drives if they had to be retired, but doesn't sound like that yet from your responses.

Link to comment
13 hours ago, tential said:

If there is no easy way to reenable the pool so that it works again, or easy way to recover the data

 

18 hours ago, JorgeB said:

Pool is in a very bad state, there are some recovery options here you can try, but also make sure to see this for better pool monitoring for the future.

 

Link to comment
On 5/14/2021 at 2:38 AM, JorgeB said:

 

 

I see, so I gave it a shot and I think I got it? I'm just not sure exactly how to start the copy and what settings to use.

I got the /x directory mounted and I want to move it to my study materials folder on Disk 17, the only disk with enough space + extra.

 

I selected the Alt+5 to copy and hit the Dive into sub dirs check list.

 

Now do I just hit OK?  Or did I choose a bad copy place (copy to a mount vs array).  I wasn't able to find array places, only these mounts?

 

Also, I have no shares anymore?  Should I be worried?

image.png

image.png

Edited by tential
Link to comment
On 5/16/2021 at 12:04 AM, JorgeB said:

Yes, but you can also browse /x and only copy just what you need.

I got a lot of errors doing that, so guessing that's not valid? 

 

I tried BTRFS repair as a last resort, but it's telling me the cache pool is still mounted, I haven't even started the array yet.

 

image.thumb.png.fc78037b4fa8d2d3af44c9d99d408454.png

Link to comment
12 hours ago, JorgeB said:

mdX would be to restore from an array disk, to restore from a cache disk you always use sdX.

 

Thanks for the clarification.  I tried, still can't get it to recognize the second disk is part of the pool, oh well.  Guess I need to start from scratch.

 

What steps do I take to reset it now that it for some reason sees both ssds, and wants me to load them, but it won't work?  So I need to wipe/rebuild the pool.  How should I do that?

image.thumb.png.4857ce4a595c28bfb89141395738328d.png

Edited by tential
Link to comment

Ok, I was  able to reset, and then follow some basic instructions to get everything back to what it used to be.  Setting up my dockers again is going to be a task.  I had my cache drive setup incorrectly I guess since I never ever used Mover, hence why my SSDs got full in the first place. 

 

I always read you can't move data between shares on unraid.  However, I always did that?  Am I misunderstanding?  I used to have a downloads share, that was on my cache only.  When a DL completed, I manually moved it over to the correct share it should be on.  I was thinking of creating a Completed Downloads share instead, that would use the cache drive to store complete downloads that then would use mover to go to the array.  Then copy from the Completed Downloads share to the correct share.  This is basically how I've always done it, just I'd be using the mover to free up space.  Is this going to cause me issues?  It's always worked in the past with no issues so not sure I'm understanding this correctly.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.