Jump to content
keyman33

Docker recovery after upgrade & proper setup of Cache drives [SOLVED]

49 posts in this topic Last Reply

Recommended Posts

Hi all,

 

I was terribly foolish and upgraded to 6.5 without doing any of the pre-work.  This is because I got lulled into the ease of upgrades by the new GUI , and didn't even think about checking things.  Very silly and stupid and I've learned a valuable lesson ( I was on 4.7 for ages, but since switching to 6, it's been so painless and even automatic).  I don't even know the previous release I was on. I think it was 6.4, but cannot be sure. How terribly sad.

 

I don't know if there's a way to recover my setup either, and if that is the case, it will be painful, but I will survive.  But if there is a way, you guys would be the ones to help me do it.

 

I hopeful that something is just pointing to the wrong place.

  • Before the upgrade, I had Plex, Duplicati, and Krusader as dockers.
  • Now, post 6.5 upgrade, I have Plex, Crashplan and HA bridge.  The latter two I deleted recently. But they are back and D and K are gone.

 

I hoping to recover my dockers setup if it is possible.

And I've also learned I need to backup my unraid itself somewhere other than duplicati. Presumably that is just taking a copy of the contents of the Flash drive? And that should also save all my dockers' settings as well, correct?

 

thanks for any assistance.

 

tower-diagnostics-20180419-2152.zip

Edited by keyman33
Solved, and lack of clear topic

Share this post


Link to post

If you just want to roll back to your previous version of unRAID and you did the update via the GUI then Tools -> Update OS give you the opportunity to downgrade to the old version.

Share this post


Link to post

Nothing about the upgrade should have broken anything in any permanent way. Your post is a little confusing to me. In particular

 

9 minutes ago, keyman33 said:
  • Before the upgrade, I had Plex, Duplicati, and Krusader as dockers.
  • Now, post 6.5 upgrade, I have Plex, Crashplan and HA bridge.  The latter two I deleted recently. But they are back and D and K are gone.

 

Are you saying that you had removed the Crashplan and HA dockers, but they somehow came back "automatically" because of the upgrade? And Duplicati and Krusader disappeared "automatically" because of the upgrade? Or are you leaving out some things? Or I am just misunderstanding what you are trying to describe.

 

18 minutes ago, keyman33 said:

I hoping to recover my dockers setup if it is possible.

 

Unless you have somehow deleted stuff from your flash drive or your appdata, you should be able to reinstall any of your dockers exactly as they were by going to Community Applications - Previous Apps.

 

19 minutes ago, keyman33 said:

And I've also learned I need to backup my unraid itself somewhere other than duplicati. Presumably that is just taking a copy of the contents of the Flash drive? And that should also save all my dockers' settings as well, correct?

 

And you can download a backup of your flash directly from the webUI by going to Main - Boot Device - Flash - Flash Device Settings and clicking Flash backup. Your docker settings are saved on flash, and that is where CA Previous Apps gets them to reinstall.

 

Share this post


Link to post

And a side note.  You really should consider changing the format of your drives from ReiserFS to XFS.  Reiser is a dead filesystem, and as time goes on with later kernel versions, it occasionally gives some users strange problems.

Share this post


Link to post

Hi all - thanks for all the feedback!

 

let me try to clarify and also, because of your answers, I think I have a better idea what is going on.

 

I’ve rewritten my answer a few times and thought I best re-start from the beginning.

 

The key thing I’ve realised, post mortem, is that my unraid was not working in top form, prior to the upgrade. My cache drive had disappeared from the system (as seen on “Main”). A new disk I had hot-inserted (in my front drive bay) was coming up as unassigned.  It had been this way for a few months, but I made some bad assumptions about internal wiring issues (and I had thought maybe I’d reached the limit on Disks, and it had forced out my cache drive) and because my more recent upgrades had gone so easily, I thought I would fix it while I upgraded.  Bad mistake and here I am.

 

So, here’s what it was before I upgraded:

6.4.x? (I think 6.4.2)

·       Cache drive is missing from the array (not shown in Unraid Main)

·       Dockers visible from Docker page in GUI:

        o   Krusader

        o   Plex ( call this “A”)

        o   dupeGuru

        o   duplicati

·       One old WD EARS disk in “unassigned” section of Unraid Main.  (I found it lying around. I had removed it as my old license had exceeded disk capacity, but I thought I would resurrect it now that I had a higher license). Because my cache was missing, I had not risked assigning it.

 

Performed upgrade to 6.5

·       Disk 3 died (another thread)

·       Replaced Disk 3

·       Old WD EARS drive causes Tower to fail on boot! That is, the BIOS said I was going to have imminent drive failure. (I’ve never experienced this before. I had thought the drive was good, but apparently not?). I removed this drive and it all booted up fine.

 

Current Status:

·       Cache drive now appears in Main page of GUI

·       Dockers visible from Docker page in GUI (different than before upgrade. Why and how to fix?):

        o   HA Bridge

        o   Crashplan

        o   Plex (call this “B”, as it appears to be configured differently than “A”)

·       A number of disks show “Some or all files unprotected” warnings. Why and how to fix?

 

I’ve provided screenshots of my 6.5 unraid if this helps at all plus a new diagnostics set.

 

Thanks in advance for any help!

 

appdata share settings.PNG

Docker configuration settings.PNG

Dockers after 6.5 upgrade.PNG

Inside cache appdata directory.PNG

Inside Disk 6s appdata directory.PNG

Location of appdata files.PNG

Shares after 6.5 upgrade with warnings.PNG

tower-diagnostics-20180422-0951.zip

Share this post


Link to post

Absolutely a bad idea to try to upgrade while you had other problems you were ignoring.

 

Your cache disk is almost completely full and very small, so it is going to take a few steps to get this all square. Do you have any VMS?

 

Assuming you don't have any VMs, We need to get everything except appdata  your system shares off your cache drive.

 

Please post a screenshot of the contents of cache.

 

Share this post


Link to post
1 hour ago, trurl said:

Absolutely a bad idea to try to upgrade while you had other problems you were ignoring.

 

 

On the other hand, it's likely these other problems you already had is the main cause of your issues, and the upgrade had little if anything to do with it. Why didn't you seek help sooner?

Share this post


Link to post
4 hours ago, keyman33 said:

Plex (call this “B”, as it appears to be configured differently than “A”)

How so?

4 hours ago, keyman33 said:

A number of disks show “Some or all files unprotected” warnings. Why and how to fix?

You mean shares, not disks.  Perfectly normal, as it means that some or all files in that share are on the cache drive and you don't have a cache-pool

 

Your screenshots showing the location of various folders on appdata share are also normal.  Haven't looked at the diagnostics, but if what @trurl states that your cache drive is small and full, then it makes sense that files are overflowing onto the array.

Share this post


Link to post

Hi trurl,

 

I don't have any VMs, and here are the contents of my cache drive.  Does it help?

 

I didn't ask for help sooner as I didn't think I needed it. I thought it was a disk limit issue that was forcing out the cache drive. Was stupid for sure.

 

The cache drive itself is not "auto-moving" as it used to do under 4.7 (to be honest I don't know if it worked since upgrading to 6.x...I think so, but am not sure (think *embarrassed* emoticon))

 

contents of cache drive.PNG

Share this post


Link to post

You need to have all of your system shares set to cache-prefer with no files on the array, and all other shares set to cache-no with no files on cache. This will require a couple of moves by mover. Follow these steps in this order. There is a lot of stuff here so read carefully and ask if you have any questions about what to do or why. If there is a problem at any step, stop and let us know about it.

 

1) Go to the Docker page and stop the docker service. Also stop all writing to your server.

2) Set the appdata, domain, and system user shares to cache-only. This will keep them from being touched by mover until we get the other stuff off of cache to make room for any of their files that may be on the array.

3) Set all other shares to cache-yes. This will prepare them for being moved off of cache. Also set these user shares to have Minimum Free larger than the largest file you will ever write to the share.

4) Go to Main - Array Operations and click Move now. Wait for it to finish. This should result in all the folders except appdata, domain, and system moved off of cache to the array.

5) Set the appdata, domain, and system user shares to cache-prefer. This will prepare them for getting any of their files on the array moved onto cache. And this is the way these shares will stay set, so if there is ever any overflow to the array they will be moved to cache when space is free.

6) Set all other shares to cache-no. Since they have already been moved to the array, they don't need to be moved. And, they don't ever need to be cached again because you don't have the space on cache for them.

7) Go to Main - Array Operations and click Move now. Wait for it to finish. This should result in any files from appdata, domain, and system moved off the array and onto cache.

 

That completes all the moves, and has your shares the way they need to be set going forward. The system shares belong on cache, and you don't have the cache space to cache any of the other user shares.

 

 

After all that, go to the User Shares page and click Compute All and when it's done post a screenshot. After we get your shares where they need to be we can work on the other problems.

Share this post


Link to post

These are great instructions. I've reached step 4, and press "move now" and the screen repaints but it doesn't appear as anything is happening.  The Cache drive remains quite full and the Main is not showing any change in disk consumption.

I've included some screenshots.

Main view of Cache drive.PNG

Cache drive settings.PNG

Share this post


Link to post

Your list of cache drive folders and the screenshot of your shares is confusing to me. Did you truncate the shares list because that was all that would fit in the browser for the screenshot? You should have shares listed for all the folders showing on your cache drive (except possibly the folders starting with a dot), and all those shares should be currently set for cache:yes if you followed through with trurl's step 3 correctly.

 

I would like to see a screenshot of a utility like windirstat pointed at \\tower\cache, it might shed some light on the current situation.

Share this post


Link to post
11 hours ago, jonathanm said:

Your list of cache drive folders and the screenshot of your shares is confusing to me. Did you truncate the shares list because that was all that would fit in the browser for the screenshot?

Essentially, yes, but here is a full view screenshot

 

11 hours ago, jonathanm said:

You should have shares listed for all the folders showing on your cache drive (except possibly the folders starting with a dot), and all those shares should be currently set for cache:yes if you followed through with trurl's step 3 correctly.

 

I would like to see a screenshot of a utility like windirstat pointed at \\tower\cache, it might shed some light on the current situation.

I deleted some bulky, unimportant files from the cache drive first, then ran Space Sniffer and have attached the screenshot along with a warning message it gave on the appdata directory (for crashplan which I no longer use since they killed home support).

All shares and all disks.PNG

Space sniffer view of cache with warning message.PNG

Share this post


Link to post

What happens now when you click on "Move Now" on the main page of the webgui? Same behaviour, or different?

Share this post


Link to post
7 hours ago, jonathanm said:

What happens now when you click on "Move Now" on the main page of the webgui? Same behaviour, or different?

Should have mentioned that as well.  Same behaviour - nothing moves.

Share this post


Link to post

Collect and post diagnostics after attempting a move. If mover logging is not enabled in settings, scheduler, enable it and hit the move now button before collecting the diagnostics.

Share this post


Link to post
Apr 24 17:40:21 Tower root: mover: cache not enabled

Stop the array, go to Settings - Global Share Settings - Cache Settings, and enable the cache drive

Share this post


Link to post
59 minutes ago, Squid said:

Apr 24 17:40:21 Tower root: mover: cache not enabled

Stop the array, go to Settings - Global Share Settings - Cache Settings, and enable the cache drive

You mean, disable it, then re-enable it?  It is already switched on. See screenshot.

 

 

Global settings for cache.PNG

Share this post


Link to post

Sure.  According to the diagnostics, it wasn't set to be yes.  Toggling should fix you up

Share this post


Link to post
3 minutes ago, Squid said:

Sure.  According to the diagnostics, it wasn't set to be yes.  Toggling should fix you up

Excellent, thanks. I've toggled it and it's working !  Now I will continue on the plan outlined and move on to Step 5 when finished!

Share this post


Link to post

Ok, back to the list:

 

On 4/23/2018 at 8:43 PM, trurl said:

4) Go to Main - Array Operations and click Move now. Wait for it to finish. This should result in all the folders except appdata, domain, and system moved off of cache to the array.

 

Unfortunately, it has now stopped moving, so I cleared a few more files manually, but still, a number remain.  Here are some screenshots.

 

Should I manually do more cleaning to move on to Step 5? 

Cache Disk content now.PNG

Cache Directory now.PNG

Share this post


Link to post

Any file in use can't be moved.  With appdata files, that pretty much means that you have to disable Docker (Settings - docker), run mover, then re-enable

Share this post


Link to post
32 minutes ago, Squid said:

Any file in use can't be moved.  With appdata files, that pretty much means that you have to disable Docker (Settings - docker) 

 

This was step 1 in my instructions. Don't know if he did it or not.

 

On 4/23/2018 at 8:43 PM, trurl said:

1) Go to the Docker page and stop the docker service. Also stop all writing to your server.

 

***Edit***

Also, he hadn't gotten to the step where he was trying to move appdata.

 

***Edit***

And instead of enabling docker after the moves I thought it would be a good idea to check everything had been moved, and then recreate docker image and reinstall dockers from Previous Apps.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.