Docker recovery after upgrade & proper setup of Cache drives [SOLVED]

[trurl]

31 minutes ago, keyman33 said:

Unfortunately, it has now stopped moving, so I cleared a few more files manually, but still, a number remain.  Here are some screenshots.

 

Should I manually do more cleaning to move on to Step 5? 

 

You should post your latest diagnostics. Did you enable mover logging as suggested?

 

4 hours ago, jonathanm said:

Collect and post diagnostics after attempting a move. If mover logging is not enabled in settings, scheduler, enable it and hit the move now button before collecting the diagnostics.

 

[keyman33]

8 hours ago, trurl said:

 

This was step 1 in my instructions. Don't know if he did it or not.

 

In fact, what I did was to stop all Dockers, not disable the function.  I've just done that and tried to do a move now, but nothing is happening (i.e., nothing is being moved).

 

Is it possible this is because a duplicate named of the file is found in the the share?  I did notice that with some of the easy cleanup I did.

If that is true, then which file do I keep, and how should I proceed?

[trurl]

9 hours ago, trurl said:

You should post your latest diagnostics. Did you enable mover logging as suggested?

 

[keyman33]

yes, mover logging has never been disabled from the start.

17 minutes ago, trurl said:

 

Pressed move, then exported diagnostics attached.

tower-diagnostics-20180425-0803.zip

[trurl]

Look at the end of your syslog. It does have a lot of things where it is trying to move a file that already exists. Not sure how you got these duplicates, maybe by trying to fix things yourself without advice.

 

The very fact that you can even browse your cache disk over the network is not necessarily a good idea. I always recommend that nobody have any disk shares and only have user shares on the network. You can lose files if you ever mix disk shares and user shares when copying or moving. And having access to a disk can also lead you to other things being out of whack if you don't know what you're doing.

 

The .DS_Store files are OK to delete, but I am concerned about the other files since it looks like you have your iTunes library on your server. I guess you are going to have to figure out which are the current versions of these for yourself.

[trurl]

We may be able to figure out which files your iTunes is using if you could show us how you have it configured.

[keyman33]

ok, I manually cleaned some files (carefully) and carried on to step 5,6, and 7.

 

On 4/23/2018 at 8:43 PM, trurl said:

7) Go to Main - Array Operations and click Move now. Wait for it to finish. This should result in any files from appdata, domain, and system moved off the array and onto cache.

 

 

At this Step 7,  I get stuck again because after the mover finished, I can still see that appdata and system have data on Disk 6 (see screenshots). (Diagnostics attached after last "move now" push)

 

Is this again because I have duplicate file names?

If so, what is the best strategy to fix the situation? I only care about these dockers, so the rest can be deleted:

• dupeGuru    
• duplicati    
• kde    
• PlexMediaServer

I would imagine I find the duplicate filenames, and then delete one according to the date (or rename it to something else like .TEMP) it might help?

 

tower-diagnostics-20180425-2045.zip

[trurl]

1 hour ago, keyman33 said:

I would imagine I find the duplicate filenames, and then delete one according to the date (or rename it to something else like .TEMP) it might help?

Try it.

[keyman33]

ok, here's my latest. I tried to do the replacement name but soon learned that the amount of files to execute this on is outrageous.

On 4/25/2018 at 10:44 PM, trurl said:

Try it.

 

So, I am at the point where I just want to clean Disk 6 of all appdata (Crahsplan and Plex) and be done with it.  After all the previous steps, all of the data should be, in one state or another, on my cache drive.

However, when I try to delete the files, I get a permissions error.  I read that New Permissions function apparently will also not affect permissions as the appdata folder is protected, for safety reasons. So, how can I proceed?

[trurl]

32 minutes ago, keyman33 said:

However, when I try to delete the files, I get a permissions error. 

 

You can do anything from the command line. I typically work with disk files and folders using the builtin Midnight Commander (just type mc at the command line). Midnight Commander is a text-based GUI file manager with plenty of information on the web.

 

Your disks are in /mnt.

 

[keyman33]

Great

 

8 hours ago, trurl said:

 

You can do anything from the command line. I typically work with disk files and folders using the builtin Midnight Commander (just type mc at the command line). Midnight Commander is a text-based GUI file manager with plenty of information on the web.

 

Your disks are in /mnt.

 

Yeah, that worked great. I'm pretty familiar with MC as I was on 4.7 for a long time and used it frequently.

W

OK, so I've reached the last step of trurl's instructions:

 

"After all that, go to the User Shares page and click Compute All and when it's done post a screenshot. After we get your shares where they need to be we can work on the other problems."

 

Here are the screenshots requested of the Compute All.  Let me know what to do next. I'm mostly worried about the Duplicati (the rest I can rebuild, if needed). I have all my settings inside the Duplicati docker and never exported my configs, stupidly.

 

[trurl]

What is on disk6 in the appdata folder and in the system folder?

[keyman33]

I don't know. I cleared out the appdata ..it was those ..DS store files.

Here's the Disk 6 System data. Can I delete?

[trurl]

Yes delete. Your system share should be cache-prefer now so when we recreate docker image it will be on cache.

[keyman33]

Ok, done.  Do you need to see another screenshot of the whole thing?  I've attached a screenshot of just appdata and system, showing they are just on the cache drive now.

[trurl]

If all of your other shares are like the previous screenshot then you should be good.

 

Now, in Settings - Docker, I suggest you delete docker image and create a new one. What path do you have for it?

[trurl]

And just to double check everything else, confirm that system, appdata, domains user shares are cache-prefer and all other user shares are cache-no.

[keyman33]

13 hours ago, trurl said:

If all of your other shares are like the previous screenshot then you should be good.

 

Now, in Settings - Docker, I suggest you delete docker image and create a new one. What path do you have for it?

 

Done. I deleted the image, and here's how it sits now.  (also doublechecked the cache settings on each share)

[trurl]

OK, you should be good to enable Docker. Then you can go to Community Applications - Previous Apps and it will let you reinstall any of your previous dockers with their previous settings.

[keyman33]

On 4/29/2018 at 9:20 PM, trurl said:

OK, you should be good to enable Docker. Then you can go to Community Applications - Previous Apps and it will let you reinstall any of your previous dockers with their previous settings.

 

Great, so I've started the process. I tried Krusader and that seems to be all fine.  Next I went to Plex, and I had some internal settings within Plex that were not the most current (remember my "A" and "B" at the beginning of the thread?  This was my A settings (essentially just directory locations), but I patched in B settings and it worked. There may be other things, but I will figure it out).

 

Ok, then moving on to duplicati, which is the one I'm most fearful of not being able to recover intact with all my settings & connections to onedrive (I switched when Crashplan for home user was shutdown by the company. My view was that getting Office 365 with 5 TB for a family subscription made the most economical sense).

 

Before I hit Apply, I have been reviewing the Container Setup because I don't want to lose my setup. Now, this might have nothing to with anything, and may not impact my setup, but I really don't understand hosts, paths, etc. The reason I was able to make this work originally is because I watched one of SpaceInvader One's excellent tutorials. 

Here's the video I followed, and his setup is far simpler (I took a screenshot of his settings), and I would have copied his exactly (am 98% sure)

 

But yet, when you look at my settings, there are 4 Host Paths, so the Container was probably updated since he did his video.

The ultimate question is should I hit Apply, or should I be making some more adjustments to ensure my old setup will return intact?

 

thanks, once more. I think we're just about at the end of this!

[trurl]

I don't use that container so haven't really researched it in any way.

 

5 minutes ago, keyman33 said:

I really don't understand hosts, paths, etc.

 

You really should make the effort. Here is the Docker FAQ:

 

https://lime-technology.com/forums/topic/57181-real-docker-faq/

 

 

[keyman33]

Ok. Great thanks for everyone's help so far. I will begin my study this weekend of dockers, hosts and paths and come back to this thread when i have more insight.

[keyman33]

Ok, all is now well in my unRaid set up.  I couldn't actually decipher where the config files were for duplicati, so I made a complete backup in case I overwrote anything. Then I learned about paths within dockers and it made sense, and I could see that my old settings were correct. So I hit apply and everything came back up just as it was.

 

Next on my agenda is to

-export my duplicati json files, in case I ever need to recover. Same goes for other configs within dockers.

-see if I can access my duplicati backups outside of the docker and unraid (I think I can), in case my unraid server was stolen.

- rebalance my drives as many are near the limit and others are under utilized.

- try to upgrade to xfs on all drives.

 

Big thanks for everyone's help here, especially trurl and jonathanm!  You guys bailed me out and got things humming again on my unraid!

 

[JonathanM]

4 minutes ago, keyman33 said:

- rebalance my drives as many are near the limit and others are under utilized.

- try to upgrade to xfs on all drives.

These two statements are a little at odds with each other. You need to unbalance so you have your largest ReiserFS drive empty. Then you reformat the empty drive to XFS, copy the next drive to it, format the drive you just copied, repeat until done.