Jump to content
casperse

Strange Docker problems?

22 posts in this topic Last Reply

Recommended Posts

Hi All

 

After replacing my cache with a large drive I have had problems with my dockers

I then did a rebuild - deleted my old docker-file and downloaded and reinstalled all my Dockers

But afterwards I see strange behaviour Dockers that will not start but seem to be running (Green arrow)

Dockers that just stops? or Emby that was running perfect before just stops and give errors? (Didn't change anything)

 

image.thumb.png.31373811bd582b479ad7e4df0d64b22f.png

 

Here is my Diagnostic files I hope someone can help me!

Your help is much appreciated this is so strange?

Gone from a stabile system to a "GHOST IN THE MACHINE" 

image.png.a43c2e31d4be9500f2b94bb31f296d9c.png

 

My container size is 40G no where near that:

image.png.eab57ab65156aec7b06a58ac115828a8.png

 

plexzone-diagnostics-20200224-2209.zip

Edited by casperse

Share this post


Link to post

This really scares the shit out of me......

 

image.png.67385540c017d26f4dba0cebf7ac9439.png

How can the user path be gone?

 

image.png.2cc265bb3dee13bfc9bf1afcdb134972.png

 

Should I reboot the system?

Seems my User shares disappeared?

 

Starting and stopping the array didn't help....

image.png.62100bceb48aa7b3869c0b789e22f94c.png

 

Rebooting and hooping that will get my user shares back....

 

Edited by casperse

Share this post


Link to post

Do a disk check on your USB stick.

Stick dropping offline / corrupt can cause weird problems.

Share this post


Link to post
8 hours ago, testdasi said:

Do a disk check on your USB stick.

Stick dropping offline / corrupt can cause weird problems.

You might be right... just got this new error

image.thumb.png.6d60442c333617e033bdf236b001ce71.png

I think the Nvidia files is on the flash so that would make sense!

 

How to check in Unraid a check command from the command line?

 

 

 

Share this post


Link to post

How to run this manually? or see the log?

I currently have a very strange problem and it might be the USB?

 

I had the user shares disappear! - I did manage to get a Diagnostic before rebooting if anyone would take a look I would be really happy

 

 

 

Share this post


Link to post

Update: Following the error leading to the Nvidia plugin which apparently also located on the USB?

I re-installed the plugin and rebooted the server, sofar it looks like everything is running again?

 

Could this cause the loss of User shares? (I didn't know that the Nvidia plugin was such a big deal of the Unraid server, if this is the case?)

 

Share this post


Link to post
Just now, casperse said:

Update: Following the error leading to the Nvidia plugin which apparently also located on the USB?

I re-installed the plugin and rebooted the server, sofar it looks like everything is running again?

 

Could this cause the loss of User shares? (I didn't know that the Nvidia plugin was such a big deal of the Unraid server, if this is the case?)

 

It is unlikely to be the Nvidia plugin as all that does is control whether the Unraid OS files on the USB stick  that are the standard version should be replaced by the version that includes the Nvidia drivers.     This replacement will only happen at the time you load a new version via the Nvidia settings - it will not have any affect the rest of the time as once a version is loaded onto the USB stick it boots just like the standard Unraid.

 

As was mentioned it is likely that the USB stick going offline for some reason caused the problem and rebooting fixed this.   I guess there could be an inherent fault in the version of Unraid OS that the Nvidia plugin has installed, but this seems unlikely or more people would be having the same problem.

Share this post


Link to post
7 hours ago, casperse said:

I did manage to get a Diagnostic before rebooting if anyone would take a look

Post them. I'll likely split this into its own thread since it is a support request not a feature request.

Share this post


Link to post
9 hours ago, itimpi said:

It is unlikely to be the Nvidia plugin as all that does is control whether the Unraid OS files on the USB stick  that are the standard version should be replaced by the version that includes the Nvidia drivers.     This replacement will only happen at the time you load a new version via the Nvidia settings - it will not have any affect the rest of the time as once a version is loaded onto the USB stick it boots just like the standard Unraid.

 

As was mentioned it is likely that the USB stick going offline for some reason caused the problem and rebooting fixed this.   I guess there could be an inherent fault in the version of Unraid OS that the Nvidia plugin has installed, but this seems unlikely or more people would be having the same problem.

My USB was placed inside a rackmount server in the middle of the motherboard - Safe place but hard to get to

I did run several test and all was clean?

image.png.60dddc66fc86526544e26003edef241d.png

 

My hope is that the logs above will give a hint to what might caused this....

 

Share this post


Link to post
3 hours ago, trurl said:

Post them. I'll likely split this into its own thread since it is a support request not a feature request.

Thanks @trurl I added my logs to the first post in this thread:

BTW: Just took my server apart to get the USB out and I did several disk check in windows all was fine no errors

Share this post


Link to post

Why do you have disks assigned in 2-7 and 10-22 but none assigned to 1,8,9?

Share this post


Link to post

I took a look at your system log and did see some messages that I shouldn't have.  Specifically I saw a kernel message relating to python:

 

Feb 24 10:26:30 PLEXZONE kernel: python[5234]: segfault at 6e6f68747988 ip 000014bccd18db98 sp 000014bcc6979c68 error 6 in ld-musl-x86_64.so.1[14bccd180000+46000]
Feb 24 10:26:30 PLEXZONE kernel: Code: 48 8b 47 10 48 39 47 18 75 14 89 f1 48 c7 c0 fe ff ff ff 48 d3 c0 f0 48 21 05 f4 ee 06 00 48 8b 57 18 48 8b 47 10 48 89 42 10 <48> 89 50 18 48 8b 47 08 48 89 c2 48 83 e0 fe 48 83 ca 01 48 89 57

 

I see you have a LOT of plugins installed.  Please try booting in safe mode, disable all plugins, and see if things go back to normal.

Share this post


Link to post
10 hours ago, jonp said:

I took a look at your system log and did see some messages that I shouldn't have.  Specifically I saw a kernel message relating to python:

 

Feb 24 10:26:30 PLEXZONE kernel: python[5234]: segfault at 6e6f68747988 ip 000014bccd18db98 sp 000014bcc6979c68 error 6 in ld-musl-x86_64.so.1[14bccd180000+46000]
Feb 24 10:26:30 PLEXZONE kernel: Code: 48 8b 47 10 48 39 47 18 75 14 89 f1 48 c7 c0 fe ff ff ff 48 d3 c0 f0 48 21 05 f4 ee 06 00 48 8b 57 18 48 8b 47 10 48 89 42 10 <48> 89 50 18 48 8b 47 08 48 89 c2 48 83 e0 fe 48 83 ca 01 48 89 57

 

I see you have a LOT of plugins installed.  Please try booting in safe mode, disable all plugins, and see if things go back to normal.

@jonp Yes I am "like many others" a Space invader fan 😊 - and moving from Synology I found many great plugin that helped me accomplish this

But you are right I need to remove the non essential plugins! - problem is that many are now "mandatory"

I guess if possible it would be better to move to dockers if I can find a similar application to replace the plugin

 

I am now running in safe mode and have attached a new Diag file - hope the error is gone now!

 

What to do next? - retracing and uninstalling every plugin, starting from the latest ones?

(If you tell me where to look for this python error I can search for it after each reboot?)

 

Again your help is much appreciated - hope we can find the source of this problem and kill it

 

@trurl I have all my drive numbers in Unraid assigned to the corresponding numbers of my servers HD cases (Make it much simpler to find a failing drive)

Drive one is a Un-Assigned Drive - so I dont spin up the array - was inspired by this post about seeding and ended up with this solution - another forum post ,(Drive 8 and 9 is temporary empty because I am slowly replacing all my 3TB drives with larger ones - So drive 8 and 9 was moved to a larger drive 6 and so on its slow but it works and it requires $ so it's a slow process 😉 )  

 

plexzone-diagnostics-20200226-0809.zip

Share this post


Link to post

Update:

This might not be related?

 

But all this started after upgrading to a new bigger M.2 cache drive, from a 1TB to a 2TB drive

Could this somehow be the problem? (I formatted it like the old one "btrfs encrypted")

Recreated the docker image and re-installed all the dockers!

Only the Apdata is copied back to the new cache drive

 

Below removed this had nothing to do with the error!

Focusing on testing Plex for my stability testing...

I just did a small test by copying the Plex Apdata folder to a UAD SSD and did a install with the same user "docker profile" but pointing to the SSD for Apdata

and then Plex started to work like it did before (Exactly the same files) now HW encoding for TV works it doesn't when running from the cache drive?

- This is so weird, it's like all logic is out the window..... Again this might not be related and just me grasping at some point of error origin 

 

I have so far removed the latest added plugins:

Custom Tab

Dynamix File Integrity

Dynamix Active Streams

Server Layout

Edited by casperse

Share this post


Link to post
8 hours ago, casperse said:

Only the Apdata is copied back to the new cache drive

This is usually spelled "appdata". If you actually used "Apdata" then you may have created some problems.

Share this post


Link to post
On 2/27/2020 at 2:58 AM, trurl said:

This is usually spelled "appdata". If you actually used "Apdata" then you may have created some problems.

No its ok "appdata" just a misspelling here....

Share this post


Link to post
On 2/26/2020 at 12:19 PM, casperse said:

Recreated the docker image and re-installed all the dockers!

Your system share has some files on the disk7. Maybe libvirt instead of docker image though. You might just check the contents of disk7 to make sure your docker image is on cache now.

 

 

Share this post


Link to post
48 minutes ago, trurl said:

Your system share has some files on the disk7. Maybe libvirt instead of docker image though. You might just check the contents of disk7 to make sure your docker image is on cache now.

 

 

Yes I can see that this is the libvirt

image.thumb.png.c8ca219a2117563d9c347cb0776d5223.png

I have disabled VM and I am running mover now, shouldn't that move the file to cache? (I have system prefer cache setting)

Share this post


Link to post
4 minutes ago, casperse said:

I have disabled VM and I am running mover now, shouldn't that move the file to cache? (I have system prefer cache setting)

If you disabled it in Settings - VM Manager then that should work.

Share this post


Link to post
5 minutes ago, casperse said:

Yes I can see that this is the libvirt

image.thumb.png.c8ca219a2117563d9c347cb0776d5223.png

I have disabled VM and I am running mover now, shouldn't that move the file to cache? (I have system prefer cache setting)

If it still will not move then check that there is not already a copy on the cache drive.    Mover will not do anything if the file already exists on the target.

Share this post


Link to post
36 minutes ago, itimpi said:

If it still will not move then check that there is not already a copy on the cache drive.    Mover will not do anything if the file already exists on the target.

Yes its on both? - which one should I delete

image.thumb.png.4c5dca907261cafca163da531710f9e2.png

Update I deleted the oldest file not updated on drive 7

Edited by casperse

Share this post


Link to post

Ups didnt like that getting a Libvirt does not start error now....

image.png.cbe119fc0834fa8f3649745289832911.png

Edited by casperse

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.