(Updating Docker container) "Error: filesystem layer verification failed for digest sha256" and is that caused by the same problem causing streaming issues in Plex?


Recommended Posts

Hi!

 

I just tried to update binhex-qBitVPN but pulling the latest image gives me this:

Pulling image: binhex/arch-qbittorrentvpn:latest
IMAGE ID [679001613]: Pulling from binhex/arch-qbittorrentvpn.
IMAGE ID [e130c81b086a]: Already exists.
IMAGE ID [f0c2d8550f0e]: Already exists.
IMAGE ID [e8b87bc620a7]: Already exists.
IMAGE ID [7b303ad07582]: Already exists.
IMAGE ID [e6cff58d3325]: Already exists.
IMAGE ID [ed093e4b8303]: Pulling fs layer.Downloading 100% of 4 KB.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [0a7e7d1b57dc]: Pulling fs layer.Downloading 100% of 12 KB.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [acdab43f922d]: Pulling fs layer.Downloading 100% of 3 KB.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [a873d56991c3]: Pulling fs layer.Downloading 100% of 11 MB.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [bf4455564214]: Pulling fs layer.Downloading 100% of 293 B.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [47be8f93f574]: Pulling fs layer.Downloading 100% of 2 KB.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [9a45b4ff2318]: Pulling fs layer.Downloading 100% of 3 KB.Download complete.Extracting.Pull complete.
IMAGE ID [d77252f0b421]: Pulling fs layer.Downloading 100% of 383 B.Verifying Checksum.Download complete.Extracting.Pull complete.
IMAGE ID [d1631fdb6136]: Pulling fs layer.Downloading 100% of 181 MB.Verifying Checksum.

TOTAL DATA PULLED: 192 MB

Error: filesystem layer verification failed for digest sha256:d1631fdb61368a5121f94f4a89dcaa24f36e2d1bd9402729aaccbba2a8403a02

I've never seen this Error: filesystem layer verification failed for digest sha256 before, and googling it doesn't give much. A few posts point towards checksum errors and doing a Memtest. Suggesting it's more of a HW issue than SW.

 

Any ideas?

 

Did a Memtest86 last night actually, just prior to this reboot of the server and prior to this error presenting itself. Showed no faults at all. Ran 4 passes.

IMG_7129.thumb.jpeg.9320ee33d9567ee319a37cc8c29f3e3f.jpegIMG_7130.thumb.jpeg.33e9ed854f1c416d7ec0874f6dd439cc.jpegIMG_7131.thumb.jpeg.0def50545e2c424c66517c3e63a58da3.jpeg

 

The thing is that I've been troubleshooting my Plex install for the last month now. That is why I did the Memtest.

I have these intermittent streaming errors/freezes/glitches/pixelations on everything I stream. Doesn't matter if it's direct play or transcoding. Doesn't matter what container I use (Hotio, Binhex, linuxserver) or if I'm HW or SW transcoding. Doesn't matter if it's played local or remote. Doesn't matter what device is used, or if it's wireless or wired. Doesn't matter what version of Plex I'm running.
More comprehensive explanation on Reddit here.

 

These streaming issues has driven me crazy and I'm more and more convinced that it's some kind of HW problem. 

If you have any suggestion regarding that issue I'm all ears.

 

Plan right now for troubleshooting that is:

- Exchange the router currently acting as a switch in the office for a 2.5 GbE QNAP switch I had already ordered. See if the router for some reason can't handle the traffic, even though it is light.

- Remove the Shelly Plug S that is used to measure power consumption of the server. If I really stretch my imagination the Plex issues might have started all the way back in October when I started using the plug... Perhaps it's introducing some weird overtones or something.... Yeah I know, but I'm desperate... 🤷‍♂️

 

Attached Unraid diagnostics and Plex server logs (confirmed playback issues occur from Dec 27, 2022 13:55:16 and forward). 

 

Best regards

define7-diagnostics-20230102-2210.zip Plex Media Server.log.zip

Edited by kim_sv
formatting
Link to comment

Ok, I managed to install Portainer and deleted a bunch of volumes that was unused. After that I was able to update the container without any problem.

What I don't understand though is why the size of the /docker directory is so different depending on where I look:

 

/mnt/user/system/docker is set to cache:prefer and is on a 250 GB SSD.

69759719_Skrmavbild2023-01-03kl_00_49_45.thumb.png.d9aba1fcad5828157c4a856bed6fc623.png

 

Checking it's size in Terminal.

72386596_Skrmavbild2023-01-03kl_00_43_37.thumb.png.1a085f3da70ca84b22b75cbbbe53ee99.png

 

Calculating /mnt/cache/system/docker in Unraid UI.

776576882_Skrmavbild2023-01-03kl_00_49_31.png.084d4f55c6df697fb3e32323ec3513e3.png

 

Something doesn't add up?!?

 

 

The streaming issues persist also... :( But perhaps I should nuke and rebuild /docker directory?

Edited by kim_sv
Link to comment

Nobody have any suggestions on the Plex streaming issue? I’ve tried multiple things to solve it without any success… 

  • I’ve nuked the docker image on the server and reinstalled Plex.
  • I’ve tried multiple fresh installs of Plex using different Docker repositories (binhex, linuxserver, official Plex).
  • I’ve run a memtest86 to make sure my RAM is good, multiple passes. No errors.
  • I’ve changed wall outlets, a desperate move I know…

I’ve also tried installing Jellyfin and tried streaming the same files through that, and that has actually been working. Haven’t been able to produce the same issues with that. So… Plex does seem to be the culprit. Unfortunately.

 

Is there some dependencies that Plex have that isn’t included in the Docker containers? Since all Plex containers I've installed have the same problem I figure it has to be something else?!

 

This is the Plex.app on my Mac. This sort of graphical errors I also get from time to time. I’d say exclusively on the artworks, posters and backgrounds like in this screenshot. Since that’s stored on the server in the appdata library, there must be either something corrupting the file when it’s transferred over the network to the Mac (or any other device I have, all present the same intermittent issues) or some sort of file corruption on the cache that is intermittent?! I really don’t get it...

59659407_Skrmavbild2023-01-09kl_07_52_00.thumb.png.90608222ffdb5adb8c352bbd89db81b4.png

Please help! I’m loosing faith here!

Link to comment
4 hours ago, kim_sv said:

Please help! I’m loosing faith here!

Lose faith in Plex, not in anything else.

 

Plex has become a necessary evil in the world and once you find a version of it that actually works (every update seems to break something else), then leave it alone and don't update again.

Link to comment
11 hours ago, DarphBobo said:

Lose faith in Plex, not in anything else.

 

Plex has become a necessary evil in the world and once you find a version of it that actually works (every update seems to break something else), then leave it alone and don't update again.

Mm, but i have been using Plex for 10+ years and generally speaking been a happy camper. So I’m willing to fight this, as in my opinion, neither Emby nor Jellyfin is even close to compete with the user experience and the WAF. 

Link to comment
13 hours ago, trurl said:

me too. And I'm not currently having any problems with it so it seems like it must be something with your setup.

 

Do you have appdata backup?

 

 

Yes, I've also tried resetting the appdata from backup. Sorry, I've tried so many things over the past weeks it's hard to keep track of what I've presented as "tried". I also have support threads regarding this going in a few different forums also, since I'm not really sure what is the problem. I could be hardware, it could be Plex, it could be Docker or the created containers, it could be my network and it could be.... I don't know, karma?! 🙄

 

Do you by any chance have any insight into the "Error: filesystem layer verification failed for digest sha256" problem?

I did switch from Docker image to directory last spring, should I try to revert back? I guess it shouldn't matter..

Edited by kim_sv
Link to comment
5 minutes ago, trurl said:

Since you have appdata backup of plex, you could try deleting your plex appdata so the container will start fresh with a new appdata and see if that works.

That is what I have done when I've spun up new containers of Plex from different repositories. Tried linuxserver, Binhex, Plex official and a clean Hotio. Just installed them plain and just added a test folder containing a few different videos in various resolution and dynamic range. Same weird streaming issues in all.

Link to comment

No, sorry! This perhaps should've been two different threads, I just put them in the same because I thought they were (are) related. But I can see how it causes problem following along. 😬

 

Originally my "only" problem was these weird streaming issues with glitches/freezings/pixelations/errors and artwork being displayed with artifacts in Plex UI. Like this:

135949094_Skrmavbild2023-01-10kl_15_26_30.thumb.png.434e606f9967ca943d7707a6c42db0f7.png983747264_Skrmavbild2023-01-09kl_07_52_00.thumb.png.e4a9038110ae54766809923da7cbf903.pngIMG_7033.thumb.PNG.69e9e19196df3c7432fa35deef9c88df.PNGIMG_7032.thumb.PNG.866fa375edbde7bc9484e0792b1a1b1a.PNG

 

But it was during my troubleshooting of these issues, and subsequently multiple different Docker container installs, that I got the:

Error: filesystem layer verification failed for digest sha256

 

So to hopefully make it clear:

- The streaming issues occur intermittently, no matter what type of device is played on, no matter what type of media is played etc etc.

- The sha256 error is occurring intermittently during some, not all, installs/updates of Docker containers. Seems to help to remove unused volumes in Portainer.

 

Since sha256 seems (?) to be a checksum problem, could that point to a network issue?

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.