Jump to content
Rich Minear

Downgraded back to 6.6.7 due to Sqlite corruption

191 posts in this topic Last Reply

Recommended Posts

I have the backup plugin installed but I do not have it configured to create backups. Per the plugin, it says it stops the container. 

Share this post


Link to post
Posted (edited)

Anyways, I know when mine corrupts.  It’s when a new show or movies is added to plex for the first time. I use sonarr, it will download a tv show to my download directory, then move it to my media directory. Plex is configure to watch for file changes, notices the new file and attempts to add it to my collection. Then db corrupts at that point. Same process with movies. Deluge downloads to the download directory then moves it to the media, at which point my db corrupts. 

 

From there, I restore from my backup db, start plex, and tell it to scan for new files. It has no problem at that point. It’s the first scenario that corrupts plex. I haven’t noticed a pattern with sonarr or the others yet. But it seems like if one is corrupted the others have a high chance of being corrupted as well. 

Edited by runraid

Share this post


Link to post

I have no plugins which could mess with it (I think). Only the folder caching plugin, which is disabled for appdata, and the turbowrite plugin.

Plex is working for ~12 hours now. (on /mnt/disk1/appdata). So I think that the issue for me is only the /mnt/user.

Share this post


Link to post
1 hour ago, saarg said:

I thought the appdata backup stopped the containers first, but I'm probably wrong and the wrath of @Squid awaits...

It does by default unless you turn it off

1 hour ago, saarg said:

So you are saying it's all @Squids fault? 😛

Seriously doubt it.

 

Share this post


Link to post
3 minutes ago, Squid said:

It does by default unless you turn it off

Seriously doubt it.

 

Me too, but where is the fun in that.

Share this post


Link to post
Me too, but where is the fun in that.
I'm down with blaming Squid.....

Sent from my Mi A1 using Tapatalk

  • Like 1
  • Upvote 1

Share this post


Link to post

 

49 minutes ago, Squid said:

Seriously doubt it.

denial is the first sign of guilt, just saying...

Share this post


Link to post

image.png.dc254c328d2291b9689a18468cb560c9.png

  • Like 5
  • Upvote 2

Share this post


Link to post
Posted (edited)
On 6/6/2019 at 2:40 PM, Lonnie LeMaster said:

 

 


Just curious, do you have any volumes mapped to your containers using the unassigned devices plugin?
 

Nope, they're mapped to my disks that are part of my array.


Also (Not explicitly directed at you Lonnie LeMaster), I just recently tried switching back to 6.7.0 again. In less than 24 hours it corrupted. I rolled it back, it went fine. Then after like 3 days I wake up one morning, to hear complaints about how the Plex server isn't working anymore. So I check the logs and indeed, the Plex server DB was corrupt. Shortly after that I checked Sonarr just to be sure, idem dito.

 

Could it be an issue with my hardware as opposed to software? In my opinion it wouldn't make sense given that 6.6.7 works totally fine and I

don't have issues with corruption on that version. My other idea would be that there is a bug in the linux kernel that is causing this

(Per info, the plugins I use: CA, Fix Common Problems, Unassigned Devices, Nerd Tools, Unraid Nvidia & User Scripts)

 

Is it perhaps better to wait for a newer version unraid? (Newer kernel?)

 

Edited by Zoremeth
Added plugins that I use.

Share this post


Link to post

Yet again my sonarr db corrupted. It’s accessed directly through /mnt/disk3

Share this post


Link to post
On 6/5/2019 at 1:28 PM, CHBMB said:

Personally I have spinning rust as my cache disk as it's used as a recording/timeshift store for my PVR and I use a SSD mounted with unassigned devices to store my docker.img and appdata and noticed a massive difference with Emby and Plex in terms of webui responsiveness, compared with when it was on the cache disk.

This might be off topic a little bit but are you suggesting Plex is faster on an unassigned SSD than on a SSD cache? I just want to make sure that I'm not missing out on something crazy, I have a RAID1 cache SSD and did notice an improvement over HDDs but if it's faster completely outside the array/cache...

Share this post


Link to post
8 minutes ago, dnLL said:

This might be off topic a little bit but are you suggesting Plex is faster on an unassigned SSD than on a SSD cache? I just want to make sure that I'm not missing out on something crazy, I have a RAID1 cache SSD and did notice an improvement over HDDs but if it's faster completely outside the array/cache...

 

No, he is comparing a spinner cache vs ssd in UD.

 

Share this post


Link to post

I would argue that a cache pool in RAID 10 is faster than a single unassigned device (I am using a 4x SSD cache pool in RAID 10).

Share this post


Link to post
3 hours ago, bonienl said:

I would argue that a cache pool in RAID 10 is faster than a single unassigned device (I am using a 4x SSD cache pool in RAID 10).

Yes and that's fine but SSDs are not slower when they're part of the cache instead of unassigned I would assume, unless BTRFS makes them slower than xfs somehow. 

Share this post


Link to post

So I have been watching this thread for a while...as I was the guy that originally had the problem.  Since I downgraded back to 6.6.7, I have had zero problems with database corruption.  I have NOT changed the data to point to a single disk, although I planning on doing it this weekend and testing.  

 

From the answers here though...that is not going to fix the issue.  The corruption is still occurring for some people.  

 

I've read through some of the other threads that are "just plex" or some other application...and people are pointing them back to the application creators for fixes.  It is NOT just happening for me with Plex, but with every application that uses the sqlite database.  

 

Like some of you, I'm questioning things in the kernel or something else that changed in 6.7.  And I'm not crazy about updating again until I see an iteration of the OS that provides some fix.  

 

Just my thoughts....and yes, I am an absolute newbie to the system.  Less than 1 year.  

 

thanks

 

rm

Share this post


Link to post

This is a major issue and seems to not be getting the attention it deserves.  I’m pretty much begging for this to get priority by the unraid devs. My 6.7.0 system is now so unreliable. Data loss or corruption needs to be a top priority. This isn’t isolated. We can point to many people who have this problem. I love unraid and this community, but right now unraid is causing me and a lot of others much pain.

 

I’m willing to donate money for a fix. 

Share this post


Link to post

I'm with rm and runraid on this one. I'm still experiencing database corruptions in Plex and restoring from backups daily, with /config on /mnt/disk1/appdata/plex like it has been suggested, on unRAID 6.7.0. This is a difficult problem to isolate, some people on the Plex forums have been discussing it with little progress, but I agree with runraid that it feels like the devs aren't giving this issue the attention it deserves. It's still happening to many people. I only started using unRAID in the past month and I'm still on the trial key. I've been meaning to finally purchase a key but I can't justify it until we figure out what's causing these database corruptions.

Share this post


Link to post

Has anyone here tried Unraid 6.7.1-rc2 release to see if the sqlite corruption issues are still there?

Share this post


Link to post
Posted (edited)

I've upgraded to 6.7.1-rc2 and restored my plex db as it just corrupted today. I'll keep you posted.

Edited by runraid

Share this post


Link to post

One thing that does stand out, it seems that everyone is changing their disks and restoring old databases, I'm curious if starting with a fresh database would fix this.

The reason I say this, is historically only people who use /mnt/user/ have ever been affected by this issue. This still seems to be the case, but for whatever reason, the issue seems to be affecting more people now, but it doesn't seem to be affecting anyone new who was already using /mnt/cache/ or /mnt/disk1/ but those that have changed from /mnt/user/ to /mnt/cache/ it doesn't seem to resolve it, so I think we need to find out if this is still the case with a fresh install.

I know that's a bit of a PITA, but it's a question worth asking.

My other thought, is I don't think it's entirely fair to dump this solely at LimeTech's feet, yes they are the distro maintainer, but that doesn't necessarily mean that every issue is their fault. This could be a weird quirk of a particular hardware combo, as it's never been possible to isolate why this has affected some, but not all, users.

Let's keep the drama down, do some troubleshooting and try and isolate the issue.

Sent from my Mi A1 using Tapatalk

Share this post


Link to post

As I said before I experienced corruption on /mnt/share when I upgraded to 6.7.0. I have changed to /mnt/disk1 with a clean install of Plex and it seems to be fixed... I'm not sure because I don't know how to test it and don't have time to test but Plex seems to be working fine. (Although it segfaulted yesterday after an update...)

Share this post


Link to post
As I said before I experienced corruption on /mnt/share when I upgraded to 6.7.0. I have changed to /mnt/disk1 with a clean install of Plex and it seems to be fixed... I'm not sure because I don't know how to test it and don't have time to test but Plex seems to be working fine. (Although it segfaulted yesterday after an update...)
That sounds promising, a clean install and so far so good.

Sent from my Mi A1 using Tapatalk

Share this post


Link to post
Posted (edited)
On 6/15/2019 at 1:10 AM, CHBMB said:
On 6/15/2019 at 1:10 AM, CHBMB said:

One thing that does stand out, it seems that everyone is changing their disks and restoring old databases, I'm curious if starting with a fresh database would fix this.

FWIW I have only been starting by deleting my corrupt sqlite database and reorganizing my library.
 

On 6/15/2019 at 1:10 AM, CHBMB said:
On 6/15/2019 at 1:10 AM, CHBMB said:

This could be a weird quirk of a particular hardware combo, as it's never been possible to isolate why this has affected some, but not all, users.

Let's keep the drama down, do some troubleshooting and try and isolate the issue.

CC:  @limetech

I am a brand new unRaid user. I started with 6.7, about a month ago. Here is what I have tried:

  1. downgrading Plex
  2. moving Plex's appdata to disk1 + disabled Tautulli
  3. upgrading Plex
  4. combining 3 movie libraries into 1.

I was able to get about a week when I did #2, but my database did corrupt after 6 or 7 days. I have been running without corruption for a few days now, but I don't believe I have solved it yet.

 

On 6/5/2019 at 12:02 PM, saarg said:

I found this from a quick Google search. It's not where I read it, but might give you a hint?

https://github.com/trapexit/mergerfs#plex-doesnt-work-with-mergerfs

Quote

Plex doesn't work with mergerfs

It does. If you're trying to put Plex's config / metadata on mergerfs you have to leave direct_io off because Plex is using sqlite which apparently needs mmap. mmap doesn't work with direct_io. To fix this place the data elsewhere or disable direct_io (with dropcacheonclose=true).

Source: https://github.com/trapexit/mergerfs#plex-doesnt-work-with-mergerfs

This sounds promising! However, I was not able to figure out if or where I could set this parameter.


Here is my hardware specifications. I bought a used Dell Precision T1700 PC and added a brand new Blu-Ray drive, two brand new 10tb shucked hard drives and one older shucked 4tb hard drive which had been in light use for a couple of years.

  • Intel Xeon E3-1226 v3 @ 3.30GHz (info)
  • 24 GiB DDR3
  • GTX 1050ti 4GB
  • 2 Western Digital 10tb drives
  • 1 Western Digital 4tb Drive
  • LG Internal Blu-Ray Drive (WH16NS40)
Edited by Brian H.
CC'ing @limetech

Share this post


Link to post

@Brian H. your experience mirrors mine. #2, moving to /mnt/disk only buys me about a week before things corrupt — and it seems all SQLite db in multiple containers corrupt at once. 

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.