Jump to content
Rich Minear

Downgraded back to 6.6.7 due to Sqlite corruption

191 posts in this topic Last Reply

Recommended Posts

Posted (edited)

I have been experiencing this since I upgraded to 6.7.0 (i think i posted in one of the reddit post as someone had this issue on reddit).

 

The thing that has reduce the amount of corruption for me was to turn off notification from sonarr and radarr to plex when a new episode and/or movie gets downloaded (can't remember the exact name for this).

 

It seems when plex trys to update its DB every minute or two it corrupts the db?

 

I now just have a library scan every hour and i think i only had a corruption or two in the last month since I did this. 

 

ps. I have started a fresh plex, sonarr and radarr and it still does it and i haven't changed my config to diskX as it is still in user.

pss. My radarr is completely screwed though the db is always corrupted and you need to restart the docker to fix it but it dies fairly quickly after that but i have just given up and since movies doesn't come out that often I am not to fussed.

 

Haven't look into this as my board carked it a week ago and i am currently running unRaid with a jerry rigged hardware with the bare minimum to get it going. Waiting on replacement to arrive before trying to look into this again.

Edited by RaihaX

Share this post


Link to post

I’ve set direct io to “no” yesterday. My plex just corrupted again.

I'm running a storage system that corrupts my data. It's doing exactly the thing it shouldn't do.

Share this post


Link to post
I’ve set direct io to “no” yesterday. My plex just corrupted again.
I'm running a storage system that corrupts my data. It's doing exactly the thing it shouldn't do.
Can you please shortly explain to me, how do you recognize the corrupt data?
Just to check my system as iam following this thread but hopefully will not affected in my system with this problem.

Sent from my MI 6 using Tapatalk

Share this post


Link to post
Just now, Toobie said:

Can you please shortly explain to me, how do you recognize the corrupt data?
Just to check my system as iam following this thread but hopefully will not affected in my system with this problem.

Sent from my MI 6 using Tapatalk
 

When your plex no longer work and you check the log and it says is corrupted

Share this post


Link to post
When your plex no longer work and you check the log and it says is corrupted
Ok thx.
As know my Plex is working fine.
Does it depend which Plex version/docker you may have installed?

Sent from my MI 6 using Tapatalk

Share this post


Link to post

No version doesn’t matter. It happens across various software that uses SQLite. 

Share this post


Link to post

And as a alternative to sqlite is mariadb may a better way to work with?
Could this may help?

Sent from my MI 6 using Tapatalk

Share this post


Link to post
Just now, Toobie said:

And as a alternative to sqlite is mariadb may a better way to work with?
Could this may help?

Sent from my MI 6 using Tapatalk
 

 

Don't think the developers of those application would be keen in re-developing their app in MariaDB just for unRaid!

Share this post


Link to post
5 hours ago, runraid said:

I’ve set direct io to “no” yesterday. My plex just corrupted again.

I'm running a storage system that corrupts my data. It's doing exactly the thing it shouldn't do.

Well, there goes that theory. You are right, a storage system shouldn't corrupt data.

 

However, it's not the software primarily, it's some interaction between the software and a subset of hardware, or some combination of software and settings, or hardware and settings, that is the issue. Not everyone has the problem, so it's difficult to pin down what exactly is the issue.

 

Earlier in the thread the developers asked for a way to duplicate the issue, but what was put forward doesn't duplicate it on all systems, so isn't especially helpful. I personally have 4 running unraid servers, all on 6.7.0, all with different hardware configurations, none of them have the issue.

 

Without a way to trigger the issue predictably, isolating the cause is very difficult.

Share this post


Link to post

it's been 5 days here since I went cache only with appdata and set it direct io to "no"... I keep randomly checking the consistency of the database, it's still coming back "ok".   Cautiously optimistic.  I've added a number of videos and recorded some via dvr, no problems as of yet.

I've been running hourly backups due to this issue... I'm on the verge of writing a script to verify the database and email me if it corrupts, but I've been feeling lazy.  Shouldn't be too hard though to do hourly backups, verify consistency and if it goes bad then email the admin with the name of the last good backup from an hour prior.  Auto restoring it wouldn't be that hard either, would just be a trick to make sure that plex wasn't doing something at the time(like recording something on the dvr).

Share this post


Link to post

Another person hit by this. The actual impact must be fairly substantial given the number of impacted people I’ve seen complain given that most people won’t comment on forums or Reddit. 

 

 

Share this post


Link to post
Posted (edited)

Am I the only one that have "solved/mitigated" the issue so far by disabling the connection between sonarr/radarr to plex and just doing a manual hourly scan for new files.

 

Have anyone tried the above and have eliminated or reduced the occurrence of corruption? It has been almost 3 weeks since my last corruption now.

 

I also figured out my radarr db corruption completely unrelated i had to dump it and remove a few values that was causing an issue and it is working fine after recompiling it.

Edited by RaihaX

Share this post


Link to post
8 hours ago, RaihaX said:

Am I the only one that have "solved/mitigated" the issue so far by disabling the connection between sonarr/radarr to plex and just doing a manual hourly scan for new files.

Looks like you are the only one based on previous posts.

 

Share this post


Link to post
9 hours ago, RaihaX said:

Am I the only one that have "solved/mitigated" the issue so far by disabling the connection between sonarr/radarr to plex and just doing a manual hourly scan for new files.

 

Have anyone tried the above and have eliminated or reduced the occurrence of corruption? It has been almost 3 weeks since my last corruption now.

 

I also figured out my radarr db corruption completely unrelated i had to dump it and remove a few values that was causing an issue and it is working fine after recompiling it.

How about the people who started to report this issue? If more people test this, maybe there is some more information to gather from this?
@Rich Minear, @Brian H., @Jackjohnsonuk, @DiRiN, @principis, @Scythe, @runraid, @Abzstrak, @Daniel Samuels, @Addy?
The more who are able to test the better :)

 

Share this post


Link to post

I am in the same boat with this.

Just setup a brand new server.... only have Plex on it. Last night I test loaded a few videos and pictures... then I got the dreaded "There was an unexpected error loading this library" error.

Today I removed Plex and manually deleted files on the server.... reinstalled and Ok so far.

 

On top of this I am Having some other weird issues where server becomes unresponsive....  I am using old hardware though... until Ryzen 3 comes to the market.

 

If you need any testing... I can test whatever is needed for a few days until vacay.

Share this post


Link to post
19 hours ago, ProZac said:

How about the people who started to report this issue? If more people test this, maybe there is some more information to gather from this?
@Rich Minear, @Brian H., @Jackjohnsonuk, @DiRiN, @principis, @Scythe, @runraid, @Abzstrak, @Daniel Samuels, @Addy?
The more who are able to test the better :)

 

I can give this a go - but I'm unsure why this would fix the issue - there is no real tether between sonarr and radarr as far as I'm aware? my plex is setup to scan disks when changes are found which is the only connection realistically? 

 

(sonarr puts a file somewhere plex can see it)

 

The backend of my plex just died with error

Thread: Uncaught exception running async task which was spawned by thread 0x14582b083700: sqlite3_statement_backend::loadOne: database disk image is malformed

yet another backup restore for me, its getting a bit out of hand. I don't want to downgrade, I'd prefer to help but we don't seem to be making much process and the dev's are no where to be seen, have they had any progress finding issues? I'd gladly upload anything they require

 

Addy 

Share this post


Link to post
Posted (edited)
8 minutes ago, Addy said:

I can give this a go - but I'm unsure why this would fix the issue - there is no real tether between sonarr and radarr as far as I'm aware? my plex is setup to scan disks when changes are found which is the only connection realistically? 

 

(sonarr puts a file somewhere plex can see it)

 

The backend of my plex just died with error


Thread: Uncaught exception running async task which was spawned by thread 0x14582b083700: sqlite3_statement_backend::loadOne: database disk image is malformed

yet another backup restore for me, its getting a bit out of hand. I don't want to downgrade, I'd prefer to help but we don't seem to be making much process and the dev's are no where to be seen, have they had any progress finding issues? I'd gladly upload anything they require

 

Addy 

I wouldn't say this fix this issue but it mitigate/reduce the corruption as mine just corrupted again last night (but its been 3 weeks since the last one and the previous corruption before then was about 2-3 weeks too). From what I can tell you guys are experiencing corruption a lot more frequent?

 

From what I can tell it seems that it only gets corrupted when plex does too many queries/update/insert in a short span of time. By only doing hourly scan it seems to be less prone to corruption.

 

I only notice this when I was redownloading all my files to 265 and I would get corruption every 30-60 min which lead me to the above and it seems to work.

 

So obviously there is a bigger underlying issue that still need to be fixed but at least now it has become less annoying...

 

Hopefully one the dev can figure it out.

 

ps. I have given up in using backup - now i just dump the db and rebuild the db from the dump. A lot easier to restore it and I can probably make a script to trigger this but i haven't been bothered as it doesn't happen as often.

Edited by RaihaX

Share this post


Link to post
Posted (edited)
8 minutes ago, RaihaX said:

I wouldn't say this fix this issue but it mitigate/reduce the corruption as mine just corrupted again last night (but its been 3 weeks since the last one and the previous corruption before then was about 2-3 weeks too). From what I can tell you guys are experiencing corruption a lot more frequent?

 

From what I can tell it seems that it only gets corrupted when plex does too many queries/update/insert in a short span of time. By only doing hourly scan it seems to be less prone to corruption.

 

I only notice this when I was converting all my files to 265 and I would get corruption every 30-60 min which lead me to the above and it seems to work.

 

So obviously there is a bigger underlying issue that still need to be fixed but at least now it has become less annoying...

I think I moved my plex to /mnt/disk1/appdata on the 17th so around 3 days for me

 

Yeah I see where you are coming from, by making sqlite scan library's less frequently we therefor reduce the chance of corruption, I say chance because we seem to think its happening randomly?

 

I added a movie to plex yesterday, was prob what caused the corruption - not many tv shows coming out and its the only thing ive added within those three days I think

 

fyi for others

someone pm'd me and asked where my plex backups where 

they here \Plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases 

 

Maybe its because the mapped locations of our server are on a share somewhere or another, what if I made a directory outside of shares?

Edited by Addy

Share this post


Link to post
Posted (edited)
6 minutes ago, Addy said:

I think I moved my plex to /mnt/disk1/appdata on the 17th so around 3 days for me

 

Yeah I see where you are coming from, by making sqlite scan library's less frequently we therefor reduce the chance of corruption, I say chance because we seem to think its happening randomly?

 

I added a movie to plex yesterday, was prob what caused the corruption - not many tv shows coming out and its the only thing ive added within those three days I think

 

fyi for others

someone pm'd me and asked where my plex backups where 

they here \Plex\Library\Application Support\Plex Media Server\Plug-in Support\Databases 

 

Maybe its because the mapped locations of our server are on a share somewhere or another, what if I made a directory outside of shares?

Yes I still don't know exactly what corrupts it as obviously my plex will scan new files just fine (i do about 3-5 new episodes every day for the shows that sonarr track and others on top so is fairly regular for me) and is only yesterday after 3 weeks that it occurred.

 

This show with regards to probability in my case that chance of corruption is fairly low.

Edited by RaihaX

Share this post


Link to post
2 minutes ago, RaihaX said:

Yes I still don't know exactly what corrupts it as obviously my plex will scan new files just fine (i do about 3-5 new episodes every day) and is only yesterday after 3 weeks that it occurred.

do you have an actual cache drive? 

 

I brought a ssd the other day and am going to put it in my tower, wondering if that will help

Share this post


Link to post
Just now, Addy said:

do you have an actual cache drive? 

 

I brought a ssd the other day and am going to put it in my tower, wondering if that will help

No cache drive and my appdata is still using "user" instead of "diskX"

 

Haven't bothered changing the above as it seems people are still experiencing issue anyway.

 

I believe from reading this thread even people with cache drive is also experiencing the same issue?

Share this post


Link to post
3 minutes ago, RaihaX said:

No cache drive and my appdata is still using "user" instead of "diskX"

 

Haven't bothered changing the above as it seems people are still experiencing issue anyway.

 

I believe from reading this thread even people with cache drive is also experiencing the same issue?

@Abzstrak you stated you added a cache drive and changed the system to cache only, have you had any issues since?

Share this post


Link to post
No cache drive and my appdata is still using "user" instead of "diskX"
 
Haven't bothered changing the above as it seems people are still experiencing issue anyway.
 
I believe from reading this thread even people with cache drive is also experiencing the same issue?
Not for me at the moment.
Using a 120gb cache ssd, but iam also not using sonarr or radarr or something like this.

Sent from my MI 6 using Tapatalk

Share this post


Link to post

Well I would be happy to get a cache drive if it resolve the issue but waiting on more confirmation. Also unRaid should probably specify that cache drive is a necessity and not optional if that is the case.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.