Jump to content
Rich Minear

Downgraded back to 6.6.7 due to Sqlite corruption

191 posts in this topic Last Reply

Recommended Posts

After finding corruption every few days in my docker containers (Plex, Sonarr, and SABnzbd), I finally bit the bullet and downgraded back to 6.6.7.  

 

I had been doing google searches for issues with 6.7 and Sqlite, and I ran across one that stated that multiple people were having the issue.  Downgrading removed the issue.  But it's weird that a search here in the forums did not find anyone really talking about it.

 

I cannot be the only one having issues....

 

I guess I will wait for the first set of updates to come out for 6.7.  

Share this post


Link to post
12 hours ago, Rich Minear said:

I had been doing google searches for issues with 6.7 and Sqlite, and I ran across one that stated that multiple people were having the issue.

Not correct

 

 

Your issue seems to be one that affects some users (definitely not all) where some applications do not like the reference to /config (edit the container(s) and then click show more settings) and switch /mnt/user/appdata to instead be /mnt/cache/appdata

 

If this is the case, then you will run into this problem on any version of unRaid.

Share this post


Link to post

So I looked at my configuration for the dockers that I have.  All of them are set to /mnt/user/appdata.  And I do not have a "/mnt/cache" at all.  

The dockers have been running stable for almost a year.  Moving to 6.7 is when I starting seeing the corruption.  Moving back....no corruption at all.  

 

I'm willing to try other things....but so far, the system has been extremely stable since downgrading.  Back to what I am used to.  

Share this post


Link to post

I have been struggling with Plex for a week (I just installed unRaid for the first time as well). If there is a problem with 'appdata', I wonder if the problem could be that the appdata folder could be split across multiple drives? I'm going to rebuild the database after I set appdata's "limit" to disk 1.

Share this post


Link to post
46 minutes ago, Brian H. said:

I wonder if the problem could be that the appdata folder could be split across multiple drives? I'm going to rebuild the database after I set appdata's "limit" to disk 1.

Before doing that, change the reference to /config on Plex to be /mnt/disk1/appdata/....

Share this post


Link to post
Posted (edited)

Edit: Oops, Ignore the below statement. My ignorance was showing. I made the wrong assumption that downgrading Plex was the solution. I now realize the suggestion was to downgrade unRaid.

 

Quote

I discovered multiple recent discussions on the plex.tv forums about database corruption after a recent update. 

The user "mongo75" in this thread says after downgrading one release he has been 6 days corruption free. So I that is the avenue I am going to pursue.

Since I am currently using the binhex/arch-plexpass template, I checked the tags on his docker hub page. Since the most recent release was from 11 days ago, I see the previous release is tagged "1.15.5.994-1-01". So I downgraded by editing the "Repository" field in the docker template to read "binhex/arch-plexpass:1.15.5.994-1-01" (a colon is required to separate the version tag from the repo name).

 

2072430027_ScreenShot2019-05-28at12_59_50PM.thumb.png.c35122d9c1133118c5352b0534860696.png

Edited by Brian H.
I made an incorrect assumption. + Deleting associated image.

Share this post


Link to post

I'm in the same boat - nothing has changed from a working environment other than an upgrade of unraid to 6.7.

 

I tried changing  /mnt/user/appdata to /mnt/cache/appdata and everything went haywire - for some reason plex losses my library files and is treating the change of director as if it can't read my old appdata file. now switching it back has lost my old server name and set up another server.  I'm sure that's reversible but it does seem to be a 6.7 issue rather than a random issue for some users; either that or the number of users it effects has grown?

 

 

Share this post


Link to post
1 hour ago, Jackjohnsonuk said:

I tried changing  /mnt/user/appdata to /mnt/cache/appdata and everything went haywire - for some reason plex losses my library files and is treating the change of director as if it can't read my old appdata file. now switching it back has lost my old server name and set up another server.  I'm sure that's reversible but it does seem to be a 6.7 issue rather than a random issue for some users; either that or the number of users it effects has grown?

This suggests that not all of the appdata share is on the cache.   With the settings at /mnt/user/appdata then files in the appdata share are found regardless of whether they are on the array or the cache.   Setting it to /mnt/cache/appdata means that any files belonging to the appdata share are only seen if they are physically on the cache, and not seen if they are on an array drive.

Share this post


Link to post
Posted (edited)

After being fed up with rebuilding my entire build almost daily, I decided to see wtf is going on with 6.7.0 Found this page. I'm going to do what was suggested above as I do not run a Cache Disk (Took it out thinking it had something to do with the corruption.) I'm going to change all my appdata to /mnt/disk1/appdata/ and see if that fixes the issue.

--------------------

24 Hour Update: Running as expected so far. No issues as of yet. Just queued up 1TB between Radarr and Sonarr. We'll see what happens after it's all finished and housekeeping kicks in.

---------------------

29 Hour Update: RIP - Task Error: database disk image is malformed.

---------------------

I stopped the Dockers having corruption issues (Radarr / Sonarr) and then deleted the logs.db file in appdata and then restarted the dockers. Seems to be working again after letting it go through its housekeeping, it picks back up where it left off before the corruption. Not sure what's causing it though?

Edited by DiRiN

Share this post


Link to post

Same issue...

 

After upgrade to 6.7.0 plex db suddenly corrupted. Reinstalled plex completely, same issue after a day. Downgrading to 6.6.7 fixed it. (Appdata is on /mnt/user)

 

I thought something had to be broken with the filesystem (XFS) but checked and it was fine. But parity check found ~300 errors. Never had this before. There is something really broken with 6.7.0 and I am not going to upgrade again.

Share this post


Link to post
5 hours ago, principis said:

Same issue...

 

After upgrade to 6.7.0 plex db suddenly corrupted. Reinstalled plex completely, same issue after a day. Downgrading to 6.6.7 fixed it. (Appdata is on /mnt/user)

 

I thought something had to be broken with the filesystem (XFS) but checked and it was fine. But parity check found ~300 errors. Never had this before. There is something really broken with 6.7.0 and I am not going to upgrade again.

 

So you did not try to use /mnt/cache or /mnt/diskX?

That is the only solution to not have this issue.

Share this post


Link to post
2 hours ago, saarg said:

 

So you did not try to use /mnt/cache or /mnt/diskX?

That is the only solution to not have this issue.

I do not have a cache disk so no. And it worked fine for years so no I haven't tested it. Shouldn't this be fixed instead of a workaround? I may test it with /mnt/diskX when I have time.

Share this post


Link to post

I'll throw my voice into the ring as well. I had plex (linuxserver) working great on unraid for about 9months prior to upgrading to unraid 6.7.0 and since then have been getting the same SQLite corruption issues mentioned in this thread.

I don't have a cache drive
Config Path set to /mnt/user/appdata/plex

Share this post


Link to post
14 hours ago, principis said:

I do not have a cache disk so no. And it worked fine for years so no I haven't tested it. Shouldn't this be fixed instead of a workaround? I may test it with /mnt/diskX when I have time.

The issue is not only with unraid, but with mergerfs also. If there is anyone that should fix it, it's sqlite. If they even see it a an issue.

Share this post


Link to post

I actually thought there was already a post some years ago telling people to use /mnt/cache instead of /mnt/user for docker config because of various issues. It was long enough ago that I only change my config to /mnt/cache out of habit when dealing with dockers without knowing why.

Share this post


Link to post
6 hours ago, saarg said:

The issue is not only with unraid, but with mergerfs also. If there is anyone that should fix it, it's sqlite. If they even see it a an issue.

Yes probably. I'm not blaming unraid, I just think they should "fix" (for unraid) it by downgrading the broken dependency if possible or put a warning in the release notes or something. I don't have the time to find the exact problem.

 

6 minutes ago, testdasi said:

I actually thought there was already a post some years ago telling people to use /mnt/cache instead of /mnt/user for docker config because of various issues. It was long enough ago that I only change my config to /mnt/cache out of habit when dealing with dockers without knowing why.

If you do not have a cache drive this is a terrible idea... I was told to put it on /mnt/user so I did. I still find it strange that it causes problems.

Share this post


Link to post
9 minutes ago, principis said:

Yes probably. I'm not blaming unraid, I just think they should "fix" (for unraid) it by downgrading the broken dependency if possible or put a warning in the release notes or something. I don't have the time to find the exact problem.

 

If you do not have a cache drive this is a terrible idea... I was told to put it on /mnt/user so I did. I still find it strange that it causes problems.

You don't have to find the problem, we've told you how to fix it.  Regardless of whether you have a cache drive or not, ideally your appdat should be confined to a single disk (prevents keeping the whole array span up all the while) If that is disk1 disk2 or disk9 it doesn't matter.

 

Use /mnt/disk1/appdata

 

Same end result.  Fixes the issue.

Share this post


Link to post
1 hour ago, principis said:

If you do not have a cache drive this is a terrible idea... I was told to put it on /mnt/user so I did. I still find it strange that it causes problems.

You misunderstood my point. It's about making sure to put appdata on a single disk (i.e. see the solution CHBMB proposed above).

Share this post


Link to post
Posted (edited)

I've tried the work around / fix. It did not fix the issue. So it's something bigger. I've rebuilt my entire server almost every day for 2 weeks straight with all possible solutions I could think of. Rolling back to 6.6.7 fixes all of the issues.

The only "fixes" that worked while on 6.7 was for Sonarr/Radarr/Lidarr was to go to appdata and delete log.db and restart the dockers and let them cycle through housekeeping a few times. Plex I had to delete the com.plexapp.plugins.library.db from the plex databases appdata and then readd my libraries and let it rescan and add all the metadata over again. Until you download or add something to the library, in which case it'll die again. For sab or nzbget I'd just delete the appdata folder all together and redo the entire thing. Forcing an update would sometimes fix it but rebuilding it would work for sure for at least another day.

Edited by DiRiN

Share this post


Link to post

Here's a reddit thread on this. There's numerous people suffering from SQLite DB corruption since upgrading to 6.7.0, myself included. 

For me, Plex, Sonarr, Radarr, and Tautulli all had corrupted databases multiple times. Moving to /mnt/disk1 does solve the issue. I never had this problem prior to 6.7.0. It wasn't until I upgraded until this happened.

 

 

Share this post


Link to post

Guys, some of you are being a little unfriendly. Someone started this thread with a problem, the second post starts with "Not correct."

 

Not cool.

 

It seems that after upgrading to 6.7.0, more people are affected by corrupt sqlite db's than before on 6.7.0. There is a correlation.

 

Besides, if making the appdata mount available on just one disk only is the only solution to this problem, why is that not default, or even documented? Someone said it even solves a problem that all disks are always spun up.

 

Share this post


Link to post
Guys, some of you are being a little unfriendly. Someone started this thread with a problem, the second post starts with "Not correct."
 
Not cool.
 
It seems that after upgrading to 6.7.0, more people are affected by corrupt sqlite db's than before on 6.7.0. There is a correlation.
 
Besides, if making the appdata mount available on just one disk only is the only solution to this problem, why is that not default, or even documented? Someone said it even solves a problem that all disks are always spun up.
 
Sqlite database corruption has been an issue for SOME people using /mnt/user/appdata/ for a long time, however nobody has been able to isolate why it affects some users and not others.

If you read what I wrote I didn't say using /mnt/disk1/appdata would fix an array being spun up all the time, but it stands to reason if you have a load of docker containers metadata then whilst those containers are running, applications will be accessing it, now if that data is spread across various array drives and not confined to a single disk, it MAY, keep drives spun up unnecessarily.

Regardless using your protected array for your appdata isn't a great idea due to the performance hit on writing to the protected array, which is why I would always recommend using at least a cache disk or, even better, an SSD for this.

Personally I have spinning rust as my cache disk as it's used as a recording/timeshift store for my PVR and I use a SSD mounted with unassigned devices to store my docker.img and appdata and noticed a massive difference with Emby and Plex in terms of webui responsiveness, compared with when it was on the cache disk.

The downside to running everything on cache, or an unassigned device is of course no parity protection, however as parity is not a backup strategy, everyone should have mechanisms in place to ensure their appdata is regularly backed up so in the event of issues they have a recent source to restore from.

I personally use CA backup/restore on a weekly basis for this, copying my backups to my protected array and then I rclone it to a cloud provider.

Sent from my Mi A1 using Tapatalk

Share this post


Link to post

We have been looking into this for the past few days.  If you have a way to make this happen on demand, please post how you do it.

Share this post


Link to post
1 hour ago, CSmits said:

Guys, some of you are being a little unfriendly. Someone started this thread with a problem, the second post starts with "Not correct."

 

Not cool. 

Pretty sure unfriendliness was not intended.

 

1 hour ago, CSmits said:

It seems that after upgrading to 6.7.0, more people are affected by corrupt sqlite db's than before on 6.7.0. There is a correlation.

Would seem so.  Many many changes in all kinds of subsystems between 6.6 and 6.7, going to take time to isolate.

 

1 hour ago, CSmits said:

Besides, if making the appdata mount available on just one disk only is the only solution to this problem, why is that not default, or even documented?

If you read the entire thread, some are reporting this does not completely solve the problem.

Share this post


Link to post
12 hours ago, saarg said:

The issue is not only with unraid, but with mergerfs also. If there is anyone that should fix it, it's sqlite. If they even see it a an issue.

Can you provide link to that discussion?

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.