Jump to content
Rich Minear

Downgraded back to 6.6.7 due to Sqlite corruption

195 posts in this topic Last Reply

Recommended Posts

2 minutes ago, Abzstrak said:

yeah, just been lazy... was thinking of doing that as well as shooting out an email/sms message to my phone letting me know its corrupt. its been 10 or 11 days since my last corruption, so... motivation isn't all there either :)

 

haha i feel you - i was extremely motivated when it was dying every day but now with 2-3 weeks in between, there is not much motivation and hopefully this gets sorted out by the dev team!

Share this post


Link to post
54 minutes ago, Abzstrak said:

its been 10 or 11 days since my last corruption

Sounds like you are attending a meeting. Stay strong.

Share this post


Link to post
1 hour ago, Abzstrak said:

I just have been running this command hourly (using user scripts addon). Obviously you'll need to create the dbbackups folder.


tar zcvf /mnt/cache/appdata/PlexMediaServer/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/dbbackups/com.plexapp.plugins.library.db-$(date +%A-%H%M).tar.gz /mnt/cache/appdata/PlexMediaServer/Library/Application\ Support/Plex\ Media\ Server/Plug-in\ Support/Databases/com.plexapp.plugins.library.db

 

Thanks. I modified it for my setup (replaced cache with user) and stuck it in a bash script under /etc/chron.daily, set it to executable and ran `update_cron`. It appeared in the list of daily tasks in scheduler, so I'm hoping it will work. :)

Share this post


Link to post

Didn't took long. Database again corrupted. Unraid 6.7.1. 

I use a r710 with a H700 Raid Card, if you want to know. 

Share this post


Link to post
6 minutes ago, Noim said:

Didn't took long. Database again corrupted. Unraid 6.7.1. 

I use a r710 with a H700 Raid Card, if you want to know. 

On which disk does your database reside? How is it formatted? What is your appdata config path for the docker? What is your DirectIO setting?

Share this post


Link to post
Just now, jonathanm said:

On which disk does your database reside? How is it formatted? What is your appdata config path for the docker? What is your DirectIO setting?

Which disk: I only have one 8TB WD Drive. One you can find in the wd my book things.

Formatted: xfs

appdata config path, don't know how this helps you: /mnt/user/appdata/PlexMedia/

direct io: Default? Never changed this settings. I am not even able to find it. 

Share this post


Link to post
47 minutes ago, Noim said:

Which disk: I only have one 8TB WD Drive. One you can find in the wd my book things.

Formatted: xfs

appdata config path, don't know how this helps you: /mnt/user/appdata/PlexMedia/

direct io: Default? Never changed this settings. I am not even able to find it. 

 

You need to change it to use /mnt/diskX, where X is your disk number. This will for most user be good enough to not get a corrupted database.

Share this post


Link to post

I want to add a datapoint for @testdasi - I am -not- having the corruption issue and never did but I use basically all the same containers (plex, sonarr, deluge.) Maybe something different in my config will help isolate what's different.

 

What CPU? How much RAM? Array config?

i5-760, P55 motherboard, 8GB ram. 

Array is 3 drives, all XFS+LUKS encrypted, 8+8+4TB, + 256GB xfs cache.

in addition, one 4TB unassigned device used mostly for deluge scratch disk, the partition is btrfs+LUKS encrypted.

All five devices connected to motherboard P55 SATA controller.

 

Roughly how large is your collection? I think file count, even a rough estimate, is more important here.

movies = 158GB / 288 files

tv = 1.7 TB / 8202 files

 

Have you set your appdata to /mnt/cache (or for those without cache, /mnt/disk1)? If you haven't, we'll ignore you.

I am using /mnt/user/appdata/<app> for -all- my containers, without corruption issue. I thought about changing it when reading the first four pages but now I won't, because I want to see if I ever do get corrupted. I don't mind rebuilding my library from scratch if it happens.

 

Do you have a link between Sonarr and Plex? If yes, have you disable it? If you haven't, we'll ignore you.

No, it's not linked. I wanted to, I couldn't get it to auth properly. Sonarr is linked to Deluge though.

 

Do you have automatic library update on change / partial change? If yes, have you set it to hourly? If you haven't we'll ignore you.

No, I manually update to avoid spinning disks up needlessly.

 

----
I repeat, I do not have the issue, I just wanted to put my info in so maybe we could see if there was any significant difference between my config and other people's configs.

Share this post


Link to post
On 6/19/2019 at 1:22 PM, ProZac said:

How about the people who started to report this issue? If more people test this, maybe there is some more information to gather from this?
@Rich Minear, @Brian H., @Jackjohnsonuk, @DiRiN, @principis, @Scythe, @runraid, @Abzstrak, @Daniel Samuels, @Addy?
The more who are able to test the better :)

 

Sorry I am in the middle of exams and didn't get an email... The issue is completely solved for me. Just moved plex db from /mnt/user to /mnt/disk1.

I'm sorry but I won't do any testing. I have way too much work to do and it's really annoying. I posted diagnostics on the different thread.

Share this post


Link to post
On 6/24/2019 at 10:10 PM, saarg said:

 

You need to change it to use /mnt/diskX, where X is your disk number. This will for most user be good enough to not get a corrupted database.

This would defeat the hole purpose of using unraid. At least for me. Btw updated unraid to 6.7.2. However, the corruption happened again. 

Share this post


Link to post
44 minutes ago, Noim said:

This would defeat the hole purpose of using unraid. At least for me. Btw updated unraid to 6.7.2. However, the corruption happened again. 

How so?  As a general rule, appdata (and that's the only path mapping that has relevance here) is confined to either the cache drive (use Cache: Prefer), or a single disk for performance reasons.  In that case, there is fundamentally zero difference between referencing it as /mnt/user/appdata or as /mnt/cache/appdata

 

That being said, this issue is a priority for limetech.

Share this post


Link to post
On 7/1/2019 at 4:57 PM, Squid said:

How so?  As a general rule, appdata (and that's the only path mapping that has relevance here) is confined to either the cache drive (use Cache: Prefer), or a single disk for performance reasons.  In that case, there is fundamentally zero difference between referencing it as /mnt/user/appdata or as /mnt/cache/appdata

 

That being said, this issue is a priority for limetech.

Your tipp didn't work. Database corruption still happens. 

Share this post


Link to post
Posted (edited)

I too have experienced the corruption issue with my Plex DBs.  Sooooo, I took the first suggestion and changed my appdata location to /mnt/disk1....rebuilt everything....lasted for not even 28 hours, I was uploading data, and then, I was back to corrupted DBs.  I deleted everything related to Plex Docker, started over using binhex-PlexPass, set appdata location to /mnt/disk1....corrupted again, but lasted for 2 days this time.  Once again, I was uploading data when issue occured.  Back to the drawing board, same set up with binhex-PlexPass and location, BUT, returned my UNRAID to 6.6.7.   Everything is running stable so far now for 3 days. 

 

Here's what I found when I went to the logs to see what was causing the issue

 

First, everything was relatively stable until I started adding media content to my library

 

After a few loads, I would see that the media was not being updated into Plex even though it was on the drive and in proper Plex format

 

Next, when I went to my TV Shows, it would show zero seasons, but if you clicked on the thumbnail, to go to the individual season folders, they'd be there, go back to home and then try to get back, you couldn't!  It would show library as empty.

 

If I went to movie folder and clicked on it, I would see the movies I had without the ones I just loaded.  When I went back to home, all the sudden my movie library showed as zero.

 

In both cases, TV Shows and Movies, once they showed zero, then all the library folders showed zero content and I would get the gray screen of death.

 

Looked at log files, and sure enough, in every corruption case, it shows Sqlite errors.  I'm wondering if media uploading, metadata uploading, etc., is causing the Sqlite issues leading to the corrupt databases?

 

Seems with what I'm seeing now, defining the appdata location to cache or diskX, and going back to unraid 6.6.7 is the way to go.  I just hope this gets fixed on the Sqlite side as I'm nervouse that this could only be a band-aid if Plex alters some of their coding.  I've been doing extensive uploading of movies and TV shows and so far, databases holding up and Plex functioning normally!

 

I hope this provides some more data to help fix the issue.

 

Edited by isrdude

Share this post


Link to post

What CPU? How much RAM? Array config?
Unraid Version: 6.7.2
Model: eVGA X58 I7-950x
M/B: EVGA 121-BL-E756 Version Tylersburg
BIOS: Phoenix Technologies, LTD Version 6.00 PG. Dated: 10/26/2010 (There is a update but the update is only for support for additional CPU's, still the newest update is from 2011)
CPU: Intel® Core™ i7 @ 3066 MHz
HVM: Enabled
IOMMU: Disabled (Can not enable on this board)
Cache: 32 KiB, 32 KiB, 1024 KiB
Memory: 12 GiB DDR2 (max. installable capacity 16 GiB*)
Kernel: Linux 4.19.56-Unraid x86_64
Parity WDC_WD100EFAX-68LHPN0_JEJXE9KZ - 10 TB (sdb)
disk1 WDC_WD100EFAX-68LHPN0_JEJXE9KZ - 10 TB (sdb) formated xfs (Free 4.43 TB)
no cache

 

Roughly how large is your collection? I think file count, even a rough estimate, is more important here.
3D Movies    ~71GB (26 Files)
Anime        ~32.7GB (25 Files)
Anime Series    ~391GB (1,099 Files)
Movies        ~294GB (167 Files)
TV Shows    ~2.61TB (2,288 Files)

Ha ve you set your appdata to /mnt/cache (or for those without cache , /mnt/disk1)? If you haven't, we'll ignor e you.
Host Path 2: /mnt/disk1/transcodes/
Host Path 3: /mnt/disk1/media/
Key 4(VERSION): plexpass
Host Key5(NVIDIA_DRIVER_CAPABILITIES): all
Host Key6(NVIDIA_VISIBLE_DEVICES): GPU-131d7e98-bc16-c9a1-d5f7-2e85a1eefa1e
AppData Config Path: /mnt/disk1/appdata/PlexMediaServer/

 

Do you have a link be tween Sonarr and Plex? If yes, have you dis able it? If you haven't, we'll ignore you .
No Link

 

Do you have automatic library update on change / partial change? If yes, have you set it to hourly? If you haven't we'll ignore you.
Scan my library automatically(Checked), Run a partial scan when changes are detected (Checked), Scan every 2 hours

 

Disks are connected directly to motherboard.

 

I continuously re-scan and start over every time a corruption happens. Trakt seems to make the corruption happen faster. I have cpu pinning to only use two of my cores however once it's corrupt all my cores jump to 100 percent at times. Even with no other dockers running.

Share this post


Link to post

I do believe the discussion should now be located here in this bug report section:

 

 

Share this post


Link to post

can anyone clarify if that only happens if your appdata is not on cache (or lets say the docker is not on cache?)

Share this post


Link to post
On 10/11/2019 at 4:51 AM, nuhll said:

can anyone clarify if that only happens if your appdata is not on cache (or lets say the docker is not on cache?)

I can verify that my Plex sqlitedb has not had an issue ever since I moved it to my SSD cache drive and off of my spinning disks in the main array.  I did this before I even knew about this thread / posts on Reddit about this issue.  I rebuilt and rebuilt, but every day or two the database would become malformed.  It's been up and stable for weeks now.

 

Shame, I lost a lot of user data, and since the backups were stored on the spinning disks they, too, were toast.  My fault, I should have kept better backups, but it is what it is.

Share this post


Link to post

I've been experiencing the same issue but i only have 1 data disk.  there should be no issues with data split across multiple disks.  I've had to restore databases probably 2 dozen times over the last 3 weeks.  Running the latest unraid (6.7.2) and also the latest Deluge, Sonarr and Plex (all binhex).

What's funny is the last 3 days have been perfect.  No corruption after my last restore of sonarr database.  Fingers crossed! The only difference is that I did not re-add my wife's pictures back into Plex yet.  Those would be roughly 70,000 items totaling 400GB 

I will try moving my docker images when it fails the next time but should not make any difference since I only have 1 disk anyway...

Share this post


Link to post
5 minutes ago, fcou said:

I've been experiencing the same issue but i only have 1 data disk.  there should be no issues with data split across multiple disks.  I've had to restore databases probably 2 dozen times over the last 3 weeks.  Running the latest unraid (6.7.2) and also the latest Deluge, Sonarr and Plex (all binhex).

What's funny is the last 3 days have been perfect.  No corruption after my last restore of sonarr database.  Fingers crossed! The only difference is that I did not re-add my wife's pictures back into Plex yet.  Those would be roughly 70,000 items totaling 400GB 

I will try moving my docker images when it fails the next time but should not make any difference since I only have 1 disk anyway...

See here.

Share this post


Link to post

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.