Database Disk Image is Malformed

[Discordant]

Hello,

 

I've started getting this error message in both Plex and Sonarr. I've tried the database rebuild troubleshooting on Plex's own support documents to no avail. I've deleted both Plex and Sonarr containers and their app data folders and then reconfigured them from the ground up and then the issue would pop up later again that day. Is something going on with my Unraid system that is corrupting these containers? How can I fix this?

 

Current plex logs are returning: 

Error: Unable to set up server: sqlite3_statement_backend::loadOne: database disk image is malformed (N4soci10soci_errorE)

blackpearl-diagnostics-20220731-0928.zip

[BigDanT]

I have the same problem.

 

My Plex, Sonarr/Radarr & Tautulli DB's are corrupting on a regular basis (I've had to become a dab hand at repairing mysql databases.).

 

Here is a link to how to repair plex db's:- youtube repair plex  DB

 

Heres how to repair the "arrs" DBs:- https://wiki.servarr.com/useful-tools#recovering-a-corrupt-db-ui

 

if anyone at Unraid could help that would be ace.

 

I use the linux server dockers.

 

 

[grover]

Was there any update to this issue?

 

I've been running Sonarr/Radarr etc for years with no issue. Recently update unraid to 6.12 (from 6.9 I think) and now no end of issues with corrupt dbs.

 

Regretting the upgrade now.

[yuka]

I'm also having this problem.  It has been occurring for roughly 1 month - I don't know when it first happened.  It only seems to be impacting my Radarr instance - Sonarr and Lidarr are having no issues and live on the same ZFS pool.

[InhaledPlane]

On 12/23/2023 at 9:43 PM, grover said:

Was there any update to this issue?

 

I've been running Sonarr/Radarr etc for years with no issue. Recently update unraid to 6.12 (from 6.9 I think) and now no end of issues with corrupt dbs.

 

Regretting the upgrade now.

I'm in a similar boat, though the issue began prior to updating to 6.12.6. The Docker container appears to start, though the web GUI won't load and then I can see the database errors in the logs.

Is it absurd to consider starting from scratch with Sonarr?!

[Karyudo]

I have the same error (that's why I'm here), and I haven't yet taken any steps to try to fix it, but I wonder if file/folder permissions have something to do with it? On the same day I believe my Sonarr started not grabbing things and throwing this error, I had Unraid support do some maintenance, including running chmod and chown on all my files.

[JC-303]

Adding myself to the list.  Sonarr ,Radarr, Bazarr are all corrupted regularly.  I wonder if this has something to do with the daily appdata backup script shutting them down for backups? That's a complete guess, but since I have nothing else to go on, I'm going to try disabling those backups for awhile and see if the corruption stops.

[PISTOL_CUPCAKES]

I am also starting to suspect that this is caused by CA Backup doing its thing. I suspect it is shutting down the dockers in the middle of sonarr/radarr performing database tasks and that leaves the sqlite database in a corrupted state. Unfortunately every time I've brought it up or seen someone else bring it up, the devs just say "well corruption shouldn't ever happen in theory and it must be something wrong with your setup." Not sure what to do at this point since backups are necessary. Here's an issue on the radarr github where I requested a notification be sent when the database is corrupted (which apparently isn't possible). https://github.com/Radarr/Radarr/issues/9007

Edited February 12 by PISTOL_CUPCAKES
fixed some weird wording

[Zakire]

I have the same issue and have had so for a long time (not sure how long, a year maybe). I don't have any backup solution in place. Both my Sonnar and Radarr databases keeps corrupting like once a week. I repair and restart, and all is fine for about a week or maybe two.

 

The databases resides on an SSD drive configured as cache, without any data movement. What's on the cache SSD always stays there.

 

I have googled the s**t out of it and it seems like only Unraid users have those problems. 

[gladrock]

Just throwing my name in here to let people know they aren't alone - I'm seeing DB corruption on radarr and sonarr every few weeks. Not related to Appdata backup because i haven't been running it. I'm going to try moving the appdata folder for these apps from `user` to a single disk. Wondering if that might help. 

[permster]

Same issue here but only impacting sonarr, no other containers have issues.  Been running fine for a year and a half.  The issue just started a few weeks ago but has corrupted the DB multiple times since.  May be related to the below post which I also replied to earlier but still don't have a solution.  Let me know how it goes on a single disk.  I may have to go that route eventually as well if this gets much worse.

 

[Zakire]

17 minutes ago, permster said:

Same issue here but only impacting sonarr, no other containers have issues.  Been running fine for a year and a half.  The issue just started a few weeks ago but has corrupted the DB multiple times since.  May be related to the below post which I also replied to earlier but still don't have a solution.  Let me know how it goes on a single disk.  I may have to go that route eventually as well if this gets much worse.

 

 

I'm running Sonarr and Radarr on a single SSD. Although it's configured as a cache drive (without any data movement), so it might not be completely single and outside of all the Unraid stuff anyway? 

[crazydave48]

I was quite relieved to find this thread and seeing others having this issue as well. It only happens in Sonarr for me. I've tired starting from scratch, deleting all my config files and starting from a new docker image and my (brand new) DB becomes corrupted in under an hour. Not exactly sure on the start date since I hadn't been using Sonarr for the past 2 weeks but it started semi recently.

[LugoCloud]

I'm having the same issue specially with Radarr now.

Any solutions yet, I don't want to rebuilt again ;-(

 

[permster]

This issue stopped for a month or more but just happened again in Sonarr last week.  I still don't know how to fix this as putting the *arr appdata on a single disk seems silly to me.

 

When you say "rebuilt again" it sounds a bit worse than it seems.  For most people that aren't constantly changing things all you have to do is restore from the last good backup and your good to go.  Before restoring I go check to make sure there aren't any newly added series or movies since the corruption occurred.  Those need to be added back after the restore.

 

One thing that might help recovery is to change the backup frequency and retention under Settings > General > Backups which by default is only every 7 days I believe.

 

Does anyone know of a good way to monitor for this issue?  The built-in health alerts in Settings > Connect don't catch this.  I tried Notifiarr which supposedly monitors for database corruption but wasn't alerted.  If I can't find anything I'll probably just write some python code to monitor the logs.db for the issue.

Edited June 28 by permster

[LugoCloud]

That is the issue don't know when the issue  started. 😁 
I just deleted the config folder and started from scratch add the season by importing the TVshow folders and follow the prompts 

I don't see a way to change the backup frequency.
This also happened to my Plex Server and Raddar, I think  is the  Backup/Restore Appdata plugin causing the database corruption.

Wonder if there is a way to use external database fro Plex, lidarr, radarr and sonarr.  

Edited June 28 by LugoCloud

[Zakire]

I've actually managed to get rid of the corruption. Not totally sure if it was an update or something I did. Although, last time I did a repair, I decided to delete the logs.db after the repair. Haven't had a single corruption in several months now. Not in Radarr, nor in Sonarr (previously had a corruption like every week on both systems).

[permster]

I don't have a solution yet but I did add some monitoring for this issue recently.  Hope this helps others.

 

The below script leverages the User Scripts plugin so if your not familiar read up and install it first.  It also uses the sqlite3 command line tool which I think may be built into Unraid but I'm not 100% sure on that.  It was already installed for me.

 

• Set *arr backups to daily instead of weekly. (only needed so your not losing as much on restore)
  • Settings > General > Backups
  • Change the Interval to 1 days.
  • Be sure to click the 'Show Advanced' button at the top or the "Backups" section won't be visible.
• Create an Unraid User Script that checks the Sonarr logs.db file for database corruption.

  • #!/bin/bash
    
    result=$(sqlite3 /mnt/user/appdata/sonarr/logs.db "SELECT Count(*) FROM Logs WHERE Message LIKE '%database disk image is malformed%' AND Time >= datetime('now','-4 hours');")
    
    if [[ $result -gt 3 ]]
    then
        echo "Database corruption detected, sending notification"
        /usr/local/emhttp/webGui/scripts/notify -s "Sonarr corruption" -i "alert" -m "Please attempt to restore Sonarr from a known good backup! Don't forget to add any shows that were added since the backup."  -d "Sonarr database corruption detected!"
    else
        echo "No corruption detected"
    fi

     

  • With a Custom cron schedule to run every 4 hours (example below).
    • 38 */4 * * *
  • Feel free to adjust the script for you needs.
    • It is easy to adapt this to radarr by changing the path to the logs.db file and then adjusting the echo and notify strings.
    • You can change the schedule as well if you think every 4 hours is too aggressive.
      • Adjust your User Script schedule accordingly.
      • Adjust the SELECT statement in the script which is currently set to look back 4 hours.

      • datetime('now','-4 hours')

Edited July 30 by permster
Adjusted script logic for better results