September 15, 201411 yr Hi All, had a power cut last night and now I find myself a bit stuck regarding some of my docker apps. PlexMediaServer, Couchpotato and headphones show as started but they are not actually running as in the web interfaces are not available. Plex log shows this over and over again Error: Unable to set up server: sqlite3_statement_backend::prepare: unable to open database file for SQL: PRAGMA cache_size=4000 CouchPotato shows this over and over again IOError: [Errno 28] No space left on device Headphones reports over and over again IOError: [Errno 28] No space left on device from hunting around I found that there are several references to my cache drive being full so presumably the apps are crashing as they are trying to write something. I double checked this by trying to just create a folder on the cache drive and I am told that the Cache drive is full. I know that the drive is not really full as main unraid page shows its 55.3GB free, so guess its stuck in a read-only state somewhere? I have rebooted my server a couple of times and the parity check runs for about 8 hours but says its ok. how do I get my cache drive back to writeable so my apps will run? if you need more info then please let me know what and how to get it
September 15, 201411 yr I wonder if as a result of the power cut you have file system corruption on your cache drive? What format is your cache drive? You may need to run the appropriate recovery tool for the type of file system it has been formatted with. A full syslog might help show if this is likely.
September 15, 201411 yr Author My Cache drive is btrfs. actually my Plex and Couchpotato seem to have resolved themselves and at least the apps are running, still no go on writing to the cache drive though so what recovery tool is required to check for corruption on btrfs? (I will start googling now )
September 15, 201411 yr Author ran the scrub thing from the main page on the cache drive, took about 10 minutes but I don't see any errors btrfs scrub start /dev/sdc1 -B -R -d -r 2>&1 scrub device /dev/sdc1 (id 1) done scrub started at Mon Sep 15 18:03:14 2014 and finished after 902 seconds data_extents_scrubbed: 1351438 tree_extents_scrubbed: 49155 data_bytes_scrubbed: 70634344448 tree_bytes_scrubbed: 805355520 read_errors: 0 csum_errors: 0 verify_errors: 0 no_csum: 7504 csum_discards: 135803 super_errors: 0 malloc_errors: 0 uncorrectable_errors: 0 unverified_errors: 0 corrected_errors: 0 last_physical: 128035586048 any further suggestions?
September 15, 201411 yr Are there any BTRFS errors in your syslog? If so, this post may help: http://lime-technology.com/forum/index.php?topic=34975.msg325308
September 16, 201411 yr Author not been able to see any errors but just did reboot so here is a fresh syslog, hope you can find something.... syslog.txt
September 16, 201411 yr hmm interesting!, i had a similar issue with cache drive showing as full, again it had 40GB free, i couldnt start a windows kvm vm and finally located the problem to /var being full, if i did a df -h /var i could see 100% was used and this was preventing the creation of the kvm vm. just incase jonp or tom are watching this thread, i was running v6 beta9 with a btrfs formatted cache drive, i had 1 kvm vm running and 6 docker containers defined, i had the docker image file (loopback mounted image) created on the cache drive also. Markyb0y - if you see the issue again please can you issue a df -h /var and see what it shows, interested to see if our issues are the same. a reboot of course did clear the issue for me but i am concerned it may happen again, got a sneaky feeling the issue is somehow related to the loopback mounted docker image but not sure how.
September 16, 201411 yr Author I still have this issue.... df -h /var Filesystem Size Used Avail Use% Mounted on - 7.8G 416M 7.4G 6% /
September 16, 201411 yr hmmm looks like a different issue then, i see you did do a reboot so that should clear down anything on /var in any case. im assuming doing a df -h /mnt/cache shows the drive has free space, yes?.
September 16, 201411 yr Author df -h /mnt/cache Filesystem Size Used Avail Use% Mounted on /dev/sdc1 120G 40G 79G 34% /mnt/cache
September 18, 201411 yr df -h /mnt/cache Filesystem Size Used Avail Use% Mounted on /dev/sdc1 120G 40G 79G 34% /mnt/cache I'm not sure if if the concept of inodes exist on btrfs formatted volumes. But you can get the appearance of being out of disk space by running out of inodes. Check to see if you are using all available inodes on the volume "df -i"
Archived
This topic is now archived and is closed to further replies.