Everything posted by kamhighway
-
Cleaning out Docker image
@ikosa I used cadvisor as well. Like you, the size of my containers never changes. I conclude that either cadvisor is reporting to us the size of the images -- which AFAIK should not change -- or there is something in docker.img besides the containers that is growing. AFAIK Lime has not said whether there is anything else in docker.img. Perhaps Lime could say whether there is supposed to be anything else in docker.img.
-
Cleaning out Docker image
@jimbobulator If this turns out to be the problem, I think you could redirect /transcode to your cache drive. My theory is if you don't redirect /transcode to someplace outside of the container, then the container starts to use up space on docker.img. From JonP's post there are two places you need to change to make the redirect. First place is in the docker settings page. The second change is made using plex's web interface to change its server settings to use /transcode.
-
Cleaning out Docker image
I think the problem may be with Plex dockers. JonP wrote a post about moving the transcode directory to a ram: http://lime-technology.com/forum/index.php?topic=37553.0 JonP's post got me thinking about where my plex docker stores its transcode files if I don't point /transcode to ram as JonP suggests. If they are stored in docker.img, it would explain why utilization of docker.img can jump by a gigabyte a day sometimes, and go for long periods growing much slower. Utilization will jump whenever someone is watching something that plex needs to transcode. I'm going to follow JonP's advice and have /transcode point to /tmp and see if that stops the increasing utilization of docker.img. To the others dealing with the same issue: 1) Are you running a plex docker? 2) If yes, do you let plex transcode your media, or does plex always just play the file directly.
-
Cleaning out Docker image
@B_Sinn3d By my count, there are at least 5 of us that have this problem. LT and the docker authors don't seem to have this issue so there must be a solution. For now, I've given my docker.img 50GB and I still have to recreate it roughly once a week. I'm looking at moving the dockers off my unraid server until a solution is found.
-
Cleaning out Docker image
@ikosa, Backups could be another source of utilization. Sonarr does schedule automatic backups, but I don't know where they go. The backup directory should also probably be mapped to a directory on the cache drive.
-
Cleaning out Docker image
@ikosa, This problem would be the same with any container that generates log files. I don't know of an easy way to look inside the running containers to see which ones are growing inside of docker.img.
-
Cleaning out Docker image
@jimbobulator Here are the docker images on my machine: REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE binhex/arch-couchpotato latest 7de43ac6e0dc 10 days ago 628.6 MB binhex/arch-sonarr latest 14b30ef80694 5 weeks ago 905.4 MB binhex/arch-moviegrabber latest f333212ec60a 6 weeks ago 518.2 MB google/cadvisor latest 175221acbf89 11 weeks ago 19.92 MB gfjardim/logitechmediaserver latest 465b1e79d3c8 4 months ago 615.9 MB needo/plex latest 8906416ebf13 4 months ago 603.9 MB gfjardim/btsync latest 69d6ec367640 5 months ago 297.7 MB needo/mariadb latest 566c91aa7b1e 14 months ago 590.6 MB I note that we have several in common: binhex/arch-couchpotato, binhex/arch-sonarr, and needo/plex. This provided me the clue to focus on these three containers. I've noticed that Sonarr seems to have a problem downloading torrents lately and the timing coincides with when I got the first message regarding utilization of the docker.img file. Could be that sonarr is writing error messages related to failed torrents into a logfile that is continually growing inside of docker.img. This could explain why sometimes the utilization jumps quickly, and sometimes it grows relatively slowly -- on days that sonarr tries to download a lot of torrents, it would generate a lot more error log entries than on days when there is nothing on sonarr's calendar to download. This would also explain why the problem only surfaced last week. Since we never commit the docker image with the error logs, the size of the image as reported by the command docker images, never changes. One way to fix this is to find the directory the sonarr container uses to write its log files and map that to a directory on the cache drive. I don't know enough about docker and sonarr to find the right directory, but maybe binhex could help us here. It may be that a container's directories for log files should always be mapped to a directory on the cache drive to prevent them from filling up docker.img.
-
Cleaning out Docker image
Running UR 6.1.0. I got the warning message "Docker high image disk utilization" this morning. First warning said utilization was 72%. Then about an hour later got a second warning saying utilization is 73%. From settings->docker i see this: Label: none uuid: 1c150737-1e3b-4d8a-a83c-81e40bf7507f Total devices 1 FS bytes used 16.49GiB devid 1 size 25.00GiB used 19.04GiB path /dev/loop0 btrfs-progs v4.1.2 From cAdvisor i see this: google/cadvisor latest 175221acbf890310cc61dc3d 19.00 MiB 7/1/2015, 5:06:45 PM gfjardim/btsync latest 69d6ec3676409cd60299b773 283.96 MiB 3/27/2015, 5:36:30 AM binhex/arch-moviegrabber latest f333212ec60ad6a58ab45984 494.22 MiB 8/7/2015, 9:14:54 AM needo/mariadb latest 566c91aa7b1e209ddd41e5b0 563.20 MiB 7/11/2014, 4:53:52 AM needo/plex latest 8906416ebf13bada755e356a 575.99 MiB 5/1/2015, 6:24:20 AM gfjardim/logitechmediaserver latest 465b1e79d3c88e69ab4c7cda 591.59 MiB 5/18/2015, 8:57:01 AM binhex/arch-couchpotato latest 7de43ac6e0dc047fbbccc125 599.44 MiB 9/9/2015, 7:24:48 AM binhex/arch-sonarr latest 14b30ef806943549665a558f 863.42 MiB 8/10/2015, 4:12:11 AM The only thing I can think of to do is wait a few hours and then check cAdvisor again to see which container has increased in size. Can anyone offer any better suggestions for diagnosing the problem? Update: I just got another warning saying that utilization is now 74%. So in 90 minutes utilization of the docker.img file increased by 1%. Looking at data for cAdvisor shows that all of the containers are exactly the same size as before. google/cadvisor latest 175221acbf890310cc61dc3d 19.00 MiB 7/1/2015, 5:06:45 PM gfjardim/btsync latest 69d6ec3676409cd60299b773 283.96 MiB 3/27/2015, 5:36:30 AM binhex/arch-moviegrabber latest f333212ec60ad6a58ab45984 494.22 MiB 8/7/2015, 9:14:54 AM needo/mariadb latest 566c91aa7b1e209ddd41e5b0 563.20 MiB 7/11/2014, 4:53:52 AM needo/plex latest 8906416ebf13bada755e356a 575.99 MiB 5/1/2015, 6:24:20 AM gfjardim/logitechmediaserver latest 465b1e79d3c88e69ab4c7cda 591.59 MiB 5/18/2015, 8:57:01 AM binhex/arch-couchpotato latest 7de43ac6e0dc047fbbccc125 599.44 MiB 9/9/2015, 7:24:48 AM binhex/arch-sonarr latest 14b30ef806943549665a558f 863.42 MiB 8/10/2015, 4:12:11 AM Settings->docker now shows Label: none uuid: 1c150737-1e3b-4d8a-a83c-81e40bf7507f Total devices 1 FS bytes used 17.00GiB devid 1 size 25.00GiB used 19.04GiB path /dev/loop0 btrfs-progs v4.1.2 Can I conclude that the containers are not growing? Is there anything else in docker.img that could be growing? Udpate 2: Its been about 3 hours and now utilization is at 76%. According to cAdvisor there has been no change is size for any of the containers. However settings->docker now shows this: Label: none uuid: 1c150737-1e3b-4d8a-a83c-81e40bf7507f Total devices 1 FS bytes used 17.42GiB devid 1 size 25.00GiB used 20.04GiB path /dev/loop0 btrfs-progs v4.1.2 Update 3: Updated to UR6.1.2. docker.img utilization continued to increase. When utilization hit 80%, I deleted docker.img and rebuilt it following directions in the sticky post in this forum. After rebuilding settings->docker now show this: Label: none uuid: 1c009fb0-4cb4-4574-8ee3-3a08847d4754 Total devices 1 FS bytes used 3.28GiB devid 1 size 25.00GiB used 6.04GiB path /dev/loop0 btrfs-progs v4.1.2 Although docker.img is now much less fully utilized, it is still growing so I don't think this solved the problem. Update 4: Left things alone overnight. In the morning settings->docker showed: Total devices 1 FS bytes used 5.48GiB devid 1 size 25.00GiB used 8.04GiB path /dev/loop0 cAdvisor says each container is the same size as it was yesterday. Don't know what else could be growing. I have now stopped each of the containers to see if utilization continues to rise even with all containers stopped. Update 5: With all containers stopped, there has been no change in utilization of docker.img for the past 2 hours. Going to start one container at a time starting with needo:Plex.
-
Preclear plugin
Very nice! Thank you for making this available to the community.
-
Docker requests
Google Drive?
-
Wake on Lan plugin for unRAID 6.1
This is very nice. Thank you!
-
unRAID 6 NerdPack - CLI tools (iftop, iotop, screen, kbd, etc.)
Is there a link to get this plugin?
-
VirtualBox Plugin for unRAID v5 and v6
Thank you for pointing me in this direction. I still can't get UltraVNC working, so the VRDP console in VBOX is a good solution. The cursor seems to really lag through. I must not have the guest additions installed properly. I see the guest addition iso on /boot/config/plugins/virtualbox, but there must be another step to install it. Update: Got guest additions installed now and that has resolved the lagging cursor problem.
-
VirtualBox Plugin for unRAID v5 and v6
First, I want to thank the author of this excellent plugin. I found it pretty easy to get a Windows 7 VM running within VirtualBox on my unraid server. I can use Remote Desktop or Teamviewer to connect to the VM, but I can't figure out how to connect with UltraVNC. If anyone is using UltraVNC to connect to their Virtualbox Windows VM, I'd be interested in learning how you made it work.
-
Shell script to perform backups using rsync
Rsync does not seem to like something about some mac application files. I have an unraid share for the installers for all the mac applications I use. When I try to rsync the share I get this: root@Vault:~# rsync -avP rsync://Tower/mnt/user/Files/Software-Mac/ /mnt/user/Files/Software-Mac | grep fail rsync: failed to set times on "/mnt/user/Files/Software-Mac/Coda/Coda 2.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr_CA.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/Coda/Coda 2.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/pt.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/FX Photo Studio PRO.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr_CA.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/FX Photo Studio PRO.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/Google Drive.app/Contents/Resources/lib/python2.6/site.py": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/MDRP.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/MDRP.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr_CA.lproj": Too many levels of symbolic links (40) rsync: failed to set times on "/mnt/user/Files/Software-Mac/MacDVD Ripper Pro/MDRP.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/MacDVD Ripper Pro/MDRP.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr_CA.lproj": Too many levels of symbolic links (40) rsync: failed to set times on "/mnt/user/Files/Software-Mac/TwistedWave/TwistedWave.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr_CA.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/TwistedWave/TwistedWave.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/Videobox/Videobox.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr_CA.lproj": No such file or directory (2) rsync: failed to set times on "/mnt/user/Files/Software-Mac/Videobox/Videobox.app/Contents/Frameworks/Sparkle.framework/Versions/A/Resources/fr.lproj/fr.lproj": No such file or directory (2) rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1508) [generator=3.0.7] Has anyone else seen this?
-
Shell script to perform backups using rsync
Thanks trurl. I did not realize I needed to install the simplefeatures email notify plugin.
-
Shell script to perform backups using rsync
This script works great between two unraid servers running rc11. The only thing I can't figure out is how to get the emails this script sends out. From where does it get the email address to send the messages to?
-
Shell script to perform backups using rsync
overbyrn, I would like to see the script you referred to in the above message. Would you mind making it available again through your dropbox? thanks, ken