GTP Posted September 18, 2020 Author Share Posted September 18, 2020 (edited) 49 minutes ago, trurl said: What are your plans for that disk, a new data slot? appdata, domains, system shares have files on the array. This will impact performance of dockers / VMs due to slower parity, and will keep array disks spunup since dockers / VMs will have open files on these. Your docker.img has 150G allocated, and is using 26G of that. I normally recommend only 20G as being more than enough. I have 17 dockers and they are using less than half of 20G. Are you sure you don't have some app misconfigured so it is writing to a path that isn't mapped? Common mistakes are specifying paths in the app using different upper/lower case than mapped container paths. Which dockers do you run? Do you have custom docker networks such as with letsencrypt / SWAG? @trurl Yes, I was thinking I should format it as a new data disk. What do you think? I have switched appdata/domains/system to "Prefer" As far as the docker allocation I think I did that because I run Plex and I was worried about it running out of room to do conversions and all the metadata it stores on the library. I am fine with lowering it if you think I should. I don't remember how to lower the amount allocated to the docker image though so if you could help me with that I would appreciate it. Also I am sure I made all the mistakes when I configured it so any mistakes I made I will correct if I can identify them. binhex-plexpass binhex-preclear Darkstat deezloaderrrmx(not working right now) duckdns FlexTV gaps Grafana grocy HOOBS Influxdb Let's Encrypt krusader mysql ombi Synclounge(not working right now) tautulli telegraf upstatsboard Varken xteve Yes, I use Let's encrypt for my reverse proxy for OMBI I believe. I hope this answers your questions please feel free to ask more if it will help. Thank you, -GTP Edited September 18, 2020 by GTP question Quote Link to comment
trurl Posted September 18, 2020 Share Posted September 18, 2020 29 minutes ago, GTP said: I have switched appdata/domains/system to "Prefer" That is the correct setting for those, but it won't be enough to get them moved to cache since mover can't move open files. We will deal with that after some more information. 24 minutes ago, GTP said: run Plex and I was worried about it running out of room to do conversions and all the metadata it stares on the library. Plex metadata (library) is in appdata assuming you have it configured correctly. Plex doesn't normally write a lot of data unless you use DVR or transcode. Those paths would need to be in mapped storage of course. You have some dockers I don't use, and I use a different Plex docker. Go to Docker page, click Container Size at the bottom, and post a screenshot of the result. Quote Link to comment
GTP Posted September 18, 2020 Author Share Posted September 18, 2020 (edited) 18 minutes ago, trurl said: That is the correct setting for those, but it won't be enough to get them moved to cache since mover can't move open files. We will deal with that after some more information. Plex metadata (library) is in appdata assuming you have it configured correctly. Plex doesn't normally write a lot of data unless you use DVR or transcode. Those paths would need to be in mapped storage of course. You have some dockers I don't use, and I use a different Plex docker. Go to Docker page, click Container Size at the bottom, and post a screenshot of the result. I have attached the screenshots for the container and my plex container. I also have been playing around with xteve which is a DVR. Edited September 18, 2020 by GTP Clarification Quote Link to comment
trurl Posted September 18, 2020 Share Posted September 18, 2020 Screenshots of Add / Edit Container page seldom show all the information. See this very first link at the Docker FAQ for a better way: When I do Container Size I get a grand total at the bottom but I don't see it with yours. Doesn't matter but that doesn't always tell us everything anyway. Maybe you need more than 20G but I am more concerned that it not grow. As mentioned you are already using 26G. I use less than 10G of 20G with my 17 dockers. The reason I asked about custom docker networks is because you will have to recreate when you recreate docker.img at a smaller size. What do you get from the command line with this? docker network ls Quote Link to comment
GTP Posted September 18, 2020 Author Share Posted September 18, 2020 4 minutes ago, trurl said: Screenshots of Add / Edit Container page seldom show all the information. See this very first link at the Docker FAQ for a better way: When I do Container Size I get a grand total at the bottom but I don't see it with yours. Doesn't matter but that doesn't always tell us everything anyway. Maybe you need more than 20G but I am more concerned that it not grow. As mentioned you are already using 26G. I use less than 10G of 20G with my 17 dockers. The reason I asked about custom docker networks is because you will have to recreate when you recreate docker.img at a smaller size. What do you get from the command line with this? docker network ls Screenshot attached reading the FAQ Quote Link to comment
trurl Posted September 18, 2020 Share Posted September 18, 2020 1 hour ago, GTP said: I was thinking I should format it as a new data disk. What do you think? With the large number of disks you have you really should consider dual parity. Quote Link to comment
GTP Posted September 18, 2020 Author Share Posted September 18, 2020 Just now, trurl said: With the large number of disks you have you really should consider dual parity. I was considering that but do I need another 12 TB disk for that? Or can I use the 8 and 12 together? Quote Link to comment
GTP Posted September 18, 2020 Author Share Posted September 18, 2020 I found the attached information in the FAQ. Which option do you recommend I do? Quote Link to comment
trurl Posted September 19, 2020 Share Posted September 19, 2020 2 hours ago, GTP said: can I use the 8 and 12 together? You can but no data drive can be larger than any parity drive. 2 hours ago, GTP said: Which option do you recommend I do? The simplest is to just use the default, transcode into appdata, which will be on cache after we get things moved. That is good enough for starters and we can discuss other options after other things are done. Quote Link to comment
trurl Posted September 19, 2020 Share Posted September 19, 2020 3 hours ago, GTP said: Screenshot attached reading the FAQ Are you actually using all of these? Probably already asked somewhere on this thread, but do you have any VMs? Quote Link to comment
GTP Posted September 19, 2020 Author Share Posted September 19, 2020 17 hours ago, trurl said: Are you actually using all of these? Probably already asked somewhere on this thread, but do you have any VMs? Br0 -unknknown bridge - I think so docker-configs - I don't even know what this one does hassio - I don't think I am anymore if its for the hassio container because I deleted that. I changed my methodology and I do run a Hassio VM and that could be the bridge to the internet. host - I think you need that one right influxdb - this one I can say for sure "yes" I use Grafana dashboards and Varken none - no idea what that is proxynet - yes for reverse proxy proxynet1 - yes for reverse proxy Yes, I run a Hassio VM instance I am sorry I don't know more I feel like I should but I was mostly just following SpaceInvadersOne's directions to install most of those containers Best, -GTP Quote Link to comment
GTP Posted September 19, 2020 Author Share Posted September 19, 2020 I set those shares to prefer and started the mover. Its been going for a while now and my cache drive is getting pretty full is that ok? Below is a screenshot of the shares and the cache drive. Quote Link to comment
itimpi Posted September 20, 2020 Share Posted September 20, 2020 Is your cache drive large enough to hold the contents of the shares set to Prefer? Quote Link to comment
trurl Posted September 20, 2020 Share Posted September 20, 2020 14 hours ago, GTP said: I set those shares to prefer and started the mover. Its been going for a while now and my cache drive is getting pretty full is that ok? Below is a screenshot of the shares and the cache drive. Unless you also disabled docker and VM service, it won't be able to move everything, since it can't move open files. But that is unrelated to filling cache. Probably should have made sure everything that didn't need to be on cache had already been moved off first before trying to move things to cache. Sorry for the delay following up. Other life going on. Post new diagnostics. Quote Link to comment
GTP Posted September 20, 2020 Author Share Posted September 20, 2020 (edited) No problem at all although the drive has just kept filling up and I tried changing the setting back but it just keeps going the opposite direction. Diagnostics attached. I have a 1tb cache and it looks like I can't set all those shares I should set to "prefer" Best, -GTP tower-diagnostics-20200920-1002.zip Edited September 20, 2020 by GTP Quote Link to comment
trurl Posted September 20, 2020 Share Posted September 20, 2020 Set appdata, domains, system to cache-only for now. That will get them ignored by mover until we can make room for them. You have some other shares with files on cache. You can see which shares have files on cache by going to Main - Cache Devices, click on the folder icon at the far right of Cache under View to see which folders (shares) are on cache. Set each of those shares to cache-yes and run mover. Wait for it to complete then post new diagnostics. Quote Link to comment
trurl Posted September 20, 2020 Share Posted September 20, 2020 Just noticed your cache is completely full and your log is full of mover messages. If mover is still running try this Mover logging can sometimes be useful if you need to debug something specific, but I don't think there is anything we don't already understand, at least for now. Go to Settings - Scheduler - Mover Settings and disable mover logging. You can reboot to clear log space, or you could just do this to get rid of these older logs rm /var/log/syslog.* Quote Link to comment
GTP Posted September 20, 2020 Author Share Posted September 20, 2020 25 minutes ago, trurl said: Set appdata, domains, system to cache-only for now. That will get them ignored by mover until we can make room for them. You have some other shares with files on cache. You can see which shares have files on cache by going to Main - Cache Devices, click on the folder icon at the far right of Cache under View to see which folders (shares) are on cache. Set each of those shares to cache-yes and run mover. Wait for it to complete then post new diagnostics. Cache is completely full I have posted two attached screenshots that should help thank you . I followed your instructions regarding stop and "only" -GTP Quote Link to comment
trurl Posted September 20, 2020 Share Posted September 20, 2020 17 minutes ago, GTP said: I followed your instructions regarding stop and "only" Also do this 35 minutes ago, trurl said: Go to Settings - Scheduler - Mover Settings and disable mover logging. You can reboot to clear log space, or you could just do this to get rid of these older logs rm /var/log/syslog.* Mover ignores cache-no and cache-only shares. It moves cache-yes shares from cache to array (we will get to cache-prefer later). As you can see, you have some cache-no shares that have files on cache. To get those moved from cache, set each of them to cache-yes. Then run mover, wait for it to complete, post new diagnostics. Quote Link to comment
GTP Posted September 21, 2020 Author Share Posted September 21, 2020 6 hours ago, trurl said: Also do this Mover ignores cache-no and cache-only shares. It moves cache-yes shares from cache to array (we will get to cache-prefer later). As you can see, you have some cache-no shares that have files on cache. To get those moved from cache, set each of them to cache-yes. Then run mover, wait for it to complete, post new diagnostics. @trurl I followed your instructions and I moved everything off of cache that I could. Two directories seem to be sticking, one directory with one movie in movies. and one directory with one tv episode in TV Shows. Other than that it only has the files it should. What should we do next? Diagnostics have been posted below and a picture of the cache contents. Best, -GTP tower-diagnostics-20200920-2010.zip Quote Link to comment
trurl Posted September 21, 2020 Share Posted September 21, 2020 Are you sure something doesn't have those files open? Go to Settings - Docker and disable dockers, then try to move them again. Quote Link to comment
trurl Posted September 21, 2020 Share Posted September 21, 2020 Looks like DVR TV Shows has all its files on cache. Go to Shares - User Shares, click Compute All. Wait for the result. If it doesn't appear after a few minutes, refresh the page. Then post a screenshot of the results. Quote Link to comment
GTP Posted September 21, 2020 Author Share Posted September 21, 2020 I have attached the screenshots from the compute all below sorry that there are 12. I ran mover again several times with docker disabled but those files in those directories just will not move. Its just a move and a couple episodes of a tv show so if we had to delete it it would not be a big deal but its weird that they are not moving. I have seen warnings about the particular movie thats stuck in the cache in the logs though. I don't know what to do but I have moved all I can and the Cache is now sitting at 335gb used. Let me know what you want me to do next. Thank you, -GTP Quote Link to comment
trurl Posted September 21, 2020 Share Posted September 21, 2020 Lets go ahead with getting appdata, domains, system moved to cache. I'm going to bed now but I will go ahead and tell you what I have in mind for next. You can go ahead with it if you want or you can wait till tomorrow if you don't want to go without dockers or VMs until then. Set appdata, domains, system shares to cache-prefer. Go to Settings - VM Manager and disable VMs. Leave them disabled Go to Settings - Docker and disable Dockers. On that same page, delete docker.img. Change its size to 20G. Leave them disabled Run mover and wait for it to complete then post new diagnostics. Quote Link to comment
GTP Posted September 21, 2020 Author Share Posted September 21, 2020 1 minute ago, trurl said: Lets go ahead with getting appdata, domains, system moved to cache. I'm going to bed now but I will go ahead and tell you what I have in mind for next. You can go ahead with it if you want or you can wait till tomorrow if you don't want to go without dockers or VMs until then. Set appdata, domains, system shares to cache-prefer. Go to Settings - VM Manager and disable VMs. Leave them disabled Go to Settings - Docker and disable Dockers. On that same page, delete docker.img. Change its size to 20G. Leave them disabled Run mover and wait for it to complete then post new diagnostics. I can absolutely wait, I understand its late here as well. I will do the following tomorrow but I do have one question before I delete the docker.img file will that delete my Plex metedata or settings? Night, -GTP Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.