IamSpartacus Posted January 4, 2019 Share Posted January 4, 2019 (edited) I've been noticing that if I kick off a manual mover process that my Plex server becomes unresponsive until the mover is complete. Now normally this doesn't affect me because I have the mover scheduled to run once a day at 8am when I never use the server. However, I started to investigate why this was happening and it's not adding up. First off, none of my plex (and other dockers) appdata is on cache or my array drives (all appdata is on an SSD set as an unassigned disk). Also, the data that was being moved to the array was not being moved to the disk where the current media I was streaming was located. I saw nothing in the syslog other than the mover activity. My system setup is as follows: Array: 8x10TB WD Gold drives (dual parity) Cache: 1TB Micron 5100 PRO Enterprise SSD Appdata: 256GB Samsung NVMe SSD CPU: Xeon D-1541 RAM: 64GB I'm kind of at a loss as to why the mover would be having any type of performance impact on my Plex server. Edited January 4, 2019 by IamSpartacus Quote Link to comment
IamSpartacus Posted January 5, 2019 Author Share Posted January 5, 2019 From the lack of response I take it this is not common Quote Link to comment
toddas Posted January 9, 2019 Share Posted January 9, 2019 I just installed unraid few days ago and am running into the same issue. I don't use the appdata share, though, but all my Plex configs and databases are indeed in the share with Cache: Prefer setting. I have no idea why Plex stops working when there is a lot of HDD activity in the array. Quote Link to comment
trurl Posted January 9, 2019 Share Posted January 9, 2019 51 minutes ago, toddas said: I just installed unraid few days ago and am running into the same issue. I don't use the appdata share, though, but all my Plex configs and databases are indeed in the share with Cache: Prefer setting. I have no idea why Plex stops working when there is a lot of HDD activity in the array. What do you mean you don't use the appdata share? So what cache-prefer share are you using then? While it is possible to do things differently, I have to ask why you would want to go against the conventions. It is just going to make it harder for you to use the community dockers, and harder for people to support you. Are you sure you know what you're doing? Quote Link to comment
IamSpartacus Posted January 9, 2019 Author Share Posted January 9, 2019 58 minutes ago, toddas said: I just installed unraid few days ago and am running into the same issue. I don't use the appdata share, though, but all my Plex configs and databases are indeed in the share with Cache: Prefer setting. I have no idea why Plex stops working when there is a lot of HDD activity in the array. 3 minutes ago, trurl said: What do you mean you don't use the appdata share? So what cache-prefer share are you using then? While it is possible to do things differently, I have to ask why you would want to go against the conventions. It is just going to make it harder for you to use the community dockers, and harder for people to support you. Are you sure you know what you're doing? Yea as @trurl said I'm not sure what you mean you don't use the appdata share. I use it i just purposely placed it on an unassigned disk and not my cache drive. This is why I'm quite confused about what's causing my issue. Quote Link to comment
limetech Posted January 9, 2019 Share Posted January 9, 2019 On 1/4/2019 at 6:57 AM, IamSpartacus said: my Plex server becomes unresponsive until the mover is complete Please elaborate. Does this mean your video stream completely freezes, or bad stutter, or webui unresponsive? And you notice this precisely when mover starts and then frees up precisely when mover finishes? What is being moved? The 'mover' is very aggressive at copying files. It uses 'sendfile()' kernel interface that can result in copying a large file in a single system call. If cache is on SSD and moving to a HDD could be RAM is rapidly and totally filling up with buffered data. If this is the case, will require some thought about how to fix. Quote Link to comment
IamSpartacus Posted January 9, 2019 Author Share Posted January 9, 2019 (edited) 8 hours ago, limetech said: Please elaborate. Does this mean your video stream completely freezes, or bad stutter, or webui unresponsive? And you notice this precisely when mover starts and then frees up precisely when mover finishes? What is being moved? The 'mover' is very aggressive at copying files. It uses 'sendfile()' kernel interface that can result in copying a large file in a single system call. If cache is on SSD and moving to a HDD could be RAM is rapidly and totally filling up with buffered data. If this is the case, will require some thought about how to fix. @limetech Ok I tested this again last night and got similar yet slightly different results. My CPU was basically idle (less than 10% usage as I had just 2 direct plays going in Plex). As soon as I kicked off the mover, within a few seconds the CPU usage jumped up over 70% as you can see from Screenshot #1. This stayed the case for about 4-5 minutes and during this time my Plex Direct plays stopped as soon as the buffers on my Shield TV's ran out. Screenshot #2 shows that virtually nothing is going on in the system that would explain the CPU usage showing as being so high at this time. The CPU usage (or lack there of) in htop not matching up what's being shown in the webUI is very strange to me. After 4-5 minutes, the CPU returned to a normal state for a few minutes or so and I was able to resume my Plex direct plays. The mover was still going at this point and would continue for another 10-15 minutes. Then, the CPU usage jumped up again (Screenshot#3). This again made Plex unresponsive. However, this time when I opened htop (sorry I didn't get a screenshot of this) I saw two Plex processes consuming a lot of the CPU and this continued for 10+ minutes. I had to actually restart the Plex docker to regain usage of the server again. I copied my Plex logs folder at this time so I could go through them and determine what was going on in Plex at that time. I'm not really sure what I'm looking for but I will try to narrow it down. I don't know why the mover starting to move files initially would be causing the CPU usage to be so high. I've got an 8C/16t CPU (Xeon D-1541) so the mover should not be taxing it so much. It then appears that the moving of the files is kicking some kind of process off in Plex that causes Plex to freak out. The Plex piece I'll have to investigate on my own because none of this is really adding up. I don't see why simply moving files around in Unraid would cause Plex to do anything at all. The WebGUI never becomes unresponsive during any of this and the syslog shows nothing other than the mover logs. Edited January 9, 2019 by IamSpartacus 1 Quote Link to comment
Viperkc Posted April 7, 2019 Share Posted April 7, 2019 I am having the same issue. When mover runs all trans coded Plex streams buffer. Has not been this way in the past, it could have started when i installed my P2000 and started trans-coding with the Nvidia version of Unraid. But i am not certain of that. I see no noticeable higher CPU or Memory usage when this happens. Thanks Quote Link to comment
brainbone Posted May 29, 2019 Share Posted May 29, 2019 (edited) I think I'm seeing this same issue. When mover is running, Plex is for the most part frozen. Happened both before and after installing an old Geforce 1050 (using Unraid-Nvidia) for transcoding, so it seems unrelated to Unraid Nvidia. I'm not certain if this happened before I upgraded to 6.7.0 as I seldom manually ran the mover when I was on 6.5.x -- though I don't recall it happening before the upgrade to 6.7.0. Edited May 29, 2019 by brainbone Quote Link to comment
bradtn Posted June 1, 2019 Share Posted June 1, 2019 I have same issue with mover, makes plex unusable. I have two unassigned disks. 1 unassigned disks runs. My docker apps and dockapps only. The other unassigned disk is where my downloads from nzbget download to and where they get unpacked. I also run the Nvidia version of unraid with a gtx 1070 doing my hw transcodes and transcode folder mapped to ram Quote Link to comment
chad4800 Posted June 4, 2019 Share Posted June 4, 2019 I am experiencing this issue as well. All Plex streams essentially lockup while the mover service runs. The streams are stuck on buffering, it may play for a second or two and then buffers again. I am using a GTX 1050 with HW transcoding enabled, however I was experiencing this issue prior to installing the GPU. My issue started maybe a month ago or so. Currently running Unraid v. 6.7.1-rc1. Plex is mapped to RAM for transcoding. I may try downgrading to a stable version prior to 6.7.0 to see if that helps. Quote Link to comment
itimpi Posted June 4, 2019 Share Posted June 4, 2019 3 hours ago, chad4800 said: I am experiencing this issue as well. All Plex streams essentially lockup while the mover service runs. The streams are stuck on buffering, it may play for a second or two and then buffers again. I am using a GTX 1050 with HW transcoding enabled, however I was experiencing this issue prior to installing the GPU. My issue started maybe a month ago or so. Currently running Unraid v. 6.7.1-rc1. Plex is mapped to RAM for transcoding. I may try downgrading to a stable version prior to 6.7.0 to see if that helps. Just a thought - what is the reason that mover is trying to run at the same time as Plex? An easy avoidance action would be to simply schedule mover to run at a time when you know Plex would not be active. Quote Link to comment
ProZac Posted June 4, 2019 Share Posted June 4, 2019 On 1/9/2019 at 2:58 PM, IamSpartacus said: @limetech Ok I tested this again last night and got similar yet slightly different results. My CPU was basically idle (less than 10% usage as I had just 2 direct plays going in Plex). As soon as I kicked off the mover, within a few seconds the CPU usage jumped up over 70% as you can see from Screenshot #1. This stayed the case for about 4-5 minutes and during this time my Plex Direct plays stopped as soon as the buffers on my Shield TV's ran out. Screenshot #2 shows that virtually nothing is going on in the system that would explain the CPU usage showing as being so high at this time. The CPU usage (or lack there of) in htop not matching up what's being shown in the webUI is very strange to me. After 4-5 minutes, the CPU returned to a normal state for a few minutes or so and I was able to resume my Plex direct plays. The mover was still going at this point and would continue for another 10-15 minutes. Then, the CPU usage jumped up again (Screenshot#3). This again made Plex unresponsive. However, this time when I opened htop (sorry I didn't get a screenshot of this) I saw two Plex processes consuming a lot of the CPU and this continued for 10+ minutes. I had to actually restart the Plex docker to regain usage of the server again. I copied my Plex logs folder at this time so I could go through them and determine what was going on in Plex at that time. I'm not really sure what I'm looking for but I will try to narrow it down. I don't know why the mover starting to move files initially would be causing the CPU usage to be so high. I've got an 8C/16t CPU (Xeon D-1541) so the mover should not be taxing it so much. It then appears that the moving of the files is kicking some kind of process off in Plex that causes Plex to freak out. The Plex piece I'll have to investigate on my own because none of this is really adding up. I don't see why simply moving files around in Unraid would cause Plex to do anything at all. The WebGUI never becomes unresponsive during any of this and the syslog shows nothing other than the mover logs. Just a quick sidenote, not sure if you have looked into this, what is your plex set to do when files in the database is changed? I assume you have set plex to scan it's library? Might it be that plex is starting to scan the files as this can take alot of resource and it might be plex itself that stalls and not the server? Quote Link to comment
chad4800 Posted June 4, 2019 Share Posted June 4, 2019 7 hours ago, itimpi said: Just a thought - what is the reason that mover is trying to run at the same time as Plex? An easy avoidance action would be to simply schedule mover to run at a time when you know Plex would not be active. I do have mover scheduled to run very early in the morning, however I did a lot of downloading recently and manually kicked of the process in the early evening to free up space on my cache drive. Quote Link to comment
IamSpartacus Posted June 4, 2019 Author Share Posted June 4, 2019 2 hours ago, ProZac said: Just a quick sidenote, not sure if you have looked into this, what is your plex set to do when files in the database is changed? I assume you have set plex to scan it's library? Might it be that plex is starting to scan the files as this can take alot of resource and it might be plex itself that stalls and not the server? Good thought but in my testing this was not the case. Yes Plex scans on library changes but with all the resources I have at my disposal the scan is done in less than 30 seconds. The issue seems to be CPU IOWait. Watching netdata during the mover process IOwait jumps up significantly and I don't understand why. 1 Quote Link to comment
chad4800 Posted June 4, 2019 Share Posted June 4, 2019 1 minute ago, IamSpartacus said: Good thought but in my testing this was not the case. Yes Plex scans on library changes but with all the resources I have at my disposal the scan is done in less than 30 seconds. The issue seems to be CPU IOWait. Watching netdata during the mover process IOwait jumps up significantly and I don't understand why. I typically don't have any issues streaming while Plex is scanning/updating my library either. I haven't noticed any high CPU, RAM, or Disk I/O utilization. I'll have to look more closely at the CPU I/O wait, and see if that's happening with me as well. Quote Link to comment
IamSpartacus Posted June 4, 2019 Author Share Posted June 4, 2019 5 minutes ago, chad4800 said: I typically don't have any issues streaming while Plex is scanning/updating my library either. I haven't noticed any high CPU, RAM, or Disk I/O utilization. I'll have to look more closely at the CPU I/O wait, and see if that's happening with me as well. It's not Plex that causes high CPU, RAM, or Disk I/O. It's the mover process. I have mitigated this for the time being by scheduling my mover to only run once a day at 5am. But now that I've moved into doing a lot of 4K, sometimes i need to manually run the mover during the day and I have to hope no one is using Plex. 1 Quote Link to comment
bradtn Posted June 4, 2019 Share Posted June 4, 2019 (edited) 1 hour ago, IamSpartacus said: It's not Plex that causes high CPU, RAM, or Disk I/O. It's the mover process. I have mitigated this for the time being by scheduling my mover to only run once a day at 5am. But now that I've moved into doing a lot of 4K, sometimes i need to manually run the mover during the day and I have to hope no one is using Plex. Same here man, I have a ryzen 1950x so I'm Quite sure I have enough cpu power. All My Dockers run off their own specific unassigned ssd. All my Downloads go to their own specific ssd so I should have zero to little io wait for any disk related things Edited June 4, 2019 by bradtn Quote Link to comment
chad4800 Posted June 5, 2019 Share Posted June 5, 2019 It seems that if the video file still resides on the cache drive it streams fine, however if the file has been moved to the array, then I have the lockup/buffering issue while the mover service is running. That would seem to point to Disk I/O contention on the array, not sure. Quote Link to comment
testdasi Posted June 5, 2019 Share Posted June 5, 2019 What is your core pinning for Plex? Try not using core 0. I noticed Unraid mover tends to use core 0 almost exclusively so if Plex (or any other CPU-intensive docker) uses it (e.g. streaming), it would lag both regardless of CPU. It has nothing to do with how powerful your CPU is. When it's stuck waiting for a response from a drive (which is slow) due to the mover, it will not be doing anything else and report 100% usage. Quote Link to comment
Teixeira Posted June 5, 2019 Share Posted June 5, 2019 (edited) I am having a similar issue. I don't think it's related to the mover since my 'media' share doesn't use a cache drive. I use a seperate HDD outside the array for SABnzbd and when Radarr/Sonarr tries to import a file, everything becomes unresponsive (yes, not only plex but all the containers). Same thing happens when i copy a file from my PC to the server. I am aware that writing data to a parity protected array requires CPU power, but not 100% of my 2700X. I thought this was a problem with Radarr / Sonarr (Mono), but then i pinned them both to only 1 core and gave them each 500MB of RAM and this still happened. Edited June 5, 2019 by Teixeira Words Quote Link to comment
chad4800 Posted June 5, 2019 Share Posted June 5, 2019 2 minutes ago, testdasi said: What is your core pinning for Plex? Try not using core 0. I noticed Unraid mover tends to use core 0 almost exclusively so if Plex (or any other CPU-intensive docker) uses it (e.g. streaming), it would lag both regardless of CPU. It has nothing to do with how powerful your CPU is. When it's stuck waiting for a response from a drive (which is slow) due to the mover, it will not be doing anything else and report 100% usage. In my case my Plex container is using 6/18, 7/19, 8/20, 9/21, 10/22, and 11/23, which I do not have pinned to any other containers or VMs. I just don't recall this happening a couple of versions ago. Quote Link to comment
testdasi Posted June 5, 2019 Share Posted June 5, 2019 19 minutes ago, Teixeira said: I am having a similar issue. I don't think it's related to the mover since my 'media' share doesn't use a cache drive. I use a seperate HDD outside the array for SABnzbd and when Radarr/Sonarr tries to import a file, everything becomes unresponsive (yes, not only plex but all the containers). Same thing happens when i copy a file from my PC to the server. I am aware that writing data to a parity protected array requires CPU power, but not 100% of my 2700X. I thought this was a problem with Radarr / Sonarr (Mono), but then i pinned them both to only 1 core and gave them each 500MB of RAM and this still happened. ALL containers becoming unresponsive is a different problem. As I mentioned, when the CPU is waiting for the drive to respond, it will report 100% usage, regardless of how powerful your CPU is. A slow-responding drive will cause what you are seeing. In terms of why a drive responds slowly, it may be due to high IO (e.g. repairing + extracting + copying strain a HDD a lot due to repeated seek) or even a failing drive (that was what happened when my old Hitachi 3TB was about to kick the dust). Quote Link to comment
Teixeira Posted June 6, 2019 Share Posted June 6, 2019 On 6/5/2019 at 6:22 PM, testdasi said: ALL containers becoming unresponsive is a different problem. As I mentioned, when the CPU is waiting for the drive to respond, it will report 100% usage, regardless of how powerful your CPU is. A slow-responding drive will cause what you are seeing. In terms of why a drive responds slowly, it may be due to high IO (e.g. repairing + extracting + copying strain a HDD a lot due to repeated seek) or even a failing drive (that was what happened when my old Hitachi 3TB was about to kick the dust). I found the problem to this issue. I have my 'system' share set to cache only, but some data was still on the array. I moved it all to the cache with the 'unbalance' plug in. Now the containers don't become unresponsive anymore when the array goes to shit. The reason i'm still here talking about something that has nothing to do with the mover is that my problem is exactly the same as the original post. The first 5 mins of the import makes the whole array unresponsive, meaning that nothing can be read from it. After 5 mins I gain access to it again but it is noticeably slower (kinda expected). After some time it goes back up to 100% CPU and nothing can be done, until it is done. You said this is probably because of slow drives. I've excluded that one drive that contains all the data, forcing unRAID to write to the next one. Plex still can't play the media that is on the "idle" drive. Quote Link to comment
testdasi Posted June 6, 2019 Share Posted June 6, 2019 58 minutes ago, Teixeira said: I found the problem to this issue. I have my 'system' share set to cache only, but some data was still on the array. I moved it all to the cache with the 'unbalance' plug in. Now the containers don't become unresponsive anymore when the array goes to shit. The reason i'm still here talking about something that has nothing to do with the mover is that my problem is exactly the same as the original post. The first 5 mins of the import makes the whole array unresponsive, meaning that nothing can be read from it. After 5 mins I gain access to it again but it is noticeably slower (kinda expected). After some time it goes back up to 100% CPU and nothing can be done, until it is done. You said this is probably because of slow drives. I've excluded that one drive that contains all the data, forcing unRAID to write to the next one. Plex still can't play the media that is on the "idle" drive. You misunderstood my point. It doesn't matter if Plex is accessing the idle drive or an active drive, the fact that the CPU is waiting for the drive to respond will exclude other activities. Think of it like this: Mover tells CPU: write file 1 to disk A CPU: ok, writing Plex tells CPU: read file 2 from disk B CPU: nope, writing Plex: are you done yet? CPU: nope, still writing Plex: are you done yet? CPU: nope, still writing Plex: are you done yet? CPU: nope, still writing Plex: are you done yet? CPU: nope, still writing Plex: are you done yet? CPU tells Mover: ok, file 1 written to disk A CPU tells Plex: ok, here is file 2 from disk B The entire time from step 4 to step 13, CPU will report 100% usage because it is fully occupied with writing. Yes, we can talk all days about how it is supposed to be multi-tasking in 2019 but the truth is some action requires the CPU (core) full attention and until it's done, it won't be able to do anything else. This is the same reason that if you plug a failing drive or SD card to Windows, it will hang the entire system while trying in vain to read the data. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.