June 5, 201115 yr Hi There, I'm running unRAID 5.0b6a on a HP MicroServer (AMD Neo N36L) with 2GB ram. I have two AFP/SMB shares configured, apps and media. The split level on these shares is at level 2, ensuring that movies and tv shows do not get split across drives. When telnetted or ssh'd into my server, I find that every few minutes the session will freeze. Any keyboard entry during this time is being buffered, but no output appears on screen. When the problem clears, any characters input appear. The 'freezes' last at most 20 seconds, but are sometimes much shorter. I have also seen the issue when playing back files on my Mac from the AFP shares. Playback will freeze for up to 20 seconds, and then resume. Has anyone seen this before? Any tips on how to diagnose why this is happening? I'm pretty sure that the glitches are not caused by a drive spinning up, as they can occur when I have no shares mounted on clients, and am simply telnetted in and ps'ing for a particular process. Any help would be much appreciated. Andy.
June 9, 201115 yr Author Done some more digging into this, and found that it only happens when I have Plex Media Server running. The entire system doesn't appear to be hanging, as I can still interact with the server from it's console. However, all network activity seems to be suspended. Plex is installed in /mnt/cache/.plex, and is IDLE when the hangs occur. If I run strace on the 'Plex Media Server' process, then I don't see the problem occurring. However, when you stop strace, you almost immediately get a hang of around 90 seconds, and then the hangs occur much more frequently after that. When idle, Plex is simply statting it's Plug-Ins directory looking for updates, and then calling gettimeofday(). It's really not doing anything else. I've spoke with Elan at Plex, and we can't see any reason why Plex should cause such problems. I'm a bit of a novice at diagnosing Linux stuff (got plenty of Solaris experience), and would really appreciate some pointers on where to look next. strace clearly isn't going to cut it, as the problem doesn't seem to be occurring in user land. If anyone out there has any low level Linux troubleshooting skills, I'd be very grateful for suggestions. Andy.
June 12, 201115 yr Author Is there really nobody out there who can help me troubleshoot this issue? Not even if I say, pretty please with sugar on top? Also, I've tried beta7, and the problem still exists.
June 15, 201115 yr Author This issue is looking more like a problem with AMD64 systems. I've found references to issues with futex() and hangs on AMD systems with 2.6.3x and upwards kernels. To try and prove this, I've also installed the demo version of unRAID on an Opteron 252 system with 2x250GB SATA drives. I see exactly the same hangs when Plex is running on this system, as I do on my HP MicroServer. I'm just going to try an installation on an Intel x64 system to see if I can replicate the problem there. Theoretically, any code that used pthread_* system calls could see this issue on AMD64 systems. Reference for this comes from here: http://forum.soft32.com/linux/Futex-hang-lockup-problem-30-AMD64-ftopict504130.html Tom, if you are reading this, I could really use some help here.
June 15, 201115 yr This issue is looking more like a problem with AMD64 systems. I've found references to issues with futex() and hangs on AMD systems with 2.6.3x and upwards kernels. To try and prove this, I've also installed the demo version of unRAID on an Opteron 252 system with 2x250GB SATA drives. I see exactly the same hangs when Plex is running on this system, as I do on my HP MicroServer. I'm just going to try an installation on an Intel x64 system to see if I can replicate the problem there. Theoretically, any code that used pthread_* system calls could see this issue on AMD64 systems. Reference for this comes from here: http://forum.soft32.com/linux/Futex-hang-lockup-problem-30-AMD64-ftopict504130.html Tom, if you are reading this, I could really use some help here. Not sure how to help with this, since I don't have any AMD64 systems. If you find a kernel patch that addresses this I'd be happy to apply it and give you a custom build to see if the problem is solved.
June 16, 201115 yr Author Hi Tom, Thanks for getting back to me. I'm trying to figure out if this is fixed, and in what version. Further reading seems to indicate that the libstdc++ library also comes into this as well, in combination with particular kernel versions. Is beta7 shipping with the latest C library? Andy.
June 28, 201115 yr Author Hi Tom, I've now managed to reproduce this issue on both 32 and 64-bit Intel systems, so hopefully this might make it easier for you to take a look at. I've also installed Plex on a full Slackware 13.1 install with kernel 2.6.33.4-smp, and the problem does not occur. This would seem to indicate that it is a uniquely unRAID based issue. My steps to reproduce are as follows: 1) Boot from vanilla unRAID 5.0-beta7. 2) Preclear all disks. 3) Create a suitable array. I've reproduced the problem with a single data disk (no parity), and with a 5 disk array (plus parity). 4) Create a user. 5) Enable AFP and SMB sharing (SMB with default config) 6) Enable user shares under 'Share Settings'. 7) Create a share and export via AFP and SMB. AFP security is private, access is read/write to previously created user. SMB security is private. No access to root. read/write access to created user. Allocation method is high-water. Min. free space is 40000000. Split level is 2. Copy a large MKV file to the share (my example file is 1920x1080 and approx 1.5GB). The next step is to install Plex. Download it from here: http://www.plexapp.com/releases/PlexMediaServer-v0.9.2.8-34480ac-slackware.tar.bz2 Once downloaded, do the following: 1) Extract the tarball to a suitable install location, either on a share or on the cache drive. 2) Create links for the Library as follows: ln -s /mnt/cache/.plex/Library /Library ln -s /mnt/cache/.plex/Library /root/Library substituting your install location as appropriate. 3) Install the ALSA sound library from here: http://packages.slackverse.org/?v=5&p=500 4) Mount your /tmp onto a physical filesystem (not necessary to reproduce the problem, but will stop memory problems if you actually stream anything with Plex). cd /tmp; tar cf - . | (cd /mnt/cache/.tmp; tar xf -) mount --bind /mnt/cache/.tmp /tmp 5) Apply fix for Plex server not showing up on iOS/LG client devices. See first item in FAQ here: http://www.lime-technology.com/wiki/index.php?title=Plex_for_UnRAID_FAQ 6) cd into plex install directory, and execute start.sh. 7) Wait for about 5 minutes to allow Plex server to perform it's initial plugin caching. Stop Plex, and re-start it using start.sh. At this point, you should start playing your test file from the media share. I used VLC on a MacPro, but I don't suppose it matters. The idea is to exercise the network. You can mount the share using AFP or SMB. The problem will reproduce regardless. When the problem occurs, the playback of the file will stop, and then resume after a number of seconds. During this time, network activity on the unRAID box appears to be completely stalled. Any ssh/telnet sessions will be hung, but any input made during this time will appear on the screen when the hang is over. I've found that it can take up to 15 minutes of playback before the problem first occurs, but once it does occur, it will happen every few minutes, and gradually get worse, with longer pauses each time it happens. Stopping Plex causes the problem to cease immediately. If anyone else can reproduce this problem, then it would be great to know. Thanks, Andy.
Archived
This topic is now archived and is closed to further replies.