Hey... I solved a very similar issue back on 4.5 by using this to start emhttp in the config/go script.

#!/bin/bash

# Start the Management Utility

ulimit -n 20000;/usr/local/sbin/emhttp &

 

Added that to my go file, but even after reboot ulimit -a shows 1024 for Open Files.  Here is my go file:

 

#!/bin/bash
# Start the Management Utility
ulimit -n 20000;/usr/local/sbin/emhttp &
echo nameserver 192.168.10.80 >/etc/resolv.conf
echo 192.168.10.10 tower >>/etc/hosts

while [[ ${LOOP:=10} -gt 1 && ! -b /dev/md1 ]]
do
        (( LOOP=LOOP-1 ))
        echo "Waiting for /dev/md1 to come online ($LOOP)"
        sleep 1
done
sleep 1

for disk in /dev/md*
do blockdev --setra 2048 $disk
done

blockdev --setra 2048 /dev/sde # Parity Drive.

 

Do I have it in the right place or do I need to move it after the last line?

 

Edit: Moving it after the last line had no effect after reboot either...I can manually type "ulimit -n 20000" through telnet and it does update and shows the new value of 20000 when I do "ulimit -a".

 

Take Care

changing the ulimit in the "go" file would not change the ulimit for your telnet session, since telnet is not started through emhttp or through the "go" file.  The smaller limit is not an issue for telnet.

 

You want the ulimit changed BEFORE emhttp is invoked, so the ulimit command belongs before it, not at the end of the "go" file.  The location you have in your sample "go" file is perfect.

If you type "testparm -sv" you'll see that Samba itself increases 'ulimit -n' to 16384.  Also, user share file system increases to it's 'ulimit -n' to 16384+4 (the + 4 for file handles possibly used by fuse).

 

The "right" way to solve this is to create /sbin/initscript (see /sbin/initscript.sample).  I have tested this by putting a "ulimit -n 20000" in /sbin/initscript and it seems to work.  Unfortunately this file has to be already present during system start up, so I would have to generate a new release to do it.  (Because it's opened by the 'init' process - ie, the mother of all processes.)

 

Why 20000?  Not sure how to make this number configurable.

I recently decided to rebuilt my Plex library from scratch. Ugh, bad idea. I'm getting this transport endpoint not connected issue all the time now.

I recently decided to rebuilt my Plex library from scratch. Ugh, bad idea. I'm getting this transport endpoint not connected issue all the time now.

I am having the same issue, as well as a friend of mine on a new build. I hope someone can shed some light on this as restarting the server everyday is not a viable solution atm.

Did either of you try any of the solutions posted?

Did either of you try any of the solutions posted?

I am in the process of adding the solution in post #1 to my friends server. I am not super keen on having to manually restart sabnzb though. I have disabled PMS on mine and it has been stable and up for 5 days without issue.

To reply back to limetech .... I have no idea where the 20000 value came from for open files ....

 

This latest release of unRaid 5.0-rc12a is unusable AFAIC, I only use unRaid for Plex media.

This problem can be created at will for me by simply starting PlexMedia via the WebGui, and

launching the PlexWeb Management GUI and simply selecting my music library.

 

It triggers the Transport endpoint problem immediately.  This should not happen on a system designed to be a file server!

I suspect that the Plex Web GUI gui would need to display the thumbnails of the music albums it may be opening lots of files.

 

Does Lime stress test samba access in their product before releasing? or is what Plex is doing unreasonable ?

I can not recreate the issue from other clients (e.g. LG TV) and seeing as I did not have issues with earlier releases unRAID

and the same music database it points towards a shoddy release from Lime.

 

My Music library has 7793 MP3's but you expect that Plex will be opening jpg's for the music albums as well.

So if you double that (worst case scenario + other Plex clients accessing my shares) =>

 

you are looking at ~ 15k max possible file items open.

 

This is suspiciously close to the samba ulimit setting "testparm -sv | egrep open"  Output "max open files = 16404"

 

setting ulimit to 20,000 and then 40,000 did not make any difference !!    (I am running a HP Microserver with 2GB DDR3 Memory single channel).

Also setting the oom_score values to -1000 does not fix this issue either.

 

What does appear to have delayed the time the problem takes to occur after clicking on the Music Folder in the Management Web GUI of Plex

is simply increasing the samba max open files param !

 

To do this create a file called:

 

/boot/config/smb-extra.conf

 

And add this text;

 

[global]

max open files = 32808

 

>>  I get more icons on the web page before the Transport errors occur .......... Still digging into this issue ..... its driving me NUTS!!  :-0

Right as well as increasing the max number of files Samba can open as above... I also;

 

1). Completely removed the Plex Media Server package.

2). Rebooted.

3). Reinstalled the Plex Media Server package.

4). Deleted all my media Libraries form Plex Media.

5). Re Added my media folders back into Plex one by one and let them rebuild.

      i)    Music Folder => OK

      ii).  Movies Folder => OK.

      iii). Home Videos => Transport Errors

      iv). Photos Folder => Transport Errors

 

Free memory steadily decreases over this time frame as Plex adds more files to the Library, and it looks like memory used by Linux for cache

is not being released for other apps. Kill off Plex and Free memory returns back to before Plex was started.

 

Plex logs do not indicate anything unusual up until it simply looses access to the user folder. At the same time in the syslog you get the usual;

emhttp: get_filesystem_status: statfs: /mnt/user/ftp Transport endpoint is not connected

 

To progress I suspect I am going to have to see if I can put the FUSE user space file system in debug mode to figure out what's going on.

It looks like a resource management issue that is triggered over time, probably memory (e.g. why is cached memory not being freed up for apps ?).

 

The question is .. which component is not doing its job!

Since I've completed rescanning my library, unRAID has had fewer transport endpoint errors, though they seem to pop up randomly. I just triggered one by browsing via Plex client to a specific show. During the scan, I was seeing them as rapidly as one per hour.

 

I've tried:

• Manually changing ulimit to 20000 after a reboot

• Putting in ulimit -n 20000 in my go script as Joe L. recommended

• Modifying  /boot/config/ident.cfg to put a greater ulimit

 

None of these have made a huge difference – in fact I'd be hard pressed to say these did anything more than placebo.

 

Plex server seems to be the easiest way to get this error to happen, but I've also had it occur when Plex was disabled, and I was tinkering with SABnzbd and SickBeard.

 

I don't use AFP or NFS, everything is via SMB shares. cache_dirs is not running either.

I have installed PlexMediaServer-0.9.7.22.511-4b5280f-unRAID.txz and it's doing a scan on a very large media library running on an Atom-based m/b with 4GB RAM.  So far no problems.  Once this has completed, what should I do to try and reproduce the problem?

 

BTW, I do notice in the /boot/config/plugins/plexmediaserver/settings.ini file this line:

 

PLEX_MEDIA_SERVER_MAX_OPEN_FILES=4096

 

Do I need to change this?

Mine seems completely random. After 3 days of uptime, I decided to try to add my music library. Scanned and added without a hiccup. Then last night when nothing was going on it randomly crashed. Rebooted this morning, and crashed again 1-2 hours later without even using it. I have some logs I am linking to if it helps.

 

https://dl.dropboxusercontent.com/u/168836/Unraid-Syslog.zip

https://dl.dropboxusercontent.com/u/168836/Plex%20Media%20Server-Logs%2BCrashreport.zip

I haven't been able to duplicate this issue, but testing continues today.  In poking around the plex server-side code, I notice that it sets up an "inotify" watch on every directory in the media library.  This might be related to the problem because the default watch limit is 8192:

 

cat /proc/sys/fs/inotify/max_user_watches
8192

 

You can raise this limit higher (to say 20000) by putting this inside your 'go' file:

 

sysctl fs.inotify.max_user_watches=20000

 

Also since plex is started from /boot/extra before 'go' is invoked, a better way might be to:

a) disable plex via it's settings page in the webGui

b) type above sysctl command via telnet/console

c) re-enable plex via settings page

 

I'm not sure this is the problem because I'd expect to see the inotify subsystem complain via syslog if it ran out of watches.  But this being the source of the problem makes sense, especially for very large music libraries where there are lots of directories.

 

Also note, this issue has nothing to do with Samba because plex is running entirely on the server and accessing the shares directly.

 

One other test:  For those of you who can make this happen at will:  (after first trying above fix), you can also set up plex to not scan the 'user share' directory, but instead set up a list of individual disk shares.  That is,

 

instead of:

/mnt/user/Music

configure plex to scan:

/mnt/disk1/Music

/mnt/disk2/Music

:

(use the disk numbers that have the Music share on them)

 

I realize this is not ideal, but it would provide more information useful for solving the problem.

 

I'm attempting to test with Tom's adjusted watch limit.

 

Here's what I've done so far:

• Confirmed I can fairly reliably cause this issue by browsing my "Videos" section using the Plex/Web view (75%+ of the time it happens within about 30 seconds of browsing, even after a fresh reboot)

• Confirmed the default watch limit was set to 8192

• Disabled Plex

• Rebooted unRAID

• Set the watch limit via the sysctl command to 20000

• Re-Enabled plex

• Verified watch limit is now at 20000

• Tried browsing my Videos section in Plex/Web again

 

I've only been at it for about five minutes, so I'll try hammering on Plex quite a bit as I can today and see if I can break it again.

So far, so good. I've been beating on Plex for a while now and things are operating normally.

So far, so good. I've been beating on Plex for a while now and things are operating normally.

Interesting...

 

If you search for "inotify" on the Plex forums there are several posts about this.

It worked so well for a few hours, so I decided to reboot and try again without changing the watch limit from the default 8192.

 

Of course now it's perfectly behaving itself again Unclear results stink!

To follow up, I was finally able to get a transport endpoint error when I reverted back to 8192 watch limit.

 

I think it's promising to bump that up. I'll test it over the next week or so and keep an eye on things.

Just FYI for others - I had tons of these Transport endpoint not connected issues when running Crashplan - it'd crash my user shares in under an hour.  Upgraded from RC8 to RC12 and everything has been AWESOME for more than 24 hours.

 

Hope this helps someone,

 

Russell

Transport endpoint not connected is driving me nuts. I'm pretty sure it has to do with the number of open files. I am accessing a number or NFS shares from a bunch of servers, and every now and then when I am copying all shares would just disappear..

 

Adding ulimit to the go file hasn't helped me at all. I just added watch limit now, curious to see how that goes.

 

 

 

 

 

 

So 6 days of uptime with the 20000 watch limit and I think that's the furthest I've gotten in quite some time. I think this may be the needed fix.

instead of:

/mnt/user/Music

configure plex to scan:

/mnt/disk1/Music

/mnt/disk2/Music

:

(use the disk numbers that have the Music share on them)

 

I realize this is not ideal, but it would provide more information useful for solving the problem.

 

Interesting, I had a similar problem a while back with XBMC, It would NOT completely scan a "TV" user share (with all the disks under under it) when I had more than 6 disks in the share (at the time I had 9). I ended up making individual entries in the XMBC settings for each disk... This gives me something to try

I am still getting the error. Longest uptime was 3.5 days, then I had it happen twice in a 24 hour period. No si bueno.

I am still getting the error. Longest uptime was 3.5 days, then I had it happen twice in a 24 hour period. No si bueno.

 

Is this with the updated watch limit (assuming you were using Plex Media Server)?

Is this with the updated watch limit (assuming you were using Plex Media Server)?

 

Yes, I invoked it manually every time though by stopping PMS, SSH in and run the command, restarting PMS. I do not currently have it in my go file. There is a new version of PMS out for Plexpass subscribers, but there is a bug that needs fixed before I can try it (I did not see anything in the changelog though pertaining to this issue)

So I've been using the increased watch limit for two weeks now, and nary an issue. I think this is certainly worth others testing who are having this issue in combination with Plex.