NFS stale file handle

[dlandon]

10 hours ago, AshranPewter said:

edit: (without removing the mover), if I don't have cache while moving these large files it affects the entire system and if the transfer take too long, can't play any media with my media server. So if possible a solution without that would be awesome...

I doubt this will help because the issue is not showing up on Unraid.  This solution was a "get around" for NFSv3 server issues on Unraid.

 

Try adding 'noacl' to the mount paramaters.  Also check your mount setup.  I don't think you are mounting using NFSv4.

 

The experience we've had on Unraid is that just using the NFSv3 server on Unraid caused the stale file handle situation.  Once we converted to NFSv4, the stale file handle issue went away.  If the stale file handle issues you are seeing are not showing on Unraid, there probably isn't much more we can do to help you.  You'll have to look on the client side to see what is happening.

[scubanarc]

Try running the following on your client:

nfsstat -c

Does it say "nfs v4"?

[AshranPewter]

10 hours ago, dlandon said:

I doubt this will help because the issue is not showing up on Unraid.  This solution was a "get around" for NFSv3 server issues on Unraid.

 

Try adding 'noacl' to the mount paramaters.  Also check your mount setup.  I don't think you are mounting using NFSv4.

 

The experience we've had on Unraid is that just using the NFSv3 server on Unraid caused the stale file handle situation.  Once we converted to NFSv4, the stale file handle issue went away.  If the stale file handle issues you are seeing are not showing on Unraid, there probably isn't much more we can do to help you.  You'll have to look on the client side to see what is happening.

 

It shows me using nfs v4 in my client, but I'm not sure how to check in Unraid. I assume it has to be reciprocated so it's the same on the other side.

 

One thing to note is that my shares look like this

And I can cd into the Media folder and then into the folders under it if I know the name. But can't auto-complete or anything.

 

2 hours ago, scubanarc said:

Try running the following on your client:

nfsstat -c

Does it say "nfs v4"?

 

Ya, it shows "Client nfs v4" just as my nfsstat -m shows it on each of my separate mounts as 4.2

[AshranPewter]

I blew away my config for unraid and then redid everything. When the mover runs it causes the same issue.

Everything using nfs 4 as far as I know.

 

It does seem strange that it's just the top level of the directory has a stale file handle.

 

Also to confirm with screenshots nfs4

 

 

 

 

 

 

[AshranPewter]

if anyone has any idea of what I can change, would appreciate it! Not using a cache drive and it's... meh to say the least.

 

Thanks~!

[AshranPewter]

For anyone running into this, plex specifically seems to cause this issue.

 

Adding 'nolock' into your fstab on the client side fixes this... so far.

[AshranPewter]

Back to this for anyone having issues. The nolock didn't work, still had issues if I wanted to use the cache drive and the mover ran. I have ideas that maybe the  mover tuner was causing issues but it was not the issue.

 

I stopped having any issues related to stale file handles in ubuntu but Plex (running in docker) was having issues where it would happen inside the container.

 

The fix? Mount the nfs shares using docker volumes. I'm sure there is probably something else that could fix it, NFS is probably mounted by docker using some specific parameters to alleviate the problems I encountered and docker volumes are probably better, bind was just simpler (or so I thought) at the time.

[falcnor]

I've just recently run into this issue as well.

Plex is not running in a docker, on Ubuntu 20.04.

Happened after a mover event on my media shares. Rebooted the plex machine and all is well. This shouldn't be the solution.

[AshranPewter]

It's probably just a plex issue overall, due to how they lock files so I would investigate lock/nolock more, it doesn't affect my Emby or Jellyfin install or any of my other dockers using the same mount points. But I'm unsure how to fix it and right now docker volumes for me fixes the issue and is probably better overall in terms of reliability for docker based apps. It also happens on all three of the commonly used docker containers available (linuxserver, hotio, official plex)

 

Samba doesn't have the same issues but has illegal characters that cause problems in Plex.

 

Overall probably not an unRaid issue and Plex probably won't help because their support forums are just like shouting into the dark in terms of official support.

Edited February 4, 2023 by AshranPewter

[AshranPewter]

Ok, I think last time I'll post about this but hopefully the following helps for someone that doesn't want to go down the Docker Volumes route. Docker Volumes fixed it and I'm fine using docker nfs volumes as it's probably the easier way in the end.

 

That being said and I haven't tested this because, it's working so I'm not gonna break it.

 

But the mounts I had that weren't working with plex:

Quote

Flags: rw,noatime,vers=4.2,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.x.x,local_lock=none,

addr=192.168.x.x

 

The Docker Mounts that are working with plex from docker:

Quote

Flags: rw,relatime,vers=4.2,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.x.x,local_lock=none

,addr=192.168.x.x

 

The only difference looks to be the noatime vs relatime in the nfs mount. So it's possible this could fix my mounts and how plex uses them. But also docker could be treating docker volumes in a different way to account for storage being over the network compared to bind mounts. So take the above as you will, haven't tested it but thought it might be useful for anyone running into this issue.

 

Cheers

[dnichols]

On 2/10/2023 at 6:44 AM, AshranPewter said:

Ok, I think last time I'll post about this but hopefully the following helps for someone that doesn't want to go down the Docker Volumes route. Docker Volumes fixed it and I'm fine using docker nfs volumes as it's probably the easier way in the end.

 

That being said and I haven't tested this because, it's working so I'm not gonna break it.

 

But the mounts I had that weren't working with plex:

 

The Docker Mounts that are working with plex from docker:

 

The only difference looks to be the noatime vs relatime in the nfs mount. So it's possible this could fix my mounts and how plex uses them. But also docker could be treating docker volumes in a different way to account for storage being over the network compared to bind mounts. So take the above as you will, haven't tested it but thought it might be useful for anyone running into this issue.

 

Cheers

 

I'm getting stale file handle errors after the mover runs (scheduled at 03:40) affecting Frigate running under docker. Unraid is v6.12.6.

 

2024-01-05 03:46:17.613617144 OSError: [Errno 116] Stale file handle: '/media/../clips'

 

Looking at your mount options I am mounting with soft whereas you are using hard mount so I'll try changing that and see if the issue persists.

 

/var/lib/docker/volumes/frigate_clips_nfs/_data from :/mnt/user/../clips
 Flags: rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.x.x,local_lock=none,addr=192.168.x.x

 

# nfsstat -m
/var/lib/docker/volumes/frigate_clips_nfs/_data from :/mnt/user/../clips
 Flags: rw,relatime,vers=4.0,rsize=1048576,wsize=1048576,namlen=255,soft,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=192.168.x.x,local_lock=none,addr=192.168.x.x

[Ben64]

I'm getting stale file handles also. Mover doesn't even have to run, just something has to change in a directory, delete a file, create a file sometimes and bam, have to remount in order to use it. 

[dlandon]

7 hours ago, Ben64 said:

I'm getting stale file handles also. Mover doesn't even have to run, just something has to change in a directory, delete a file, create a file sometimes and bam, have to remount in order to use it. 

Give more information.  Post diagnpstics.