Help with NFS and SMB on the same Windows 11 workstation...

[ibbanez]

Hello guys, I've been a user for several years.  Just recently, I figured out how to setup NFS on Windows 11. I always used SMB, and never had much issue, other than speed issues, and using the Server name, versus IP address, and ultimately using IP address as it was more stable.  However, I found out about NFS, and set it up, and wanted to test them side by side.  

So I started using NFS, and started download some ROMs and moving things around, and everything seemed fine.  In some instances, NFS seemed faster, in other, SMB seemed faster.  However, after a few days of using NFS, moving a bunch of files around, one thing I started noticing was that the NFS share, everything was there and showing up, however on the SMB share, drastically less files are showing up in the SMB share.  

Why is this happening, or better yet, what is happening?  If I view the files in Krusader or DoubleCommander, they are all there.  But still when viewing on Windows through the SMB share, they just don't show up.  Some examples:

NFS                                                      SMB
Roms - 643 files                                    Roms - 47 files

[Frank1940]

You have not really provide us with much information.  However, I want to point out one thing.  Linux is case sensitive and Windows is not case sensitive.  As an example, Test.txt and test.txt are two files to Linux BUT they are only one file to Windows!   Which one will be seen is a largely a matter of chance.  (It appears that the one 'discovered' first is the one that will be seen and used.)  Oh, and complete directories can go missing for the same reason.

 

If this is not the issue, please provide some screenshots of a common directory showing what files are missing in Windows compared to Linux.

Edited June 7, 2022 by Frank1940

[ibbanez]

Sorry, I wasn't really sure what data I would need to provide in this case, as I tried searching for a similar issue, but maybe I just couldn't find the right search terms.  One sec and I will add screen shots of what I'm referring to.

[ibbanez]

Here are pics of what each see's...

Here is NFS

Here is SMB

 

[Frank1940]

I don't see anything obivious.  Let's look at things from the Linux viewpoint.

 

Open up the Unraid Terminal.  (It is on the GUI toolbar at the top right with the     >_       icon.)  Type the following command:

ls -al /mnt/user

 

You will see a list of your shares.  (Forgive me if you are familiar with command line usage at this point...) push the <up-arrow> key and that line will reappear.  Add a forward-slash,    '/ '    , and the name of the share in question.  (If the share name contains a space, the space must be proceed by a back-slash   '\'   .)   Continue down the tree until you get the the directory that you were the screen shots in your last post.   Make another screen shot,  you should get something like this:

 

Please note that there are a mixture of capital and small letters in the share name (Folder_Tree).  This is because Linux is case sensitive!

[ibbanez]

Here you go....  Don't worry about sharing the cmd line stuff.  while I've used it before, I don't live in linux, so I don't have these things memorized, and Im not above taking some help  This only started once I added NFS and tried it out. I didn't notice anything at first, but its almost after I started moving things under NFS, that SMB stopped seeing the changes.  Here is the screen shot of the ls -al /mnt/user and ls -al "/mnt/user/Emulation/Roms/Nintendo 3DS/Decrypted 3DS Games" ....  Now side note, I did recently upgrade to 6.10.1 and then 6.10.2 recently, and did have some issues with some dockers and permissions, but in searching, it seemed like several people ran into that issue as well.  But I did upgrade about a week ago.  Just throwing that out there in case it may help. I have run the docker safe perms recently to see if that helped as I have to run that from time to time.

First is ls -al /mnt/user


Now the next command is 9 pages, but here you go.









 

[ibbanez]

I also don't know if any other folders are affected. I haven't run into them. I have so much data and subfolders, it would take a while to go through each one. I spot checked a few, and they seem fine. It really feels like things that Ive moved while on NFS aren't showing up on the SMB for some reason.  But, I've also added TV shows and movies since NFS and they appear to be showing, but I'm going to investigate a little more thoroughly now that you're helping to see if there are more folders, than this one.  The only reason I caught it was I was testing, and happening to be messing with this roms and moving things around and decrypting all these from my encrypted copies from my 3ds install, and testing out Citra.

[ibbanez]

OK I just found something interesting, and maybe can point smarter people than me in the right direction. It appears the SMB share is only showing the roms from Disk4, instead of all of them, in this folder only.  Because the other folders display fine under SMB.  Does that help?

 

[Frank1940]

Look at it from the disk perspective

 

Look at this command:

 

In the command, replace     /user/     with /diskX/    where the X is the disk number (e.i., 1,2,3...) and you want to look at these directories on each disk---  "Emulation", "Roms"  and "Nintendo 3DS".  Not all of your disks may have the complete path depending on whether the disk is being required to store files on it.

 

I am suspecting that there is some permission/owner/group issue that was created with how you set up your NFS access parameters.  That is what you need to be looking for as go through the directory tree on each of your data disks.  (For example, the user 'root' is not allow as a SMB user!)

 

I am going to make a comment at this point, there are not many Unraid users who are using NFS with Windows.  (As I remember it, it is only available with Windows PRO and up.  I believe it was added only because there are a few UNIX/Linux NFS environments where Windows computers needed it to be compatible.) 

 

As an aside, SMB performance was inferior to NFS many years back.  (Network overhead was much, much higher for SMB than for NFS.) Today, there is not a lot of difference in performance.  99.9% of the Windows world uses SMB. I understand Apple is moving to SMB.  Linux has had SMB capability for years now and I suspect it now comes preinstalled on most distributions. 

[ibbanez]

So funnily enough, I liked SMB performance better, but the problem I ran into was after I removed the NFS drive, the files I've been working on, disappeared. So currently, Im keeping NFS on as it allows me to access all the files in my share through my Windows machine. I only did this as a test and I was curious how it works to be honest.  As far as setting anything up, on the UnRaid side, I literally just enabled it, and exported each Share, just like I did for SMB.  On the windows side, you have to enable the feature, and do a couple things, but other than there, there wasn't any "setup" per se, or permissions to do.  Everything just worked once I turned it on and enabled in Windows.  The only other thing I did was create a batch script to auto mount the drive when Windows starts.  And yes, I run Windows 11 Pro.  I do play around with Linux, but haven't ran a daily driver in many years, as I now fully work in Windows/Mac environment for work and my business. 

 

How could I check permissions and fix it?  Did you see my other post that after looking deeper, it appears to only show the files from Disk 4? So also under this Share, this appears to be the only folder selectively not showing everything.  There are other folders in the same //Emulation/Roms/Nintendo 3DS/ folder tree that are showing everything, just this 1 singular folder that is having an issue.  I don't know enough about how to fix that, without moving everything to a temp folder, deleting the offending folder, and renaming a new one and move everything back and see if it fixes it.

[Frank1940]

First, I would run the New Permissions tools (   Tools   >>>  New Permissions  ) on the 'Emulation' share and see if that fixes it.  (For some reason, I thought you had already run it but, looking back, I could not find that you had.) 

[ibbanez]

Hey, yeah I actually ran that after the docker errors I got after going to 6.10.1, but I didn't type that I don't think. What I did mention I tried was the docker safe perms in tools, I did run that, because usually with SMB, if I run into issues with permissions to copy/delete/rename, etc, that will usually solve it, but I will try the New Permissions tool now...Thanks, and I will let you know how it turns out.  Thanks for taking time and trying to help me at least, I do greatly appreciate your help.

[ibbanez]

OK, so I ran the New Permissions tool as well as the Docker Safe Perms again, and neither worked.  For some reason, any SMB connection to this one folder, is only showing the files found on drive 4, but viewing the share from NFS, or even inside a file manager from UnRaid, shows all the files, its only this SMB connection from this one folder as far as I can tell. Im stumped.  I almost want to just create another folder, move everything to it, delete the folder, recreate it, move everything back, and check again, but at the same time, I want to find the cause and a solution for this, because stuff like this bugs me and not knowing why....

[ibbanez]

OK, so I tried making a new folder under Emulation called Temp, and copied them into there from  Unraid, not the SMB connection, and now shows 97 files, but not the full 643, wth.  I don't understand why SMB is only showing certain drives files'.  This is frustrating.  I don't understand how a brand new folder I just created is now experiencing this.  Any thoughts?

[Frank1940]

22 hours ago, Frank1940 said:

n the command, replace     /user/     with /diskX/    where the X is the disk number (e.i., 1,2,3...) and you want to look at these directories on each disk---  "Emulation", "Roms"  and "Nintendo 3DS".  Not all of your disks may have the complete path depending on whether the disk is being required to store files on it.

 

I am suspecting that there is some permission/owner/group issue that was created with how you set up your NFS access parameters.  That is what you need to be looking for as go through the directory tree on each of your data disks.  (For example, the user 'root' is not allow as a SMB user!)

 

 

You need to go through this procedure and look at the owner, group and the permissions for each of the directories in the path to the directory with the problem.  I am suspecting that there is an problem with one of them. 

[ibbanez]

OK, I may need some help here. I went through each disk and found the new temp folder I created, and which one housed all the roms.  Interesting, every folder I normally have  shows the owner and permissions as "nobody     drwxrwxrwx"  Full read/write on Owner, Group, Other....  However, all of these specific folders were showing "nobody    drwxr-xr-x" and then read as Owner: Read/Write    Group: Read-Only    Other: Read-Only.  So I selected the folder in question that was messed up, and then edited the permissions to all be Read/Write like the others.  The folder then showed as "nobody     drwxrwxrwx", but the files inside show as "nobody     -rw-rw-rw-", which is how all my other folders show up, that are fine. 
What is odd is even creating a new folder and moving the files to it, while about twice as many showed up, still a fraction.  So now 96 show up, versus 643 or whatever the whole number is.  And that was from creating a new folder.  After changing the permissions on the files/folders on each disk individually, nothing changed.  I am so confused as this has never happened in the many years I've used UnRaid, and I'm just lost as to why and how to fix it.  If the permissions all match what all my other files/folders now say, I don't get why they don't show up now.

[Frank1940]

What was the name of the Group?  It should always be users.

 

Another thing.  Post up the command string that you used to setup the NFS connection to your Unraid server from your Windows computer. 

 

All directories (They begin with a 'd' to indicate that they are not a file but a directory) should have the execute bit (the 'x' in the permissions) set.  If they don't, they are not browse-able and weird stuff will happen.

[ibbanez]

I was simply looking through the GUI in Unraid, it never shows the group name.  I'll post a screenshot of what I was seeing.  Is there a cmd or way I can find out? I'm sure there is, I'm just unfamiliar. 

Now regarding the cmd.  I can simply open up the folder and \\192.168.1.135\mnt\user, but what I found online about setting this up was to create a batch script and use this.
mount -o anon \\192.168.1.135\mnt\user\ Z: ....  I just would have thought if a permission issue running the New Permission tool and the New Docker Perms tool would have resolved, or doing them manually through the GUI, but I'm a total Linux noob, so who knows.
 

 

[Frank1940]

I was giving you instructions to look at things using the command line in the GUI terminal program that you used earlier.   You really can't depend on seeing the actual settings in Linux unless you are at the command line level.  Most of the higher level tools conceal/(filter) certain information for simplicity. 

 

11 minutes ago, ibbanez said:

I just would have thought if a permission issue running the New Permission tool and the New Docker Perms tool would have resolved, or doing them manually through the GUI, but I'm a total Linux noob, so who knows.

 

I would have thought so too but at this point, I am not willing to assume anything!

 

@JorgeB, @bonienl do you have any thoughts?  Or do you know of someone who is knowledgeable about NFS who might be able to help?    

[ibbanez]

ok, all the files are in the group users

[ibbanez]

sorry, im bouncing between searching and looking at stuff.  Sorry, I didn't realize all the users was the group when I ran the ls -al /mnt/user/Emulation/temp cmd, they all belong to the users group, like the should.

[ibbanez]

I also tried turning NFS off on UnRaid to see if somehow it would revert, or was interfering, but it didn't help.  now I kinda wish I hadn't thought up testing this to begin with 😜 

[ibbanez]

instead of posting 9 screenshots, I just piped the command to a txt folder if that would help.

 

lscommand.txt

Edited June 9, 2022 by ibbanez

[ibbanez]

What is so weird is that UnRaid see's everything fine.  When viewing the share through a file manager, or docker file manager, it reads all the files in the right places, just SMB for whatever reason only reading from a few drives.  I even double checked the settings, and made sure anything obscure was off.  I had some stuff in the extra config in case I ran MacOS for compatibility, but I commented all that out to just check.

SMB Settings

Enable SMB: Yes(Workgroup)
Hide "dot" files: No
Enable SMB Multi Channel: No
Enhanced macOS interoperability: Yes
Enable NetBIOS: No
Enable WSD: Yes
WSD options [experimental]:

So my SMB extra config is this:

[rootshare]
     path = /mnt/user
     comment = 
     browseable = yes
        valid users = ibbanez
        valid list = ibbanez
#vfs objects = catia fruit streams_xattr
#unassigned_devices_start
#Unassigned devices share includes
       include = /tmp/unassigned.devices/smb-settings.cong
#unassigned_devices_end

        

Edited June 9, 2022 by ibbanez
making neater

[JorgeB]

6 hours ago, Frank1940 said:

do you have any thoughts?

Can't really help with NFS as I never used it, as for the SMB issue, did you confirm permissions were OK for these?

 

On 6/7/2022 at 10:14 PM, Frank1940 said:

In the command, replace     /user/     with /diskX/    where the X is the disk number (e.i., 1,2,3...) and you want to look at these directories on each disk---  "Emulation", "Roms"  and "Nintendo 3DS".  Not all of your disks may have the complete path depending on whether the disk is being required to store files on it.