luca Posted January 4, 2010 Share Posted January 4, 2010 Sometime when copying large group of (small) files from my workstation onto the server (even if the transfer is server to server), I get the following Windows error: "Too many files are currently in use. Quit one or more programs, then try again". If I click on "try again" the copy goes through, but it's likely to give the same error with a different file a while later. This problem only seem to happen when I copy several small files. The fact that this behavior may happen without anything substantial running in the (windows) background, makes me think that this is not a matter of a lack resources, as the error message implies. I am running windows 7 Ult. x64. The unraid server is 4.5 final, but I remember seeing this error with some of the pre 4.5 betas. Event viewer does not report this problem. Has anyone encountered this, and is there anything I can look at on the server side? Link to comment
starcat Posted January 4, 2010 Share Posted January 4, 2010 You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like # rsync -av --stats --progress fromDirectory toDirectory Link to comment
purko Posted January 4, 2010 Share Posted January 4, 2010 ...even if the transfer is server to server... For server to server, it's easiest to telnet into your server and use the mc command. I am running windows 7 Ult. x64. Go shoot some microsoft programmers and you'll feel much better. Link to comment
abeta Posted January 4, 2010 Share Posted January 4, 2010 I saw this problem today copying from the unRAID drive to my desktop machine. It wasn't server to server and I'm on Win7 64Bit as well. Link to comment
starcat Posted January 4, 2010 Share Posted January 4, 2010 Must be something windows related as I am using 4.5 myself, straight from first avail and never had this issue. However, I am not using windows at all. Sorry, can't be of more help! Link to comment
luca Posted January 4, 2010 Author Share Posted January 4, 2010 You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like # rsync -av --stats --progress fromDirectory toDirectory I am not familiar with linux, my bad. I will try to use rsync next time. BTW, why rsynch and not cp or mv? However, this problem happens copying from windows as well. Yes, I know, ditch Windows... I realize that this is windows related, OTOH I get this error only when I am using the unraid server. Link to comment
abeta Posted January 4, 2010 Share Posted January 4, 2010 You copy server to server through your workstation? Not using rsync directly on your unRAID box? Something like # rsync -av --stats --progress fromDirectory toDirectory I am not familiar with linux, my bad. I will try to use rsync next time. BTW, why rsynch and not cp or mv? However, this problem happens copying from windows as well. Yes, I know, ditch Windows... I realize that this is windows related, OTOH I get this error only when I am using the unraid server. Same. I have this problem copying to/from the unRAID only. If I copy the same subdirectory to my Drobo unit I don't have this problem. Link to comment
starcat Posted January 4, 2010 Share Posted January 4, 2010 Is the Drobo connected via Network or directly via USB/Firewire? Link to comment
abeta Posted January 4, 2010 Share Posted January 4, 2010 Is the Drobo connected via Network or directly via USB/Firewire? Droboshare (Ethernet) and uses Samba too. Link to comment
Kaygee Posted January 4, 2010 Share Posted January 4, 2010 [bug 462172] Re: samba "Too many files are currently in use." Bug Watch Updater Wed, 30 Dec 2009 17:41:00 -0800 ** Changed in: samba Status: In Progress => Fix Released Link to comment
abeta Posted January 4, 2010 Share Posted January 4, 2010 [bug 462172] Re: samba "Too many files are currently in use." Bug Watch Updater Wed, 30 Dec 2009 17:41:00 -0800 ** Changed in: samba Status: In Progress => Fix Released Pardon me, but where do you go to look for these bugs and where do I go to get the fix? Thx! Link to comment
Joe L. Posted January 4, 2010 Share Posted January 4, 2010 [bug 462172] Re: samba "Too many files are currently in use." Bug Watch Updater Wed, 30 Dec 2009 17:41:00 -0800 ** Changed in: samba Status: In Progress => Fix Released Pardon me, but where do you go to look for these bugs and where do I go to get the fix? Thx! You might try creating a file named smb-extra.conf and in it, put this [global] max open files = 20000 Put that smb-extra.conf file in the config folder on your flash drive. If there is already a file there, add those two lines to it. Then type smbcontrol smbd reload-config to have it read the new contents of the file you created. Apparently, there is a limit to the number of open files in SAMBA, and files are not being closed in a given session, so it runs out. Upping the limit might be a work-around. Linux itself has its own limit, you can see it with sysctl fs.file-max You can set it with sysctl fs.file-max=50000 Don't get too crazy... you can run out of other resources if too many files are kept open. But the odds are you'll be able to get past the file copy you were attempting until the newer version of unRAID, with a newer version of SAMBA, can be deployed. This bug is described here (with the work-around): https://bugs.launchpad.net/samba/+bug/462172 Joe L. edit: according to the fix, the old limit was 10,000 open files. Win7 needs 16,384 + a few extra (20 in the patch) So.. setting the max open files to 16500 should handle it. Actual patch is here: https://bugzilla.samba.org/attachment.cgi?id=5098&action=edit Link to comment
abeta Posted January 5, 2010 Share Posted January 5, 2010 My output is greater than your command of 50000, did you mean to suggest 500000? I didn't set that value, it was what was already there root@unRAID:/boot/config# sysctl fs.file-max fs.file-max = 346736 root@unRAID:/boot/config# I'm still getting errors after putting in the smb-extra.conf file you suggested and the other commands. I tried renaming the file to smb.conf since google suggested smbcontrol smbd reload-config would try to read the smb.conf file. I also bumped the max open files up to 50000 without any success. Link to comment
BRiT Posted January 5, 2010 Share Posted January 5, 2010 abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.: [global] # configurable identification include = /etc/samba/smb-names.conf << ... snip ... snip ... >> # hook for user-defined samba config include = /etc/samba/smb-extra.conf # auto-configured shares include = /etc/samba/smb-shares.conf Link to comment
abeta Posted January 5, 2010 Share Posted January 5, 2010 abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.: [global] # configurable identification include = /etc/samba/smb-names.conf << ... snip ... snip ... >> # hook for user-defined samba config include = /etc/samba/smb-extra.conf # auto-configured shares include = /etc/samba/smb-shares.conf Doh! My apologies. I'll put it back to smb-extra.conf as indicated but it still doesn't seem to like it for me. Link to comment
luca Posted January 5, 2010 Author Share Posted January 5, 2010 You might try creating a file named smb-extra.conf and in it, put this [global] max open files = 20000 Put that smb-extra.conf file in the config folder on your flash drive. If there is already a file there, add those two lines to it. Then type smbcontrol smbd reload-config to have it read the new contents of the file you created. Apparently, there is a limit to the number of open files in SAMBA, and files are not being closed in a given session, so it runs out. Upping the limit might be a work-around. Linux itself has its own limit, you can see it with sysctl fs.file-max You can set it with sysctl fs.file-max=50000 Don't get too crazy... you can run out of other resources if too many files are kept open. But the odds are you'll be able to get past the file copy you were attempting until the newer version of unRAID, with a newer version of SAMBA, can be deployed. Thanks guys. I applied the fix Joe L described; hope that does it. This fs.file-max business reminds me of the FILES= command in DOS' config.sys... Link to comment
Joe L. Posted January 5, 2010 Share Posted January 5, 2010 abeta, you should have already had a file names /etc/samba/smb.conf. It is generated by unRAID upon boot. It should have some lines in it like the following, which specifies to use additional configuration file snippets.: [global] # configurable identification include = /etc/samba/smb-names.conf << ... snip ... snip ... >> # hook for user-defined samba config include = /etc/samba/smb-extra.conf # auto-configured shares include = /etc/samba/smb-shares.conf Doh! My apologies. I'll put it back to smb-extra.conf as indicated but it still doesn't seem to like it for me. You must edit it using an editor that DOES NOT put carriage returns at the ends of the lines. also, the file name that unRAID "includes" from under /boot/config changed at some point. If using 4.5 unRAID, the file is /boot/config/smb-extra.conf After adding the lines to the file, you must either stop and re-start the array, OR issue the smbcontrol smbd reload-config command to have it re-read the new values. To see if it was entered correctly, type testparm -s | head -20 and it will print all the global parameters. You should see the "max open files" parameter in the listing. Joe L. Link to comment
abeta Posted January 5, 2010 Share Posted January 5, 2010 I used mcedit to edit the file. It looks like it is correct with the max open files line but I'm still getting errors when I copy that subdirectory. See cut/paste: Loaded services file OK. Server role: ROLE_STANDALONE [global] workgroup = MSHOME server string = Media server security = SHARE null passwords = Yes passdb backend = smbpasswd guest account = root syslog = 0 syslog only = Yes unix extensions = No max open files = 20000 load printers = No printcap name = /dev/null disable spoolss = Yes show add printer wizard = No local master = No force user = root create mask = 0711 directory mask = 0711 guest only = Yes root@unRAID:/boot/config# ETA: I bumped it up to 100000 to see if that mattered, I still get errors. Link to comment
luca Posted January 5, 2010 Author Share Posted January 5, 2010 Just to report that the max files does not seem to help matters. The problem occurs with the same frequency as before. Link to comment
purko Posted January 5, 2010 Share Posted January 5, 2010 Put that smb-extra.conf file in the config folder on your flash drive. If there is already a file there, add those two lines to it. Then type smbcontrol smbd reload-config to have it read the new contents of the file you created. I've noticed that I don't really need to issue the `smbcontrol smbd reload-config` command. Looks like unRAID's samba reloads its settings every time the smb-extra.conf changes. Link to comment
chopeta Posted January 13, 2010 Share Posted January 13, 2010 I was having this same issue with my Win7 x64 + UnRAID 4.5 I created the smb-extra.conf file with the params. mentioned here, but the issue was still ocurring... Then I executed this command: root@Tower:~# ulimit -n 50000 root@Tower:~# ulimit -n 50000 root@Tower:~# And now I am not having issues anymore... looks like it fixed the problem. Another solution would be use rsync from inside the UnRAID, find bellow a example that I used before (assuming your copying to/from a Windows share already mounted inside your UnRAID under /mnt/winshare) # rsync --numeric-ids --perms --owner --group -D --times --block-size=2048 --recursive --checksum-seed=32761 --progress -v --modify-window=1 /mnt/winshare/* /mnt/disk1/ I normally do my copies using Windows Explorer and then run this rsync command to check that everything was really transfered. Windows copy seems to be faster then rsync to direct copy my files... I don't know why. Regards, chopeta Link to comment
purko Posted January 13, 2010 Share Posted January 13, 2010 Then I executed this command: root@Tower:~# ulimit -n 50000 root@Tower:~# ulimit -n 50000 root@Tower:~# And now I am not having issues anymore... looks like it fixed the problem. Executing that command does not accomplish anything globally: ulimit is used to set the resource usage limits of the shell. Per shell. To see what I am talking about, just after you've executed your command in your shell, open another shell (another telnet session) and see what it says when you type ulimit -n In the new session: root@Tower:~# ulimit -n 1024 It doesn't have anything to do with samba. Link to comment
purko Posted January 14, 2010 Share Posted January 14, 2010 I normally do my copies using Windows Explorer and then run this rsync command to check that everything was really transfered. If that is the case, then do yourself a favor and get TeraCopy. You can set it up to do CRC checks after it finishes all copying. Not to mention that it does indeed copy stuff much faster than windows explorer. And, it also has nice speed indicators. Link to comment
chopeta Posted January 14, 2010 Share Posted January 14, 2010 Executing that command does not accomplish anything globally: ulimit is used to set the resource usage limits of the shell. Per shell. To see what I am talking about, just after you've executed your command in your shell, open another shell (another telnet session) and see what it says when you type ulimit -n In the new session: root@Tower:~# ulimit -n 1024 It doesn't have anything to do with samba. All right, sorry for possibly create confusion. I will test TeraCopy right now. Thanks for the tip! Regards, chopeta Link to comment
tyrindor Posted January 21, 2010 Share Posted January 21, 2010 I'm having the same issues... no fix yet... Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.