May 31, 20224 yr I have just move my database backup to my UNRAID server, on to the disk array. The database has pictures that are linked to the database and stored in a folder next to the database files. Over time the picture collection has grown to 29,900 images in one folder. Viewing the server and folder via the Web browser I can view the 29,900 files, it takes a few moments for the window to show all the images and Krusader happily moves the folder from cache pool to the array. The problem starts when I try and view the folder from my Mac running OS 12.4 (This also happens with OS 12.3.1) via a SMB share. When opening the folder it hangs with a loading symbol but never completes to show the 29,900 images, I can close the window but the finder than crashes and I have to restart my computer. I am currently running UNRAID Version: 6.10.2 but this also happened under Version: 6.10.1. I have only just moved this large folder to UNRAID so unsure about previous versions. Also I moved the folder to a ssd cache pool and made no difference. All the other shares on the cache pool or array work at speed but do not have the quantity of files that the database contains, Probably a 1,000 files at the most. Is there a maximum number of files you can have in a folder with a SMB share? I have already added the following code to the SMB Extras under setting to get the search, copy to and from working. #unassigned_devices_start #Unassigned devices share includes include = /tmp/unassigned.devices/smb-settings.conf #unassigned_devices_end [global] vfs objects = catia fruit streams_xattr fruit:nfs_aces = no fruit:zero_file_id = yes fruit:metadata = stream fruit:encoding = native spotlight backend = tracker [share name] path = /mnt/user/share name veto files = /._*/.DS_Store/ delete veto files = yes spotlight = yes Even in Version: 6.10.1. I need the SMB Extras code to get the search working. Edited May 31, 20224 yr by wow001 add image showing the total number of files
May 31, 20224 yr Author Just check the UNRAID Version: 6.10.2. The SMB Extras code is still need to get the search working in MAC OS, but no longer required for copying the files to or from the server.
May 31, 20224 yr Since Day 1, more than ten years ago, unraid has never handled huge amounts of files well. I have massive collections of music, tv show episodes, and family pictures spanning a century. Unraid has performed slowly when accessing them since the very beginning.
May 31, 20224 yr Author That not good news. I thought the job of a server was to handle large data sets with easy. Are you accessing the data via SMB on Mac or Windows?
May 31, 20224 yr You might also want to try installing Dynamix Folder Caching to help things out a a bit
May 31, 20224 yr Windows. Browsing directories with massive amounts of files, or massive nested directories, or both, has always been an issue.
May 31, 20224 yr Author 9 minutes ago, Squid said: You might also want to try installing Dynamix Folder Caching to help things out a a bit Could not find "Dynamix Folder Caching" did you mean "Dynamix Cache Dirs"
May 31, 20224 yr Just now, tucansam said: I just downloaded it as well. Gonna just go with defaults? NO. Only enable it on often used problem shares. Enabling it on everything will likely result in worse performance, and drives that never spin down. It works by reading the directory listing at frequent intervals, so that when a listing is asked for, it will likely be in RAM. If you ask it to read too many items, they won't all stay in RAM, and reading them again will keep the drive spun up as it's read over and over again.
May 31, 20224 yr 8 minutes ago, tucansam said: I just downloaded it as well. Gonna just go with defaults? Only enable it on the shares that need to be cached. It has a bad habit of caching a lot more that you want, like appdata and system shares that are normally on an SSD and is a waste of resources to cache.
May 31, 20224 yr Author Thank you for the information I have now "include" just the one share that contains the 29,000 images, will check back in the morning and see how it goes. As you say it only this share that is a problem, all my other shares work at speed. The folder with 29,000 images is not at the top level of the share and is 4 folders deep but it effects the whole share.
June 1, 20224 yr Community Expert On 5/30/2022 at 8:12 PM, wow001 said: When opening the folder it hangs with a loading symbol but never completes to show the 29,900 images Are you trying to display 29,000 thumbnails? A complete aside: Why do you have 29,000 photos in a single folder/directory? How do you ever find anything in it?
June 1, 20224 yr 18 minutes ago, Frank1940 said: Why do you have 29,000 photos in a single folder/directory? How do you ever find anything in it? Perhaps the joy of random discovery?
June 1, 20224 yr One thing you might also want to consider is changing the SMB setting for the share to be case sensitive (rather than the usual default of case insensitive). On my system this resulted in a significant performance boost when accessing files on the SMB shares. However you can obviously only do that if you access the files with pathnames of the correct case; in your case if the pathnames in the database have been stored with the correct case.
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.