competent-bailout3425 Posted December 3, 2023 Share Posted December 3, 2023 (edited) The speed of traversing the /mnt/user file system is very slow. For instance, in my array, I have a top-level directory ASD, which contains a large number of nested small files. The entire ASD directory is in the cache disk. When I use time find /mnt/cache/ASD -type f, it takes roughly 8 seconds. However, time find /mnt/user/ASD -type f takes 168 seconds, consuming 20 times more time, even though there is no ASD directory under /mnt/user0 at all. I believe that the algorithm in Unraid should cache whether each directory exists on each disk, especially for directories that have been accessed recently. This should at least be an option, even though it might bring about some issues. For instance, writes through /mnt/disk1 might require a delay to be observable through /mnt/user0 and /mnt/user. However, this is acceptable in many scenarios because it can significantly speed up the opening of files and the traversal of the user file system. If such an operation is too aggressive, at least for the top-level directories, acceleration should be applied automatically according to the settings in the share tab. Typically, I would establish a special directory to store some data that would only be stored in the cache (i.e., only primary storage is set in the share). For example, the compilation directory of OpenWRT, which would contain a large number of files stored in a multi-level directory structure. If every level of directory and file access requires check the existence on each array disk, it would waste a lot of unnecessary time. Edited December 3, 2023 by competent-bailout3425 Quote Link to comment
trurl Posted December 3, 2023 Share Posted December 3, 2023 Dynamix Cache Dirs plugin Quote Link to comment
Solution ChatNoir Posted December 3, 2023 Solution Share Posted December 3, 2023 And for shares only on a pool, the Exclusif access share option could be helpful for file access time. Also, this has nothing to do with the forum's config or whatever. Let's move that to general Quote Link to comment
competent-bailout3425 Posted December 4, 2023 Author Share Posted December 4, 2023 12 hours ago, trurl said: Dynamix Cache Dirs plugin This plugin does indeed improve access speed, but it's still quite slow. The command time find /mnt/user/fast -type f|wc -l takes 106s, while time find /mnt/cache/fast -type f|wc -l takes 6.5s. Compared to the original speed(160), this improve a lot, but it's still significantly slow compare to the original 7s. Quote Link to comment
competent-bailout3425 Posted December 4, 2023 Author Share Posted December 4, 2023 Optimizing access through plugins, in my opinion, won't yield very good results. My memory is already quite large, and I'm using ZFS, so I basically don't trigger IO. I believe that the algorithm should internally add a cache to store which files and directories have been recently accessed and on which hard drives they reside. Instead of trying to test whether the directory exists from all hard disks every time, this would greatly improve the access performance of a multi-disk file system. Quote Link to comment
competent-bailout3425 Posted December 4, 2023 Author Share Posted December 4, 2023 12 hours ago, ChatNoir said: And for shares only on a pool, the Exclusif access share option could be helpful for file access time. Also, this has nothing to do with the forum's config or whatever. Let's move that to general what do you mean "Exclusif access"?my share option is already pool only. Quote Link to comment
ChatNoir Posted December 4, 2023 Share Posted December 4, 2023 4 hours ago, competent-bailout3425 said: what do you mean "Exclusif access"? https://docs.unraid.net/unraid-os/release-notes/6.12.0/#exclusive-shares Quote Link to comment
itimpi Posted December 4, 2023 Share Posted December 4, 2023 5 hours ago, competent-bailout3425 said: what do you mean "Exclusif access"?my share option is already pool only. You do need to enable Exclusive Access option under Settings->Global Share settings to get this option enabled for user shares which meet the criteria. Quote Link to comment
competent-bailout3425 Posted December 4, 2023 Author Share Posted December 4, 2023 2 hours ago, itimpi said: You do need to enable Exclusive Access option under Settings->Global Share settings to get this option enabled for user shares which meet the criteria. After I enable the Exclusive Access option Settings->Global Share setting, the Exclusive access option in share tab is still unconfigable. Quote Link to comment
JorgeB Posted December 4, 2023 Share Posted December 4, 2023 1 hour ago, competent-bailout3425 said: After I enable the Exclusive Access option Settings->Global Share setting, the Exclusive access option in share tab is still unconfigable. It's not configurable, it's automatic if the share only exists in a pool, it won't work for shares that exist only (or also) in the array. Quote Link to comment
trurl Posted December 4, 2023 Share Posted December 4, 2023 4 hours ago, competent-bailout3425 said: Exclusive access option in share tab is still unconfigable. Try stopping and starting the array. Quote Link to comment
competent-bailout3425 Posted December 5, 2023 Author Share Posted December 5, 2023 13 hours ago, trurl said: Try stopping and starting the array. that work,thank Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.