stev067 Posted May 13, 2021 Share Posted May 13, 2021 Hey everyone, I hope this is an easy one since I'm new... I'm trying to dump a 30TB collection of 600+ folders each containing files, onto a share using Krusader. It will span multiple disks, and I set the share to split only on top level directory. When I initiate the transfer, it immediately creates all 600+ folders onto the first disk, and then begins to add the files. Since I'm using highwater, once the drive reaches 50% it will switch to writing to the next drive, but I don't want all of these empty folders being left behind. Will it delete those? Am I doing something wrong, or do I just need to clean up empty folders afterwards? Thanks! Quote Link to comment
JorgeB Posted May 13, 2021 Share Posted May 13, 2021 8 minutes ago, stev067 said: When I initiate the transfer, it immediately creates all 600+ folders onto the first disk That's normal with Krusader (and rsync), depending on the source you may be able to use Windows explorer, it won't do that. Quote Link to comment
stev067 Posted May 13, 2021 Author Share Posted May 13, 2021 23 minutes ago, JorgeB said: That's normal with Krusader (and rsync), depending on the source you may be able to use Windows explorer, it won't do that. So I have a windows 10 VM running on this machine. I could pass through this external drive which is the source, and then transfer to the share via windows. And that would solve my problem? Quote Link to comment
FreeMan Posted May 13, 2021 Share Posted May 13, 2021 Unless you're accessing the DISKxx shares directly (and generally you shouldn't be), you'll never notice the empty directories. The UNRAID OS will aggregate the contents of the splits and it'll look seamless to you. I get that simply knowing that there are empty directories there can be annoying, but it won't really impact anything. Krusader will, most likely, copy the files far faster than even Win10 running in a VM, if speed is of any consideration. Quote Link to comment
stev067 Posted May 13, 2021 Author Share Posted May 13, 2021 (edited) 3 hours ago, FreeMan said: Unless you're accessing the DISKxx shares directly (and generally you shouldn't be), you'll never notice the empty directories. The UNRAID OS will aggregate the contents of the splits and it'll look seamless to you. I get that simply knowing that there are empty directories there can be annoying, but it won't really impact anything. Krusader will, most likely, copy the files far faster than even Win10 running in a VM, if speed is of any consideration. Thanks. Another thing I realized though is that after the first drive reaches capacity during this initial write, because all of the folders are on that drive and because of the way my split is set at top level only, it will just stop and say the drive is full. And the reason I want to use the top level only split is to minimize multiple drive spin-ups. I'm just writing in chunks now to try and manually balance the drives for this initial write. But I'm a little disappointed this hasn't been baked into the programming yet! Edited May 13, 2021 by stev067 Quote Link to comment
JonathanM Posted May 13, 2021 Share Posted May 13, 2021 46 minutes ago, stev067 said: I'm a little disappointed this hasn't been baked into the programming yet! What would you change? Quote Link to comment
stev067 Posted May 14, 2021 Author Share Posted May 14, 2021 (edited) 40 minutes ago, jonathanm said: What would you change? I'm not sure if this is a Krusader or Unraid thing, but when writing anything to a share, the entire folder structure of the source shouldn't just be dumped onto the first destination drive at the start. I imagine the folders can be added on a file-by-file basis, mirroring the source's folder structure as the write progresses. That way, when you reach write capacity of the first drive, (or 50% in the case of highwater), the split level rule won't prevent the next-in-line folders from being written to the next-in-line drive, because those empty folders won't have been pre-written to the first drive. Edited May 14, 2021 by stev067 a word Quote Link to comment
JonathanM Posted May 14, 2021 Share Posted May 14, 2021 Unfortunately Unraid has no control over what gets written first. It's totally controlled by the application doing the writing. Some applications work like you want, but unfortunately many don't. Quote Link to comment
stev067 Posted May 14, 2021 Author Share Posted May 14, 2021 1 minute ago, jonathanm said: Unfortunately Unraid has no control over what gets written first. It's totally controlled by the application doing the writing. Some applications work like you want, but unfortunately many don't. Yeah I kinda figured it was a Krusader thing. I'm very new so that is the one I've heard recommended. Can you recommend another that works more like the way I'm describing? Quote Link to comment
JonathanM Posted May 14, 2021 Share Posted May 14, 2021 6 hours ago, JorgeB said: Windows explorer, it won't do that. Quote Link to comment
stev067 Posted May 14, 2021 Author Share Posted May 14, 2021 (edited) 5 hours ago, FreeMan said: Krusader will, most likely, copy the files far faster than even Win10 running in a VM, if speed is of any consideration. Ok and is this true? @jonathanm Edited May 14, 2021 by stev067 Quote Link to comment
JonathanM Posted May 14, 2021 Share Posted May 14, 2021 Depends on the source, destination, and transfer media speed. If you have valid active parity during the transfer that's pretty much the limiting factor. Quote Link to comment
JorgeB Posted May 14, 2021 Share Posted May 14, 2021 If the external drive is mounted with UD you can even use Windows explorer on a networked computer and all the data will still be copied locally on the server, without using the network, at the same speed it would be with Krusader and without creating the empty folders. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.