dcruff Posted January 6, 2022 Share Posted January 6, 2022 I'm copying 10 TB of media files to my new Unraid server using external USB hard drives. I think that the Samsung SSD 1TB cache drive is in the way of the process. The SSD heats up and has stopped the server at least once and therefore I am only copying the files in small batches - also because the cache fills up and stops the process. And now I'm reading that I may be wearing out my brand new SSD. Can I turn off the cache and copy directly to the array? Right now, my SSD has 644 GB used and 355 GB free and I'm wondering if it will ever finish writing to the array - It is going very slowly. Quote Link to comment
trurl Posted January 6, 2022 Share Posted January 6, 2022 There is no requirement to cache anything, and caching is NOT recommended if you are loading more than cache can hold at one time. Attach diagnostics to your NEXT post in this thread for more detailed recommendations. Quote Link to comment
Shonky Posted January 6, 2022 Share Posted January 6, 2022 Turn cache off or if doing it on the machine itself, copy direct to /mnt/user0/... Quote Link to comment
dcruff Posted January 6, 2022 Author Share Posted January 6, 2022 Since my cache is currently showing 866 GB Free and 132 GB used, how can I make sure that my array is updated? The cache is not changing and I'd like to bypass it to finish copying over all of my media files. Also, I'm still not sure how to bypass the cache. Do I somehow just shut it off while doing the copying? I've been using Krusader and copying to /mnt/user/Media Library. embyserver-diagnostics-20220106-1136.zip Quote Link to comment
JonathanM Posted January 6, 2022 Share Posted January 6, 2022 2 hours ago, dcruff said: Also, I'm still not sure how to bypass the cache. Do I somehow just shut it off while doing the copying? I've been using Krusader and copying to /mnt/user/Media Library. In the GUI click on the shares tab, select your Media Library share, and set cache:no I believe you may need to stop and start the array to get the setting to stick because of a current bug. Quote Link to comment
trurl Posted January 6, 2022 Share Posted January 6, 2022 14 minutes ago, JonathanM said: select your Media Library share, and set cache:no That will bypass cache for all new files. But Mover ignores cache-no shares so won't move any of the existing files from cache until you set it back to cache-yes. Quote Link to comment
dcruff Posted January 6, 2022 Author Share Posted January 6, 2022 Thank you. That looks like it will work. But I'm still concerned about the data in the cache that doesn't seem to want to move. I've got 12 folders (Media files) that seem stuck in the cache. Right now there is 57.8 GB used and the Reads and Writes counters are not moving. I select the MOVE button on the bottom and it greys out for a while but does not seem to change anything in the cache. Maybe I'll just leave it be for a while and shut off caching while I copy the remainder of my media files to the array. Quote Link to comment
dcruff Posted January 7, 2022 Author Share Posted January 7, 2022 Follow up question - It is still really slow copying data from my USB drive to the array without the cache. Would it speed up if I temporarily shut off the Parity while copying the data? Right now, it has taken a few hours to copy 200 GB to the array with another 10TB to go. Quote Link to comment
trurl Posted January 7, 2022 Share Posted January 7, 2022 Initial data load should be done without caching, and is often done without parity. Quote Link to comment
dcruff Posted January 7, 2022 Author Share Posted January 7, 2022 That's what I was thinking. The parity process must be what's slowing the data load to a crawl. I'll just disable it for the time being - assuming that I can find the right switch. Quote Link to comment
trurl Posted January 7, 2022 Share Posted January 7, 2022 You have to stop the array and unassign the parity disk. Quote Link to comment
Nedry Posted September 7, 2023 Share Posted September 7, 2023 (edited) On 1/6/2022 at 1:26 AM, Shonky said: Turn cache off or if doing it on the machine itself, copy direct to /mnt/user0/... Are there any potential drawbacks I'm not seeing to using this method to bypass the cache drive for USB transfers? Occasionally I will move large amounts of finished media to the server, and under normal settings this will lead to dumping the data to the mirrored NVMe cache and then later to the HDDs. This seems wasteful, in terms of NVMe i/o, because the cache is serving no real purpose here. I know I can disable caching for the share, and reenable later, but I'm hesitant to be continually messing with the mover settings like that, especially if they may require an array restart to take effect. So... can I just use the WEBUI filesystem to copy the files directly from my attached SSD drive to the appropriate user0 folder and bypass the cache that way whenever it's necessary? Because that sounds like a far better solution than toggling mover settings every time. Edited March 12 by Nedry Quote Link to comment
trurl Posted September 10, 2023 Share Posted September 10, 2023 On 9/7/2023 at 7:04 PM, Nedry said: can I just use the WEBUI filesystem to copy the files directly from my attached SD drive to the appropriate user0 folder yes 1 Quote Link to comment
RealActorRob Posted September 15, 2023 Share Posted September 15, 2023 (edited) Similar situation. SMB copy from Mac OS to Unraid share, flipped off cache mid copy and copy was not interrupted. HTH. Was doing a copy via MC from an NVME to an /mnt share of 250+ GB with a 120GB cache, so it filled it. Aborted that, changed share to not use cache and re-did. Edited September 15, 2023 by RealActorRob Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.