Can I bypass the cache when copying large amounts of data to the Unraid server?


Recommended Posts

I'm copying 10 TB of media files to my new Unraid server using external USB hard drives.  I think that the Samsung SSD 1TB cache drive is in the way of the process.  The SSD heats up and has stopped the server at least once and therefore I am only copying the files in small batches - also because the cache fills up and stops the process.  And now I'm reading that I may be wearing out my brand new SSD.  Can I turn off the cache and copy directly to the array?  Right now, my SSD has 644 GB used and 355 GB free and I'm wondering if it will ever finish writing to the array - It is going very slowly.  

Link to comment

Since my cache is currently showing 866 GB Free and 132 GB used, how can I make sure that my array is updated?  The cache is not changing and I'd like to bypass it to finish copying over all of my media files.
Also, I'm still not sure how to bypass the cache.  Do I somehow just shut it off while doing the copying?  I've been using Krusader and copying to /mnt/user/Media Library.

 

 

embyserver-diagnostics-20220106-1136.zip

Link to comment
2 hours ago, dcruff said:

Also, I'm still not sure how to bypass the cache.  Do I somehow just shut it off while doing the copying?  I've been using Krusader and copying to /mnt/user/Media Library.

 

In the GUI click on the shares tab, select your Media Library share, and set cache:no

 

I believe you may need to stop and start the array to get the setting to stick because of a current bug.

Link to comment

Thank you.  That looks like it will work.  But I'm still concerned about the data in the cache that doesn't seem to want to move.  I've got 12 folders (Media files) that seem stuck in the cache.  Right now there is 57.8 GB used and the Reads and Writes counters are not moving.  I select the MOVE button on the bottom and it greys out for a while but does not seem to change anything in the cache.

Maybe I'll just leave it be for a while and shut off caching while I copy the remainder of my media files to the array.

Link to comment

Follow up question - It is still really slow copying data from my USB drive to the array without the cache.  Would it speed up if I temporarily shut off the Parity while copying the data?  Right now, it has taken a few hours to copy 200 GB to the array with another 10TB to go.

 

Link to comment
  • 1 year later...
On 1/6/2022 at 1:26 AM, Shonky said:

Turn cache off or if doing it on the machine itself, copy direct to /mnt/user0/...

 

Are there any potential drawbacks I'm not seeing to using this method to bypass the cache drive for USB transfers? Occasionally I will move large amounts of finished media to the server, and under normal settings this will lead to dumping the data to the mirrored NVMe cache and then later to the HDDs. This seems wasteful, in terms of NVMe i/o, because the cache is serving no real purpose here. 

 

I know I can disable caching for the share, and reenable later, but I'm hesitant to be continually messing with the mover settings like that, especially if they may require an array restart to take effect. So... can I just use the WEBUI filesystem to copy the files directly from my attached SSD drive to the appropriate user0 folder and bypass the cache that way whenever it's necessary? Because that sounds like a far better solution than toggling mover settings every time.

Edited by Nedry
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.