I'm copying 10 TB of media files to my new Unraid server using external USB hard drives.  I think that the Samsung SSD 1TB cache drive is in the way of the process.  The SSD heats up and has stopped the server at least once and therefore I am only copying the files in small batches - also because the cache fills up and stops the process.  And now I'm reading that I may be wearing out my brand new SSD.  Can I turn off the cache and copy directly to the array?  Right now, my SSD has 644 GB used and 355 GB free and I'm wondering if it will ever finish writing to the array - It is going very slowly.  

There is no requirement to cache anything, and caching is NOT recommended if you are loading more than cache can hold at one time.

 

Attach diagnostics to your NEXT post in this thread for more detailed recommendations.

Turn cache off or if doing it on the machine itself, copy direct to /mnt/user0/...

 

Since my cache is currently showing 866 GB Free and 132 GB used, how can I make sure that my array is updated?  The cache is not changing and I'd like to bypass it to finish copying over all of my media files.
Also, I'm still not sure how to bypass the cache.  Do I somehow just shut it off while doing the copying?  I've been using Krusader and copying to /mnt/user/Media Library.

 

 

embyserver-diagnostics-20220106-1136.zip

2 hours ago, dcruff said:

Also, I'm still not sure how to bypass the cache.  Do I somehow just shut it off while doing the copying?  I've been using Krusader and copying to /mnt/user/Media Library.

 

In the GUI click on the shares tab, select your Media Library share, and set cache:no

 

I believe you may need to stop and start the array to get the setting to stick because of a current bug.

14 minutes ago, JonathanM said:

select your Media Library share, and set cache:no

That will bypass cache for all new files. But Mover ignores cache-no shares so won't move any of the existing files from cache until you set it back to cache-yes.

Thank you.  That looks like it will work.  But I'm still concerned about the data in the cache that doesn't seem to want to move.  I've got 12 folders (Media files) that seem stuck in the cache.  Right now there is 57.8 GB used and the Reads and Writes counters are not moving.  I select the MOVE button on the bottom and it greys out for a while but does not seem to change anything in the cache.

Maybe I'll just leave it be for a while and shut off caching while I copy the remainder of my media files to the array.

Follow up question - It is still really slow copying data from my USB drive to the array without the cache.  Would it speed up if I temporarily shut off the Parity while copying the data?  Right now, it has taken a few hours to copy 200 GB to the array with another 10TB to go.

 

Initial data load should be done without caching, and is often done without parity.

That's what I was thinking.  The parity process must be what's slowing the data load to a crawl.

I'll just disable it for the time being - assuming that I can find the right switch.

 

You have to stop the array and unassign the parity disk.

On 1/6/2022 at 1:26 AM, Shonky said:

Turn cache off or if doing it on the machine itself, copy direct to /mnt/user0/...

 

Are there any potential drawbacks I'm not seeing to using this method to bypass the cache drive for USB transfers? Occasionally I will move large amounts of finished media to the server, and under normal settings this will lead to dumping the data to the mirrored NVMe cache and then later to the HDDs. This seems wasteful, in terms of NVMe i/o, because the cache is serving no real purpose here. 

 

I know I can disable caching for the share, and reenable later, but I'm hesitant to be continually messing with the mover settings like that, especially if they may require an array restart to take effect. So... can I just use the WEBUI filesystem to copy the files directly from my attached SSD drive to the appropriate user0 folder and bypass the cache that way whenever it's necessary? Because that sounds like a far better solution than toggling mover settings every time.

Edited March 12, 20242 yr by Nedry

On 9/7/2023 at 7:04 PM, Nedry said:

can I just use the WEBUI filesystem to copy the files directly from my attached SD drive to the appropriate user0 folder

yes

Similar situation. 

 

SMB copy from Mac OS to Unraid share, flipped off cache mid copy and copy was not interrupted. HTH.

 

Was doing a copy via MC from an NVME to an /mnt share of 250+ GB with a 120GB cache, so it filled it. Aborted that, changed share to not use cache and re-did.

 

Edited September 15, 20232 yr by RealActorRob