Moving large amount of data, some issue..

[je82]

Hi,

 

I've started migrating my data over to unraid and i've came across some weird issue.

 

I've setup FreeFileSync to simply mirror a path locally to a path on the unraid setup via a smb share.

 

Mirror d:/mystuff1/ > //unraid/mystuff1/

 

For each file FreeFileSync writes a filename.randomhash.tmp first and starts building the file, then once the file is build it checks crc and then renames the file to filename.extension

 

Sometimes during this renaming process the file becomes inaccessible and freefilesync tells me it has no permission to rename the file. This does not occur for every file and appears to be random.

 

The SMB share has no caching enabled, and i have enabled "CA Turbo Mode" plugin for faster write speeds.

 

Any ideas?

 

Thanks.

Edited November 4, 2019 by je82

[itimpi]

I had a similar problem at one point, and it turned out to be an issue with the network card at the client end sometimes resetting itself under load.

 

having said that is could be a good idea to post your system diagnostics zip file (obtained via Tools >> Diagnostics) covering a period when you have had this error to see if anyone can spot an issue at the Unraid end.

Edited November 4, 2019 by itimpi

[je82]

28 minutes ago, itimpi said:

I had a similar problem at one point, and it turned out to be an issue with the network card at the client end sometimes resetting itself under load.

 

having said that is could be a good idea to post your system diagnostics zip file (obtained via Tools >> Diagnostics) covering a period when you have had this error to see if anyone can spot an issue at the Unraid end.

Alright, i though there might be something at the unraid end that checks for signal that written file has been completed and it want to apply some meta to the file and the file gets locked by the unraid system for a very short period of time and sometimes ffs is so quick it tries to rename the file at the exact same time as unraid is applying the meta resulting in file being locked and permission denied.

 

I have not noticed any network drops from neither unraid or the server sending the files and it seems to only occur when using ffs that has this thing where it writes a temp file first then renames it. It's not a big problem, i've setup ffs to retry if it occurs.

Edited November 4, 2019 by je82

[je82]

After more checking it's definitely not a network or connectivity issue, i setup a simple ping to run while it's running the filesync and ping never looses any packages but it still is occurs. I believe the issue is with the samba protocol.

 

I SSH'd into the unraid install and went to /var/log/samba/ and checked the logs but they appear empty, where can i find logging for samba?

 

Thanks