Best practices for moving data?

 

Issue: Moving 11 TB of data video / photo over to our new UNRAID server.

 

  a. Some folders/files moved by plugging in USB drive into the server and using Krusader Docker App

  b. Some folders/files moved via Network

  c. Some folders/files moved by placing old drive into hot-swap slot directly into the server and copying from it

 

For most I turned off cache drive so it could directly write to the drives as there were so many TB to move.

 

Now that I'm testing data on the UNRAID Server I'm seeing a ton of problems. See attached photos.

 

Does not matter the format of the file they do not open and show as corrupt.

 

I know moving data is not rocket science; drag and  drop and wait a few days for all to be moved so I'm at a loss!

 

Any help would be greatly appreciated. Thanks in advance everyone!

Run memtest for at least a few passes to rule out memory issues

Sent from my NSA monitored device

Thanks Squid.

 

I just noticed something interesting.. The folders/directories that have the issue for some reason are not set to block size 4096 bytes (ext4). They are  random or as it looks. 6 bytes or 28 bytes, I also have a 208 bytes directory..  Here's a image. 

Could it be one of the drives was not formatted properly and the data that it getting copied to it has this random inode. If so, is there a fix?

 

On typical ext4 file system the default inode size is 256 bytes, block size is 4096 bytes.

What strange is that even some of the data in directories with the strange random inode size are working. Any ideas would be awesome...

 

Example: The 80thBay... data within is working and so is the the others that are set to 4096.

 

Edited September 20, 20196 yr by JCristina
added photo

7 minutes ago, JCristina said:

ext4 file system

unRaid uses xfs / btrfs / reiserfs.

yes. The files come from many sources and are being archived on the UNRAID server. Strangely. If I copy "OVERWRITE" over a directory that is not working with the same directory data, the files still fail. But, if I copied the data into another new directory, they work file... I'm lost. 

Sounds like it could be an issue with specific drive(s). Are you using shares that are spread across all your drives?

Yes. I them set to fill 50% and move onto the next and so on. 

Try testing the files drive by drive and see if there is a pattern to which ones work and which ones don't.

Is there a command / way of show wish drive a specific folder / directory is housed?

8 minutes ago, JCristina said:

Is there a command / way of show wish drive a specific folder / directory is housed?

If you click on the folder icon at the end of a line on the shares tab then you will be shown where folders and/or files are located.

 Thank you itimpi.. I will check it out now to see if it is only one drive or something random across multiple