Did I transfer data from USB the wrong way?

[siege801]

Hi All,

 

I'm new to unRAID - but loving it so far!

 

I'm not new to the *nix world, so to me the best way to transfer data from my USB drive to my data array was to mount the USB (console -> mkdir /mnt/usb && mount /dev/sdx1 /mnt/usb); and then rsync the data to /mnt/disk1/SHARE/. This seems to have worked fine, but all the data is, of course, only on one drive.

 

I'm looking into unBalance to 'scatter' the data across the other drives. My issue is, if I understand the unBalance process correctly, I can't seem to scatter the data to the drives in such a way that includes the source drive. Eg):

 

2Tb Disk0 - parity, so N/A in this context.

2Tb Disk1 - contains 1Tb of data transferred from USB

2Tb Disk2 - blank

2Tb Disk3 - blank

2Tb Disk4 - blank

2Tb Disk5 - blank

 

In unBalance, I select Disk1 as the source, but I then cannot select Disk1 as a target in the scatter process.

 

So my question is, how do I scatter the data correctly? Or should I delete the lot and upload the data differently? I went with this method so that my speed was only limited by the USB transfer, not USB plus network.

 

Please let me know if I've done wrong or what I need to do to share out the data. That said, I've seen Reddit posts stating "why bother?" - just let unRAID do its thing over time. Thoughts?

 

Bonus question - when I transfer data from my next USB - should I be disabling the parity drive temporarily to help with transfer?

 

 

Thanks in advance!

[Turnspit]

Hi!

To help speed up copying large amounts of data, go to Settings -> Disk Settings and set "Tunable (md_write_method)" to "Reconstruct write" for the transfer process.

 

I've never used Unbalance myself, but when copying files over directly I would advise to copy to the UnRAID-managed share, not directly to a single disk - this way unRAID will manage the balancing itself according to you settings.

The target directory would then be /mnt/user/SHARE/.

 

You could also take a look at the Krusader docker, which adds a nice GUI to help with file management directly in UnRAID.

Edited July 15, 2021 by Turnspit

[siege801]

Hi,

 

Thanks for all of that.

 

I've adjusted the Tunable - not sure I've easily seen a difference in the upload speed, but again, the bottleneck is most likely the USB speed.

 

Meanwhile, rather than re-uploading the data I'm going to try creating a new, temporary share which should then be accessible at /mnt/user/TEMPSHARE. I'll rsync from /mnt/disk1/SHARE to /mnt/user/TEMPSHARE. Then I'll rsync back to /mnt/user/SHARE.

 

If I understand you correctly, that should upload the files more evenly across the share's underlying array?

[ChatNoir]

7 minutes ago, siege801 said:

If I understand you correctly, that should upload the files more evenly across the share's underlying array?

By default, Unraid does not balance data on the drives. The Shares follow the Allocation method that you set in the Share settings.

https://wiki.unraid.net/Manual/Shares#Allocation_method

 

The default is Highwater.

 

You can also see the details of each setting by clicking on the option name in the GUI.

[siege801]

53 minutes ago, ChatNoir said:

By default, Unraid does not balance data on the drives.

 

Yup, now that I've RE-read the definitions of the Share methods, that makes sense, and the goal of Highwater (spinning up the disks as little as possible) makes sense too. Thanks for that. Looks like I sent myself on a wild goose chase.