Copy Data to unraid from unassigned devices - seems slowly

[Fuxdom]

Hey forum,

 

I have installed an unRaid NAS on new hardware in the last few days.

(And I'm completely excited)

Especially thanks to the good information here and the relaxed / factual interaction of the community with one another, as a newbee I was able to solve some problems with construction / installation / furnishing.

Now it's running ... and I'm starting to deal with VMs, Docker, etc. (never worked with NAS, Linux ...).

 

Now I am copying my existing data on the array - around 45 TB.

 

-the disks are connected (mounted) as unassigned devices directly to the NAS

-Parity and cache are (for now) off

-I copy everything over the Krusader to the 1st NAS disk (e.g. want media folder on Disk1, then my own files from the next old disk on Disk2, etc.)

-the array HDDs are Seagate Exos 18TB (CMR)

-the old HDDS are Seagate Skyhawk 8TB (SMR)

- both directly on the mainboard

- the other hardware: 8 core Intel Rocket Lake and 32 GB Ram

 

So should actually go 'fast'.

 

In fact, I only achieve around 23-30 MB / s (read) and that seems pretty slow to me, especially with large files. (because of SMB?)

 

Would be grateful for an assessment and, if necessary, a few tips on how I can do it better.

 

(Sorry for my english i am geman)

Edited September 29, 2021 by Fuxdom

[Vr2Io]

1 hour ago, Fuxdom said:

- the other hardware: 8 core Intel Rocket Lake and 32 GB Ram

 

1 hour ago, Fuxdom said:

especially with large files. (because of SMB?)

Seems you transfer file through network, as all disk connect locally, then you should transfer through locally for best performance.

 

Edited September 30, 2021 by Vr2Io

[Fuxdom]

Quote

Seems you transfer file through network, as all disk connect locally, then you should transfer through locally for best performance.

Both - the Array Disk1 (18TB CMR) and the unassigned Disk (Seagate 8TB SMR) are conected directly on the Mainboard (via SATA)....

 

oh, i have seen thaat i wrote SMB - was false - i meant SMR (Shingled Magnetic Recording )

 

But, thanks for your reply

[Vr2Io]

Then no much reason for such low speed, SMR disk won't have issue in read speed.

[TheSkaz]

what hardware are you running?

how is your Array connected to the computer?

how are the unassigned drives connected to the computer? (sata? sas? usb3/2?)

how is your "share" configured? (most free/etc)

how are you copying the files? (/mnt/user/xxxx or /mnt/disk1/xxxxx)

 

i have the Exos 16TBs and I can confirm they can write at 100MB sustained, through a SAS controller

 

 

note: I looked up Krusader and It looks like a SFTP application like WinSCP. if it works just like WinSCP you could have network issues. where the file is copied to the computer running krusader and then copying back to the new disk. 

 

try SSHing into the box and do something like 'mv -v /external/drive/folder /mnt/disk1/media' and see if that is any faster. (change the paths obviously)

[Fuxdom]

Quote

what hardware are you running?

Mainboard: Gigabyte W480M VISION W

32 GB ECC Ram

Intel Core 7 11700K

Parity and cache are (for now) off

Quote

how is your Array connected to the computer?

8x SATA port at the Mainboard available (for now 4x seagate Exos 18TB HDD)

Quote

how are the unassigned drives connected to the computer? (sata? sas? usb3/2?)

SATA

Quote

 

how is your "share" configured? (most free/etc)

how are you copying the files? (/mnt/user/xxxx or /mnt/disk1/xxxxx)

 

 /mnt/unassigned/media/xxxxx    --->  /mnt/disk1/xxxxx

(i installed/configured the unassigned Devices Plugin and Docker binhex krusader)

 

Quote

i have the Exos 16TBs and I can confirm they can write at 100MB sustained, through a SAS controller

👍

Yes, the Exos Enterpreis Hdds are great, have in contrast to the WD Pro, a better, lower failure probability and are even cheaper here in Gemany.

I do not understand why in the forun we get so often to the WD NAS HDDs and you hardly read what from the Exos.

Thank you for your advice.

 

But I think I found the problem. When copying with the krusader, I selected 10-20 folders with about 200 media files (each about 2-4 GB) at the same time.

I suspect that the individual files are physically in different places on the unassigned HDD, Krusader has started to copy everything at the same time. That means the read head of the HDD had to jump back and forth constantly.

That went on all day. Now after 15 hours only one folder with approx. 500 GB (25 files) is copied.

Result:

unassigned disk read 189 MB / s

Array Disk1 write 199MB / s

I suspect the file manager Krusader was overwhelmed ... (also got quite slow)

But I'm a beginner and two weeks ago I had never heard of unraid, xfs etc. - what do I know ...

 

[Vr2Io]

13 minutes ago, Fuxdom said:

I suspect that the individual files are physically in different places on the unassigned HDD, Krusader has started to copy everything at the same time. That means the read head of the HDD had to jump back and forth constantly.

👍