In have loaded my server with data and got good write speeds once I discovered turbo write.  But now when backing up a share from the server to a disk attached to my local machine, it is too slow.   I take it there is no turbo read but is there some other tuning that can speed things up?

 

The share in question is a 500GB photo archive which contains lots of small thumbnail images as well as large ones.  The backup is initiated on my mac mini with Carbon Copy Cloner and is the only process going on.  Backup speed was averaging 15-30MBs and for fisrt 5 minutes was in the kb range

so I just stopped the backup; I would expect 80-90MBs over ethernet.  Backup speed on  local disk to local disk is a steady 130MBs.

 

Hardware config:  Intel Core 2 Duo 2.66Ghz, 4GB ram; 1Gib nic

Unraid: array: 1 x 5TB parity HDD  2x 4TB  1 x 3TB HDD

           cache pool: 2 x 512GB SSD

 

Diagnostic file attached.

 

 

tower-diagnostics-20230215-1445.zip

Edited February 15, 20233 yr by hendrst1

12 minutes ago, hendrst1 said:

The share in question is a 500GB photo archive which contains lots of small thumbnail images as well as large ones.  The backup is initiated on my mac mini with Carbon Copy Cloner and is the only process going on.  Backup speed was averaging 15-30MBs so I just stopped the backup; I would expect 80-90MBs over ethernet.  Backup speed on  local disk to local disk is a steady 130MBs.

 

When copying files of this size, this is about the same speed I get using Windows 10 when copying to and from my Unraid server.  (When I copy 15-40GB .mkv files, the speed is always above 100MB/s.)  I have always attributed the problem to the file overhead on both ends.  One end is finding the file and the other is trying to figure where to store it and writing that data to the file allocation tables.  Once all of that is done, the actual transfer of the files can begin...

 

Local disk to Local disk--- Remember that the disks involved have transfer speeds of at least 150 MB/s so you are seeing some overhead effect there also. 

I experimented with a few more downloads.  

 

It looks like a batch of old Apple Aperture library files that are causing the download to slow to a crawl (2 hrs est. to download a 2GB library file). Other directories with just photos download at the expected speeds (60-80MBs).   These Aperture library files have a large number of small files composed of not just .jpg previews and master files but all sorts of database, xml files and other peculiar system files.    But, even so, why do these present such a problem for downloading?   I don't recall there was a problem on the initial upload but maybe I didn't notice it.