Anyone using it and feel like it's much slower than crashplan for local backups? WIth crashplan, I hit consistent 400-800mbps to a local source. With duplicati, it appears to be 1/10th that or less.

 

 

Yeah, it isn't fast.  I kept restarting one of my local backups trying to figure out a way to make it faster.  Tried SSH, WebDAV, and the next step was to do it to just a disk mount, but we'll see.  Doesn't help that the first machine I'm backing up is an ancient macbook that I'm afraid will die any day and Mono + Duplicati want all the resources.  Wasn't sure if the issue was networking, or just that system not being able to encrypt/compress the files fast enough.

Edited August 28, 20178 yr by Skraut

I ran into that issue initially then realized I was transfering via my external router instead of internal - got a 10x speed improvement using internal routing.  D'Oh!  :-)

 

If I recall correctly some of CrashPlan's reported speeds were based on "effective" speeds not ACTUAL speeds.  In other words, if you had an already backed up 100MB file with a 2MB change and CrashPlan backed up that 2MB in 5 seconds it would report a 100MB/sec effective backup rate - VERY different from the 5MB/sec actual transfer speed.

 

Assuming that's NOT the case here, it's also likely that since Duplicati does everything (except store files) locally I would expect all of the CrashPlan load handled on the server (version maint., verification, etc.) now gets done on the local machine.

 

I have no proof of any of those, of course - but hopefully it make sense.

I've tweaked some settings in the backup setup as I'm still in the testing phase. It's gotten "better"... but still slow.

 

Right now, it appears I'm hitting about 6.5MB/s using duplicati. This is local over my network to a remote drive server. When crash plan managed that same destination, on that same server, it averaged 30MB/s. This was monitored via data transmission rates from the server, and not via crash plan. So, actual speed.

 

My initial backup on crash plan took about 18 hours. At this rate, it will take 105 hours.

 

Just took 2 hours to run 43GB:

Last successful run:Today at 3:37 PM (took 02:03:49)
Source:43.40 GB
Backup:38.46 GB / 1 Version

 

 

The setup, server, storage are all identical between crash plan and duplicati. The only difference is the program.

 

As a test, I ran dupicati w/out encryption/compression on the 43GB test backup. Took 90 minutes.

 

I installed Clouberry. Took 10:44 to backup with no compression/encryption.

 

I get that they both handle the files differently, but seriously. So close to buying the pro license from cloudberry.

A few more observations:

 

Observing disk behavior, it appears to read the data at about 15-25MB/s, breaking it into chunks and creating a vulpine which is then placed in ram. Then once finished, it transmits the data to the local server on the network (via a 10gbe connection, so no bottleneck.) once complete it moves to the next section. (For contrast, crashplan writes directly to the network share.)

 

Sounds great, but being idle while sending and waiting is what is killing the backup times. I've looked into creating multiple volumes at the same time, but that appears to not be possible. Because if it would create several volumes at simultaneously, it could be really fast.

 

Also, duplicati is causing the following errors in my syslog:

 

 

 

Lots of those, using the same path/credentials that crash plan and cloudberry use, but without errors. I'm believe this is also causing slowdowns but unsure why it is occurring only with duplicati.

Edited September 2, 20178 yr by 1812