Duplicacy - Slow Upload

[witalit]

So I am trialling out Duplicacy to replace Duplicati as I have found it to be a pain. One thing I have noticed though is my upload speeds to Google Drive are like 2.5MB/s. its possibly to hopefully increase this by setting some options within Duplicacy. 

 

Does anyone know how to set these options I can't find where to input it as described in this thread on the Duplicacy forums -

https://forum.duplicacy.com/t/unraid-speed-drop-off/3221/2

 

Also why is there no support thread for the Duplicacy docker?

Thanks

Edited March 2, 2020 by witalit

[JonathanM]

12 minutes ago, witalit said:

Also why is there no support thread for the Duplicacy docker?

Ask here.

 

[witalit]

On 2/29/2020 at 7:09 PM, jonathanm said:

Ask here.

 

 

0 help in that thread I hit a brick wall for asking that question.

[JonathanM]

1 hour ago, witalit said:

 

0 help in that thread I hit a brick wall for asking that question.

Unfortunately that's one of the stark differences between the publishers of templates in the Unraid app ecosystem. Some are way more helpful than others. I find LSIO and binhex to be very responsive to their users.

[witalit]

2 minutes ago, jonathanm said:

Unfortunately that's one of the stark differences between the publishers of templates in the Unraid app ecosystem. Some are way more helpful than others. I find LSIO and binhex to be very responsive to their users.

 

Thanks anyways jonathanm. I think I know the solution to the problem but I will need to investigate how I implement it tonight, I'll report back on this thread. Duplicacy seems like a great tool, relatively simple gui and tbh I could probably get over the slow upload once the bulk of my backup is completed. Nice to push some more speed though if its possible!

[JonathanM]

Yeah, you asked why there was no support thread, and you got your answer. The template publishers don't care to support their Unraid users in the Unraid forum, they just use the Unraid ecosystem without really giving back. Not ideal, but it is what it is.

[Squid]

2 hours ago, jonathanm said:

Yeah, you asked why there was no support thread, and you got your answer. The template publishers don't care to support their Unraid users in the Unraid forum, they just use the Unraid ecosystem without really giving back. Not ideal, but it is what it is.

TBH, that's not 100% fair to selfhosters.  They do support their templates.  Generally via their discord, but are gravitating towards separate threads here.  There's many other apps available from established well known authors / maintainers with support threads here and for all intents and purposes do not support the apps at all...

[JonathanM]

2 minutes ago, Squid said:

TBH, that's not 100% fair to selfhosters.

Did you read their response to witalit? Rather dismissive.

 

They did tell him to use their catchall thread which I referred him to, but that thread is not what is listed in the template as their support thread.

[hotio]

Without selhosters giving back to the unraid ecosystem there wouldn't even be a duplicacy template, that's progress in my eyes.

[Roxedus]

On 2/29/2020 at 7:56 PM, witalit said:

Also why is there no support thread for the Duplicacy docker?

Because when it got created we saw no real reason to, since there isn't any Unraid specific thing about this container.

On 2/29/2020 at 7:56 PM, witalit said:

https://forum.duplicacy.com/t/unraid-speed-drop-off/3221/2

Reading the documentation for Duplicacy, makes me believe the option box appears for that backup after the initial setup of it. 




Totally unrelated stuff:

4 hours ago, witalit said:

0 help in that thread I hit a brick wall for asking that question.

If that's your definition of a brick wall...

 

3 hours ago, jonathanm said:

Unfortunately that's one of the stark differences between the publishers of templates in the Unraid app ecosystem. Some are way more helpful than others. I find LSIO and binhex to be very responsive to their users.

Yes it is. But comparing image maintainers and template creators is not a valid compassion. I would rather not support a application I do not use. 

 

2 hours ago, jonathanm said:

they just use the Unraid ecosystem without really giving back.

I do give back, this whole project is based on the community. Not everything happens on the forums, and frankly, accusations like this could discourage future people from doing it. 

 

35 minutes ago, jonathanm said:

support thread

If I knew someone who used duplicacy and were willing to spend their free time (and tackle the beef that sometimes happens), I would create one. I greatly believe that the support currently offered with our template is sufficient, and better than a stale forumpost. 
 

Now back to the the last question in the OP, I above stated why there isn't one currently, but that doesn't mean there cant be one.  As i said in the "cacth-all" post: 

Quote

 some might get its own dedicated thread in based on feedback.

I take this post as feedback enough to be willing to create one, however I still cant guarantee app-support from me personally. 

[DarphBobo]

1 hour ago, jonathanm said:

Rather dismissive.

I can think of many other examples across the board of "why not ask in the support threads hosted by the app itself?"

 

 

[hotio]

please do tell, all of them isn't needed, just a few is enough, let's say 10

EDIT: just kidding

 

EDIT2: why are there no smileys? Maybe that's why people flock to dicord 🙂

 

EDIT3: there it is, lots of work though for a smiley

 

EDIT4: *discord, a dicord is probably something else

Edited March 2, 2020 by hotio

[JonathanM]

3 minutes ago, DarphBobo said:

I can think of many other examples across the board of "why not ask in the support threads hosted by the app itself?"

 

 

True. In this instance the poster thought his question was directly related to it's usage in unraid, and asked why there wasn't a support thread on the forum. I pointed out that there was indeed a support thread from the template maintainer, but it wasn't linked in the template.

 

Over and over we tell people to click on the support link in the container instead of posting questions to the general forum, but there isn't an unraid support thread linked to the template in question.

 

Inconsistency causes frustration.

[JonathanM]

33 minutes ago, Roxedus said:

I take this post as feedback enough to be willing to create one, however I still cant guarantee app-support from me personally. 

At the very least could you link your templates that don't have individual support threads to your catch-all thread, and keep the OP in that thread up to date with whatever support options you are currently offering for each container? List all your containers offered in CA, and if the only support you can offer is a link to the app support site itself, then state, "we don't use the app, go here to ask questions"

 

That way when someone clicks on the support link in either CA or the container drop down it gives them a complete picture of what support options they have.

[dockerPolice]

10 minutes ago, jonathanm said:

Over and over we tell people to click on the support link in the container instead of posting questions to the general forum, but there isn't an unraid support thread linked to the template in question.

There is no requirement that the support thread be hosted here and cannot be pointed at the app's own thread.   Probably the majority of questions regarding any given app aren't actually applicable to the unRaid installation, but are rather application specific questions.  On the face of it based upon the title of this thread (Slow Uploads), I'd call that app specific.

 

But that is rather immaterial as it is semantics.  

 

Personally, I'd love for every single app to have a dedicated support thread per app.  In practice, its not going to happen.

 

Where and how the support threads are hosted and what I am able to discern about any given support level does however highly impact any moderation that may (or may not) one day be required on.

 

 

[Squid]

27 minutes ago, jonathanm said:

posting questions to the general forum,

You don't think that the war on that is already lost?

[JonathanM]

1 minute ago, Squid said:

You don't think that the war on that is already lost?

Sigh. Seems to be so.

 

Having a single unambiguously correct answer to the question "Where do I go for support" is too lofty a goal.

 

Tying things up in neat and tidy consistent packages sure is satisfying when you can achieve it though.

[hotio]

Let's hope we don't have to send him off to Google for support...it's Google Drive related afterall.

[witalit]

1 hour ago, Roxedus said:

Reading the documentation for Duplicacy, makes me believe the option box appears for that backup after the initial setup of it. 

Seems you are right I think I have found the 'options' I am looking for if only I had read the support documentation properly. I will try tonight, I find the Duplicacy GUI to be a little odd some of the functions are not that apparent on first glimpse. 

 

Quote

The following Backup page shows the new backup just added. You can click the Include/Exclude column to specify which files will be included in or excluded from the backup, and the Options column to specify extra options for the backup operation.

 

 

1 hour ago, Roxedus said:

0 help in that thread I hit a brick wall for asking that question.

In all honesty I misread your reply when you said 'Then use this post.' I thought you was referring to the Duplicacy forums so sorry for that. 

 

1 hour ago, Roxedus said:

and better than a stale forumpost. 

I have to disagree here, I have got very good information from the support threads of Plugins/Templates/Dockers on these forums. I guess my question is not related to the template and is in fact the functionality of the app. 

 

1 hour ago, Roxedus said:

I take this post as feedback enough to be willing to create one, however I still cant guarantee app-support from me personally. 

I didn't think this post would spiral as it has and my question probably would not fit into that support thread :-)… 

 

 

Edit: All in all.. I have got some responses to my thread but not the ones I expected haha.. What's a Monday without a good debate right? 

Edited March 2, 2020 by witalit

[Squid]

2 minutes ago, witalit said:

didn't think this post would spiral as it has and my question probably would not fit into that support thread :-)… 

I think that @jonathanm and I are arguing just for the sake of arguing.  😀

[JonathanM]

1 minute ago, Squid said:

I think that @jonathanm and I are arguing just for the sake of arguing.  😀

I wasn't arguing with you, it's @DarphBobo that I had an issue with. I agree with you and @dockerPolice

 

[witalit]

23 minutes ago, dockerPolice said:

There is no requirement that the support thread be hosted here and cannot be pointed at the app's own thread.   Probably the majority of questions regarding any given app aren't actually applicable to the unRaid installation, but are rather application specific questions.  On the face of it based upon the title of this thread (Slow Uploads), I'd call that app specific.

 

But that is rather immaterial as it is semantics.  

 

Personally, I'd love for every single app to have a dedicated support thread per app.  In practice, its not going to happen.

 

Where and how the support threads are hosted and what I am able to discern about any given support level does however highly impact any moderation that may (or may not) one day be required on.

 

 

 

What I think happened is I am so used to the docker templates having support threads.  When I downloaded Duplicacy (recommended by a lot of UnRaid users) I was confused to see no support link just a link to the Duplicacy forums. As far as I can tell the support threads for most templates are rife with support questions, a lot of those probably are not related to the template. 

 

57 minutes ago, jonathanm said:

Over and over we tell people to click on the support link in the container instead of posting questions to the general forum, but there isn't an unraid support thread linked to the template in question.

 

Inconsistency causes frustration.

 

Exactly that I clicked the support thread and went over to the Duplicacy forums. I can't thank the contributors to templates enough and have donated to a few for a couple beers! I guess I have been spoilt with the helpful people on this forum, I wanted to avoid asking questions elsewhere but seems like I am in the wrong if its app specific. One of the reasons I have also stuck with UnRaid as the forums are active and informative. 

 

Quote

You don't think that the war on that is already lost?

 

To be honest that's the same with the General forum for forums across the web :-).

[witalit]

 

Turns out it was super simple. Setup your storage, add a backup to the storage. Once the backup is created you can click the - under the Options column as highlighted below and enter the Command Options as per my image

 

-threads 8 -limit-rate 8680

 

 

 

After making this change I can now upload at around 5MB/s so over double what I was seeing before. I suspect the thread limit is CPU dependant so 8 is my limit speeds not to bad for 200GB I need to backup on initial seed. 

 

Thanks all sorry for any trouble caused, seems it was simple in the end. 

 

 

[GilbN]

1 hour ago, witalit said:

 

Turns out it was super simple. Setup your storage, add a backup to the storage. Once the backup is created you can click the - under the Options column as highlighted below and enter the Command Options as per my image

 

-threads 8 -limit-rate 8680

 

 

 

After making this change I can now upload at around 5MB/s so over double what I was seeing before. I suspect the thread limit is CPU dependant so 8 is my limit speeds not to bad for 200GB I need to backup on initial seed. 

 

Thanks all sorry for any trouble caused, seems it was simple in the end. 

 

 

Btw. I read some of the documentation and the threads does not have anything with the CPU. Its how many upload threads you allow. See: https://forum.duplicacy.com/t/backup-command-details/1077 

The -limit-rate is the global upload speed for all threads in kB/s

And according to this https://duplicacy.com/issue?id=5670666258874368 more threads on HDD's means more head seeking and slower uploads. So 1 thread is best for HDDS and multiple for SSDs but YMMV.. I don't use use duplicacy. 

[witalit]

11 hours ago, GilbN said:

Btw. I read some of the documentation and the threads does not have anything with the CPU. Its how many upload threads you allow. See: https://forum.duplicacy.com/t/backup-command-details/1077 

The -limit-rate is the global upload speed for all threads in kB/s

And according to this https://duplicacy.com/issue?id=5670666258874368 more threads on HDD's means more head seeking and slower uploads. So 1 thread is best for HDDS and multiple for SSDs but YMMV.. I don't use use duplicacy. 

 

Thanks I had a read up last night after I kicked off the initial backup of the same information. Your right the threads are nothing to do with CPU, I found 8 threads to peak out around 6.5MB/s. I will try some testing but my 200GB backed up fine overnight so speeds are not really too bad for me it completed in around 8-9 hours. It would be nice to max out my Internet connection as I have 1Gb up/down fibre but hey it works I wont complain.

 

Next test is to restore my data fully and compare. When I was attempting to restore even just a portion of my Duplicati data backup the damn restore kept failing so Cloud backup hasn't been great so far.