CrashPlan

[xamindar]

Ah... so the tarring of the current install is just to create a restore point in case of a bad update.  In the event of an update, I can assume that the .identity file stays intact?

It should stay intact. If the .identity file was changed or erased crashplan will think this computer is a completely new one. If you didn't save this file on reboots then a new computer would appear on your crashplan account every time you rebooted. After a while you would have a LOT of "Tower" names on your crashplan account.

[LaurentHU]

Anybody else has a memory leak issue with CrashPlan on Unraid? I managed to install (this week) CrashPlan like described here, works marvelously well until it eats the whole memory (2 GB here) and makes the whole system crash... Bad bad bad...

I'm running 4.5.6 with only rtorrent / rutorrent and CrashPlan installed, I figure 2GB is more than enough no?

Any easy way to figure where the memory leak is (sorry Linux noob question, didn't figure out how to classify by memory use % in top).

[boof]

Anybody else has a memory leak issue with CrashPlan on Unraid? I managed to install (this week) CrashPlan like described here, works marvelously well until it eats the whole memory (2 GB here) and makes the whole system crash... Bad bad bad...

I'm running 4.5.6 with only rtorrent / rutorrent and CrashPlan installed, I figure 2GB is more than enough no?

Any easy way to figure where the memory leak is (sorry Linux noob question, didn't figure out how to classify by memory use % in top).

 

No, no problems at all. I imagine it *may* depend on how many files crashplan is tracking in terms of it's memory usage.

 

I'm backing up ~100k files.

 

What makes you think it's crashplan eating all your memory / leaking memory? And not something else?

 

You can use 'top' 'ps' and 'free -m' to check memory usage.

 

I'd hazard a guess that what's more likely is you have something writing a log file / temp file to your ram disk and that's eating memory...

[aiden]

Likewise I'm backing up ~10k files and have no leak issues.

[LaurentHU]

I suspect CrashPlan because I had no issues before...

 

Yes most probably it's a log file being written, I am currently backing up almost 1 TB of data and approx 300 000 files, which might explain the issues that CrashPlan has. I'll check tonight and report on findings - but best practice might really be to install CrashPlan on the cache disk.

[prostuff1]

If you have rtorrent and Crashplan running at the same Time I suggest stopping rtorrent for the time being.  The initial scan from Crashplan can take a while and does some serious work.

 

I stagger the startup of my applications also.  Crashplan starts up right away on my server and then I start Transmission (a BitTorrent client just like rtorrent) a couple minutes later.  This gives Crashplan time to do what it needs to do and I have not run into any problems yet.

[aiden]

Based on Prostuff's recommendation in the wiki entry for crashplan, I too added a 30 second delay after Crashplan initializes to make sure it has a chance to get settled before moving on.

[gfjardim]

I had this memory leak (1.6GB used), and Crashplan was using a lot of files (20k opened files).

 

I had to:

 

a) change the version of the Java RE: downloaded the current version for Slackware 13.1 here and installed it. Old versions of JRE are known to have "memory leaks" and "too many open files" problems.

 

b) configured Crashplan to use this instead of the own JRE version: changed the value JAVACOMMON in /usr/local/crashplan/install.vars to /usr/lib/java/bin/java.

 

Re-tared the files and restarted Crashplan, and it's all good again.

 

Since this approach uses a different slackware version build, it's moderately risky.

 

[boof]

20k *open* files at one time? or 20k files that were to be considered in your backup?

[gfjardim]

20k *open* files at one time? or 20k files that were to be considered in your backup?

 

20K simultaneously opened files after I raised "ulimit -n" to 50k files, and 1.6GB of used RAM.

 

Obviously a memory leak with JRE.

[LaurentHU]

And now it's working again, no memory leak, just going through a very long and painful resync between the client and the backup server... Not sure at all what happened!

[shire]

Hi!

 

I  use Crashplan to backup my Mac. Speaks something against using a cache drive for the backup share on the unRAID machine?

 

Bye.

[boof]

Hi!

 

I  use Crashplan to backup my Mac. Speaks something against using a cache drive for the backup share on the unRAID machine?

 

Bye.

 

It depends how you view it and how you're using the cache drive.

 

If you're using a hidden directory on the cache drive (to avoid it being moved onto the array) then does it bother you your backups aren't protected? They are a second copy of the data after all. Only you can answer that one.

 

If you're using the cache drive as normal then the backups will be moved onto the array eventually.

[shire]

Hi!

 

Hi!

 

I  use Crashplan to backup my Mac. Speaks something against using a cache drive for the backup share on the unRAID machine?

 

Bye.

 

It depends how you view it and how you're using the cache drive.

 

If you're using a hidden directory on the cache drive (to avoid it being moved onto the array) then does it bother you your backups aren't protected? They are a second copy of the data after all. Only you can answer that one.

 

If you're using the cache drive as normal then the backups will be moved onto the array eventually.

 

I would like to see the data to be copied into the array. I wouldn't like to have TB of data saved to the unRAID server. Due to the fact I'm already using TM on my Mac to save my data to an external USB drive.

 

Hm. I don't know if you can limit the backups going backwards over one or two weeks. I installed crashplan yesterday. I'm just testing a bit...

 

Bye.

[prostuff1]

I would like to see the data to be copied into the array. I wouldn't like to have TB of data saved to the unRAID server. Due to the fact I'm already using TM on my Mac to save my data to an external USB drive.

 

Hm. I don't know if you can limit the backups going backwards over one or two weeks. I installed crashplan yesterday. I'm just testing a bit...

 

Bye.

 

Yes, you can set Crashplan to keep a certain amount of backups.

[shire]

Hi!

 

I would like to see the data to be copied into the array. I wouldn't like to have TB of data saved to the unRAID server. Due to the fact I'm already using TM on my Mac to save my data to an external USB drive.

 

Hm. I don't know if you can limit the backups going backwards over one or two weeks. I installed crashplan yesterday. I'm just testing a bit...

 

Bye.

 

Yes, you can set Crashplan to keep a certain amount of backups.

 

Well I can't change the backup frequency. I guess that this is a pro feature. Hm. That is somehow uncool cause I don't need crashplans online update abilities.

 

Bye.

[boof]

Hi!

 

I would like to see the data to be copied into the array. I wouldn't like to have TB of data saved to the unRAID server. Due to the fact I'm already using TM on my Mac to save my data to an external USB drive.

 

Hm. I don't know if you can limit the backups going backwards over one or two weeks. I installed crashplan yesterday. I'm just testing a bit...

 

Bye.

 

Yes, you can set Crashplan to keep a certain amount of backups.

 

Well I can't change the backup frequency. I guess that this is a pro feature. Hm. That is somehow uncool cause I don't need crashplans online update abilities.

 

Bye.

 

Changing the backup frequency is a crashplan+ feature (not pro, pro is an entirely different product). They've removed crashplan+ from their store for some reason. Announcement coming this week about it and what it means.

 

Crashplan central is their online data store and is, again, a seperate product from crashplan, crashplan+ and crashplan pro.

 

Otherwise you get a once daily scan / backup.

 

You *may* be able to bypass this by restarting the crashplan engine each time you want to start a backup, this will certainly have it reevauluate all files - whether or not it will then start a backup I'm n ot sure.

[shire]

Hi!

 

Hi!

 

I would like to see the data to be copied into the array. I wouldn't like to have TB of data saved to the unRAID server. Due to the fact I'm already using TM on my Mac to save my data to an external USB drive.

 

Hm. I don't know if you can limit the backups going backwards over one or two weeks. I installed crashplan yesterday. I'm just testing a bit...

 

Bye.

 

Yes, you can set Crashplan to keep a certain amount of backups.

 

Well I can't change the backup frequency. I guess that this is a pro feature. Hm. That is somehow uncool cause I don't need crashplans online update abilities.

 

Bye.

 

Changing the backup frequency is a crashplan+ feature (not pro, pro is an entirely different product). They've removed crashplan+ from their store for some reason. Announcement coming this week about it and what it means.

 

Crashplan central is their online data store and is, again, a seperate product from crashplan, crashplan+ and crashplan pro.

 

Otherwise you get a once daily scan / backup.

 

You *may* be able to bypass this by restarting the crashplan engine each time you want to start a backup, this will certainly have it reevauluate all files - whether or not it will then start a backup I'm n ot sure.

 

Thanks...I will look forward to their announcement...

 

Bye.

[shire]

Hi!

 

Hi!

 

I  use Crashplan to backup my Mac. Speaks something against using a cache drive for the backup share on the unRAID machine?

 

Bye.

 

It depends how you view it and how you're using the cache drive.

 

If you're using a hidden directory on the cache drive (to avoid it being moved onto the array) then does it bother you your backups aren't protected? They are a second copy of the data after all. Only you can answer that one.

 

If you're using the cache drive as normal then the backups will be moved onto the array eventually.

 

I recognized that the backup process takes a while and therefore keeps all my drives up and running. I think I will go for the cache drive and let the data been moved be the mover during the night.

 

Bye.

[boof]

I recognized that the backup process takes a while and therefore keeps all my drives up and running. I think I will go for the cache drive and let the data been moved be the mover during the night.

 

Bye.

 

If you're backing up *from* your array then all your drives will spin up anyway to read the backup data (pending on amount of changes to be backed up and how distributed amongst your drives they are).

 

In terms of what crashplan *writes* during a backup it writes into single 4GB encrypted files (with a small handful of other files scattered around for metadata).

 

Depending on your split levels it would be very easy to force this onto a single drive in the array, meaning you would only every have one drive spinning as a consequence of crashplans *write* operations. Of course you'd be limited by the physical space on that drive for your backups.

 

I don't think using the cache drive will have the consequence you expect with 100% determinance. Once the data is moved onto the array if crashplan needs to :

 

- append to an existing backup file

- read across all the backup files during backup verification

- perform a restore

- prune versions

 

It will spin up the array drive rather than anything in cache.

 

If you set your split level to distribute this across your drives theres a good chance you'll end up with lots of them spinning up during a crashplan operation. If anyone else is doing that they can probably comment. I force all my writes from crashplan onto a specific drive.

 

 

[prostuff1]

I force all crashplan backups to a single drive in my unRAID server so that only one drive is spinning when I am writing to the array and doing a Crashplan backup.

[Loch]

Just realized Crashplan got updated to the new Crashplan 3

 

http://support.crashplan.com/doku.php/release/m25_r01

 

looks like there may be some ability to modify the back-up schedule for the free version.

[c234rmf]

Any of you wizards out there have advice on how to save the new crashplan3 so it survives over re boot?

thanks

 

[prostuff1]

It should install automagically.  Once it has you can use the Stop Crashplan button in the User Scripts section of unMenu; once you have stopped it use the Tar Crashplan button to bundle all the new stuff up.

[boof]

It should install automagically.  Once it has you can use the Stop Crashplan button in the User Scripts section of unMenu; once you have stopped it use the Tar Crashplan button to bundle all the new stuff up.

 

Exactly that, I've been running it with no problems since it updated.