***EDIT***

See post #3, array not spinning down is due to crashplan

********

 

I am on 4.7, my array is about a month old, I have been tinkering with it.

 

Up until about a couple days ago, there were no problems, but during the last couple of days, I noticed that disk4 and parity drives are not spinning down. I see the spin down commands issued to them right after the mover at around 4:40am, but in the morning, the two drives are still seen as spinning in unMenu.

 

I do have a cache drive and all my user shares are set to use it.

 

I do have plugins such as sab/cp/sick and vm, crashplan, but they are all installed on the cache drive.

 

I did try to install inotify tools, but for some reason slackware.org.il has been down for the last 12 hours.

 

I also did a full reboot last night to no avail.

 

Any ideas?

 

btw, my syslog (through unMenu) currently contains only these entries as it was restarted after mover this morning:

 

Feb 24 04:40:01 Tower syslogd 1.4.1: restart.
Feb 24 04:41:40 Tower kernel: mdcmd (25): spindown 0 (Routine)
Feb 24 04:41:41 Tower kernel: mdcmd (26): spindown 1 (Routine)
Feb 24 04:41:42 Tower kernel: mdcmd (27): spindown 2 (Routine)
Feb 24 04:41:43 Tower kernel: mdcmd (28): spindown 3 (Routine)
Feb 24 04:41:43 Tower kernel: mdcmd (29): spindown 4 (Routine)
Feb 24 07:17:48 Tower kernel: NTFS driver 2.1.29 [Flags: R/O MODULE]. (System)

 

It shows the spindown issued to all after the mover, but disk4 and parity are still spinning

 

(Is there a more detailed syslog somewhere? even when the syslog was full after a reboot last night, there was little info about the plugins. I only saw a few lines about sab, but nothing on couch or sickbeard, etc)

 

Thanks in advance

you can use the lsof command to see what files are still open (I forget what flags you may need to add to lsof.

but something like > lsof |grep disk4

may give you enough information.  or search further on the forums about using lsof.

I just pinpointed the problem to crashplan.

 

It is writing directly to disk4, ignoring the cache drive. I feel stupid that I didn't realize this sooner.

 

Now, to use the cache drive, am I supposed to select mnt/user/backup or mnt/user0/backup?

 

Thanks

 

 

(In crashplan, I had selected the backup save folder as  > mnt > user > backup, I'll go ahead and try user0 instead)

If you want to guarantee use of the cahe drive use /mnt/cache/<user share name>

Note if the cache drive gets full it will fail.

 

Sent from my SGH-I727R using Tapatalk

But I would like the backups to reside on the array with parity protection, not on the cache drive.

 

When mover moves the files from the cache drive to the array, next time crashplan checks the folder, it will see it as empty and do an entire backup all over again, right?

 

 

I tried user0, but crashplan still seems to ignore that and write directly to disk4. Oh well, I gotta go to work, I'll try to figure it out in the afternoon.

 

Thanks

Then check your user share settings for the backup folder. Make sure it is set to use the cache drive and what the minfree space is set to

 

Sent from my SGH-I727R using Tapatalk

Crashplan is one of the few things that as far as I am concerned should not be used in conjunction with the cache drive.

I'm confused. If you write to /mnt/user/backup and it goes to disk4 and you want it to use the parity protected array then what's the problem?

 

It's using disk4 because that the disk currently picked due to the share settings.

 

As for the cache drive, if you use /mnt/user/backup and the share is properly set to use the cache then the new data will go to the cache. The mover will move it to the array. The data will still appear in /mn/user/cache and Crashplan won't know the difference.

 

Once again, the share settings dictate which disk gets used. Your backup share settings are currently selecting disk4.

 

Peter

I'm confused. If you write to /mnt/user/backup and it goes to disk4 and you want it to use the parity protected array then what's the problem?

 

It's using disk4 because that the disk currently picked due to the share settings.

 

As for the cache drive, if you use /mnt/user/backup and the share is properly set to use the cache then the new data will go to the cache. The mover will move it to the array. The data will still appear in /mn/user/cache and Crashplan won't know the difference.

 

Once again, the share settings dictate which disk gets used. Your backup share settings are currently selecting disk4.

 

Peter

 

I want the backups to eventually go to the array for parity protection, but not right away. Crashplan is doing 15 min backups from multiple computers around the house to the unraid server. If it goes directly to the array, that means constant writes to the array and the parity drive.

 

I would like crashplan to keep doing the backups, have them get written to the cache drive throughout the day, and at the end of the day, it all gets moved to the array, so the parity drive and array drives are written to once a day. That is how sab/couchpotato/sickbeard and all my other plugins work.

 

If my cache drive crashes, I only lose up to a day's worth of data, which is fine.

 

With that said, my backup user share is set to use the cache drive (all of my user shares are). If I copy something into that user share from a network computer, it all ends up in the cache drive first.

 

I realize that the backups are ending up in disk4 because it has the most empty space. I wasn't really asking why they were going to 4 instead of 3, but more of a why are they going to an array drive directly, rather than the cache drive first.

 

Within crashplan gui, I can browse to the location where I want the backups to be saved. I browse to mnt and underneath, I get all the disk1, disk2, etc. plus the cache drive, and two folders named "user" and "user0"

 

"user" folder has all the user shares plus the folders in the cache drive (my hidden sab, mysql etc. folders)

"user0" folder has just the user shares.

 

I tried selecting my backup folder through either user and user0, and both times, I got somethings written to the cache drive, some went straight to the array disks. I figured this out right before I left for work, so I didn't get a chance to properly test it yet.

 

The only thing I am not sure about is how the cache drive handles overwrites. Does it create a duplicate/newer file on the cache drive, that eventually overwrites the old one when mover script runs, or does it just overwrite the file on the array right away?

Because crashplan does not just copy the files directly, it converts them to its proprietary secure format that looks more like split rar archives, so it does a lot of overwriting on those files. If overwrites are done directly to the array, bypassing the cache drive, that would explain everything.

 

Thanks

Then check your user share settings for the backup folder. Make sure it is set to use the cache drive and what the minfree space is set to

 

Sent from my SGH-I727R using Tapatalk

 

user share is set to use the cache drive. the minfree is i believe 20gb (I'll have to double check when I get home) and I have about 450gb free space on the cache

 

Thanks

I'm pretty sure that an edited or overwritten file stays on the disk where it started.

 

Peter

 

 

You are right lionelhutz, I just found this thread that explains how the cache drive behaves when overwriting:

 

http://lime-technology.com/forum/index.php?topic=2572

 

I guess crashplan is opening the existing archive, appending to it and then saving, hence the direct write to the array drive.

 

Whereas when I first set up crashplan, and started the initial backup, all the files were new and they ended up on the cache drive first.

 

I guess I will have to rethink my strategy as there is no easy fix on the unraid side of things and I can't just make crashplan delete the old file and copy a new one (without doing full backups each time).

 

Maybe I will switch to a once a day type of backup and have crashplan not bother with the cache drive, as it will keep writing directly to the array after the initial backup anyway.

 

OR

 

Have crashplan save the backups to the cache drive in a hidden directory (untouched by the mover). And then have a cron script to rsync the backups to the array all at once, once a day or once every two days (Just like the mover script but without deleting the copy on the cache drive so crashplan can keep rsyncing to it). It would double the space used, but it would accomplish minimum writes to the array.

 

 

 

I just came up with an idea. How about a modification to the mover script, so that certain folders can be designated to not be deleted after the move, instead just copied. I don't know what other applications there could be for it, but it would certainly help in this case. What do you guys think?

 

Thanks

 

 

***EDIT***

I just came up with other applications for it. Currently I have the sab, couch potato, sickbeard, mysql, virtualbox and all those other plugins' data saved on the cache drive to prevent excessive writes to the array and the flash drive. If my cache drive goes belly up, I lose all that plugin data. The modified mover could backup those folders on the array as or in a "hidden" user share(s).

 

So if the cache drive dies, I can restore that plugin data (along with the crashplan backups) from the hidden user share(s)

 

Thanks

You could have crashplan save to a hidden folder on the cache drive then run a cronjob rsync each night to copy the changes to the array.

 

Sent from my SGH-I727R using Tapatalk

You could have crashplan save to a hidden folder on the cache drive then run a cronjob rsync each night to copy the changes to the array.

 

Sent from my SGH-I727R using Tapatalk

 

Yep, that's exactly what I did.

 

First I switched the backup locations to the cache drive in a folder that starts with a "."

 

Then I created a script to rsync from that folder to a new user share that does not use the cache drive.

 

Finally, I added a line in my go script to copy the rsync script to the cron.daily folder at boot.

 

Every night at 3am (hopefully) backups will be rsynced to the array

 

Thanks for the help guys