I need to use 3 external USB drives as one drive for backup


Recommended Posts

Hi all,

I want to backup my movies/series/Photos/documents stored on my Array to external drives and I cannot figure out the best way to do what I need to do.

Here is the situation:

My array contains about 12Tb of movies/series/Photos/documents.

My "Movies" share is the largest with 11Tb.

I own 3 Lacie external USB drives of 5Tb each.

 

I would like to use these 15Tb as a backup, ideally with 4 folders: Movies backup/Series backup/Photos backup/Documents backup.

I would like to use a sync tool to sync these 4 folders to my shares.

 

My problem, if you had not guessed it already, is that I have 3 external drives that are each smaller than the biggest folder (movies=11Tb) I need to backup.

 

Here is what I thought would be the ideal solution:

Plug the 3 Lacie drives to my windows PC, have windows recognise them as one drive (here is the guide found for this: https://lifehacker.com/5986883/how-to-combine-multiple-hard-drives-into-one-volume-for-cheap-high-capacity-storage) then use a sync tool to synhronize/transfer accross my network.

The problem is that the transfer rate/speed over the network to these USB3.0 drives is really slow (6Mb/s).

 

I noticed however than when one of these drives is directly plugged into the Unraid server via USB, the transfer (krusader) is about 40Mb/s

Krusader has a sync tool, so I am thinking I could use it.

 

HOWEVER I have no clue as to how I can make these 3 external drives appear as ONE drive in Unraid.

I don't want them constantly plugged in, and I guess I cannot mount them in the array and regularly disconnect them, rigtht?

Can I create a share on unassigned drives?

 

I am a little lost as to what solution would work.

 

Thank you in advance for your help.

 

D.

 

 

 

 

Link to comment

Thank you very much Johnnie I shall try that.

Could you just tell me if, once I have backed up all my shares into the unique pool I have created, I can unplug the devices, plug them back later and obtain the exact config?

Link to comment
  • 2 years later...

Thanks Johnnie this is exactly the info I needed. 

 

I have created a "Frankenstore" backup solution (pic below), using 5 USB 16 TB drives.  These are cheap drives, at ~$310 each, and even with 3D printing an exoskeleton for portability, wiring up a single power supply, and using a 7-port USB 3.0 hub with toggle switches, my total cost for an 80 TB backup solution is under $1700.  The final solution is extremely portable, making it easy to take offsite for security.  The 10A 12V power supply could easily support 6 drives, and possibly even 7, so I have a bit of room to grow to 96 or even 112 TB of backup capacity in the future, though for the next year 80 TB is plenty.

 

The toggle switches on the USB hub are really cool, as it allows me to control the power-up order and get the same disk ID each time, though I'm not sure if that matters with the BTRFS pool.

 

Of course, at such a low cost, I am expecting drive failures.  Since this is primarily just an offline backup for my main array, I'm cool with taking that risk.  When I read through your linked instructions, you talk about replacing a drive, but not specifically replacing a failed drive.  Is the process the same, or will it be different?  I'm assuming with a JBOD, I only lose the data on the failed drive, plus any files that might have been split across two drives onto the failed drive - I don't suppose there is a way to prevent splitting files across drives in a pool, is there?

 

Also, with Unraid v6.9 in the wings, is using UD still the right way to go?  I'm running 6.8.3, and do not run beta or even RC on my production server.  Do you know if my UD BTRFS JBOD pool will migrate to v6.9's new multi-pool functionality, or would I have to recreate it from scratch and re-do my backup?

 

Thanks!

Paul

 

image.thumb.png.dcf06f7edc505f0a1e9c3731f1976644.png

Link to comment
31 minutes ago, Pauven said:

as it allows me to control the power-up order and get the same disk ID each time, though I'm not sure if that matters with the BTRFS pool.

It doesn't, btrfs saves all info on the device metadata, so you can even clone one or more devices with dd and the pool would start without complaining.

 

32 minutes ago, Pauven said:

you talk about replacing a drive, but not specifically replacing a failed drive.  Is the process the same, or will it be different?

Same or similar if the device is missing, though now with the latest beta and multiple pool support you can more easily add, remove or replace devices using the GUI.

 

33 minutes ago, Pauven said:

I'm assuming with a JBOD, I only lose the data on the failed drive

Yes, if the metadata is redundant, still files might be split on multiple drives (there's no way to prevent that), and the pool won't mount, you'll need to use recovery options.

 

35 minutes ago, Pauven said:

Also, with Unraid v6.9 in the wings, is using UD still the right way to go? 

I'm already using v6.9 for my pools, good news i that you can import any pool create on UD.

 

  • Thanks 1
Link to comment
  • 1 year later...

Hey Johnnie/@JorgeB, I could use some help on this.

 

Side note, your new username and logo had me all confused, I couldn't figure out how you seemed to have been here for years/decades, yet I didn't recognize the name.  I finally figured out your provenance, though I'm still baffled by the user name change.

 

Anyway, to my issue.  I created a portable backup drive pool, as described above, with Unassigned Devices back when I was running 6.8, using the directions you linked to above.   I plug it in 2-4 times a year and do a backup, it's fantastic.  Those instructions (which I think you wrote) have since been deleted, since the preferred way is to use the multiple drive pools feature in 6.9.

 

But the functionality in 6.9 is not the same.  If you create a drive pool and then unplug it, Unraid is unhappy about the missing drives.  You can make the warnings go away if you delete the pool, but then you have to make sure you add recreate the pool with all the drives back in the correct order before doing your next incremental.  You also have to stop/start the array to do any changes to the pool.

 

Unassigned Devices did this particular task so much better.  No warnings, don't have to delete the config, just plug it in and mount it, don't have to stop the array.  While I can understand that the preferred method is to use Unraid for multiple permanent drive pools, I don't understand why the documentation for doing it with UD was deleted, as that still serves a niche.

 

I'm trying to help some other users get up and going with the solution I'm using, and since I can't find the documentation I can't fully help them.  I think there were some command lines I used when setting up the btrfs pool as jbod, possibly related to formatting but I don't recall.  I also need to expand my UD backup drive pool soon, almost ran out of space on my last backup so I need a 6th drive, and I'm worried I won't be able to do this correctly without the instructions.

 

Even the UD support thread points to the now deleted instructions, and the internet archive doesn't have any successful copies of the FAQ.

 

Is this something you can help with, or point me to someone who can?

 

Thanks!

Paul

Edited by Pauven
Link to comment
12 hours ago, Pauven said:

You can make the warnings go away if you delete the pool,

You can also just leave the pool as was and start the array without any assigned devices there, there won't won't be any errors and next time you just need to re-assign the same devices to that pool and Unraid will import it, you do need to start/stop the array to remove it though.

 

12 hours ago, Pauven said:

I don't understand why the documentation for doing it with UD was deleted, as that still serves a niche.

I didn't delete the entry, just hid it since I though it was no longer needed, but it's now visible again.

 

https://forums.unraid.net/topic/46802-faq-for-unraid-v6/?do=findComment&comment=462135

 

 

 

 

  • Thanks 1
Link to comment

Awesome, thank you Johnnie (should I still call you Johnnie, or Jorge, or something else?), that's exactly what I needed.  I was surprisingly close in my recreation of the steps based upon my research, but was full of doubt. You're extremely helpful as always.  😊

 

Another user did a test and discovered he was able to mount a pool created in Unraid using UD, no big surprise I guess since these are just standard btrfs pools.  So for some users it might be easier to create the pool using Unraid, remove it, delete the definition, and then use UD from then on for hot-plugging.

 

I would definitely use the Unraid pools feature if it more gracefully handled hot-pluggable backup pools, and didn't require the stop/start.  I'm not complaining, though, since UD does this extremely well.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.