Moving data from Drobo to unRAID

[Josh7890]

I currently have my 6 TB media library on an HFS-formatted Drobo connected to my Mac mini. To copy the data over to unRAID, I'm planning on doing it over the network, but I expect this will take a very long time.

 

Does anyone have any suggestions on how to speed this up? For example, are there any good Finder alternatives for copying files? Are there any ways to mount the Drobo (and HFS file system) on my unRAID server? If I go ahead with copying over the network, would it be better (and possible?) to skip my non-gigabit router and go directly from the gigabit ethernet port on the Mac mini to the gigabit ethernet port on the unRAID server?

[JonathanM]

If I go ahead with copying over the network, would it be better (and possible?) to skip my non-gigabit router and go directly from the gigabit ethernet port on the Mac mini to the gigabit ethernet port on the unRAID server?

I'm not a mac guy, so someone else may have a better answer, but I think your best bet is to plug a good cat5e or cat6 cord directly from the mac to unraid.

 

Before you do that, you will need to give both devices a fixed IP for that interface, in unraid just go to settings, network, and see what is currently in the ip address, subnet mask, default gateway, and dns server fields. Stop the array, go back to that page, and change obtain ip address automatically to no, and just fill in the same info that was there automatically.

 

Do the same thing on the mac, then you can connect them directly and it should work. You may have to use the IP address for unraid instead of the name. I don't know the procedures for setting the static ip on the mac.

[Josh7890]

Thanks! It's similarly easy on the Mac side. I'll go that route unless someone knows of a way to make unRAID read the Drobo directly, which I doubt is possible.

[Josh7890]

Do any Mac users here have suggestions for Finder alternatives that will let me queue up copy jobs, pause and resume, etc?

[archedraft]

If it was me I would use terminal and rsync.

[CHBMB]

If it were me, I'd pull the drives and mount them in Unassigned devices.... (Unsure if it can mount HFS)

 

But then, I'm pretty impatient sometimes.. 

[garycase]

Buy yourself an inexpensive Gb switch [e.g. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122608 ]

 

Then just plug your server, Drobo, and Mac into the switch ... and the switch into one of your router ports.

 

Then you can still use DHCP -- the router will assign addresses -- but the transfers will be at Gb speed

 

[Josh7890]

If it were me, I'd pull the drives and mount them in Unassigned devices.... (Unsure if it can mount HFS)

 

But then, I'm pretty impatient sometimes.. 

 

I'm impatient too! But sadly the HFS file system is wrapped in Drobo's proprietary "BeyondRAID" system.

 

Buy yourself an inexpensive Gb switch [e.g. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122608 ]

 

Then just plug your server, Drobo, and Mac into the switch ... and the switch into one of your router ports.

 

Then you can still use DHCP -- the router will assign addresses -- but the transfers will be at Gb speed

 

Right now I've started copying files over with the Mac and unRAID box connected via Gb ethernet ports and a cat6 cable, using self-assigned IP addresses. (Using Finder to copy a batch of files.) Unfortunately, the network link isn't the bottleneck here. I'm getting somewhere around or under 20 MB/s (Finder doesn't tell me). I'll try rsync for my next batch and see how that works.

 

I suspect I could connect everything to the router again, even wirelessly, and I'd get the same speeds.

[CHBMB]

If it were me, I'd pull the drives and mount them in Unassigned devices.... (Unsure if it can mount HFS)

 

But then, I'm pretty impatient sometimes.. 

 

I'm impatient too! But sadly the HFS file system is wrapped in Drobo's proprietary "BeyondRAID" system.

 

Buy yourself an inexpensive Gb switch [e.g. http://www.newegg.com/Product/Product.aspx?Item=N82E16833122608 ]

 

Then just plug your server, Drobo, and Mac into the switch ... and the switch into one of your router ports.

 

Then you can still use DHCP -- the router will assign addresses -- but the transfers will be at Gb speed

 

Right now I've started copying files over with the Mac and unRAID box connected via Gb ethernet ports and a cat6 cable, using self-assigned IP addresses. (Using Finder to copy a batch of files.) Unfortunately, the network link isn't the bottleneck here. I'm getting somewhere around or under 20 MB/s (Finder doesn't tell me). I'll try rsync for my next batch and see how that works.

 

I suspect I could connect everything to the router again, even wirelessly, and I'd get the same speeds.

 

Have you currently got a parity disk?  If so you could do a New Config and don't assign parity and that will help speed things up some..

[Josh7890]

Have you currently got a parity disk?  If so you could do a New Config and don't assign parity and that will help speed things up some..

 

I disabled the parity disk before I started the transfer. I've also disabled the cache for this share since my 120 GB SSD will fill up quickly.

 

The first batch is turning out to be more like 12 MB/s. At this rate, the transfer is going to take a week (if there aren't any hiccups).

 

Edit: It took pretty much exactly 60 minutes to copy 40 GB. That's about a week for 6 TB.

 

I'm using SMB, with AFP disabled entirely. Is there any chance AFP would perform better than SMB? From what I understand even Apple has switched to SMB as the default between Macs now (with the exception of network Time Machine backups), so I wouldn't expect the old AFP to do better.

 

I suspect the Drobo is the bottleneck, which is unfortunate as it's the only thing I can't change until the data is off it.

[trurl]

Have you currently got a parity disk?  If so you could do a New Config and don't assign parity and that will help speed things up some..

 

I disabled the parity disk before I started the transfer. I've also disabled the cache for this share since my 120 GB SSD will fill up quickly.

 

The first batch is turning out to be more like 12 MB/s. At this rate, the transfer is going to take a week (if there aren't any hiccups).

Sounds like 100Mb ethernet speed to me, check your cables and plugs.

[Josh7890]

Sounds like 100Mb ethernet speed to me, check your cables and plugs.

 

The Mac mini and the unRAID box are both reporting 1000 Mbps ethernet connections.

 

After the next batch I'll try a transfer from the Mac mini's internal drive to unRAID to isolate the Drobo or point my troubleshooting somewhere else.

 

I appreciate everyone's help, btw! This is a very friendly community.

[CHBMB]

Sounds like 100Mb ethernet speed to me, check your cables and plugs.

 

The Mac mini and the unRAID box are both reporting 1000 Mbps ethernet connections.

 

After the next batch I'll try a transfer from the Mac mini's internal drive to unRAID to isolate the Drobo or point my troubleshooting somewhere else.

 

I appreciate everyone's help, btw! This is a very friendly community.

 

What about the connection between the Drobo and the Mac Mini?

 

Yeah, it's pretty chilled in here for the most part.... 

[Josh7890]

What about the connection between the Drobo and the Mac Mini?

 

It's Firewire 800. The only other option is USB 2.0.

 

I'm trying rsync now without any other changes and so far (after just a few files), I'm getting around double the speed - around 23 MB/s.

[littlebluebro]

I have almost the exact same scenario now. I have a Drobo with FW800 and USB 2.0 that I'm trying to migrate all it's data to my new unRAID server. I haven't used the Drobo and years but I tried connecting to my Mac and going over the network. I used both Finder and cp -Rv to copy stuff and after copying a large batch I'd come back to find it stalled out and not seeming to make any progress. I'd have to reboot the Drobo to get things to work again and I'd find that a large chunk of data that had copied over (presumably near the end of the batch) had become corrupted on the destination side. I'm scared now of copying data over and thinking things are intact when there's corruption there and wondering if anyone has any recommendations on validating the data. I could try comparing file sizes of the source to the destination but in a few small tests I did the file sizes matched but still I could see visible corruption in the destination side. Perhaps comparing a file hash?