Jump to content

Best way to load unraid server with data?


oksurb

Recommended Posts

Hey everyone,

 

Just finishing up building my 1st unRAID box and will be starting to load it soon.

 

Was wondering what most people do...do you just copy files (movies, say) from a separate pc, like a windows pc, over the network, to do people actually put DVD drives in the unRAID box and somehow load from DVD?

 

Hope that made sense...thanks!

 

Andy

 

Link to comment

definately want to copy across the network. DVDs have a slower transfer speed (i believe) than the network, plus you'd need to burn the DVD then copy it off.

 

There are some people that disable their parity for the initial data transfer, simple because the write speeds are quicker...though if you have a drive failure you will loose data.

 

If you have a relatively small amount of data you could copy it to a HDD and put the HDD inside the UnRaid and copy it to the array using the console, but i think the network is the best way

Link to comment

Over the network is easiest... If you really have a motherload of data you could add the drives containing the data to the unraid server and copy them inside the server towards the array drives.. This will be a bit quicker..

 

Most easiest however is to bite the bullet and use the network... Use a tool like teracopy instead of plain vanilla windows to avoid a broken copy makes it necessary to restart the whole copy sequence ..

Link to comment

unRAID has no drivers to mount/read/rip DVDs.  Do not even bother to add a DVD drive to the unRAID server, as it will be useless unless you decide to install a full version of Slackware and compile your own kernel.

 

I would NOT disable parity, as with it ENABLED any un-readable disk blocks should be re-allocated as you load your files.  Without it enabled, you just get corrupt files if a data disk has an un-readable sector.    use of the pre-clear script should find any un-readable sectors, but many people skip that step, trusting the new drive.  That is, in my opinion, a mistake.  (one recent post told of a 33% failure rate in a set of 6 new drives)  I do not trust ANY new drive.

 

Use the network to copy your files, use a program like "terracopy" to perform the copy and verify the transfer was error free.

 

Joe L.

(translation... adding a DVD file system driver and the needed utilities and tools to read DVDs is VERY Complicated, not for beginners, or even fairly experienced...  I don't think I've read of anybody ripping DVDs from a drive on the unRAID box itself using a custom kernel.)

Link to comment

Hey guys...sorry for not getting back sooner.  Had to walk away from this thing for a while.  Having the same issue a lot of noobs have in being able to access it thru my browser on my windows machine...but that's for a different post...SMILEYFACE!!!

 

Anyway...thank you very much for your input on my question...message received!

 

Thanks,

Andy

Link to comment

just to add, if your copying from a windows machine.  download and use teracopy (its free)

 

its liked a souped version of copy

 

Advantages are:

Copy files faster.

Pause and resume file transfers.

Error recovery. In case of copy error, TeraCopy will try several times and in the worse case just skip the file, not terminating the entire transfer.

http://codesector.com/teracopy

 

Thanks for this. I'll try this out.

 

I've had a few files fail on me when I've copied them so this will be handy.

Link to comment
  • 2 months later...

Hi all,

 

I'm also about to move data over to my freshly built Unraid and was going to use Terracopy as suggested. Question though...

 

Most of my data is on a 2TB DNS-323 attached via cat5 to my router. If I use Terracopy from my PC (wireless N), will the data first move through the PC enroute to the Unraid? Or will it copy directly from the DNS to the Unraid?

 

Basically, what would be the most efficient way to move the DNS323 data over?

 

Thanks.

Link to comment

Hi all,

 

I'm also about to move data over to my freshly built Unraid and was going to use Terracopy as suggested. Question though...

 

Most of my data is on a 2TB DNS-323 attached via cat5 to my router. If I use Terracopy from my PC (wireless N), will the data first move through the PC enroute to the Unraid? Or will it copy directly from the DNS to the Unraid?

 

Basically, what would be the most efficient way to move the DNS323 data over?

 

Thanks.

unless you have a login onto the NAS, and access to a copy command on it, the only transfer method is via your PC.

 

Just use Teracopy, assign the parity disk in the unRAID server before transferring the data, and just queue everything up in Teracopy and let it run overnight while you sleep.

 

Joe L.

Link to comment

Well... you would need to set a preclear signature, then format riserfs and then hope it works.. And you will need to set parity afterwards... I guess it could be possible but it sounds like a collossal waste of time and effort..

 

In any case this will take a ton of time to transfer this much data, the way I did it how Joe described. Just start the copy before you go to bed.. If you need to do a couple of drives it will take a couple of days.. Much easier and fully standard so less risk.. Use something like teracopy though to make the transfer more robust..

Link to comment

Well... you would need to set a preclear signature, then format riserfs and then hope it works.. And you will need to set parity afterwards... I guess it could be possible but it sounds like a collossal waste of time and effort..

 

In any case this will take a ton of time to transfer this much data, the way I did it how Joe described. Just start the copy before you go to bed.. If you need to do a couple of drives it will take a couple of days.. Much easier and fully standard so less risk.. Use something like teracopy though to make the transfer more robust..

Actually you don't need the preclear signature, that's only used to tell unraid that the disk you are adding to an already parity protected array is currently all zeros so parity is still valid. As soon as any data is added, the preclear is no longer valid, and parity would have to be recalculated based on the contents of the disk.

 

If you partition and format the drive in a system that can properly set up a single reiserfs volume on the drive, it should be able to be inserted into an UNPROTECTED unraid array without issues, and parity calculated after the fact.

 

There is a much better reason to copy files over the network though. It is much better to beat on the system and verify it works properly BEFORE you delete your source data. It would suck to load precopied drives into the box, and only find out afterwards that something is wrong with the system and it is corrupting your data. Since the primary use is going to be accessing the data over the network, it serves as a good proofing method to load and verify that data over the network to begin with.

Link to comment

One of the experts could explain if you can format the drives ReiserFS then put them in your windows box and load them that way first?  :-)

 

I'm curious to know if that's possible too!

 

Russell

In addition to all that the others have mentioned (having to first create an unRAID specific reiserfs file-system on the disk) there are no drivers for windows that are able to "write" to reiserfs file-systems.

 

Best you can do is boot your window's PC with a copy of unRAID, mount the NTFS drive and the target reiserfs formatted data drive, then copy using "mc" or native linux copy utilities. (cp, or rsync)

 

However, risk exists you'll clobber your window's hard-disk.

 

Joe L.

Link to comment

Hi all,

 

I'm also about to move data over to my freshly built Unraid and was going to use Terracopy as suggested. Question though...

 

Most of my data is on a 2TB DNS-323 attached via cat5 to my router. If I use Terracopy from my PC (wireless N), will the data first move through the PC enroute to the Unraid? Or will it copy directly from the DNS to the Unraid?

 

Basically, what would be the most efficient way to move the DNS323 data over?

 

Thanks.

 

Raid 1 or 0?

 

If it's Raid 1, you can do what I did.  I shutoff the DNS323 and pulled one of the drives.  You can mount the drive in the system and copy it locally.  At that point you're looking at hours instead of days.

 

Alternative is to login to the unraid box console and use a FTP or network mount to the DNS323 and copy the data over.  In that case, you aren't dependent on your desktop.

Link to comment

Archived

This topic is now archived and is closed to further replies.

×
×
  • Create New...