I am ready to build a new UnRaid system but I really need to use my existing WHS server, which will be replaced by unraid when finished.  

 

I copied all of the stuff (photos, software source disks etc) off of the WHS to a drive on another system but I have three 1 gig disks with video, just plain unprotected, shared disks.  

 

What I need to do is start UnRaid and assign the three brand new 1 gig disks that I purchased for the UnRaid system, let it build the UnRaid storage pool, assign a handful of other 640 gig drives I have hanging around, let it all settle down and then...

 

attach these disks directly to the UnRaid server (not add to the storage pool) and copy the contents of these video disks onto the unRaid storage pool.

 

Yes, I could find another machine, add them into one of my other machines and then copy them across the network but it would just be a lot easier / cleaner to directly attach to the UnRaid system as non storage pool disks and drag and drop the files from them into the storage pool.  Probably a lot faster too as it is not across the LAN.

 

So is this possible?  I have read all the FAQs etc but see nothing about doing something like this.

 

Any assistance in doing this would be much appreciated.

 

Thanks,

 

JWC

You did not look in the "How To" section of the wiki.

 

Here is how to do it using command line commands:

http://lime-technology.com/wiki/index.php?title=Copy_files_from_a_NTFS_drive

 

You'll want to use the midnight-commander

mc

built into unRAID to perform the actual copy of the files from your old disks to the unRAID data disks.

 

If you install unMENU it will perform all those steps for you with you just needing to click on a few buttons on its web-interface.  

The link to the wiki entry is here: http://lime-technology.com/wiki/index.php?title=UnRAID_Add_Ons#UnMENU

Here is an example image of its disk management page showing a 250Gig NTFS USB drive I have mounted using it.

http://i35.tinypic.com/2w3n5zp.jpg

 

Good luck in getting everything assembled and configured.

 

Joe L.

Let me start by saying that I am by no means an expert but I did find some info that you might find useful.  I'm assuming that you have been using these drives with windows and they are NTFS formatted.  From what I've read you can mount NTFS formatted drives in unRAID and copy the files from them onto a disk in the unRAID array.  See this wiki where a guide is provided to assist you.

SNAP is an option.

This is a "new build".  I have three brand new 1 TB drives about to be delivered(today).  I have two 1.5 TB drives with video on them (completely full) and another 1 TB drive partially filled.

 

Is it possible to add two new 1 TB drives, get them set up *without* parity, copy data from a 1.5 TB drive onto the two 1 TB drives, then drop that 1.5 TB drive in as the parity disk? 

 

IOW I eventually want to use one of the 1.5 TB drives as the parity drive, but they are both full.  I am willing to risk the possibility of losing data for the day or two it will take to do it this way but I do not know how to proceed.

 

I have read that you can do things without the parity drive and add it later but I cannot seem to find what it was I read.  I am not finding any instructions on running without parity in the manual.

 

jwc

Yes,

SNAP is an option.

 

SNAP lets you mount a NTFS drive outside the parity protected array.  You can then copy the data over to the unRAID disks.  You can assign parity right away, or you can wait until all your data is copied over, that's your choice.

 

So here's what you can do:

1) Remove all NTFS drives from your former WHS.  Install the two new 1 TB drives.  Preclear them (optional, but highly recommended - if they don't pass preclear, don't trust them with your data).

2) Start your new unRAID array with both 1 TB drives assigned as data drives (disk1 and disk2).  Don't assign a parity drive.

3) Use SNAP to mount one of the 1.5 TB NTFS drives outside the unRAID array.  Use Midnight Commander to copy all the files over onto the 1 TB data drives.  You'll have about 500 GB of free space left between the two data drives after this is done.

4) Preclear the 1.5 TB drive (again optional but recommended).  This is the point where your data is at risk (though no more at risk than it was on your WHS).

5) Assign the precleared 1.5 TB drive as data disk3.  You should now have about 2 TB of free space between all three data disks.

6) Use SNAP to mount the other 1.5 TB NTFS drive.  Again, use MC to copy over all the files onto the data disks.

7) Preclear the second 1.5 TB.

Assign the second 1.5 TB drive as a parity drive.  Start the array, and let the parity sync complete.  Once it does, run a parity check.  Once this parity check completes with no errors, your data is protected.

 

Alternatively, you could mount the NTFS drives onto a Windows or Mac computer and copy the data to the unRAID array over the network.  This will be much slower, but it may be easier for you (no need for Midnight Commander).  If you do this, use TeraCopy on Windows and enable the 'test' option.  This will run CRC checks on all your data after the transfers complete to ensure that the data all transferred over without corruption.

Thanks for the detailed outline of how to get this done. 

 

My video has been unprotected on the WHS and the window for failure is just a few days as I do this stuff.  It just seems like doing the parity at the end is the way to go.

 

Thanks again,

 

jwc

I agree.  The only way in which you may be putting your data at more risk than your current WHS situation is that you are trusting some of it to the brand new 1 TB drives.  Some drives fail within the first month, whereas others last years.  That is one of the primary purposes of the preclear script - to catch those that fail early.  So again, I HIGHLY recommend that you preclear at least the two new 1 TB drives before trusting any data to them.  It is less important that you preclear the 1.5 TB drives since you have been using them for a while already, and I presume that you already trust them.  Still, if you are patient, it wouldn't hurt to preclear all of them (at the right time and in the order I explained above, of course).

If you wanted to reduce your risk you could install your third 1TB drive and compute parity with it before proceeding to step 4 in Rajahal's instructions.  This way if either of the two 1TB data drives failed while you were precelaring or computing parity you could put the third 1TB drive back in and you could reconstruct the lost drive thus avoiding any data loss.  This can be done but it's more work to avoid what amounts to a small risk.  Not sure if it's worth the extra effort but I wanted to make you aware anyway.

I am actually copying the data off of one of the 1.5 tb drives as we speak, and if the preclear takes awhile I may have the 1.5 TB drive cleared and ready to use by the time I am ready to do the build.

 

On another note, does anyone know how much cores / memory affects the speed?  It just happens I am using an "old" server that has a quad core AMD and 16 gigs of RAM in it.  I will likely replace the quad core with an older dual core and yank some of the memory if it is going to be wasted in the UnRaid. 

 

For example can UnRaid preclear multiple drives simultaneously?

 

jwc

On another note, does anyone know how much cores / memory affects the speed?

It will have a big impact on the speed at which your power meter turns. 

 

A stock unRAID system will run perfectly on a single core CPU and 1 GB of RAM.  If you are going to use a lot of add-on apps then you might benefit from a dual core CPU and more RAM (maybe 4 GB max) depending on what you are doing.  Anything beyond that is probably big time overkill.

On another note, does anyone know how much cores / memory affects the speed?

It will have a big impact on the speed at which your power meter turns. 

 

A stock unRAID system will run perfectly on a single core CPU and 1 GB of RAM.  If you are going to use a lot of add-on apps then you might benefit from a dual core CPU and more RAM (maybe 4 GB max) depending on what you are doing.  Anything beyond that is probably big time overkill.

 

LOL.  Probably not.  I have a home office with a pretty massive SQL Server, 20 hard drives etc.  I will be dumbing it down more so that I can use the hardware elsewhere than that it will noticeably affect my power bill.

 

hwc

OK, am I missing something?  I pays my money and I waits...  :'(

 

What am I waiting for?

 

And I don't even see a place to enter a key in the user interface... 

are you looking for the license key?

  Once your order has been processed, you will receive a separate email from Lime Technology with your registration key data file(s) attached. Please allow 24-48 hours for receipt of your registration key (usually the same business day). If you have not received your key, please check your email spam filter.

 

To install the Registration Key, simply copy the key data file to the config directory of your Flash and reboot your server.

Other info..

http://lime-technology.com/wiki/index.php?title=FAQ#How_is_unRAID_licensed.3F

OK, am I missing something?  I pays my money and I waits...   :'(

 

What am I waiting for?

 

And I don't even see a place to enter a key in the user interface...   

You will get back a .key file.  You will simply copy that file to the flash drive's config folder and then stop the array and reboot.  Then you'll have the ability to add more drives, etc.

 

You can use file-explorer in windows to get to the config folder.  It is at

\\tower\flash\config

 

To answer a previous question of yours, you can preclear up to 6 drives simultaneously.  unRAID doesn't really preclear the drives, you do by running the preclear script written by Joe L.  unRAID has no knowledge of any preclear processes happening in the background save for the final step when you add the precleared drive to the array and unRAID recognizes a special signature that tells it the drive has already been cleared.

 

To preclear multiple drives at once, either at the system console or in a telnet session you would:

1) Start the first drive preclearing as normal.

2) Press alt-F2 to switch to a new login console.  Log in again and start the second preclear.

3) Press alt-F3, login, start the third preclear...

...etc.

 

You can then press alt-F1 through alt-F6 to check on the progress of each of your preclears.  It is recommended that you run preclears either from the system console or using screen (available through unMenu, another of Joe L's beautiful creations).  While you can run preclears from telnet over the network, if your computer goes down for some reason (power loss, accidental restart, etc.) the preclear sessions will all be canceled partway through.  If that happens you just have to start over again, there's no way to pick them up from where they left off.  It doesn't hurt the drives at all, it is just annoying.

 

You don't need any serious hardware to preclear multiple drives at once, just enough open SATA ports.  I believe a few users reported problems when using very small amounts of RAM (128 mb or so), but it doesn't sound like that will be a problem for you.  Technically unRAID requires a minimum of 512 mb anyway (even though it will actually run on less).

Technically unRAID requires a minimum of 512 mb anyway (even though it will actually run on less).

Although unRAID will run in 512Meg, and perhaps a bit less I've seen several posts where 512Meg was NOT sufficient to run a file-system-repair on a 2TB drive.  The data structures needed on the 2TB drive were just too huge to fit in memory and the file-system repair tool just crashed when the repair was attempted.

 

The user ended up upgrading the memory to 2Gig in order to repair their disk corruption.

 

With 2TB drives, I do not feel 512Meg is enough.... Yes it will run but you might find yourself in a bind if you ever need to rebuild the file-system tree using reiserfsck --rebuild-tree.  I do not know if 1TB of ram is enough...  Today with memory being relatively inexpensive, I'd go with 2GB or more of RAM.

 

Joe L.

...I do not know if 1TB of ram is enough...

 

Guess we're all screwed then

More info on multiple "screen" sessions..

 

There are several ways to do this:

 

1. use multiple "telnet" sessions to log onto unRAID.  Run one preclear_disk.sh script in each session.   (This is what I usually do)

 

2. Log into the system console using mutiple "consoles"  (Control-Alt-F1 through Control-Alt-F6 will switch between the six availale system consoles)

Run one preclear_disk.sh per console.  (Switch between them as needed to review their progress)

 

3. Install and run "screen" a program designed to allow you to have as many virtual "screens" as desired and switch between them with a hot-key-sequence.   It is described in this post: http://lime-technology.com/forum/index.php?topic=2817.msg24825#msg24825

 

Once you invoke it with "screen" you can start up a preclear_disk.sh, then type "Control-A c" to get a new console, start another preclear_disk.sh, type "Control-A c" to again get a new virtual screen, start a third preclear_disk.sh, etc.

 

You can at any time type "Control-A n" or "Control-A p" to switch to the next or previous virtual screen to track their progress.  You can type "Control-A ?" to get a list of possible commands to manage the screen consoles.

 

A brief tutorial on how to use screen is here: http://www.rackaid.com/resources/linux-tutorials/general-tutorials/using-screen/

 

You can even detach from screen, allowing you to close the telnet session and re-attach later.  To detach type "Control-A d"   Then, as a later time, type

screen -r to re-attach.

 

Another good article on "screen" can be found here:

http://www.linuxjournal.com/article/6340

 

It can do a lot more. You can "name" the screen sessions, list the sessions

Control-A "

(Control-A followed by a "quote")

 

Joe L.

 

I like this part of screen

You can even detach from screen, allowing you to close the telnet session and re-attach later.  To detach type "Control-A d"   Then, as a later time, type

screen -r to re-attach.

 

because I would have a tendency to do this:  

 

Sorry for all what may seem like simple questions. I'm just not familiar with Linux/UNIX but I am learning here. Another question I have is about logging into a telnet session that is actively running a command. I just started a pre-clear on a drive but I used OS X to telnet over to the server to run it. I can see that it shows me the progress. I will need to close out of my telnet window which I am assuming the pre-clear will still be running.

You are assuming incorrectly. The pre-clear will terminate when you end the telnet session.

I should actually confirm that. Would it still continue to run the pre-clear? And if it does, how can I telnet back in and view the progress?

You can't.  It is terminated when you log out.

 

Either

1. Do not log out

or

2. Use the system console  (you have 6 virtual consoles accessible as Alt-F1 through Alt-F6)

or

3. Use the "screen" program to manage virtual screens so you can disconnect and re-connect at will.

It is described here: http://lime-technology.com/forum/index.php?topic=2817.msg24827#msg24827

 

 

You guys do understand that this is a bit daunting for the non-linux guy (me).

 

I need the functionality so I will prevail but sheesh...

 

I built my first two computers with a soldering iron and parts from Popular Electronics - 1976 and 1981.  I ran CPM and learned to program with Turbo Pascal on the CPM home built system.  I build all of my own stuff, from parts from Newegg.  I program in C# and .Net for a largish SQL Server system that I put together from parts - a dual core motherboard with an 8 core AMD (6000) and 16 gigs RAM with another 16 gigs on order, running Windows 2008 and SQL Server 2008.

 

I am not a technophobe...

 

 

And... I have firefox open with about a dozen tabs -

 

1) The manual (jumps right in but never does anything remotely like 1( Add a disk to the system.  2) Open the interface in a browser by... 3) Click X to cause it to start adding the disk...

2) The Unraid add-ons

3) BJPs "high level list of steps"

4) a half dozen threads from the forums

 

Man you gotta want this bad.   And I do! 

 

And then I am reading the "bitching" threads about how the author is absent... (what have I gotten myself into?)

 

But I am here, I have purchased the Pro key and I will get this thing to work.  I will need some hand holding without a doubt.

 

I have to say the forum is an awesome group of folks contributing to the cause.

 

OK, back to business...  and thanks for the pointers.

 

jwc

You guys do understand that this is a bit daunting for the non-linux guy (me).

I don't know anything about Linux either.  That's why I built a test server out of some old HDDs (one of them was a 13GB IBM Deskstar) and played around with it for a few weeks to make sure that I could hack it.  Depending on your situation you may not have that luxury.  I'm now experimenting with some add-in features.  My recommendation is to take small, methodical steps.  Start small with just a few drives and get your server up and running.  Once you're more comfortable with unRAID you can start to add drives and move over all your data.  The last thing I'll say is that if it were not for the support/help available on this forum I would not have even attempted to use unRAID.  If I can successfully build an unRAID server anybody can!

The author is absent? That is news to me. I am int the midst of deploying this and hope that isn't the case. I have a pretty significant investment in this hardware i am about to buy a license for real.

So I was booting into UnRaid.  Added two disks.  Got in, got as far as formatting them.  Copied a bunch of stuff trying to set up UnMenu.  Ended up rebooting and then... no boot out of the bios.  Somehow adding the disks pushed the SanDisk flash down in the hard drive order.  Finally found that and pushed it back up to drive one and it boots again.

 

I got my key and dragged it over and rebooted, I am now seeing "Unraid Server Pro".  Yea.

 

I'm trying to get the UnMenu thing happening but the instructions assumes user / operator linuxness.   I have no linuxness.  

 

We definitely need a "How to for Windows people".

 

jwc

LOL.  The author exists, that was a flippant remark.  I will leave others to say more.

 

jwc

 Ended up rebooting and then... no boot out of the bios.  Somehow adding the disks pushed the SanDisk flash down in the hard drive order.  Finally found that and pushed it back up to drive one and it boots again.

What motherboard do you have? The ASUS board I have allows you to "Force" (& retain) a boot order. There may be a similar option in your MB bios.

 

I'm trying to get the UnMenu thing happening but the instructions assumes user / operator linuxness.   I have no linuxness.  

Where are you stuck?

We definitely need a "How to for Windows people".\

jwc

Sounds like you are volunteering once you get it figured out!