Hi, I'm thinking about purchasing Unraid and am currently testing the trial version to see if it is something I want to commit to. I have about 20TB of data across 10 drives so, based on what I've read, it would take a long time to transfer all that data over.

 

One of the questions I had that I was unable to find an answer to by searching is whether or not it is possible to change the name of the disk assignment. With the trial version all I see is disk1, disk 2 and parity. Is it possible to change the name "disk1" to something more memorable? I had previously named my drives Alpha, Bravo, Charlie, etc... Just having hard drive assignments makes it a little difficult to correlate with physical location in the server.

 

In an unrelated note, the current server I am using is what I will be converting to the unraid server. I don't have an empty hard drive around to transfer data out of and back into the server (which would also add extra time to the data transfer procss) and it's my understanding that I have to format every drive I put into the server. So any recommendation on how to do this without going out and buying another hard drive to use as an interim transfer?

One of the questions I had that I was unable to find an answer to by searching is whether or not it is possible to change the name of the disk assignment. With the trial version all I see is disk1, disk 2 and parity. Is it possible to change the name "disk1" to something more memorable? I had previously named my drives Alpha, Bravo, Charlie, etc... Just having hard drive assignments makes it a little difficult to correlate with physical location in the server.

As of right now, this is not possible.  You might be able to do it by creating your own SMB shares in a smb-extra.conf file.  I can kind of see the point in naming disks but just take note of the serial number of the drive and you could do the same thing.

 

I think what you are trying to get at though is that you want the drive to show up over the network as Alpha, Bravo, etc correct?

 

 

In an unrelated note, the current server I am using is what I will be converting to the unraid server. I don't have an empty hard drive around to transfer data out of and back into the server (which would also add extra time to the data transfer procss) and it's my understanding that I have to format every drive I put into the server. So any recommendation on how to do this without going out and buying another hard drive to use as an interim transfer?

Short answer no, long answer...

 

Move data around on your current drives to free up at least one drive.  Preclear said drive, add to unRAID array, format said drive, start copy from one drive to the other using something like TeraCopy in check/verify mode.

Move data around on your current drives to free up at least one drive.  Preclear said drive, add to unRAID array, format said drive, start copy from one drive to the other using something like TeraCopy in check/verify mode.

Just to add, if you do not have enough free space to free up a drive, you need another drive anyway. Performance really goes into the crapper when you utilize more than 90% of available space. Also, I can't over emphasize the need to check your existing drives for any faults before you add them to the unraid array. A drive that has been perfectly fine in another application can have problems in unraid because of the way protection is implemented. Unraid uses ALL the bits on the disk, where your previous application may have only used the area that was actively storing data. Personally I would not try to migrate data between servers without having a backup, or verify each file as it's copied before you delete the original. You may see posts describing how to do initial data copying without parity enabled to speed things up, but that's asking for data loss.

"Move data around on your current drives to free up at least one drive.  Preclear said drive, add to unRAID array, format said drive, start copy from one drive to the other using something like TeraCopy in check/verify mode."

 

 

I've never used TeraCopy before but based on the website, doesn't it require Windows to run? Can I run it within Unraid since all the HDs are attached to the Unraid server even though they are not formatted or in the array?

 

I shifted data around and am now preclearing 1 drive. Preclearing is sufficient to ensure a drive is in working condition right? It's been about 15 hours now and still not complete. This process is taking forever and was hoping for a faster way to do it. This may be a bad place to ask this but are there other alternatives that are as flexible as unraid? Previously I was using Flexraid but that was a snapshot system rather than a live backup system.

I've never used TeraCopy before but based on the website, doesn't it require Windows to run? Can I run it within Unraid since all the HDs are attached to the Unraid server even though they are not formatted or in the array?

 

I shifted data around and am now preclearing 1 drive. Preclearing is sufficient to ensure a drive is in working condition right? It's been about 15 hours now and still not complete. This process is taking forever and was hoping for a faster way to do it. This may be a bad place to ask this but are there other alternatives that are as flexible as unraid? Previously I was using Flexraid but that was a snapshot system rather than a live backup system.

As you've seen, TeraCopy is a windows program, and would be used to copy data over a network connection to the unraid server. Since presumably you will be using a network to access your data from unraid, it would be a good idea to be sure that your data is not being corrupted as it passes over the network. You haven't said what format your drives are currently using, but you could probably mount them using the community authored SNAP addition to unraid, then copy the data with a networked machine to the unraid protected drives.

 

Preclearing exercises the entire drive and verifies it's capable of holding data reliably. It takes a long time because it's reading and writing the entire drive. Anything less and you can't be sure that unraid can accurately use the drive to reconstruct a failed drive, and all other drives in the unraid array must be healthy to get back a single failed drive.

 

You seem to be in a rush, which is fine if you don't care about the integrity of your data. The process of moving 20TB of data to unraid is going to be a long slow operation if you do it in a safe manner. If you have backups of all your data elsewhere, then let us know and we can give you shortcut methods that will speed things up, at the risk of losing your data if anything goes wrong.

 

One of the questions I had that I was unable to find an answer to by searching is whether or not it is possible to change the name of the disk assignment. With the trial version all I see is disk1, disk 2 and parity. Is it possible to change the name "disk1" to something more memorable? I had previously named my drives Alpha, Bravo, Charlie, etc... Just having hard drive assignments makes it a little difficult to correlate with physical location in the server.

 

While you can not that I am aware of, you can hide the default share of "disk1, Disk2, disketc." and make newshare called alpha, bravo, etc. and assign it to a single drive. essentually doing the same thing.

 

I shifted data around and am now preclearing 1 drive. Preclearing is sufficient to ensure a drive is in working condition right? It's been about 15 hours now and still not complete. This process is taking forever and was hoping for a faster way to do it.

 

Preclear is optional, but it ensures your drive is in good condition and ready to use in unraid.

The best solution is to pick up a few extra drives and preclear those all at once, transfer data, then preclear the next batch.. transfering data and precleaing the next drive(s) at the same time is the most efficient/fastest.

 

almost any software raid or hardware raid based system will take a lot of work and time to migrate that much data if you have no extra drives.. some raid arrays /drive  pools you cant even expand once you build it.

You haven't said what format your drives are currently using, but you could probably mount them using the community authored SNAP addition to unraid, then copy the data with a networked machine to the unraid protected drives.

 

My drives are in NTFS format. I was under the impression that SNAP was for drives that are connected to the Unraid machine. If I'm transferring data from a networked machine why would SNAP be necessary? Sorry if that's an ignorant question but I'm just reading through the Wiki and there's a lot of information to figure out. I figured transferring data using SNAP by mounting drives to the Unraid machine would be faster but I wouldn't be able to transfer with Teracopy. If I used Teracopy then I would have to remove the HDs from the Unraid machine, hook it up with USB to the other machine, and transfer through the network. That seems like it would take much longer. I would buy more HDs but have been waiting for prices to drop since the flood in Thailand.

 

I'm not really in a rush, but it just seems like it could take an entire month for me to move all my data over. Lots of things could happen in that time. I don't have a backup elsewhere. Was previously using Flexraid but the whole thing got disabled when they decided to move to a pay version. I figure if I was going to pay for something I'd like something more dependable and with more support. Always thought about doing Unraid but Flexraid seemed sufficient.

You haven't said what format your drives are currently using, but you could probably mount them using the community authored SNAP addition to unraid, then copy the data with a networked machine to the unraid protected drives.

 

My drives are in NTFS format. I was under the impression that SNAP was for drives that are connected to the Unraid machine. If I'm transferring data from a networked machine why would SNAP be necessary?

It is not needed if you are performing copies to unRAID over the network.

Sorry if that's an ignorant question but I'm just reading through the Wiki and there's a lot of information to figure out. I figured transferring data using SNAP by mounting drives to the Unraid machine would be faster but I wouldn't be able to transfer with Teracopy.

True

If I used Teracopy then I would have to remove the HDs from the Unraid machine, hook it up with USB to the other machine, and transfer through the network. That seems like it would take much longer. I would buy more HDs but have been waiting for prices to drop since the flood in Thailand.

If you used Teracopy you would NOT need to physically move the existing drives to the new unRAID server.

 

I'm not really in a rush, but it just seems like it could take an entire month for me to move all my data over. Lots of things could happen in that time. I don't have a backup elsewhere. Was previously using Flexraid but the whole thing got disabled when they decided to move to a pay version. I figure if I was going to pay for something I'd like something more dependable and with more support. Always thought about doing Unraid but Flexraid seemed sufficient.

If you can only move a single disk's worth of data at a time, you are correct.  It will take longer.  Fortunately, you can queue up all the transfers for a disk, then let it run while you sleep.  I personally copied about 5.5TB of data from my older server to my new one.  Only took a few days for the actual copy, but it was simplified in that the new had all the disks initially installed I'd need for the copy.

 

Joe L.

Ok, maybe I misunderstood. What I read into your first post was that you were booting your current server using the unraid boot flash, and wanted to convert the drives into the unraid array one at a time. My suggested process was to temporarily assign a network share in unraid to each ntfs data drive one at a time using snap, and copy using teracopy over the network from the temporary snap share to the permanent unraid share on the same machine. That would accomplish multiple goals at the same time, first it would stress the network connection and drives by copying the data across the network, and also allow you to have two copies of your data until you finish copying the NTFS drive, verify the copy was intact, then you could start the preclear on the newly empty drive, add it to the unraid, and start the next drive.

 

This presupposes you have your largest drive already precleared and installed as your parity drive, and another blank drive assigned as your first data drive ready to receive the data from your first NTFS drive.

 

If you only have one blank drive with no parity assigned, you are running too great a risk in my opinion.

 

If you have two machines running, then teracopy from the old server to the unraid server.  The time you might gain by mounting the drives internally to the unraid server isn't worth the risk of losing data. I've seen MANY posts here by people inducing drive errors by loose cabling while physically moving drives around. It's best not to disassemble a working server until you have a full backup of the data. If that's not possible, I'd suggest taking any precaution you can to keep your data safe.

Thanks jonathanm! I think that was a little more clear. So what you're saying is for me to have at least a parity drive and an empty drive in an array. My drive with data would be in the unraid machine but not part of the array. Use snap to mount that drive. Then from a Windows machine that is on the same network, move data from the mounted drive to the array with teracopy. Then preclear that drive and add it to the array. Keep doing that for all the drives.

 

If that's correct then I'll try that. I'll probably have to pick up another hard drive though, I have enough space to leave one HD clear but don't have enough to have a parity drive also. Is there really that great a danger of losing data when initially transferring without a parity drive?

 

On another unrelated note, does Unraid have a feature to export a list of all the items in a drive so you know what you need to recover in the event of a two hard drive failure?

Thanks jonathanm! I think that was a little more clear. So what you're saying is for me to have at least a parity drive and an empty drive in an array. My drive with data would be in the unraid machine but not part of the array. Use snap to mount that drive. Then from a Windows machine that is on the same network, move data from the mounted drive to the array with teracopy. Then preclear that drive and add it to the array. Keep doing that for all the drives.

 

That sounds like what I understood you were trying to do in the first place. My apologies if you were trying to ask about a different method.

 

If that's correct then I'll try that. I'll probably have to pick up another hard drive though, I have enough space to leave one HD clear but don't have enough to have a parity drive also. Is there really that great a danger of losing data when initially transferring without a parity drive?

 

Well, danger of losing data is relative, so I'll turn the question back to you like this. Is the risk worth the added cost of picking up a drive now at flood inflated prices versus what you will eventually pay per MB when you run out of room in the future? You will most likely buy another drive anyway later when you run out of room. If you buy it now, you can use it to lower your risk during the transfer.

 

How much is your data worth to you?

 

On another unrelated note, does Unraid have a feature to export a list of all the items in a drive so you know what you need to recover in the event of a two hard drive failure?

Not built in as an easy to push button, no. It would be a fairly easy thing to send a recursive directory list to a text file every once in a while though.

ls -R>list.txt

executed in the directory you want to list might work. Each disk shows up as a directory under /mnt when you are logged into the console either physically or telnetted in.

I would NOT mount the drive via SNAP and then use teracopy.  That would copy the data from unRAID to your PC and then back.  It would take twice as long.

 

As already mentioned by another, leave the drives in your current Window's PC until you verify it is safely copied to the unRAID server and protected by parity.  You are FAR more likely to cause data loss in moving drives physically between servers. 

I would NOT mount the drive via SNAP and then use teracopy.  That would copy the data from unRAID to your PC and then back.  It would take twice as long.

 

As already mentioned by another, leave the drives in your current Window's PC until you verify it is safely copied to the unRAID server and protected by parity.  You are FAR more likely to cause data loss in moving drives physically between servers.

 

Now I'm totally confused. I thought he said the drives were already in his current server, which he wants to migrate to use unraid instead of whatever current OS he is using (he never said). I took that to mean he didn't have the ability to boot both systems, because there was only one system.

Now I'm totally confused. I thought he said the drives were already in his current server, which he wants to migrate to use unraid instead of whatever current OS he is using (he never said). I took that to mean he didn't have the ability to boot both systems, because there was only one system.

 

jonathanm is correct. I only have one system. I do not have a separate Unraid and Windows system. I am turning the Windows system into Unraid. If using Teracopy from a separate PC would copy the files from Unraid -->Windows--> Unraid, that would be redundant. Should I just remove the harddrive from the Unraid machine, hook it up to my windows machine via USB and Teracopy it over? Or should I just leave it in the Unraid machine, mount it with SNAP, and copy it over to a drive in the array?

I think this quote-

You are FAR more likely to cause data loss in moving drives physically between servers. 

means Joe really does agree with me, he just doesn't see it yet. 

 

If you do it my way, you never even touch the drives and cabling, you just assign them to their roles using software as you migrate the data.

 

There are faster ways to copy the data internally, but then you will never know until it's too late if you have an issue with your particular network setup. If you migrate a drive or two and encounter no issues, then maybe I would start copying using MC or rsync in a screened telnet session. With teracopy over the network, you get feedback if a copy fails the crc check, and can investigate why. Silent data corruption when everything looks good is the worst, you never know how much data you lost.

 

If using Teracopy from a separate PC would copy the files from Unraid -->Windows--> Unraid, that would be redundant. Should I just remove the harddrive from the Unraid machine, hook it up to my windows machine via USB and Teracopy it over? Or should I just leave it in the Unraid machine, mount it with SNAP, and copy it over to a drive in the array?

 

Yes, that is redundant, and I suggested that on purpose to fully stress your network. If you remove the drive, you are still copying over the network, but now you add the possibility of losing data with the physical removal and USB connections. I'm pretty sure snap over the network is faster than most USB HDD adapters unless you are using USB 3.0.

 

Doing one drive using one method doesn't exclude trying another method for the next drive. As long as you don't erase the source disk before you are sure the data is intact on the destination, you should be fine.