Migrating from Windows


Recommended Posts

I'm looking to finally set up unRAID, and I've got some questions.
My current volumes look like this (Windows software raid):
 

Volume A - 2 disk mirror, 4TB usable:
Disk 1 - 4TB
Disk 2 - 4TB
 

Volume B - 2 disk mirror, 4TB usable:
Disk 3 - 4TB
Disk 4 - 4TB


These was set up 5 years apart, so it made sense at the time to do it like this. For various reasons, I'd like to only have 2 data drives and 2 parity drives. 
As I understand it, the array will continue to work even if both data drives fail, and only the 2 parity drives are working. So it doesn't matter which physical drive is assigned what role (I have 2 new drives, 2 old).

Does it make sense to set up the drives in this order?:

  1. Remove disk 2/4 from my Windows machine, they will hold the only copies of my data.
  2. Install unRAID, preclear disk 1/3.
  3. Set up a new volume with 1 data drive, and 1 parity drive.
  4. Connect disk 2.
  5. Use the "Unassigned devices" plugin to mount disk 2 (NTFS file system).
  6. Move data from disk 2 to my unRAID array. (How do I do this, can I do it natively in unRAID, or do I need the Krusader plugin?)
  7. Preclear disk 2
  8. Expand my array (It will now have 2 data drives, 1 parity drive).
  9. Connect disk 4, mount it, and move data to my array.
  10. Preclear disk 4, set it up as 2nd parity drive.


I'll have to migrate Plex and qBittorrent to Docker. Plex should be relatively easy, but I can imagine having to add all torrents in qBittorrent manually since I'm moving from Windows to Docker.


To sum up:

  1. Does it make sense to move my data in this order?
  2. How do I set up the filesystem on the individual disks. Is everything done by setting up shares, or can I manually create folders?
  3. Do I use "Unassigned devices" and "Krusader" plugins to move data to my array? I don't want to accidentally loose anything...
  4. Is there a way to migrate torrents from qBittorrent Windows to Docker?
  5. Can I see the GUI of programs running in Docker? For example, how do I see the qBittorrent UI? (apart from the Web UI).
  6. Is there a popular way of doing backups? I'm considering running backups from a Windows machine (network or VM) to be able to use SpiderOak.
  7. How does fan control on a normal pc case work? Is there a way for unRAID to control fans connected to motherboard fan connections? Or perhaps I will need to use a dedicated fan controller and USB-passthorugh to a VM.
  8. Anything else I must know about before installing?

 

Thanks for any and all feedback!

Link to comment
31 minutes ago, Ademar said:

I don't want to accidentally loose anything...

What you have outlined may work, but has a high risk of failure due to any number of things going wrong.

 

Windows RAID or Unraid do not mean you have a backup, it's high availability where a drive failure can be tolerated without losing data. You are proposing operations that can cause data loss quite easily just by typing or clicking something incorrectly.

 

I STRONGLY recommend not breaking the Windows RAID until all your data is copied (not moved) safely to Unraid. That means purchasing more storage drives, but most people running Unraid need more storage fairly regularly anyway. If you purchase 2 8TB drives, assign 1 as parity, 1 as data, you can copy all your files and continue to use your current drives as a real backup, one that can survive various oopsies that don't involve drive failure.

Link to comment
8 minutes ago, jonathanm said:

Windows RAID or Unraid do not mean you have a backup, it's high availability where a drive failure can be tolerated without losing data.

 

I STRONGLY recommend not breaking the Windows RAID until all your data is copied (not moved) safely to Unraid. That means purchasing more storage drives, but most people running Unraid need more storage fairly regularly anyway. If you purchase 2 8TB drives, assign 1 as parity, 1 as data, you can copy all your files and continue to use your current drives as a real backup, one that can survive various oopsies that don't involve drive failure.

That's why I'm currently using SpiderOak to backup everything that is important. Most of it can be replaced. I don't have any extreme storage needs, so spending 500$+ (yes, this is correct) to get new 8TB drives is out of the question. Does it make sense setting up the system in the order I listed? What are some examples of operations that can lead to data loss? (misclicking).

Link to comment

I'd say grab a 8TB or so USB drive, use it for backups now and in the future.... runs about $150.  Remember, redundancy is not backups.

 

This will give you some margin for error if something goes wrong in your process and provide you with a place to run backups in the future.

Link to comment

Reading back over this, I caught a couple things I missed the first read through.

 

It sounds like you only have a single computer, not a windows machine and a second computer you are setting up for unraid. If this is the case, then I'm very uneasy about your plan to migrate data, I'm not at all positive that a degraded NTFS RAID member can be mounted in linux or read outside of a windows installation. I googled for answers, but didn't find a good solid positive response about an intact NTFS RAID set, let alone a degraded set. Single NTFS disks, sure, but nothing about RAID sets.

 

Second, you clearly state

3 hours ago, Ademar said:

Remove disk 2/4 from my Windows machine, they will hold the only copies of my data.

So, at the very least, before you even attempt this, I second @Abzstrak's recommendation of backing things up in total to a USB external drive. At least that way you can recover without massive downloads from spideroak.

Link to comment

I appreciate the feedback, but both of you seem to misunderstand somewhat. The most likely cause of data loss for me would be drive failure. Both unRAID and Windows software raid protects against that (the chance of 2 drives failing at the same time is low enough to not be an issue) . I'm perfectly aware of the difference between redundancy and backup. That's why I have been doing offsite backups of everything I cannot afford to loose, for 10+ years. I only have about 6TB of data, most of which can be replaced. If my house burns down, having another 8TB backup disk laying around won't help. It would be nice to have more than one copy of my data when setting up unRAID though, but it's simply not worth the price. Is there anything in particular that I should be extra careful about when setting up unRAID?

Link to comment
4 minutes ago, jonathanm said:

Reading back over this, I caught a couple things I missed the first read through.

 

It sounds like you only have a single computer, not a windows machine and a second computer you are setting up for unraid. If this is the case, then I'm very uneasy about your plan to migrate data, I'm not at all positive that a degraded NTFS RAID member can be mounted in linux or read outside of a windows installation. I googled for answers, but didn't find a good solid positive response about an intact NTFS RAID set, let alone a degraded set. Single NTFS disks, sure, but nothing about RAID sets.

 

So, at the very least, before you even attempt this, I second @Abzstrak's recommendation of backing things up in total to a USB external drive.

My first post might have been a little unclear. Yes, I'm turning a Windows machine into a unRAID machine. In Windows disk manager, I can "break mirror", which effectively just creates 2 identical basic NTFS drives. I'm 99% sure this would be no different than plugging any other Windows drive into unRAID. Buying a new 8TB is not worth it, but I might copy some of the data over on other unused drives, SSD in another computer etc. to have some form of backup before doing this.

Link to comment
4 hours ago, Ademar said:
  • Move data from disk 2 to my unRAID array. (How do I do this, can I do it natively in unRAID, or do I need the Krusader plugin?)

mc (Midnight Commander) is available in the Unraid console

http://trembath.co.za/mctutorial.html

A disk successfully mounted with Unassigned Devices plugin will appear in the /mnt/disks/* path, your array shares that you define will appear in the /mnt/user/* path. You can directly access individual array disks at the /mnt/disk1 /mnt/disk2 /mnt/cache etc paths. Since the individual disks all participate in the user shares, it will appear that data is duplicated between the /mnt/diskX and /mnt/user paths. DO NOT copy or move between the two, they are the same file presented in two different views.

 

Files in the /mnt/disks/ paths can be copied to either /mnt/diskX/share or /mnt/user/share.

Link to comment
2 hours ago, jonathanm said:

mc (Midnight Commander) is available in the Unraid console

http://trembath.co.za/mctutorial.html

A disk successfully mounted with Unassigned Devices plugin will appear in the /mnt/disks/* path, your array shares that you define will appear in the /mnt/user/* path. You can directly access individual array disks at the /mnt/disk1 /mnt/disk2 /mnt/cache etc paths. Since the individual disks all participate in the user shares, it will appear that data is duplicated between the /mnt/diskX and /mnt/user paths. DO NOT copy or move between the two, they are the same file presented in two different views.

 

Files in the /mnt/disks/ paths can be copied to either /mnt/diskX/share or /mnt/user/share.

Alright. I remember reading something like this, will definitely keep this in mind. Thanks for the heads up.

Link to comment
  • 3 weeks later...
On 6/6/2020 at 7:27 AM, Ademar said:

I'm looking to finally set up unRAID, and I've got some questions.
My current volumes look like this (Windows software raid):
 

Volume A - 2 disk mirror, 4TB usable:
Disk 1 - 4TB
Disk 2 - 4TB
 

Volume B - 2 disk mirror, 4TB usable:
Disk 3 - 4TB
Disk 4 - 4TB


These was set up 5 years apart, so it made sense at the time to do it like this. For various reasons, I'd like to only have 2 data drives and 2 parity drives. 
As I understand it, the array will continue to work even if both data drives fail, and only the 2 parity drives are working. So it doesn't matter which physical drive is assigned what role (I have 2 new drives, 2 old).

Does it make sense to set up the drives in this order?:

  1. Remove disk 2/4 from my Windows machine, they will hold the only copies of my data.
  2. Install unRAID, preclear disk 1/3.
  3. Set up a new volume with 1 data drive, and 1 parity drive.
  4. Connect disk 2.
  5. Use the "Unassigned devices" plugin to mount disk 2 (NTFS file system).
  6. Move data from disk 2 to my unRAID array. (How do I do this, can I do it natively in unRAID, or do I need the Krusader plugin?)
  7. Preclear disk 2
  8. Expand my array (It will now have 2 data drives, 1 parity drive).
  9. Connect disk 4, mount it, and move data to my array.
  10. Preclear disk 4, set it up as 2nd parity drive.


I'll have to migrate Plex and qBittorrent to Docker. Plex should be relatively easy, but I can imagine having to add all torrents in qBittorrent manually since I'm moving from Windows to Docker.


To sum up:

  1. Does it make sense to move my data in this order?
  2. How do I set up the filesystem on the individual disks. Is everything done by setting up shares, or can I manually create folders?
  3. Do I use "Unassigned devices" and "Krusader" plugins to move data to my array? I don't want to accidentally loose anything...
  4. Is there a way to migrate torrents from qBittorrent Windows to Docker?
  5. Can I see the GUI of programs running in Docker? For example, how do I see the qBittorrent UI? (apart from the Web UI).
  6. Is there a popular way of doing backups? I'm considering running backups from a Windows machine (network or VM) to be able to use SpiderOak.
  7. How does fan control on a normal pc case work? Is there a way for unRAID to control fans connected to motherboard fan connections? Or perhaps I will need to use a dedicated fan controller and USB-passthorugh to a VM.
  8. Anything else I must know about before installing?

 

Thanks for any and all feedback!

Hello, I will be in a similar situation except my drives have not been in Raid.  Did you use the process you outlined in this post? Did you run into any problems? Were you able to transfer in this way?

 

Thanks!

Link to comment
On 6/27/2020 at 2:59 PM, SPOautos said:

Hello, I will be in a similar situation except my drives have not been in Raid.  Did you use the process you outlined in this post? Did you run into any problems? Were you able to transfer in this way?

 

Thanks!

Since my drives had been mirrored, only one drive in each mirror could be read by unRAID. In your situation, that shouldn't be a problem. You should be able to mount the drives with "Unassigned devices" plugin, and use something like "binhex-krusader" to move the files over to your shares on the array disks. I ended up connecting my old NTFS drives to another Windows machine on the network, and copying the files over to my unRAID shares. That works fine, but it takes longer time because it's limited to 1 gb/s, so it would be quicker to connect the drives directly to your unRAID machine.

Be aware that Windows and Linux treats folders with capitalized letters differently. I had a folder named "folder", and accidentally created "Folder". When I deleted "Folder" in the unRAID share, from my Windows pc, everything in "folder" also got deleted.

Link to comment
3 hours ago, Ademar said:

Since my drives had been mirrored, only one drive in each mirror could be read by unRAID. In your situation, that shouldn't be a problem. You should be able to mount the drives with "Unassigned devices" plugin, and use something like "binhex-krusader" to move the files over to your shares on the array disks. I ended up connecting my old NTFS drives to another Windows machine on the network, and copying the files over to my unRAID shares. That works fine, but it takes longer time because it's limited to 1 gb/s, so it would be quicker to connect the drives directly to your unRAID machine.

Be aware that Windows and Linux treats folders with capitalized letters differently. I had a folder named "folder", and accidentally created "Folder". When I deleted "Folder" in the unRAID share, from my Windows pc, everything in "folder" also got deleted.

 

Thank you for the info! I'm a bit confused regarding the folder getting deleted. So "folder" and "Folder" were two seperate files but when you deleted the "Folder" file it also deleted the "folder" file as well?  Since they had the same name did it link them together or sync them or something?

Link to comment
1 hour ago, SPOautos said:

 

Thank you for the info! I'm a bit confused regarding the folder getting deleted. So "folder" and "Folder" were two seperate files but when you deleted the "Folder" file it also deleted the "folder" file as well?  Since they had the same name did it link them together or sync them or something?

"Folder" and "folder" were to separate folders. "folder" contained some files, while "Folder" were empty. I accessed the share from my Windows pc over the network. When I deleted "Folder", everything inside "folder" was deleted. You should install the "recycle bin" plugin in case you accidentally delete something.

 

Edit: they were not linked in any way.

Edited by Ademar
  • Like 1
Link to comment

Because of the fact that Linux is case sensitive and Samba is not then If you have two different folders with the same spelling and different case then there’s is a level of pot-luck which one gets picked up by Samba to link to a share.   You will find that you can only see the contents of one of the folders.

  • Like 1
Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.