UFGrayMatter Posted May 5, 2007 Share Posted May 5, 2007 If I switch from XP to unRAID, how can I copy the data currently on 2 500gb drives now? I will be setting up with 3 new 500gb drives, then copy the data over, then add the old drives to the unRAID. ?? Quote Link to comment
erikatcuse Posted May 6, 2007 Share Posted May 6, 2007 Depending on how much data you need to move the simplest way would be to just copy the data over the network. If you had a usb enclosure you might be able to connect it directly to the server to copy the files Quote Link to comment
BillK Posted May 6, 2007 Share Posted May 6, 2007 It sounds like you have the right order....build your new unRAID server using three NEW drives (one parity, two data), copy the data from your XP system over from your existing two drives, then pull the existing drives out of your PC and install in your unRAID system. You can't do it the other way (i.e., just add a parity drive and install all three in your UNRAID system), since unRAID uses the ReiserFS file system and will need to reformat all the data drives. Quote Link to comment
UFGrayMatter Posted May 6, 2007 Author Share Posted May 6, 2007 Depending on how much data you need to move the simplest way would be to just copy the data over the network. If you had a usb enclosure you might be able to connect it directly to the server to copy the files hmm - oh well - thought it would be this way. It's 1TB worth of movies . Guess it will take a while Quote Link to comment
Joe L. Posted May 14, 2007 Share Posted May 14, 2007 Depending on how much data you need to move the simplest way would be to just copy the data over the network. If you had a usb enclosure you might be able to connect it directly to the server to copy the files hmm - oh well - thought it would be this way. It's 1TB worth of movies . Guess it will take a while You might be able to install the drives in the unRaid server but DO NOT ASSIGN THEM in the disks page on the web-management utility. Instead, write down their disk assignment slots. In other words, they will be at /dev/hda, /dev/hdb, /dev/hdc, etc... but the actual hd will be listed along side of the drive's serial number in the drop-down-field on the disk assignment page. Again, do not assign the disks, but write down their position in the array. (you must temporarally stop the array to see the physical device names of the disks in the drop-down-lists, but leave them unassigned to unRaid slots) As an example... let's assume your old data disk was put in the physical device /dev/hdj ... type the following... mkdir /mnt/old_data mount -r -type ntfs /dev/hdj /mnt/old_data or... if a FAT filesystem mount -r -type vfat /dev/hdj /mnt/old_data then, you can use the unix cp command to copy your files to an existing data drive you previously added to your unRaid array. To copy all your files to disk1 type this: cd /mnt/old_data cp -r * /mnt/disk1 Lastly, to unmount the old disk after copying all the files from it type: umount /dev/hdj (Yes, it is umount, not unmount) Once the data is off of the old NTFS/FAT drive and copied to an unraid data drive, and the old drive is then unmounted, you can then use the devices web-management page to asssign the old drive to a slot in the unraid array. (you must stop the array to assign it) It will then be cleared and formatted and made available as part of the unRaid array. Now, just to let you know... I've never done this on my array, but I see no reason why it should not work. use of the -r option to the mount command will mount the drive as read-only so you should be safe... only way to mess up would be to assign the old drive to unRaid and clear/reformat it before you got your data copied from it. Let us know how it works for you... Joe L. Quote Link to comment
parsec Posted May 14, 2007 Share Posted May 14, 2007 awesome Joe !!!, if we have reports of it working, is it ok if I quote you on the wiki ? (or you could put it in :-) /Rene Quote Link to comment
Joe L. Posted May 15, 2007 Share Posted May 15, 2007 awesome Joe !!!, if we have reports of it working, is it ok if I quote you on the wiki ? (or you could put it in :-) /Rene if it works, sure, you may add it to the wiki. Quote Link to comment
John Kotches Posted June 10, 2007 Share Posted June 10, 2007 How were you guys mounting the NTFS file systems? When I tried to mount, I was told that ntfs was an unknown file system type. Were you using fuse (I think that's it) from SourceForge? Cheers, Quote Link to comment
Joe L. Posted June 11, 2007 Share Posted June 11, 2007 How were you guys mounting the NTFS file systems? When I tried to mount, I was told that ntfs was an unknown file system type. Were you using fuse (I think that's it) from SourceForge? Cheers, I was pretty sure that NTFS used to be in there as a "read-only" filesystem driver, but I just looked at my current 4.0 final server. It's not in there... Tom, is my memory failing me? Joe L. [pre] root@Tower:/proc# cat filesystems nodev sysfs nodev rootfs nodev bdev nodev proc nodev debugfs nodev sockfs nodev usbfs nodev pipefs nodev futexfs nodev tmpfs nodev inotifyfs nodev eventpollfs nodev devpts reiserfs ext2 nodev ramfs msdos vfat iso9660 nodev smbfs nodev mqueue [/pre] Quote Link to comment
Joe L. Posted June 11, 2007 Share Posted June 11, 2007 Tom, It looks like the ntfs-3g project has made a lot of progress and as of February 2007 has a stable release of a read/write NTFS user space driver. Perhaps you can take a look. This would make migration of data much easier for many users of unRaid. Check it out here: http://www.ntfs-3g.org/index.html Joe L. Quote Link to comment
Albin Posted July 7, 2007 Share Posted July 7, 2007 How were you guys mounting the NTFS file systems? When I tried to mount, I was told that ntfs was an unknown file system type. Were you using fuse (I think that's it) from SourceForge? Cheers, I was pretty sure that NTFS used to be in there as a "read-only" filesystem driver, but I just looked at my current 4.0 final server. It's not in there... Tom, is my memory failing me? Joe L. Joe, Is there an earlier version that you know has the ntfs file system. I wanted to use your proposed method for moving data to an unraid drive. I have an extra tower where I can use my additional unraid os to move the files and then move the file between two unraid systems (one unraid would have the current build, and the other would have the build you suggest that has the ntfs drive support) Thanks Unraid Quote Link to comment
orb Posted July 7, 2007 Share Posted July 7, 2007 You could also use a live CD to do that. Choose one that have ntfs support and you're done. (or a flash based one if you don't have a cd drive handy) Quote Link to comment
Albin Posted July 9, 2007 Share Posted July 9, 2007 You could also use a live CD to do that. Choose one that have ntfs support and you're done. (or a flash based one if you don't have a cd drive handy) Orb, This looks like a great option for moving files from NTFS to an UnRaid system. I downloaded "Live CD" and then booted using the iso. I had an NTFS drive and an unraid drive hooked up. I was able to mount both, but I could not copy data from the NTFS drive to the Unraid drive. I made sure to turn on read/write permissions, but that still did not allow for files to be created, or copied to the unraid drive. I am sure it is a setting in Live CD that I am missing. Are there any tips that you can give? Albin Quote Link to comment
orb Posted July 9, 2007 Share Posted July 9, 2007 I guess it depends on which LiveCD you used, Ubuntu ? Slax ? Knoppix ? DSL ? I would use Slax because it is Slackware based as is unRAID, but I will check that reiserFS is compiled with write support in the LiveCD you currently have, and also look into access permission on hte drive you want to copy to (to makes things easy you could put it to 777 if it's not already, and change it back after the copy to whatever it was) Quote Link to comment
Albin Posted July 9, 2007 Share Posted July 9, 2007 I guess it depends on which LiveCD you used, Ubuntu ? Slax ? Knoppix ? DSL ? I would use Slax because it is Slackware based as is unRAID, but I will check that reiserFS is compiled with write support in the LiveCD you currently have, and also look into access permission on hte drive you want to copy to (to makes things easy you could put it to 777 if it's not already, and change it back after the copy to whatever it was) Orb, I downloaded Slax and burned the iso to a cd. I then tried to boot to the iso but got the "blank Screen" issue that seems to be related to my on-board video card. Instead of waisting to much time on that I managed to try Slax in a different tower (different MB and different video card) and it booted fine (I booted using the StartX command). After booting I mountedthe unraid formated drive and a ntfs drive and then copied a few files from the ntfs to the unraid drive. (if anyone needs pointers on mounting drives in the slax gui, just let me know. I am sure I am not the anyone struggling as a newbie with this kind of stuff) I then powered down and placed the unraid drive back in the unraid tower and powered up unraid. I then browsed to the mapped network unraid drive and the files that I copied over from the ntfs drive to the unraid drive (with the use of slax and www.slax.org ) were present. (I downloaded SLAX Standard Edition v 5.1.8.1) So I had success with your proposed method. Thanks so much for the tip. I also noticed that when playing around with Slax I will be able to learn many of the copy, move, and other commands that will most likely be helpful with UnRaid. Again thanks for the tip, and now I can move (consolidate) all my files off of several ntfs drives and on to my UnRaid drives at incredible speeds, and without the use the network or an xp machine. By the way, All I did with the UnRaid machine, was to power down, pull the hard drive (not the parity drive) that I was going mount in Slax, then when I was done with the moving/copying of files from the ntfs drive to the unraid drive and had powered down slax, I then pulled the UnRaid drive from the machine that I used for booting to Slax and placed it back in the unraid machine and booted up Unraid and the files showed up. I guess I could perform a parity check, but unraid says parity is valid (I am still learning about when to check parity, so an comments on this would be great, and if anyone see issues with what I did above please comment). One other note. The above process maybe a lot easier for others if they can use their current unraid machine to boot to slax. I would of done it this way, but I would be sure to disconnect all drives except for the drive you want to move data to, and connect the ntfs drive. If you want to have parity when moving files, then any of the above methods mentioned will not be helpful. I know this is a long post, but I figured it may help others. Albin Quote Link to comment
Joe L. Posted July 9, 2007 Share Posted July 9, 2007 I guess I could perform a parity check, but unraid says parity is valid (I am still learning about when to check parity, so an comments on this would be great, and if anyone see issues with what I did above please comment). Albin Albin, Good to know your file copy worked. Hopefully, Tom will add NTFS filesystem support back in soon. However... Your parity is now out of date. (almost worse than not having it at all) You must now re-calculate party on the unRaid server. You did your file copy when it was not looking (booted up on the other CD), so the current Parity contents do not know of any of your new files. Please, force a parity check now... sooner even, as your existing, out of date parity data would clobber/manngle the contents of any drive you attempt to recover using it. Expect the parity check to find many errors. In this case they are expected, and will be because the current parity calcs were done on the original contents of your target drives. (before you copied the ntfs data to on the unRaid disks) Once the parity check has run to completion, you will be protected again. Joe L. Quote Link to comment
orb Posted July 9, 2007 Share Posted July 9, 2007 Glad i could help. I would second Joe advise, since you worked out of unraid, unraid is not aware of changes, so you should definatly run a parity check. Quote Link to comment
Albin Posted July 9, 2007 Share Posted July 9, 2007 I guess I could perform a parity check, but unraid says parity is valid (I am still learning about when to check parity, so an comments on this would be great, and if anyone see issues with what I did above please comment). Albin Albin, Good to know your file copy worked. Hopefully, Tom will add NTFS filesystem support back in soon. However... Your parity is now out of date. (almost worse than not having it at all) You must now re-calculate party on the unRaid server. You did your file copy when it was not looking (booted up on the other CD), so the current Parity contents do not know of any of your new files. Please, force a parity check now... sooner even, as your existing, out of date parity data would clobber/manngle the contents of any drive you attempt to recover using it. Expect the parity check to find many errors. In this case they are expected, and will be because the current parity calcs were done on the original contents of your target drives. (before you copied the ntfs data to on the unRaid disks) Once the parity check has run to completion, you will be protected again. Joe L. Joe, Thanks for filling me in on the need for a Parity Check. I am going to copy over several gigs to the UnRaid drive via Slax and then I will be sure to force a Parity Check. Albin Quote Link to comment
melechmet Posted July 10, 2007 Share Posted July 10, 2007 Tom, It looks like the ntfs-3g project has made a lot of progress and as of February 2007 has a stable release of a read/write NTFS user space driver. Perhaps you can take a look. This would make migration of data much easier for many users of unRaid. Check it out here: http://www.ntfs-3g.org/index.html Joe L. bump Quote Link to comment
huyazo Posted July 14, 2007 Share Posted July 14, 2007 Hi, as a happy new unRAID user, I would like to share my experience of moving some gigs of music and movies the past couple of days. A lot of the files have non-English characters, like ñ in Spanish or ø in Norwegian. As suggested in the forums and the wiki, I decided to use cp command from the console for speed reasons. After copying, I wanted to check that everything had been moved. First I did an ls command on the console, and a lot of text rushed over the screen, looked promising. Then I used my alternative browser, FreeCommander, which also contains a synchronization tool. I was very surprised when all the files turned up as missing in target. Looking in the ordinary explorer part of FreeCommander, I could see them. Very strange. So, I did a compare using SyncBack and got a totally different result. SyncBack reported that most of the files were in the target, but I discovered quite a few files missing - and then whole directories missing. A closer inspection showed that all files and directories containing Norwegian characters æ, ø and å had not been copied. Files with Spanish, Portuguese, French and German special letters were copied, but the letters had been replaced with an underscore. My next step was to go to Windows Explorer to copy the missing files. Bug surprise again: No files in the target directories! After some investigation, where FreeCommander gave me the clue as it shows the Attributes, it turns out that the cp command turns on a the Hidden flag and the System flag on each and every file, resulting in all files being invisible in Windows Explorer. I managed quite easily to change the attributes with the chmod command, but when I realized how long it would take me to correct all the underscores and manually copy the missing files, I decided the far easiest solution was to delete everything I had moved, and use Windows Explorer on the source PC to copy files to the unRAID server. I didn't have any means to take the time on any of my transfers, but my gut feeling is that the difference is not big, and the result turned out perfect with Windows Explorer. You can even start several copy sessions in parallel before going to bed, as Windows does a great job in this respect. Well this turned into a long post, but my conclusion is very simple: avoid the Linux commands for file operations, as they don't give you the needed language and Windows support. I have been working professionally in the IT industry since 1983, and I have seen the lack of language support from US developers being repeated again and again. Now, as far as I know, Linus Torvald is Finnish, so he should know better... Hakon (my name is actually Håkon, but many mail systems still have problems with the å...) ps. I have updated the wiki description of transfer from network shares to unRAID. Quote Link to comment
limetech Posted July 14, 2007 Share Posted July 14, 2007 Hi, as a happy new unRAID user, I would like to share my experience of moving some gigs of music and movies the past couple of days. A lot of the files have non-English characters, like ñ in Spanish or ø in Norwegian. As suggested in the forums and the wiki, I decided to use cp command from the console for speed reasons. ... Use of the 'cp' command is documented in the wiki, but we never intended this to be the primary means of moving files. This is mainly because one of our design goals is to hide the underlying linux OS as much as possible. Also, as you have observed, there is not much of a speed improvement over moving files with Explorer anyway. Nevertheless, as unRAID OS evolves, we plan to make it easier to cusstomize the platform and fix these annoying problems, such as... My next step was to go to Windows Explorer to copy the missing files. Bug surprise again: No files in the target directories! After some investigation, where FreeCommander gave me the clue as it shows the Attributes, it turns out that the cp command turns on a the Hidden flag and the System flag on each and every file, resulting in all files being invisible in Windows Explorer. This is just an artifact of the default file create permissions for 'root'. What's happening is that Samba (which implements Windows Networking protocol) is "overloading" some of the unix file permission bits to implement the DOS 'Hidden' and 'System' attributes (which are meaningless in the unix world). This is a known "issue" for which you can search to find more info on. In a nutshell, since unRAID is meant to be used in a Windows Networking environment, we enable the Samba feature that implements 'Hidden' and 'System'. We plan to make this configurable in an upcoming release. ... Well this turned into a long post, but my conclusion is very simple: avoid the Linux commands for file operations, as they don't give you the needed language and Windows support. I have been working professionally in the IT industry since 1983, and I have seen the lack of language support from US developers being repeated again and again. Now, as far as I know, Linus Torvald is Finnish, so he should know better... Don't blame Linus!! We take full responsibility for not including all the necessary code pages in the unRAID build. This was done to conserve space, but again, we plan to correct this in a future release. Hakon (my name is actually Håkon, but many mail systems still have problems with the å...) I guess those would be the mail systems created by US developers ps. I have updated the wiki description of transfer from network shares to unRAID. Thank you! Quote Link to comment
huyazo Posted July 14, 2007 Share Posted July 14, 2007 Thanks Tom, I have no problems at all that these features are not being "built into" the underlying OS, I think you have done the right decision to prioritize a small footprint. I just want others to be aware that they can lose some files in the transfer process if they are "cheating" by using Linux commands rather than Windows OS tools. I assume what happens is that you are bypassing the layers 5-7 in the OSI model if you do this, which are taking care of formats and presentation. Now I am more worried about the disk I put in yesterday, which was working fine, but now it turns up as Unformatted. It was a brand new 500G Samsung, and I have successfully moved my music and pictures there. Of course, I still have the source files, so I will push the Format button. If this fixes the problem, I actually get worried. If it is not a HW problem, why did it suddenly turn unformatted? I assume that the array should recover the lost data after formatting... Well, I assume this belongs in another thread. Hakon Quote Link to comment
Joe L. Posted July 14, 2007 Share Posted July 14, 2007 To possibly save a lot of time, do NOT re-format/reload. (at least not yet) Instead, take the array off-line, power it down, and then power up once more. See if the disk is detected as being formatted once more. It could be that the web-interface is confused to the disk status. You might want to run a reiserfsck command on the file-system (yes,another OS command) to check the file-system for errors. (you can do a search for how to do this rather than take this thread off topic too far. Joe L. Quote Link to comment
huyazo Posted July 15, 2007 Share Posted July 15, 2007 Joe, thank you for your advices! Unfortunately, I had already formatted the disk. I had to reformat 2-3 times, but now it has been stable for 2 days after I checked the connectors and changed the power connector. I will let you all know (in the hardware forum if I get any more problems with the Samsung disk, as I have seen some more posts reporting problems with it. Håkon Quote Link to comment
kenshin Posted August 17, 2007 Share Posted August 17, 2007 How were you guys mounting the NTFS file systems? When I tried to mount, I was told that ntfs was an unknown file system type. Were you using fuse (I think that's it) from SourceForge? Cheers, I was pretty sure that NTFS used to be in there as a "read-only" filesystem driver, but I just looked at my current 4.0 final server. It's not in there... Tom, is my memory failing me? Joe L. [pre] root@Tower:/proc# cat filesystems nodev sysfs nodev rootfs nodev bdev nodev proc nodev debugfs nodev sockfs nodev usbfs nodev pipefs nodev futexfs nodev tmpfs nodev inotifyfs nodev eventpollfs nodev devpts reiserfs ext2 nodev ramfs msdos vfat iso9660 nodev smbfs nodev mqueue [/pre] I ran into the same issue on unraid 4.1 however I discovered that the kernel module for ntfs support IS included. It is just not loaded by default. I loaded it temporarily using the following command: modprobe ntfs and the response to cat /proc/filesystems: root@tower:~# cat /proc/filesystems nodev sysfs nodev rootfs nodev bdev nodev proc nodev debugfs nodev sockfs nodev usbfs nodev pipefs nodev anon_inodefs nodev futexfs nodev tmpfs nodev inotifyfs nodev devpts reiserfs ext2 nodev ramfs msdos vfat iso9660 nodev smbfs nodev mqueue ntfs I hope this helps someone else trying to copy data from an ntfs disk. -kenshin Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.