patseguin Posted February 20, 2012 Share Posted February 20, 2012 It's been a small disaster copying files to my shares. I was getting around 40MB/s yesterday across network and then after looking through the manual, I decided to connect a temporary HD, copy the files from this machine, then connect it to the unRAID server and copy that way. Well, I was getting a whopping 20MB/s (wtf?) and got up this morning and MC said "lost connection to host" which I guess means the hard drive. I still have the computer connected to my monitor so I switched to its input and the screen is black. If I try to open a telnet session it just says connection failed. I can't bring up the web interface either. I appear to have no choice but to hard reboot the system. Is there anything else I can do? EDIT: Well the system appeared to be frozen so I had no choice but to hard reboot. The array is online and all seems fine but now when I try to replace a files on one of my shares I get "you need permission to perform this action". EDIT #2: I am copying a new file and it's going at 13MB/sec. I can't believe how slow this system is. Every time I rip a new movie I can expect it to take 3 hours to copy to unRAID? Something doesn't seem right. EDIT #3 not only can't I seem to replace any files, but copying a new file is going at 18.8MB/sec right now plus the main screen is showing Parity-Check in progress with an estimated remaining time of 1,983 minutes. Something seems very wrong. Should I just break apart the array and rebuild it? Quote Link to comment
Ryan00 Posted February 20, 2012 Share Posted February 20, 2012 Post your syslog , also are the hard drives your copying to becoming full? Quote Link to comment
patseguin Posted February 20, 2012 Author Share Posted February 20, 2012 The drives are nowhere near full. I'm not feeling very good about unRAID right now to be honest. 30-40 MB/s transfers across gigabit doesn't sound right at all and hard drive to RAID copy was even SLOWER at about 25MB/s before it froze. I had no choice but to hard reboot and I started the array and it seemed fine. I tried overwriting the partial copy of the last movie and it said "you need permission to do that". I tried deleting it and it said "that file is open in another program". Furthermore, I looked at the main screen and it said something about checking parity and it was at 1% with about 10 hours to go or something silly like that. I really wanted unRAID to be a good solution but if I do get everything copied over in a week, I can really expect a single rip to take about 3 hours to copy over? I was previously running hardware RAID10 on Win 7 and a new rip would take about 2-3 minutes to copy over the network to the server. Anyone able to help on these issues? I really want unRAID to work but it's not looking too good. Someone told me yesterday that my 40GB/s over gigabit ethernet was great but when I decided to try a hard drive copy as per the manual all help broke loose. Not only was it slower, but it froze part way through one of the files and I can't delete it or copy the file over it now. Quote Link to comment
lionelhutz Posted February 20, 2012 Share Posted February 20, 2012 Provide a syslog and hardware specs. Quote Link to comment
patseguin Posted February 20, 2012 Author Share Posted February 20, 2012 I'll find syslog when I get home. Hardware is 5x 3TB Seagate Barracuda drives, AMD990FX mobo with FX8150 CPU and 16GB RAM, mobo giga ethernet, nVidia 430 gfx. All the drives are set to AHCI in the BIOS. Actually could this be a problem with unRAID? SHould they just be set as SATA type? Quote Link to comment
S80_UK Posted February 21, 2012 Share Posted February 21, 2012 EDIT #3 not only can't I seem to replace any files, but copying a new file is going at 18.8MB/sec right now plus the main screen is showing Parity-Check in progress with an estimated remaining time of 1,983 minutes. Something seems very wrong. Should I just break apart the array and rebuild it? Copying files while a parity check is in progress is going to be very slow unless you copy to a cache drive since the drives are already busy trying to read different areas from that which you are writing to, and the writing of new data to the array also requires data and parity reads for each data and parity write. I would not break the array, but I would be inclined to let the parity check finish before you write more data. At present each operation is also making the other take a lot longer than running the two separately. Quote Link to comment
JonathanM Posted February 21, 2012 Share Posted February 21, 2012 Any time you shut down the machine without properly stopping the array, it will start a parity check. That's just the way it is. Always use the web interface to stop the array and shut down the machine whenever possible. If it's not possible, the parity check will run on next boot to account for any errors that may have occurred during the unplanned or unmanaged shutdown. Quote Link to comment
bigdog66 Posted February 21, 2012 Share Posted February 21, 2012 in all my noobness I am jumping in to just offer my opinion and all of you who know more feel free and disagree if you see fit...lol sounds to me like patseguin has a lot of data to copy over but more important sounds like there is enough room to copy everything over and still have the original files I would think it would prob be best to let the parity check finish (which as stated whenever the pc is just shutdown without shutting down unraid will kick in a parity check) I would disable parity which would allow full write speeds so all of the data can be copied over a lot faster...then when that is done turn parity back on and let it build main point to patseguin write speeds are going to be a lot slower with parity in place...no lying about that but with the current situation doing what i think would probably lead to a happier new user again all of the experts feel free and elaborate or please tell me im wrong...dont be mean though as im probably bigger than you hahahaha Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 How do I just start all over again? Do I just format the USB drive and copy over my Pro Key again? The whole thing is fubar right now lol. Quote Link to comment
dgaschk Posted February 21, 2012 Share Posted February 21, 2012 How was the HDD attached to the server for the local copy? Please post a syslog. Starting over will not correct hardware problems. Quote Link to comment
SSD Posted February 21, 2012 Share Posted February 21, 2012 in all my noobness I am jumping in to just offer my opinion and all of you who know more feel free and disagree if you see fit...lol sounds to me like patseguin has a lot of data to copy over but more important sounds like there is enough room to copy everything over and still have the original files I would think it would prob be best to let the parity check finish (which as stated whenever the pc is just shutdown without shutting down unraid will kick in a parity check) I would disable parity which would allow full write speeds so all of the data can be copied over a lot faster...then when that is done turn parity back on and let it build main point to patseguin write speeds are going to be a lot slower with parity in place...no lying about that but with the current situation doing what i think would probably lead to a happier new user again all of the experts feel free and elaborate or please tell me im wrong...dont be mean though as im probably bigger than you hahahaha This is a perfectly valid approach. By keeping the original source files until the array is fully protected you minimize the chance of data loss due to a failed drive. I did this myself in a much earlier version of unRaid and recommended it in a number of posts. But things have changed alot since those days and I no longer recommend this option. The major change has been write speed improvements. In the old days write speeds were in the 10 MB/sec range. Current unRaid versions are 3-4 times that fast. At current speeds the advantages of burning in the array with high volume file transfers using unRaid's parity mechanisms combined with the experience and confidence gained by the new user in the process outweigh the speed advantages in my opinion. Case in point - this thread. Here is a new user experiencing problems because he didn't fully understand all the nuances of parity and parity checks. He obviously has some type of problem in his array (which could be as simple as a lose cable) and is now asking about starting over (a common urge but seldom a good idea). I think working through these types of issues is better done now, when his data is being copied than after copying terabytes of data and having a false sense of security. Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 Fwiw, I am keeping all my media on my local PC until unRAID is good to go. bjp is totally right though. What happened was I decided I was going to try a hardware raid and I configured it in BIOS but then changed my mind after realizing all the space I could be using with unRAID. Well, when I rebooted back into unRAID my configuration was fubar. It listed drives missing, wrong, etc. Additionally, I originally skipped preclearing because I thought since they were fresh shrink wrapped drives that I would be OK. This time I am doing everything by the book. I cleared my unRAID USB stick and reinstalled it. I am now preclearing all 6 drives. Once that is done, I will create my shares and copy my media over, followed by creating the parity disk. Does this sound like a good plan of action? Also, just in case something happens - what if Windows crashes or I close the telnet window? How will I be able to tell when preclear is done? I am running all 6 preclears using Screen. Quote Link to comment
mbryanr Posted February 21, 2012 Share Posted February 21, 2012 Also, just in case something happens - what if Windows crashes or I close the telnet window? How will I be able to tell when preclear is done? I am running all 6 preclears using Screen. The benefit of using screen is that the session continues, and you can reattach to the session. http://lime-technology.com/wiki/index.php?title=Configuration_Tutorial#Preclearing_With_Screen Additional screen info http://kb.iu.edu/data/acuy.html Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 OK thanks. As to my other question, is it a good idea to copy over my data and then create the parity drive? Or vice versa? Quote Link to comment
wsume99 Posted February 21, 2012 Share Posted February 21, 2012 Fwiw, I am keeping all my media on my local PC until unRAID is good to go. If you have the space to do this then it is ideal. As to my other question, is it a good idea to copy over my data and then create the parity drive? Or vice versa? As bjp said previously there is a benefit to establishing parity first and then copying all the data over. The downside is that it will take longer to copy over the data but it may be worthwhile to take a little extra time now and take all the steps you can to make sure that the array is in good condition. There is nothing wrong with the method you mentioned - copy data first then establish parity - and I personally followed that method when I first setup my array. Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 OK thanks. I have right around 2TB or so data so how long will that take to establish parity? I assume the web gui will show me progress? Quote Link to comment
lionelhutz Posted February 21, 2012 Share Posted February 21, 2012 Does this sound like a good plan of action? Only if you find some kind of hardware problem during your testing. You were having copy issues and crashing. If you don't figure out why then it's likely to just happen again. Peter Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 Does this sound like a good plan of action? Only if you find some kind of hardware problem during your testing. You were having copy issues and crashing. If you don't figure out why then it's likely to just happen again. Peter No, I wasn't having crashing issues. It seemed to be copying really slow over the network and people here told me the speed was actually good but me being the dumbo that I am decided to cancel that and try a hard drive to raid copy. I think all of my problems were a result of rushing into it. Quote Link to comment
wsume99 Posted February 21, 2012 Share Posted February 21, 2012 OK thanks. I have right around 2TB or so data so how long will that take to establish parity? I assume the web gui will show me progress? Yes, the web GUI will show progress. The time required to compute parity depends on the size of your parity drive not how much data you have. An array with a 2TB parity drive normally takes 7-8 hours (if you are using green drives). If you are using a 3TB parity drive then I'd expect it to take 10-12 hours to compute parity. Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 OK so is my plan of preclearing, copying data, creating parity, in that order a good plan? I was thinking doing parity last because file copy might be faster? Quote Link to comment
Joe L. Posted February 21, 2012 Share Posted February 21, 2012 OK so is my plan of preclearing, copying data, creating parity, in that order a good plan? I was thinking doing parity last because file copy might be faster? It is a good plan ONLY if you also take the time to verify the copied data was copied correctly. (reading it back and comparing it, or a checksum of it with the original on the source drive) Otherwise, I think it is a poor plan. You'll not know that the copy on the unRAID disk is un-readable until you attempt to calculate parity and you see read-errors (un-readable blocks on the disk) At that point you'll know the disk block involved, but I know of no easy way to figure out what file that might represent. Hopefully all your original data will still be there so you could compare. If you had already established parity, unRAID would use the parity drive in combination with all the other data drives to re-construct the un-readable sector and write it back to the un-readable drive. This gives the drive's SMART firmware the opportunity to re-allocate that un-readable sector to one in its pool of spare sectors. None of that can occur is there is no parity drive assigned. My advice, start the copy going, go to bed, wake up with it completed. If you use something like "teracopy" it can even re-start if something were to occur in the middle AND also perform checksums to verify the copy was successful. (not a bad idea either way you perform the copy. Just be aware, it takes almost as much time to verify as an additional step as it does to perform the original copy. Personally, I established parity first when I copied my entire older unRAID array to my second, newer server. Then, I let it copy about 9GB while I slept. Joe L. Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 OK so it sounds like creating parity first is probably a better idea? Is teracopy something I just need to find and copy to the flash drive somewhere? Quote Link to comment
Joe L. Posted February 21, 2012 Share Posted February 21, 2012 OK so it sounds like creating parity first is probably a better idea? Is teracopy something I just need to find and copy to the flash drive somewhere? no, it is a program you use on your window's PC. (download and install on the window's PC you are copying from) Quote Link to comment
patseguin Posted February 21, 2012 Author Share Posted February 21, 2012 Do you mind if I ask what advantage unRAID has over Windows Home Server 2011? Quote Link to comment
prostuff1 Posted February 21, 2012 Share Posted February 21, 2012 protection from 1 drive failure for all data on all drives without the need to mirror all that data. WHS has no efficient way of protecting all data without essentially doing a raid1 on every drive. There are other advantages but I'm not going to list them out here. This has been asked before, try a search and see if you can find them. Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.