Newb... Got my array started and have a few questions

[Ryan_M]

I just got an array started yesterday. For now just using a 4TB parity drive and a 3TB data drive to try out unraid. If all goes well I'll be buying a pro version key and adding the rest of my drives to the array. My questions are about array start up and transferring files. When I started the array for the first time it has to build the parity drive - took about 6 hours (seemed excessive but alright). After that was complete I tried copying some files to the array using teracopy. Speeds were kinda slow and then got worse after copying about 2Gb. Teracopy told me transfer time to copy a 35Gb movie would be 17 days! So there is my first concern. I figured something wasn't right so I thought I'd reboot and try again. The array didn't automatically start. I has to go into web interface to tell it to start and it's doing a parity check... says it's going to take about 9 hours this time though with the way the time is counting down it's looking more like 13 hours. That's a really loooong restart time after a reboot. I'm concerned what the restart time is going to be after I add a dozen drives to the array. These speeds can't be typical, can they? Also is there a way to have the array start automatically after a restart.

 

Thanks for any help.

Ryan

[trurl]

Settings - Disk Settings - Enable Autostart

 

If you don't stop the array before shutting down you will always get a parity check, since the parity may be in an unknown state if you don't let everything finish. You should powerdown or reboot from the webGUI.

 

The webGUI can slow things down a bit if it is refreshing since it has to access the disks. Just open it occasionally to check and your parity build and copies should go faster. 6 hours for parity doesn't seem too bad. As for the 35GB copy, if you let it run for a while it would probably settle down to something more reasonable if everything is working correctly. Also, writes will be significantly slower than reads because every write to a parity protected array actually causes a read from the both the data disk and the parity disk so the new parity can be generated, and then the data write to the data disk and parity write to the parity disk.

 

Do some more testing and if it is still unreasonable ask again and maybe we can help track down the problem.

 

 

 

 

[Fireball3]

It often happens that the GUI is going unresponsive.

Then you either have to console-fu your way through and unmount the drives or reboot the hard way.

The latter will automatically cause a parity check after unclean shutdown - you don't want that.

Follow the powerdown link in my sig and install the powerdown script.

Make sure it works for you and you're safe next time when you press the power switch or call powerdown on the console.

 

My start-up time is <60s with 11 drives.

 

My transfer speeds via GB-LAN to the protected array varies between 20-45 MB/s. (depends on file size)

Use a fast cache drive and you will max out the GB LAN. Then run the mover script when the server is idle.

 

A syslog is always useful so that knowledgable people here can check if there is a basic

problem during startup. Follow the "useful sig" link in my sig to learn how to extract the syslog.

 

 

[french_guy]

When you say fast cache drive, do you mean SSD?

[DaleWilliams]

A 'cache' drive is enabled with a Pro key ...

It can be a hard drive or an SSD...its in the box with your other unRAID drives.  (I use an old 250Gig hard drive)

 

It is a drive that is NOT protected by the Parity Drive. 

Once installed, then when you 'copy' files to the array, they FIRST go to the cache drive at full speed. (depends upon the drive, but maybe 30-40 Meg per...it can be twice as fast with an SSD.)

 

A chron job then automatically moves the files from the cache to the protected array...since this move happens more slowly as parity is calculated, the 'Mover' typically runs automatically at 3 AM, although that's configurable.

 

The tradeoff is that your 'write' operations to unRAID are at full speed, but while the data is on the cache, it is not yet protected by Parity check.

 

Files on cache look as if they are part of the regular protected share. They can be read/changed/deleted, etc.,

While data is on the cache drive, 'share', 'folder' and 'file' interface management of the cache drive is all but 'invisible'.

 

Many folks also use their cache drive for certain write intensive operations that don't require parity protection. (For example, I keep my PLEX database on the cache. Other users use their cache as part of their torrent collection.  Until the final file is put together a final file doesn't need to be on the regular array.)

[Ryan_M]

Thanks for the advice everyone. I wasn't aware of the 'clean shutdown' but it makes sense. I suppose I could have canceled this second parity check as there was no write operations going on at the time of shut down and both drives are empty anyway but I'll let it finish now since I'm this far in.

 

I did expect the writes to be slower vs. the standard drive to drive over Gb LAN but I think this was way out. I used to get mid 60's before, I assumed I'd get a little less than half that with the array having to write to two disks. When I tried doing the copy I was getting mid 30's (which was better than I expected) but after ~2GB was copied it slowed to about 500KB/min and stayed like that for well over an hour but TBH I don't think there was anything going on at that point. I'll try again and if it persists I'll post back with some better info (thanks for the links Fireball).

[Fireball3]

When you say fast cache drive, do you mean SSD?

 

Not necessarily. Fast is probably the wrong term - let's say a drive with good throughput.

I have an old server grade 160 GB drive that writes @100MB/s.

That is enough to max out the GB LAN.

[Ryan_M]

OK got the array back up and the proper shut down figured out but still can't copy anything smoothly to the array.

 

Copying with teracopy I'm getting the same issue as before except it doesn't slow right down as I had previously thought - it just stops after about 2GB. When I try to delete the failed copy it says the "file is in use by another program" on the file that the copy froze on. Rebooting the array lets me delete the rest of the failed copy. Using standard Windows copy will get farther than teracopy before I get an error message saying "there is a problem accessing \\tower\disk1. Make sure you are connected to the network...". However there is no problem communicating with the tower with the web interface. I still get the same "file is in use by another program" error message when trying to delete the file the copy froze up on.

 

Any ideas?

 

Thanks,

Ryan

syslog_1.txt

[Ryan_M]

syslog file was too big to be attached as one file. I didn't know which part was the important part so I split it in half and here is the second half.

syslog_2.txt

[trurl]

You should zip your syslog. Text compresses very well. Messages about a mouse seem to be the bulk of your syslogs. Don't think that is typical.

 

Your network seems to be renewing the lease every 30 minutes for some reason.

 

Post the results of

ifconfig

and

ethtool eth0

[Ryan_M]

Screen shots are attached. Sorry made with cell phone. Looked into ways to do a screen capture, but from what I googled it  involved running scripts etc. and got over my head quickly. That might be a battle for another day.

[trurl]

Nothing obvious from your photos. Maybe someone else will have an idea.

 

I do have some advice for you on how to better provide information for troubleshooting. One I have already mentioned. Zip your syslogs.

 

Since you are using Teracopy I assume you are on Windows. Download, install, and learn to use PuTTY. It will allow you to work with your server from your PC as if you were at the console. More importantly, it will allow you to copy to your clipboard just the text on the "remote" console instead of having to take photos of the screen. It will get very cumbersome very quickly for everyone if you are always attaching blurry photos that may not even show the entire screen in response to every request for information.

 

It is also sometimes useful to have screenshots from the webGUI so if you are unsure how to do that you should learn.

 

If you have any specific questions about how to do any of this, just ask.

[Ryan_M]

Zipping the syslog makes sense... don't know why I didn't think of it. PuTTY sounds exactly like what I just needed, I'll start looking into that. I'm OK on a windows machine but quite useless when it comes to linux. Thanks for the advice.

[Ryan_M]

Ok here's the info again... in better format I hope this time.

syslog.zip

ifconfig.txt

ethtool.txt

[DaleWilliams]

Not sure what's happening.

A couple of things to try.

1. Login to your router and find the setting for DHCP lease time. Set it to 24 hours...or 24x60= 1440 minutes....that's would make the address for each device good for a full day... more than enough time.

2. Powerdown cleanly. Remove the Logitech mouse from unRAID. You can drive unraid from the console with a keyboard using MC (Midnight Commander) without a mouse...

If the copy problem isn't gone post the log again.

 

 

[Ryan_M]

Thanks Dale,

 

Already tried both of those things and still the same thing. I'm going to start a new thread since this has escalated beyond a general "does this seem right?"/"how do I do this?" thread. I have a specific issue now.