August 3, 200718 yr Author Great information! I'll give that a go tonight and see what it does. Another note I wanted to make (And this is my plan for testing tonight BTW) is that apparently when you move data onto a Samba share windows "Reserves" blocks in chunks, then overwrites the reservation with the real data when it gets to it....I suppose that's sort of a "Write ahead buffer". Anyway, there are some emails on the Samba list that indicate ReiserFS can have issues with this over many different "write" operations. I have 5 WD 500G drives on the same controller, so what I'm going to do is move ALL data on one of them to a second (with unRAID running) via the command line on the server. Then I'll move them /back/. This should allow for any read numbers to reflect the samba method of writing to disk in the first test...then the "clean" version in the second. Once the data is back on the original drive, I will then do another pull to the Windows box. All this is in an attempt to see if Vista is causing it's own problems with the current version of Samba.
August 3, 200718 yr Im not sure what my read speeds are but im using Vista64 Media Center as my client and a couple Xp boxes and they can all stream HD over my gigabit network at the same time. Craig
August 3, 200718 yr Author I'm not sure that Joe L. and I are really thinking this will improve my network connection, but with Tom out of town...we are having fun tweaking things while waiting for the next build . Joe L. You ready for something VERY interesting? /dev/md1 is the drive that I dumped my files onto from Vista. /dev/md2 is a copy of the Dogma directory over to a disk that before this had been unused totally. They are both the exact same size and make of drive connected to the same SATA controller. Check out these results. root@Storage:/mnt/disk1# dd if=/mnt/disk1/D Distinguished Gentelman/ Dogma/ root@Storage:/mnt/disk1# dd if=/mnt/disk1/Dogma/VIDEO_TS/VTS_01_2.VOB of=/dev/null 2097148+0 records in 2097148+0 records out 1073739776 bytes (1.1 GB) copied, 54.8361 seconds, 19.6 MB/s root@Storage:/mnt/disk1# dd if=/mnt/disk2/Dogma/VIDEO_TS/VTS_01_2.VOB of=/dev/null 2097148+0 records in 2097148+0 records out 1073739776 bytes (1.1 GB) copied, 25.1323 seconds, 42.7 MB/s root@Storage:/mnt/disk1# df -h Filesystem Size Used Avail Use% Mounted on /dev/sda1 980M 29M 952M 3% /boot /dev/md4 466G 33M 466G 1% /mnt/disk4 /dev/md2 466G 8.2G 458G 2% /mnt/disk2 /dev/md3 466G 9.1G 457G 2% /mnt/disk3 /dev/md5 466G 308G 158G 67% /mnt/disk5 /dev/md1 466G 193G 273G 42% /mnt/disk1 /dev/md6 234G 33M 234G 1% /mnt/disk6 root@Storage:/mnt/disk1# blockdev --getra /dev/md1 2048 root@Storage:/mnt/disk1# blockdev --getra /dev/md2 2048
August 3, 200718 yr So... either the two channels on the SATA controller perform differently, or the two physical drives perform differently, or the filesystem on one has the entire file in one contiguous extent and on the other it is fragmented, or the disk performance is far better on the nearly empty drive since the blocks occupied by the files are on an area of the disk that can be accessed faster (perhaps an outer cylinder where more data passes undedr the read head in a given rotation), or one of the disks is doing re-ahead, the other is not, even though identical physically. So many variables... Joe L.
August 3, 200718 yr Author Yea, there are. I'm in the middle of doing a full copy of all the data on the drive to the second drive. They are both exactly the same drive, can I use hdparam to verify they are both functioning at the same level to the OS? I'm leaning toward a combination of Fragmentation and Location as the cause. 50% is still a large number however.
August 3, 200718 yr And no, none of this directly improves network speed... but if we can't get data off the disk fast enough, it will never be able to get to the network clients. Before I set the read-ahead buffer size I was able to stream 4 separate ISO images to 4 clients on my LAN while rebuilding a missing drive. Oh yes, the 4 ISO images I was streaming were on the disk being rebuilt and therefore all of my drives had to be read to supply the data. With the improved/tweaked read-ahead buffer size, I expect it would do even better. I have seen no effect at all on playing a movie on my media player... The improved disk performance is not even being utilized for normal operations in my household. (There's never more than two movies being served at a given time, and 99% of the time, only 1 stream being served.) Your 50% difference is a bit more than I would have guessed due to fragmentation and location. (although location is a big factor) If you feel comfortable, you might switch SATA cables to the two drives and see if the better performance moves to the other SATA channel. You can still assign the same physical drives to the same logical slots in the unRaid drive config page even though they are on a different controller cable. Joe L.
August 3, 200718 yr Author I may try that...and you are correct, if I am seeing 20mb/sec reads...even if that is slower then 1/2 of my lowest "Parity Sync" speed...it is hardly going to cause the absurdly slow file transfers to Vista. The crazy part about all of this is that if I mount the share on the vista box and do the copy again...I get the same results. However, if I run IOMeter against that mounted drive, I see the speed I would expect with this disk performance. I guess I'm getting too far into the weeds. Even if these options are sub-optimal, they are clearly not causing my slow network performance to Vista. For that, I wait on Tom to put out the release with NFS support so I can test another method of file sharing. I can't think of any other options, unless anyone else knows a non-samba way to have a folder in Vista refer to a remote system so I can test .
August 3, 200718 yr Just wanted to say thanks for these detailed posts and that someone else is very interested in all this. I'm using an Asus P5W,E6600 running Vista 32 ultimate using media center with an unraid (v3 at the moment) server with 3 400 GB recent release seagate pata drives. I have netgear gigabit switches and also some xp machines with gigabit cards. I haven't really tested client file reading from unraid, only a single vista 32 client reading 20 Mbps or less hd .ts and .dvr-ms files which work fine though that's only requiring ~8MB/sec from the unraid server I guess. Anyway I hope to do some of the same testing as described here. I do have a desire to read ~20 MB/sec from unraid with my vista 32 client for large file copies etc.
August 3, 200718 yr Based on your experiment, I decided to see if performance of the "md" device could be tuned using the blockdev command. I'm going to add a "sleep 30;blockdev --setra 2048 /dev/md1 --setra /dev/md2 --setra /dev/md3 .. etc" line to my "go" script. Joe, you're my "hero of the day"!! I've tested your blockdev idea - but not locally on the Unraid server, but I've simply measured how quickly I can copy files from the Unraid server to my XP client. Here are my test results: test 1: blockdev 256 - 19.27 MB/s test 2: blockdev 256 - 18.96 MB/s test 3: blockdev 2048 - 26.02 MB/s test 4: blockdev 2048 - 24.79 MB/s So, your little change has resulted in 33% real life performance improvement for me. Do you happen to have some more such ideas up in your sleeve? Some questions about your "go" script addition: (1) Why is it "sleep 30;blockdev --setra 2048 /dev/md1 --setra /dev/md2"? Shouldn't it be: "sleep 30;blockdev --setra 2048 /dev/md1 --setra 2048 /dev/md2"? (2) Does it all have to be in one line? I'd prefer to put it in separate lines. (3) Is the "sleep" necessary? Why did you put it there? (4) Does the sleep only work on what is written after it in the same line? Or does it also affect on the lines following the sleep line? Thanks a bunch!!
August 3, 200718 yr Author Actually, it should look like this... sleep 30;blockdev /dev/md1 /dev/md2 ... The sleep is to make sure that the unRAID process has a chance to start and get settled I believe. But Joe can correct me if I'm wrong
August 3, 200718 yr Based on your experiment, I decided to see if performance of the "md" device could be tuned using the blockdev command. I'm going to add a "sleep 30;blockdev --setra 2048 /dev/md1 --setra /dev/md2 --setra /dev/md3 .. etc" line to my "go" script. Joe, you're my "hero of the day"!! I've tested your blockdev idea - but not locally on the Unraid server, but I've simply measured how quickly I can copy files from the Unraid server to my XP client. Here are my test results: test 1: blockdev 256 - 19.27 MB/s test 2: blockdev 256 - 18.96 MB/s test 3: blockdev 2048 - 26.02 MB/s test 4: blockdev 2048 - 24.79 MB/s So, your little change has resulted in 33% real life performance improvement for me. Do you happen to have some more such ideas up in your sleeve? Some questions about your "go" script addition: (1) Why is it "sleep 30;blockdev --setra 2048 /dev/md1 --setra /dev/md2"? Shouldn't it be: "sleep 30;blockdev --setra 2048 /dev/md1 --setra 2048 /dev/md2"? (2) Does it all have to be in one line? I'd prefer to put it in separate lines. (3) Is the "sleep" necessary? Why did you put it there? (4) Does the sleep only work on what is written after it in the same line? Or does it also affect on the lines following the sleep line? Thanks a bunch!! Very happy it makes a real-world difference. I'll take a 33% improvement in performance any day. In shell scripting, two commands on the same line, separated by a semicolon as I showed, are executed sequentially Commands on subsequent lines are also executed each in turn, and in the order written. The purpose of the sleep command is to allow the emhttp process to start up the array and initiate the "md" devices. I gave it 30 seconds for that task. Since I reboot very seldom, I really did not care if the first 30 second or so had less efficient read performance. I actually did put a series of commands in my go script as you described, and when they are invoked the commands listing slots that are not populated fail. The alternate syntax, of one command listing each attribute to set, the value and affiliated disk parameters would also work, but I do not know what happens when a slot mentioned is not populated. Besides, when I started typing it, it wrapped around and was not as neat looking. My "go" file is as shown below... Glad to have made your day. Joe L. #!/bin/bash # Start the Management Utility /usr/local/sbin/emhttp & sleep 30; find /mnt -print >/dev/null 2>/dev/null blockdev --setra 2048 /dev/md1 blockdev --setra 2048 /dev/md2 blockdev --setra 2048 /dev/md3 blockdev --setra 2048 /dev/md4 blockdev --setra 2048 /dev/md5 blockdev --setra 2048 /dev/md6 blockdev --setra 2048 /dev/md7 blockdev --setra 2048 /dev/md8 blockdev --setra 2048 /dev/md9 blockdev --setra 2048 /dev/md10 blockdev --setra 2048 /dev/md11 blockdev --setra 2048 /dev/md12 blockdev --setra 2048 /dev/md13 blockdev --setra 2048 /dev/md14 To be honest, the following lines would work for everybody and set all of the existing drives in a loop, as the array expands, the loop would automatically include them. No need for the individual lines for each drive. sleep 30 for i in /dev/md* do blockdev --setra 2048 $i done
August 3, 200718 yr sleep 30 for i in /dev/md* do blockdev --setra 2048 $i done That looks neat - thanks!
August 6, 200718 yr Just wanted to say thanks for these detailed posts and that someone else is very interested in all this. I'm using an Asus P5W,E6600 running Vista 32 ultimate using media center with an unraid (v3 at the moment) server with 3 400 GB recent release seagate pata drives. I have netgear gigabit switches and also some xp machines with gigabit cards. I haven't really tested client file reading from unraid, only a single vista 32 client reading 20 Mbps or less hd .ts and .dvr-ms files which work fine though that's only requiring ~8MB/sec from the unraid server I guess. Anyway I hope to do some of the same testing as described here. I do have a desire to read ~20 MB/sec from unraid with my vista 32 client for large file copies etc. An update: I tested my aforementioned vista machine reading from unRaid v3 default config and got 20 MB/sec sustained with a 14 gig (total) windows explorer copy of 1 gig .vob files so Tybio's issues seem to be specific to his setup. I presume the 'blockdev --setra 512 /dev/md1' trick cannot be used with unRaid v3? I get the below when I try it. root@raid_01:/usr/src/linux/drivers/md# blockdev --setra 512 /dev/md1 BLKRASET: Invalid argument I may try v4.1 unRaid but am wary of the disconnect problem people are seeing. http://lime-technology.com/forum/index.php?topic=888.0
August 6, 200718 yr Author It is a sporadic Vista problem from what I'm seeing. Does not seem linked to hardware...but there are reports of it all over...even if there are not enough of them to make it a big deal. While it is not a common problem, I don't think it is linked to hardware (mostly cause I've swapped everything out on both Vista systems). It effects an nforce4 AMD system AND a 775 Intel system. I've swapped out all cables, the switch, and all the network cards....so while I accept it is not a pervasive problem, I don't think it is implementation specific. It is a Samba issue of some nature. The primary reason I think that is that when transfering files between the two Vista boxes there are 20+ SMB requests for "chunks" of the file at a time. When going from the Vista boxes to the unRAID box OR to the XP box there are only 2 at a time. When pulling files from the unRAID system to the XP system it is perfectly fine...and FTP from the Vista boxes to the unRAID server is also fine.
August 6, 200718 yr FWIW, my Vista machine has the same problems. Fast write speed. Very slow read speed (2-3 MB/s). My XP machine is fast in reading and writing.
August 6, 200718 yr I have the same problem with Vista, I can write at 16 MB/S But when I read it is only 4-5 MB/s never more. I have tested it on XP and the reads are normal.
August 6, 200718 yr Thanks for the great system tweak Joe L. I went (on VERY rough calculations) from about 20mbps to 28mbps using the 2048 in the recursive script you posted. Now just to get write speeds up
August 7, 200718 yr Thanks for the great system tweak Joe L. I went (on VERY rough calculations) from about 20mbps to 28mbps using the 2048 in the recursive script you posted. Now just to get write speeds up Happy to have helped. Going from 20 to 28 Mbps is about a 40% increase in speed. Not too shabby. I'm curious, when writing to a disk, the existing data blocks in both the disk being written to, and the parity drive being written must be read first for the parity calculations to occur, therefore, the improved "read" performance might just also improve write performance. Have you done any tests to see if setting the read-ahead buffer size helped improve the effective speed of writing data to the array too? Joe L.
August 7, 200718 yr Going from 20 to 28 Mbps is about a 40% increase in speed. Not too shabby. Yes, it knocked about a minute off copy time of a 5.81gb movie folder I'm getting fairly decent write speeds, but hey, we'd all like to be able to shoot a 6 gig DVD over in 1 minute or less, hehehe. Takes me about 7 or 8 to do a 6gb movie. Network usage is a sawtooth (never loose connectivity, but usage goes from around 28% down to 0 and back up, about a 8 second period). I believe this is because the unRaid box just can't write and do the new parity calcs, etc, as fast as the network can deliver. What command do I use to set the read-ahead buffer? (I know next to nothing about Linux). Thanks again sir!!
August 8, 200718 yr Author While waiting for Tom to get the version with kernel NFS support built, I'm going to go into pure "I don't care if I have to reinstall" mode and try out a bit of a nuclear option. I'm downloading the pre-release Beta of SP1 for Vista....I'll install it and let you all know if copy speeds change at all! wish me luck
August 8, 200718 yr I'm downloading the pre-release Beta of SP1 for Vista....I'll install it and let you all know if copy speeds change at all! Would either the performance or compatibility packs help out? Saves you from a reinstall. http://arstechnica.com/news.ars/post/20070807-vista-performance-and-compatibility-packs-released.html Michael
August 13, 200718 yr i actually have the same issue with vista running to my windows server 2003 machine, and those updates did not help at all. Transferring from vista to server gets normal speeds (60+MB/sec), but going from server to vista i get only 6-12MB/sec. if even that when using mapped drives and explorer. It's a problem with vistas explorer, b/c when using ftp to trasnfer everything works the same speed going from server to vista or vista to server.
Archived
This topic is now archived and is closed to further replies.