July 29, 200718 yr Greetings all, I've just started up an unRAID server with 7 disks using the ASUS P5B-VM DO motherboard. While moving my media around I've found that between my two desktops I can get 30MB/sec+, while I only get at most 12MB/sec to the unraid server. Also, reading is even slower then that (which makes no sense, I admit. All devices are connected to the same netgear gig switch. I understand that the Ethernet driver on the Asus board does not support jumbo frams with the drivers we are using. Is that a driver limitation? Or is there somewhere else I should be looking for a problem? On a slightly tangential note, I got some error messages about running out of RAM. Is 512M still enough to run 4.1 or should I get the second stick to bump it to a Gig? Any pointers would be great!
July 29, 200718 yr I improved by write performance to my unRAID box by using a regular Linux swap partition... and that should eliminate your memory problems too. If you have a spare disk of any size, 4GB or more) just put it in the case, don't add it to he array, then fdisk it as a swap partition, and add the swapon <device name> command to your go script.
July 29, 200718 yr Author Good tip, I'll see if I have a drive around for that. In the meantime I've upgraded to 2G of DDR2 RAM (Was on sale while I was out shopping . The problem has not gone away. I did note these lines in the log, does this mean that rather then SATA II speeds I'm running slower then dirt to the drives? Jul 30 14:03:51 Storage kernel: [ 48.156260] ata1.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.157298] ata1.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.687843] ata2.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.689718] ata2.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 49.224163] ata3.00: ATA-7: WDC WD5000AAKS-22TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 49.225166] ata3.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.092595] ata5.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.093560] ata5.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.631648] ata6.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.632623] ata6.00: configured for UDMA/133
July 29, 200718 yr Author Quick followup, I'm now pushing data to the server at 12MB/sec+...which is fine. Reads are only at 3.5MB/sec...which is not good enough for stutter free HD playback.
July 29, 200718 yr Good tip, I'll see if I have a drive around for that. In the meantime I've upgraded to 2G of DDR2 RAM (Was on sale while I was out shopping . The problem has not gone away. I did note these lines in the log, does this mean that rather then SATA II speeds I'm running slower then dirt to the drives? Jul 30 14:03:51 Storage kernel: [ 48.156260] ata1.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.157298] ata1.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.687843] ata2.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 48.689718] ata2.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 49.224163] ata3.00: ATA-7: WDC WD5000AAKS-22TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 49.225166] ata3.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.092595] ata5.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.093560] ata5.00: configured for UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.631648] ata6.00: ATA-7: WDC WD5000AAKS-00TMA0, 12.01C01, max UDMA/133 Jul 30 14:03:51 Storage kernel: [ 50.632623] ata6.00: configured for UDMA/133 Yup... might be the underlying reason for the slow performance. Check other posts, in most cases it was a BIOS setting that caused the motherboard to treat the SATA drives as emulated IDE drives rather than SATA drives. As far a running out of memory... make sure you DO NOT accidentally write to a user-share. To make certain, export user-shares as READ-ONLY until Tom releases version 4.2 of unRaid (Which is supposed to have writable user-shares) If you accidentally write to a user-share, odds are good the file you create will exist only in RAM, and if you create a large file in RAM, larger than your physical memory, you will crash your server.
July 29, 200718 yr Author Huh, I wonder how much I would have to beg to get the Limetech guys to post the BIOS config they are using on the Asus board . I'll mess with it and see what I can do. I don't have any user shares as yet, so I'm safe on that count
July 29, 200718 yr Author Red herring on the UDMA/133 thing: Estimated speed: 58,161 KB/sec Estimated finish: 136.9 minutes No way I would get that with IDE emulation right? I'm back to lost!
July 29, 200718 yr Red herring on the UDMA/133 thing: Estimated speed: 58,161 KB/sec Estimated finish: 136.9 minutes No way I would get that with IDE emulation right? I'm back to lost! You are correct... you would not get that kind of disk performance with IDE emulation... Your disks are working fine. OK on the user-shares... Look to your network card driver or configuration for the inefficiency. (log-shot, but easy to check... could also be a poorly performing LAN cable with lots of re-tries on errors. Try a different one)
July 29, 200718 yr Author Ok, some more information. Between the two Vista boxes on the same switch, file transfers in ether direction stay over 30MB/sec. From ether Vista box to unRAID (writes) average about 12MB/sec, which I am fine with... From unRAID to the Vista boxes (reads) average about 4MB/sec, that's where the problem rests. I'm not seeing any errors on the unRAID ethernet interface. I have tried this with both the Intel onboard NIC and adding in a D-Link 530T with Jumbo frames enabled. See below for more information on things, log shows boot, and then the next message is me logging in after testing file transfers to save for posting. eth0 Link encap:Ethernet HWaddr 00:0F:3D:F1:98:56 inet addr:192.168.1.109 Bcast:192.168.1.255 Mask:255.255.255.0 UP BROADCAST NOTRAILERS RUNNING MULTICAST MTU:1500 Metric:1 RX packets:2578544 errors:0 dropped:0 overruns:0 frame:0 TX packets:4183561 errors:0 dropped:0 overruns:0 carrier:0 collisions:0 txqueuelen:1000 RX bytes:1893211475 (1.7 GiB) TX bytes:1489825082 (1.3 GiB) Interrupt:17
July 29, 200718 yr Author Another bit of information. Pulling a folder (4G) from the unRAID box to ether of my two vista computers takes about 20 minutes (3.5MB/sec). Doing the same thing from an XP box takes about 4/5 minutes. I don't know the speed, as XP file copy does not show it...but I think this is clearly isolated to a Vista issue.
July 29, 200718 yr Author Ok, after doing 3 things to my vista systems I think I've resolved the issue. For others who may be running Vista systems, if you are not getting acceptable network performance try these settings: 1> Disable "Remote Differential Compression": Control Pannel --> Programs and Features --> Turn Windows Features on or off (Bottom of left options) and uncheck the box. 2> Open a command prompt and enter the following commands: netsh interface tcp set global rss=disabled netsh interface tcp set global autotuninglevel=disabled 3> Reboot. Doing these things improved my writes to the server by ~2MB/sec. My reads from the unRAID server went from 3.5MB/sec to 25MB/sec. Going to do a HUGE copy now, I'll report back if these settings hold up for both systems, and over a long transfer.
July 29, 200718 yr Author Nevermind, didn't help after all. Not sure what's going on...but this is unusable for me with Vista clients
July 30, 200718 yr Apparently, a number of people are reporting problems with Vista network speed, under some circumstances. Here's one link: http://www.avsforum.com/avs-vb/showthread.php?t=881089. I saw somewhere else about someone regaining good speed, by turning off iTunes, obviously not a good solution.
July 30, 200718 yr Author Aye, I'm sure at this point that it is a Vista computability issue. As I am running Vista for Media Center...I guess I just have to hope that the Linux Samba team can figure out what junk Vista is expecting...or SP1 comes out. Till then, no HD streaming for me. Even SD DVD rips are having issues from time to time. Quite disheartening really.
July 30, 200718 yr Author Well, I'm done. Unless anyone has some ideas on this question I'm just going to suffer with bad speed until something updates. I guess the /last/ option I have is to try a new NIC rather then the nForce4 onboard NIC. Perhaps for $20 it is worth giving a shot.
July 31, 200718 yr I'm having the same problem.. only I average about 6MB/s for read speeds. Still, this is not acceptable. I've noticed this also seems to have something to do with Vista. I'm using the Intel Pro/1000 GigE adapter. Is there some tweak that needs to be applied to this NIC in Vista to get it to work properly with unRAID? Enable jumbo frames or flow control or...
July 31, 200718 yr Author Anyone know if it is possible to upgrade Samba to 3.0.25 on a running system to test it? I know Linux fairly well, but I'm unsure which flavor of pre-built distribution package I would need to look for.
August 1, 200718 yr unRAID version 4.1 uses Samba version 3.0.24. In reading the release notes for Samba 3.0.25, there is: o Improved compatibility with Windows Vista clients including improved read performance with Linux servers. We will upgrade Samba along with the next unRAID release, which should be sometime next week.
August 1, 200718 yr unRAID version 4.1 uses Samba version 3.0.24. In reading the release notes for Samba 3.0.25, there is: o Improved compatibility with Windows Vista clients including improved read performance with Linux servers. We will upgrade Samba along with the next unRAID release, which should be sometime next week. Gotta love the frequent releases - thanks Tom. Bill
August 1, 200718 yr Author Thanks a ton Tom, I'm 9 hours into an 18 hour file copy to pull 180G of data off one of my drives. This is not really going to work well until I can resolve the Vista thing .
August 2, 200718 yr Author Well, I installed 3.0.25b on my box today to test it out and it is "better" from 3.5MB/sec reads to 4.5MB/sec reads on Vista. XP /flys/. FTP (via vsftpd) averages 17MB/sec with spikes well above that...so it's clearly not a network problem. Vista is the stink
August 2, 200718 yr Author Ok, some more information Tom. While messing around with an old drive today I set it up outside the array formatted as ext2. I then added it to smb.shares and reloaded the samba binary that comes with 4.1. I copied a directory from my first unRAID disk onto the ext2 partition and pulled it down with the Vista Media Center in full (4.02GB): Transfer Speed: 7.8MB/sec Once that was done, without changing anything at all I opened up a window to //server/disk1 and did another copy of the same directory off of the unraid md1: Transfer Speed: 4.7MB/sec Odd tidbits: While transferring off of md1: (unRAID array disk) CPU was between 75% and 100% wait at all times (mostly over 90%, but it's hard to judge). Load Average was .9 to 1.1 by the end of the transfer. Not sure if this is useful, as hdparm is not supposed to work with SATA drives in non-IDE mode (I think that's what these errors mean) root@Storage:~# hdparm -Tt /dev/md1 /dev/md1: Timing cached reads: 3556 MB in 2.00 seconds = 1777.40 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device Timing buffered disk reads: 38 MB in 3.08 seconds = 12.33 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device While transferring off of sdh2 (ext2 non-raid disk) the CPU was between 40% and 90% wait at all times. (averaged at about 65%). Load Average was .7 to .85 by the end of the transfer root@Storage:~# hdparm -Tt /dev/sdh /dev/sdh: Timing cached reads: 3572 MB in 2.00 seconds = 1785.15 MB/sec Timing buffered disk reads: 172 MB in 3.01 seconds = 57.10 MB/sec While 8MB/sec is not even close to what it should be, as I get 20+MB/sec via FTP so I'd think 15MB/sec would not be unreasonable to expect via samba, it is twice as fast from non-raid filesystems.
August 3, 200718 yr Based on your experiment, I decided to see if performance of the "md" device could be tuned using the blockdev command. First, I looked at how the current re-ahead buffer was set. root@Tower:/usr/src/linux/drivers/md# blockdev --getra /dev/md1 256 Next, I performed a test of the read speed by copying an ISO image to /dev/null. I ended up with a speed of 24.9 MB/s. This speed was pretty consistent with repeated "dd" commands. root@Tower:/usr/src/linux/drivers/md# dd if=/mnt/disk1/Movies/URBANCOWBOY.ISO of=/dev/null 9988316+0 records in 9988316+0 records out 5114017792 bytes (5.1 GB) copied, 205.588 seconds, 24.9 MB/s Next, I set the read-buffer to 512k and tried the test again. Now I was getting 36.1 MB/s. A nice improvement root@Tower:/usr/src/linux/drivers/md# blockdev --setra 512 /dev/md1 root@Tower:/usr/src/linux/drivers/md# dd if=/mnt/disk1/Movies/URBANCOWBOY.ISO of=/dev/null 9988316+0 records in 9988316+0 records out 5114017792 bytes (5.1 GB) copied, 141.468 seconds, 36.1 MB/s I tried a bit bigger... 1024k, this time read speed improved to 39.6 MB/s root@Tower:/usr/src/linux/drivers/md# blockdev --setra 1024 /dev/md1 root@Tower:/usr/src/linux/drivers/md# dd if=/mnt/disk1/Movies/URBANCOWBOY.ISO of=/dev/null 9988316+0 records in 9988316+0 records out 5114017792 bytes (5.1 GB) copied, 128.595 seconds, 39.8 MB/s A bit bigger still and I ended up with 40.1 MB/s read speed. This is a lot better than the 24.9 I started with. I found the speed did not improve when setting the read-buffer bigger than 2048k root@Tower:/usr/src/linux/drivers/md# blockdev --setra 2048 /dev/md1 root@Tower:/usr/src/linux/drivers/md# dd if=/mnt/disk1/Movies/URBANCOWBOY.ISO of=/dev/null 9988316+0 records in 9988316+0 records out 5114017792 bytes (5.1 GB) copied, 127.655 seconds, 40.1 MB/s OK, now... put the buffer size back to the original 256k value and try using the hdparm command to measure speed root@Tower:/usr/src/linux/drivers/md# blockdev --setra 256 /dev/md1 root@Tower:/usr/src/linux/drivers/md# hdparm -tT /dev/md1 /dev/md1: Timing cached reads: 2692 MB in 2.00 seconds = 1346.21 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device Timing buffered disk reads: 38 MB in 3.05 seconds = 12.45 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device re-check the "dd" speed. yup, it's back to the original slower performance I started with root@Tower:/usr/src/linux/drivers/md# dd if=/mnt/disk1/Movies/URBANCOWBOY.ISO of=/dev/null 9988316+0 records in 9988316+0 records out 5114017792 bytes (5.1 GB) copied, 206.27 seconds, 24.8 MB/s Now, set the buffer to 2048k and check using hdparm -tT. root@Tower:/usr/src/linux/drivers/md# blockdev --setra 2048 /dev/md1 root@Tower:/usr/src/linux/drivers/md# hdparm -tT /dev/md1 /dev/md1: Timing cached reads: 2692 MB in 2.00 seconds = 1346.49 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device Timing buffered disk reads: 184 MB in 3.01 seconds = 61.07 MB/sec HDIO_DRIVE_CMD(null) (wait for flush complete) failed: Inappropriate ioctl for device Wow... according to the hdparm test, read speed increased from 12.45 MB/sec to 61.07 MB/sec. According to my "dd" command, we went from 24.9 MB/sec to 40.1 MB/sec. Obviously, they disagree in the specific speeds being reported, but both show substantial improvement with a 2048k read-ahead buffer set by the blockdev command. I'm going to add a "sleep 30;blockdev --setra 2048 /dev/md1 --setra /dev/md2 --setra /dev/md3 .. etc" line to my "go" script. Performance was decent before, this should make it even better. Tom... might the default read-ahead buffer size be configured to 2048k in the next release? I'm guessing that the difference in buffer size between the "md" device and the /dev/hd device cause multiple requests to be issued to the physical disk slowing everything down. Increasing the read-ahead buffer size on the "md" device makes everything match better. Joe L.
Archived
This topic is now archived and is closed to further replies.