alphazo Posted December 6, 2009 Share Posted December 6, 2009 Hello, I just finished my new unRAID server (4.5-beta12).I notice a 50% performance drop when running Linux OS instead of MAC OS on the same hardware platform. Do you see the same thing? Does it come from the samba client implementation in Linux or NIC card support? Would getting a separate PCMCIA Gigabit ethernet (for the Dell machine) solve the problem? Server configuration Motherboard: Intel D945GCLF2 (Atom330 @ 1.6GHz) Memory: 2GB Hard drive controller #1 (on-board) : Intel 82801GB, ICH7 SATA controller Hard drive controller #2 (PCI) : Promise SATA 300 TX4 disk1 (connected to ICH7) : Samsung EcoGreen HD154UI, 1.5TB, 32MB cache disk2 (connected to Promise SATA300) : Samsung EcoGreen HD154UI, 1.5TB, 32MB cache disk3 (connected to Promise SATA300) : Samsung EcoGreen HD154UI, 1.5TB, 32MB cache disk4 (parity drive, connected to ICH7) : Samsung EcoGreen HD154UI, 1.5TB, 32MB cache No cache drive! NIC (on-board) : Gigabit Realtek 811C. MTU can be set to maximum 7200 on D945GCLF2. Case: Chenbro ES34069 Mini-ITX Gigabit switch: D-Link DGS-1008D Guests Guest # 1 : Macbook 13" (Core2Duo) with Marvell Yukon 88E8053 NIC, Mac OS 10.5.7 and Ubuntu 9.10 LiveCD Guest # 2 : Dell D430 with Broadcom BCM5752 Gigabit Ethernet NIC. ArchLinux and Ubuntu 9.10 LiveCD To run the benchmark I copied 5 files of 693.8MB each from and to the unRAID server. On unRAID, I started bwm-ng and hit 't' three times to get an average speed over 30s then I would note the peak average speed. I also started a timer manually to get an idea of the transfer time. For mounting remote filesystem on guests I used the built-in client under MacOS and Ubuntu and mount.cifs method under ArchLinux. My conclusions 100Mbit/s is slow MacOS beats Linux by 50% on the same hardware (at least on two architectures) Hard drive controller impacts performance (PCI vs on-board), especially when writing to an array that doesn't have cache drive. Increasing MTU doesn't boost performance significantly. In some cases it even lowers overall performances (might be switch related though). Disabling flow control appears to increase performance by a very small factor. Encryption (encFS) has a very significant impact on throughput (like running on a 100Mbit/s network). So what do think about the benchmark down there and more importantly what do I need to do in order to get 34MB/s download on Linux? Alphazo ************************************** * Speed: Gigabit * * MTU: 1500 on both unRAID and guest * * Flow control: ENABLED on MAC OS * ************************************** ------------------------------------------------------------------------------------------------- | MacBook | MacBook | Dell D430 | Dell D430 | Dell D430 | | MAC OS 10.5.7 | Ubuntu 9.10 | ArchLinux | ArchLinux | Ubuntu 9.10 | | int. HDD | ext. HDD | int. HDD | ext. HDD | ext. HDD | ------------------------------------------------------------------------------------------------------- | Prot. | Disk mode | User share | Disk mode | Disk mode | Disk mode | Disk mode | --------------------------------------------------------------|-------------------------------|-------------------------------| | To disk2 | SMB | 2:40 | 23MB/s | 2:31 | 25MB/s | | | | | | | | | |-------------------------------------------------------------|-------------------------------|------------------------------ | | To disk2 (encFS) | SMB | 6:25 | 9MB/s | 6:44 | 9MB/s | | | | | | | | | |-------------------------------------------------------------|-------------------------------|-------------------------------| | From disk2 | SMB | 1:50 | 34MB/s | 2:03 | 25MB/s | | | | | | | | | |-------------------------------------------------------------|-------------------------------|-------------------------------| | From disk2 (encFS) | SMB | 4:23 | 14MB/s | 4:15 | 14MB/s | | | | | | | | | |-------------------------------------------------------------|-------------------------------|-------------------------------| | To disk1 | SMB | 2:08 | 29MB/s | 2:15 | 30MB/s | 3:44 | 16MB/s | 4:00 | 14MB/s | 2:37 | 23MB/s | 4:05 | 15MB/s | |-------------------------------------------------------------|-------------------------------|------------------------------ | | To disk1 | FTP | 1:56 | 31MB/s | | | 2:03 | 29MB/s | 3:38 | 20MB/s | 1:59 | 31MB/s | 2:15 | 27MB/s | |-------------------------------------------------------------|-------------------------------|------------------------------ | | To disk1 (encFS)* | SMB | 6:56 | 9MB/s | 7:53 | 8MB/s | | | | | | | | | |-------------------------------------------------------------|-------------------------------|-------------------------------| | From disk1 | SMB | 1:48 | 34MB/s | 1:48 | 34MB/s | 2:53 | 22MB/s | 4:00 | 14MB/s | 4:01 | 15MB/s | 3:32 | 16MB/s | |-------------------------------------------------------------|-------------------------------|-------------------------------| | From disk1 | FTP | 1:51 | 33MB/s | | | 2:20 | 28MB/s | 3:52 | 16MB/s | 2:07 | 29MB/s | 2:30 | 26MB/s | |-------------------------------------------------------------|-------------------------------|-------------------------------| | From disk1 (encFS)* | SMB | 4:45 | 13MB/s | 5:19 | 12MB/s | | | | | | | | | |---------------------|-------|-------------------------------|-------------------------------|-------------------------------| *************************************** * Speed: Gigabit * * MTU: 1500 on both unRAID and guest * * Flow control: DISABLED on MAC OS * *************************************** --------------------------------- | MacBook / MAC OS | --------------------------------- | Disk mode | User share | -----------------------------------------------------| | To disk2 | 2:31 | 24MB/s | - | - | |----------------------------------------------------| | From disk2 | 1:40 | 38MB/s | - | - | |----------------------------------------------------| | To disk1 | 2:14 | 28MB/s | - | - | |----------------------------------------------------| | From disk1 | 1:48 | 34MB/s | - | - | |----------------------------------------------------| *************************************** * Speed: Gigabit * * MTU: 7200 on both unRAID and guest. * * Flow control: ENABLED on MAC OS * *************************************** --------------------------------- | MacBook / MAC OS | --------------------------------- | Disk mode | User share | -----------------------------------------------------| | To disk2 | 2:29 | 24MB/s | - | - | |----------------------------------------------------| | From disk2 | 1:45 | 36MB/s | - | - | |----------------------------------------------------| | To disk1* | 4:25 | 18MB/s | - | - | |----------------------------------------------------| | From disk1* | 4:02 | 19MB/s | - | - | |----------------------------------------------------| * For some reasons, troughtput dropped to 3MB/s toward the end. *************************************** * Speed: Gigabit * * MTU: 7200 on both unRAID and guest. * * Flow control: DISABLED on MAC OS * *************************************** --------------------------------- | MacBook / MAC OS | --------------------------------- | Disk mode | User share | -----------------------------------------------------| | To disk2 | 2:27 | 26MB/s | - | - | |----------------------------------------------------| | From disk2 | 1:40 | 39MB/s | - | - | |----------------------------------------------------| | To disk1 | 2:14 | 27MB/s | - | - | |----------------------------------------------------| | From disk1 | 1:47 | 34MB/s | - | - | |----------------------------------------------------| *************************************** * Speed: 100Mbit * * MTU: 1500 on both unRAID and guest. * * Flow control: ENABLED on MAC OS * *************************************** --------------------------------- | MacBook / MAC OS | --------------------------------- | Disk mode | User share | -----------------------------------------------------| | To disk2 | 6:31 | 9MB/s | - | - | |----------------------------------------------------| | From disk2 | 5:40 | 10MB/s | - | - | |----------------------------------------------------| | To disk1 | 6:31 | 9MB/s | - | - | |----------------------------------------------------| | From disk1 | 5:40 | 10MB/s | - | - | |----------------------------------------------------| [EDITED] on Dec 10 to add FTP transfer rate (same filezilla version running on MacOS, ArchLinux & Ubuntu) and benchmarks when using external drive on Dell D430. Link to comment
Yulquen Posted December 6, 2009 Share Posted December 6, 2009 have you tested ftp transfer speed? at least it always gives me the highest speed, much faster than samba. Link to comment
alphazo Posted December 10, 2009 Author Share Posted December 10, 2009 Thank you for pointing at the FTP protocol. I went through a round of new tests and discovered few more thing. I've updated the first table above to include FTP performance and external vs internal drive. Here is a quick summary of what I have obtained: SMB MacBook / MAC OS X: 30MB/s up, 34MB/s down MacBook / Ubuntu 9.10 / Ext. HDD: 16MB/s up, 22MB/s down Dell D430 / ArchLinux / Int. HDD: 14MB/s up, 14MB/s down Dell D430 / ArchLinux / Ext. HDD: 23MB/s up, 15MB/s down Dell D430 / Ubuntu 9.10 / Ext. HDD: 15MB/s up, 16MB/s down FTP MacBook / MAC OS X: 30MB/s up, 34MB/s down MacBook / Ubuntu 9.10 / Ext. HDD : 29MB/s up, 28MB/s down Dell D430 / ArchLinux / Int. HDD: 20MB/s up, 16MB/s down Dell D430 / ArchLinux / Ext. HDD: 31MB/s up, 29MB/s down Dell D430 / Ubuntu 9.10 / Ext. HDD: 15MB/s up, 16MB/s down So my additional conclusions are: On MacBook/MAC OS, same (good) performance level for both FTP and SMB Running Ubuntu on the MacBook (with an external drive) kills the SMB performance by 50%. On the plus side, FTP performance is close to the one under MAC OS. Low performance drive (1.8", 4200rpm) on the Dell machine slows down the transfer to/from unRAID. In SMB mode, switching to an external drive on the Dell PC increased performance when writing to the array (still not as good as MAC OS). However it didn't increase performance when reading from the array. In FTP mode, switching to an external drive on the Dell machine doubled the performance. Ubuntu on Dell machine is still very slow (both SMB and FTP) even after switching to an external drive So changing drive on the Dell machine allows me to get respectable performance over FTP (only under ArchLinux). However SMB is still behind and pretty bad when reading from the array. Is this something that can be improved? Alphazo PS: I guess the first thing to try is the new 4.5 release! Link to comment
alphazo Posted December 16, 2009 Author Share Posted December 16, 2009 I think I'm gonna rename the thread to "Why is Samba on Linux is 50% slower than MacOS?" Since last time I went to another round of benchmarks that I will post soon (based on 4.5 release). In the meantime I just wanted to share some numbers obtained on a different hardware platform. I've also included NFS and surprisingly it gave very good results (out of the box, no tweakings) - Lenovo T400 with ArchLinux 64-bit SMB disk2 -> T400 internal drive: 24MB/s, 2:38 T400 internal drive -> disk2: 26MB/s, 2:24 For some reasons an external USB drive gave slightly better results when gathering data from unRAID server with up to 28MB/s, 2:07. FTP disk2->T400 internal drive: 58MB/s, 1:00... yep one single minute for those 3.93GB T400 internal drive -> disk2: 36MB/s 1:43 In that case the USB drive shows its limit by capping to 33MB/s, 1:51 (to disk2) and 28MB/s, 1:56 (from disk2). But still better than SMB with internal drive NFS disk2->T400 internal drive: 58MB/s, 55s... better than FTP! T400 internal drive -> disk2: 28MB/s 2:16... slower than FTP I noticed that during 3.93GB might not be big enough for the bwm average throughput estimate. During the transfer I've seen it jumping to 70MB/s pretty often. I'm really puzzled by the low performance obtained on Linux Samba clients (even on new hardware) compared to MacOSX. NFS seems a nice alternative. Is this still considered as experimental on unRAID even 4.5 release? Thanks Alphazo [EDIT] I think I found the explanation for lower SAMBA performance on Linux (vs other platforms)...It was even in the wiki http://lime-technology.com/forum/index.php?topic=4996.0 Didn't have time to test it though! Link to comment
Recommended Posts
Archived
This topic is now archived and is closed to further replies.