ryan
-
Posts
64 -
Joined
-
Last visited
Content Type
Profiles
Forums
Downloads
Store
Gallery
Bug Reports
Documentation
Landing
Posts posted by ryan
-
-
I am actually in the process and have spent most of the weekend debugging and looking into this.
rsync will use the encryption cipher that ssh uses by standard.. to remedy this, i have enabled all cyphers on the synology box in DSM (choose Terminal > SSH > Advanced > Low Encyption Ciphers), also this on the unRAID box and then used the rsync command with ssh to force to use a different encryption algorithm.. the result?
I get the full 115MB/s over rsync to the Syno box, by running two and three of these on a trunk ethernet, i am sending at very high speeds.
rsync -av -e 'ssh -c arcfour' /mnt/user/Media/ [email protected]:/volume1/Media/ --progressGive it a try,
-
3 minutes ago, bonienl said:
In your settings I see Transmit Hash Policy: layer2, which makes balancing purely on MAC addresses.
I know you have tried layer2+3, did you change back? Also the interface needs to be set down/up to make the change effective.
OK, just tried that and confirmed it was layer2 (probably because of the reboot.. but i change, and reset the interface state.. What interesting is now when i have set this,
Transmit Hash Policy: layer2+3 (2)
Actor Churn State: none
Partner Churn State: none
root@ATLAS:~# cat /sys/class/net/bond0/bonding/xmit_hash_policy layer2 0 root@ATLAS:~# ifconfig bond0 down;echo 'layer2+3' >/sys/class/net/bond0/bonding/xmit_hash_policy;ifconfig bond0 up root@ATLAS:~# cat /sys/class/net/bond0/bonding/xmit_hash_policy layer2+3 2 root@ATLAS:~# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2+3 (2) ... -
Not the case, after i swap interfaces back, i get traffic on eth0 but not eth1..
-
1 hour ago, bonienl said:
The specifications of my motherboard say dual LAN with support of teaming. What does it say for your motherboard?
I had a look at "Actor/Partner Churn State" which in your case is "none" for eth0 and "churned" for eth1. I believe the correct state for both interfaces should be "monitoring", perhaps you do have a hardware limitation?
Interesting, i had a look around the web - this unRAID is on a HP NL54 - and i can see others have setup bonding too, so it should work OK.
I think the first post i must have looked at the output too eariy, i now see monitoring in the Actor/Partner state:root@ATLAS:~# cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable System priority: 65535 System MAC address: 00:24:9b:1a:cf:69 Active Aggregator Info: Aggregator ID: 2 Number of ports: 2 Actor Key: 9 Partner Key: 3327 Partner Mac Address: ec:08:6b:e4:f1:36 Slave Interface: eth0 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 00:24:9b:1a:cf:69 Slave queue ID: 0 Aggregator ID: 2 Actor Churn State: monitoring Partner Churn State: monitoring Actor Churned Count: 0 Partner Churned Count: 0 details actor lacp pdu: system priority: 65535 system mac address: 00:24:9b:1a:cf:69 port key: 9 port priority: 255 port number: 1 warning: this system does not seem to support IPv6 - trying IPv4 port state: 61 details partner lacp pdu: warning: this system does not seem to support IPv6 - trying IPv4 system priority: 32768 system mac address: ec:08:6b:e4:f1:36 oper key: 3327 port priority: 32768 port number: 12 port state: 61 Slave Interface: eth1 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 38:ea:a7:a9:2c:f9 Slave queue ID: 0 Aggregator ID: 2 Actor Churn State: monitoring Partner Churn State: monitoring Actor Churned Count: 0 Partner Churned Count: 0 details actor lacp pdu: system priority: 65535 system mac address: 00:24:9b:1a:cf:69 port key: 9 port priority: 255 port number: 2 port state: 61 details partner lacp pdu: system priority: 32768 system mac address: ec:08:6b:e4:f1:36 oper key: 3327 port priority: 32768 port number: 10 port state: 61I have swapped the interfaces around - that is to say eth0 <> eth1 in the unRAID configuration to test.. still the same, i do not get the full speed.
A few tests I can also see that the links are not "shared"
bond0: flags=5187<UP,BROADCAST,RUNNING,MASTER,MULTICAST> mtu 1500 inet 192.168.1.10 netmask 255.255.255.0 broadcast 0.0.0.0 ether 00:24:9b:1a:cf:69 txqueuelen 1000 (Ethernet) RX packets 3842122 bytes 5830623843 (5.4 GiB) RX errors 0 dropped 570 overruns 0 frame 0 TX packets 1395401 bytes 101876433 (97.1 MiB) TX errors 0 dropped 9 overruns 0 carrier 0 collisions 0 eth0: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500 ether 00:24:9b:1a:cf:69 txqueuelen 1000 (Ethernet) RX packets 65645 bytes 94136919 (89.7 MiB) RX errors 20 dropped 0 overruns 0 frame 20 TX packets 2664714 bytes 199981043 (190.7 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 eth1: flags=6211<UP,BROADCAST,RUNNING,SLAVE,MULTICAST> mtu 1500 ether 00:24:9b:1a:cf:69 txqueuelen 1000 (Ethernet) RX packets 3842029 bytes 5830605446 (5.4 GiB) RX errors 0 dropped 570 overruns 0 frame 0 TX packets 464779 bytes 32597478 (31.0 MiB) TX errors 0 dropped 0 overruns 0 carrier 0 collisions 0 device interrupt 18
Looks like eth0 is not used at all.. I will try and swap around again, it could be something to do with the eth0 interface!
-
In addition, i have tried bonding on my workstation.. and tested the opposite way around, this works fine, i get the full 2gbp/s speed. So i know it's not the switch setup, could it perhaps be the onboard ethernet device on the unRAID that doesn't support this somehow ?
-
1 hour ago, bonienl said:
See also this topic about traffic balancing at unRAID side (there is no GUI support).
Hmm i have just tried this, seems to make no difference.
I can see also with testing the opposite i.e. unRAID as the iperf3 client, and two seperate machines. I get the same.. although it does not seem to be balancing through the interfaces!
-
Thanks for the clarificaiton on STP.
The MAC Address is also fixed:
details actor lacp pdu:
system priority: 65535
system mac address: 38:ea:a7:a9:2c:f9
port key: 9
port priority: 255
port number: 2
port state: 61
details partner lacp pdu:
system priority: 32768
system mac address: ec:08:6b:e4:f1:36
oper key: 2536
port priority: 32768
port number: 12
port state: 61So, within the TP-LINK:
-
@bonienl Thanks for you observations - 192.168.1.10 is indeed my unRAID system.
The switch was setup with LAG and STP enabled, i also tested without. I have a TP-LINK T1600G-28TS.
MAC Address is strange too.. i didnt see that. If it is the switch not setting up the LAG properly, what other settings do i need to consider ? AFAIK i should only select two ports which need to be aggregated?
-
11 minutes ago, bonienl said:
A single pc to pc connection will not exceed 1 Gb/s. To take advantage of the bonded channel with link aggregation, you need multiple concurrent connections to/from different PCs.
There are three different machines connecting to two different iperf3 processes within the above test. The link should meet 2gbps ) i actually need 4 but testing with three.
-
I'm having some issues with my bonding.. i have two NIC which are:
root@ATLAS:/mnt/cache/cache_only# sudo cat /proc/net/bonding/bond0 Ethernet Channel Bonding Driver: v3.7.1 (April 27, 2011) Bonding Mode: IEEE 802.3ad Dynamic link aggregation Transmit Hash Policy: layer2 (0) MII Status: up MII Polling Interval (ms): 100 Up Delay (ms): 0 Down Delay (ms): 0 802.3ad info LACP rate: slow Min links: 0 Aggregator selection policy (ad_select): stable System priority: 65535 System MAC address: 38:ea:a7:a9:2c:f9 Active Aggregator Info: Aggregator ID: 1 Number of ports: 1 Actor Key: 9 Partner Key: 3502 Partner Mac Address: ec:08:6b:e4:f1:36 Slave Interface: eth0 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 0 Permanent HW addr: 38:ea:a7:a9:2c:f9 Slave queue ID: 0 Aggregator ID: 1 Actor Churn State: none Partner Churn State: none Actor Churned Count: 0 Partner Churned Count: 0 details actor lacp pdu: system priority: 65535 system mac address: 38:ea:a7:a9:2c:f9 port key: 9 port priority: 255 port number: 1 port state: 61 details partner lacp pdu: system priority: 32768 system mac address: ec:08:6b:e4:f1:36 oper key: 3502 port priority: 32768 port number: 10 port state: 61 Slave Interface: eth1 MII Status: up Speed: 1000 Mbps Duplex: full Link Failure Count: 1 Permanent HW addr: 00:24:9b:1a:cf:69 Slave queue ID: 0 Aggregator ID: 2 Actor Churn State: churned Partner Churn State: churned Actor Churned Count: 2 Partner Churned Count: 2 details actor lacp pdu: system priority: 65535 system mac address: 38:ea:a7:a9:2c:f9 port key: 9 port priority: 255 port number: 2 port state: 69 details partner lacp pdu: system priority: 65535 system mac address: 00:00:00:00:00:00 oper key: 1 port priority: 255 port number: 1 port state: 1However i am only gettng 1gbit throughput total from three different clients:
root 2312 0.7 0.0 6500 1696 pts/1 S+ 10:52 0:52 iperf3 -s -B 192.168.1.10 -p 5053 root 23263 1.6 0.0 6500 1764 pts/3 S+ 12:20 0:25 iperf3 -s -B 192.168.1.10 -p 5054 root 29199 2.3 0.0 6500 1596 pts/2 S+ 10:40 2:58 iperf3 -s -B 192.168.1.10 -p 5202Three different instances of iperf3, gives:
Computer 1 [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 390 MBytes 327 Mbits/sec 71 sender [ 4] 0.00-10.00 sec 388 MBytes 326 Mbits/sec receiver Computer 2 [ ID] Interval Transfer Bandwidth Retr [ 4] 0.00-10.00 sec 198 MBytes 166 Mbits/sec 1023 sender [ 4] 0.00-10.00 sec 198 MBytes 166 Mbits/sec receiver Computer 3 [ ID] Interval Transfer Bandwidth [ 4] 0.00-10.00 sec 861 MBytes 722 Mbits/sec sender [ 4] 0.00-10.00 sec 861 MBytes 722 Mbits/sec receiverAny ideas? Note the retries too!?
-
Noted! So this is a mix of incorrect usage.. and filesystem types.. Thanks! I'll have a play this weekend and see what i can figure out for my usage case.
-
What format was the cache using while you had a single drive? If you want to use a cache Pool then the only supported format is BTRFS so it is possible that trying added the second drive caused unRAID to switch the format (I think it is a bug that you do not get a prompt warning you about this) which is why the old data is no longer accessible and unRAID wants to format the drives.
To preserve the contents you need to
- Stop any services using the cache (docker/VMs)
- copy the current contents off the drive to somewhere safe (probably an array data drive)
- set up the cache pool as you want it
- start the array and select the option to format the cache pool
- copy the saved data back to the cache drive
- restart any stopped services that use the cache drive
Yes i think this may be the case.. I have actually figured out that if i take the slots down to 1, then mount the drive changed back to XFS then it mounts OK..
I do not wish to use cache pool, but seperate drives cache1, cache2.. is this possible.. or do i have the usage of cache drives incorrect?
- Stop any services using the cache (docker/VMs)
-
I have added a second cache drive, to be out of the current array.. however upon adding the drive and starting the array.. the existing cache drive was starting to clear?? now both drives are unmountable?!
I have seem to have lost my entire app directory for docker.. and all other content on there ??
Why has this happened?!
I have temporarily mounted the cache, and i can see that the data is still there
root@ATLAS:/mnt/test# ls -la total 0 drwxrwxrwx 6 nobody users 70 Nov 24 20:24 ./ drwxr-xr-x 8 root root 160 Nov 24 22:48 ../ drwxrwxrwx 14 nobody users 247 Nov 24 21:20 appdata/ drwxrwxrwx 2 nobody users 24 Oct 22 17:54 cache_only/ drwxrwxrwx 10 nobody users 332 Oct 15 17:43 downloads/ drwxrwxrwx 8 nobody users 242 Nov 24 22:25 vmware/
How can i get this back into the cache section? Every time i add it says i need to format..
-
perhaps try
lsof | grep /mnt/ and see if there is anyhting you recognise?
-
-
Thanks - I totally missed this. I'll post my findings when completed.
-
[reserved] for solution/followup
-
Mount gets forced into read-only state.. this is the errors i get in syslog:
Oct 22 12:47:58 ATLAS sshd[11194]: Accepted password for root from 192.168.1.40 port 63738 ssh2 Oct 22 12:56:27 ATLAS kernel: BTRFS critical (device sde1): corrupt leaf, slot offset bad: block=234264772608,root=1, slot=34 Oct 22 12:56:27 ATLAS kernel: BTRFS critical (device sde1): corrupt leaf, slot offset bad: block=234264772608,root=1, slot=34 Oct 22 12:56:27 ATLAS kernel: ------------[ cut here ]------------ Oct 22 12:56:27 ATLAS kernel: WARNING: CPU: 1 PID: 11780 at fs/btrfs/extent-tree.c:2927 btrfs_run_delayed_refs+0xa8/0x22f() Oct 22 12:56:27 ATLAS kernel: BTRFS: Transaction aborted (error -5) Oct 22 12:56:27 ATLAS kernel: Modules linked in: ipt_MASQUERADE nf_nat_masquerade_ipv4 iptable_nat nf_conntrack_ipv4 nf_nat_ipv4 iptable_filter ip_tables nf_nat md_mod bonding e1000e tg3 ptp pps_core kvm_amd kvm k10temp ahci i2c_piix4 libahci i2c_core acpi_cpufreq [last unloaded: pps_core] Oct 22 12:56:27 ATLAS kernel: CPU: 1 PID: 11780 Comm: kworker/u8:7 Not tainted 4.4.23-unRAID #1 Oct 22 12:56:27 ATLAS kernel: Hardware name: HP ProLiant MicroServer, BIOS O41 07/29/2011 Oct 22 12:56:27 ATLAS kernel: Workqueue: btrfs-extent-refs btrfs_extent_refs_helper Oct 22 12:56:27 ATLAS kernel: 0000000000000000 ffff8800a7133cb8 ffffffff8136ad0c ffff8800a7133d00 Oct 22 12:56:27 ATLAS kernel: 0000000000000b6f ffff8800a7133cf0 ffffffff8104a486 ffffffff812b147c Oct 22 12:56:27 ATLAS kernel: ffff880203dd44c0 ffff8802054a8800 00000000fffffffb 0000000000000000 Oct 22 12:56:27 ATLAS kernel: Call Trace: Oct 22 12:56:27 ATLAS kernel: [<ffffffff8136ad0c>] dump_stack+0x61/0x7e Oct 22 12:56:27 ATLAS kernel: [<ffffffff8104a486>] warn_slowpath_common+0x8f/0xa8 Oct 22 12:56:27 ATLAS kernel: [<ffffffff812b147c>] ? btrfs_run_delayed_refs+0xa8/0x22f Oct 22 12:56:27 ATLAS kernel: [<ffffffff8104a4e2>] warn_slowpath_fmt+0x43/0x4b Oct 22 12:56:27 ATLAS kernel: [<ffffffff812afb1b>] ? __btrfs_run_delayed_refs+0xc3d/0xc7f Oct 22 12:56:27 ATLAS kernel: [<ffffffff812b147c>] btrfs_run_delayed_refs+0xa8/0x22f Oct 22 12:56:27 ATLAS kernel: [<ffffffff812b163a>] delayed_ref_async_start+0x37/0x76 Oct 22 12:56:27 ATLAS kernel: [<ffffffff812eb052>] btrfs_scrubparity_helper+0xc9/0x17e Oct 22 12:56:27 ATLAS kernel: [<ffffffff812eb131>] btrfs_extent_refs_helper+0x9/0xb Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105ae70>] process_one_work+0x194/0x2a0 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105b826>] worker_thread+0x26b/0x353 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105b5bb>] ? rescuer_thread+0x285/0x285 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105fab6>] kthread+0xcd/0xd5 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105f9e9>] ? kthread_worker_fn+0x137/0x137 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8162167f>] ret_from_fork+0x3f/0x70 Oct 22 12:56:27 ATLAS kernel: [<ffffffff8105f9e9>] ? kthread_worker_fn+0x137/0x137 Oct 22 12:56:27 ATLAS kernel: ---[ end trace 322b5af6ac00c643 ]--- Oct 22 12:56:27 ATLAS kernel: BTRFS: error (device sde1) in btrfs_run_delayed_refs:2927: errno=-5 IO failure Oct 22 12:56:27 ATLAS kernel: BTRFS info (device sde1): forced readonly Oct 22 12:56:27 ATLAS kernel: pending csums is 1363968 Oct 22 12:56:36 ATLAS shfs/user: err: shfs_open: open: /mnt/cache/vmware/isos/.lck-00217dfb00000000 (30) Read-only file system Oct 22 12:56:36 ATLAS shfs/user: err: shfs_open: open: /mnt/cache/vmware/Win 10/.lck-00144c0401000000 (30) Read-only file system Oct 22 12:56:36 ATLAS shfs/user: err: shfs_open: open: /mnt/cache/vmware/Win 10/.lck-008b7b0b01000000 (30) Read-only file system
The only way i solve this is by remounting.. but it eventually happens again. Any ideas anyone ?
-
What is the largest HDs the HP 40nl can handle running? 3tb, 4tb, etc
128,000TB.
I believe that's the current 48-bit LBA BIOS limit.
Have 4 of those
In all seriousness, there comes a limit to how much you want to spend/loose or waste time on. 6TB is tempting but the downtime on recovery is big.
-
Ok, very useful info there.. I'll do a bit of product research and come back with some results then
thanks once again. -
It does port multiplier, albeit choppy. The silicon image linux drivers & chipsets do port multipliers smoother.
In any case, I was testing it last night on my N54L for benchmarks.
I was getting 285MB/s on the internal N54L port with a Samsung Pro 256GB
When I moved it to this card I was getting 450MB/s
First of all, thanks for replying and the advice!
Looks like an excellent solution for a ssd cache drive - i've no experience with PMP, that may be something to investigate for external drive arrays.
i see there are two pcie slots on the N54L? It doesnt matter which one you connected it to?
-
has anyone installed a PCIe into with an SSD into these?
comes with low-profile bracket, could be nice to use one of these guys - any experience in these anyone?
-
Maybe add support to start/stop docker containers. I can provide a nice json api to it.
Yupi! I second that, could be great.
-
definitely.
general info, network speed, temp etc is a must. I'm up for testing, and helping if coding help is needed
Rsync slow between unRaid and second server
in General Support
Posted
No need to assume! you can check with the -vvv option in the ssh commands to determine which cipher is used.
Have you tried with FTP for comparison? FTP doesnt transfer all the good stuff (ownership, timestamps etc) but it's good for just comparing speeds to those of rsync. Furthermore, rsync is not a performace transfer tool rather a tool used to sync data and check if all is OK.