Feedback please! Using 10Gb adaptor in PCIe 1x slot


KptnKMan

Recommended Posts

On 9/25/2021 at 7:54 PM, Ford Prefect said:

I agree, that there is some Kind of confusion.
Maybe caused by the Statement that the L3-MTU, although sitting on a higher layer needs to be smaller as L2-MTU, which sits on a lower layer.

What you need to understand, at least that is the way I understood it, is that a communication on L3 will not travel on L3 physically, only logically.
Hence MTU on the respective layer must be the same for each communication device (in oder to avoid fragmentation).
Physically the communication travels down on the respective local device stack until it reaches the lowest layer and only then the packet gets actually transported to the next hop.

A data packet on L3 with 1500MTU can have 1480bytes of net data without risking fragmentation. L3 envelope is an additional 20bytes.
As depicted oin the diagram you linked earlier, from MT, L2 adds another envelope.

Yeah, I don't think your statements were confusing for me in that way exactly, because I think I get the L2-L3 relationship (I'm not new to networking, but with this particular use case I've been trying to be "correct".) but I guess there is no correct answer.

I was mainly looking for a recommendation of a way forward to make sense of.

You indicated what I shouldn't do, so I was really trying to understand what a recommendation of what I should do would be.

 

Anyhow, thinking of a 20 byte IP header + payload combination makes enough sense that I can go on with that.

I also found some more examples like this article that support that understanding is a good way to go forward.

 

On 9/25/2021 at 7:54 PM, Ford Prefect said:

This is what interfaces on my CRS326 look like:

<image>

notice, that L2 MTU is shown with higher numbers (where L3 MTU is 9000 and 1500 respectively).

As much as this makes sense, the MTUs in the image raise new questions for me, like how are the odd numbered MTUs related to 1500 and 9000 respectively. But as it is, I'm not going to worry too much about that right now.

 

As it is now, I've set the MTU to 9000, and L2-MTU to 9800 (Max L2 apparently being 10218), and the sky doesn't seem to have fallen yet. 😅

Everybody seems to be behaving, and the network appears to be performing as desired.

 

I've also re-enabled the active-backup/10Gb-1Gb bond0, and that seems to be working without issues. 👍

I'll be monitoring the bond setup and seeing what happens.

 

Next I guess will be to return the 1Gb transceivers, get 10Gb RJ45 transceivers, and get an additional CSS610 for across the house. 🤔

Link to comment
33 minutes ago, KptnKMan said:

As it is now, I've set the MTU to 9000, and L2-MTU to 9800 (Max L2 apparently being 10218), and the sky doesn't seem to have fallen yet. 😅

Everybody seems to be behaving, and the network appears to be performing as desired.

Yes, nice...my numbers are different, because - for L3-MTU 1500, the 1592 for L2-MTU seemed to be the defaults in ROS, at least all my MT devices have them like that on 1G interfaces and I didn't set them myself.

For L3-MTU 9000 between 10G links of unraid, my switches and Router, I did set L2-MTU to the max of my RB4011 (which is different/lower from the max values of CRS3xx and CSS610).

 

33 minutes ago, KptnKMan said:

I've also re-enabled the active-backup/10Gb-1Gb bond0, and that seems to be working without issues. 👍

I'll be monitoring the bond setup and seeing what happens.

 

Next I guess will be to return the 1Gb transceivers, get 10Gb RJ45 transceivers, and get an additional CSS610 for across the house. 🤔

...sounds like a plan. Good luck with your project...keep us posted as it progresses.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.
Note: Your post will require moderator approval before it will be visible.

Guest
Reply to this topic...

×   Pasted as rich text.   Restore formatting

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.