Gfrktyl

I was doing an experiment about linux bridge and my network topology is like:

As you can see, there are two hosts located in a LAN, Host1(10.74.68.58) and Host2(10.74.68.47). On Host1, I created a bridge br0 and assigned an IP for it (192.168.3.101). Then I attached eth0 to the bridge:

[root@10.74.68.58:~] # bridge link

2: eth0 state UP : <BROADCAST,MULTICAST,PROMISC,UP,LOWER_UP> mtu 1500 master br0 state forwarding priority 32 cost 2

I set the default route as br0 and it is ok to ping 10.74.68.47:

[root@10.74.68.58:~] # ip r

default dev br0  scope link

172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.42.1

192.168.3.0/24 dev br0  proto kernel  scope link  src 192.168.3.101

But things became unexplainable when I set default route to eth0 :

[root@10.74.68.58:~] # ip r

default dev eth0  scope link

172.17.0.0/16 dev docker0  proto kernel  scope link  src 172.17.42.1

192.168.3.0/24 dev br0  proto kernel  scope link  src 192.168.3.101

When eth0 is the default route interface, I try to ping host2 in two different ways:

1, ping 10.74.68.47

Failed. After I checked tcpdump file (captured on interface br0), I found that br0 only received ARP response. So there is no ARP info on interface eth0, thus it can not get the mac of host2. I think this is the right behaviour, is my understanding right?

2, Then I tried ping -I br0 10.74.68.47

I wanted to use -I option to steer clear of default route, but also failed. After I check the tcpdump file (captured on interface br0), I found there is already a pair of icmp echo request and echo reply packet. This confused me a lot. Now that br0 have received echo reply, why can't I ping to host2 successfully?

[root@10.74.68.58:~] # ping -I br0 10.74.68.47

2 packets transmitted, 0 received, 100% packet loss, time 1006ms

Can you guys give me some pointers?

Bridging doesn't work the way you think it works. :-)

A bridge is only concerned with OSI level 2 (Ethernet frames). On this level, IP addresses etc. don't exist. Conceptually, you can think of a bridge as a collection of ethernet interfaces. Each interface is called a port, and a packet that goes into one port comes out on all other ports. (Actually, in the Linux implementation, there's an optimization that keeps a table of seen MAC addresses, but conceptually, it doesn't matter).

So a bridge can connect ("bridge") several ethernet segments into one big segment.

Then what does it mean to "give a Linux bridge an IP address"? In the Linux implementation, the bridge is not a separate hardware device (like they originally were), it's also accessible from the host itself. That means it acts like a kind of "super-ethernet interface" with many ports, but packets that go into the kernel or out from this kernel to or from any of these ports reach the Linux OS under a single IP address.

So as soon as you make an ethernet interface a slave (port) of a bridge, it ceases to have its own address. The only thing that counts is the IP address of the bridge.

In other words, making a bridge with only a single port makes no sense (you could have used the interface by itself). Trying to route packets to a port of a bridge makes no sense (as far as the kernel is concerned, the bridge is a single device).

If you want to play around with a bridge, you need a structure like this:

  10.0.2.1/23    10.0.2.2/23    10.0.3.254/23     10.0.3.1/23    10.0.3.2/23 



  ............   ............   ...............   ............   ............

  .  Host A  .   .  Host B  .   .  Host X     .   .  Host C  .   .  Host D  .

  .          .   .          .   . <-- br0 --> .   .          .   .          .

  .   eth0   .   .   eth0   .   . eth0   eth1 .   .   eth0   .   .   eth0   .

  .....|......   .....|......   ...|......|....   .....|......   .....|......

       |              |            |      |            |              |      

  -----+--------------+------------+      +------------+--------------+------



  <-------- left Segment  --------->      <------- right Segment ----------->

Here the left segment with hosts A and B is bridged by host X to the right segment with hosts C and D, and each host is accessible by a single IP address (which is assigned to interfaces or the bridge as a whole).

When you ping -I br0 10.74.68.47, Protocol stack find that the ip(10.74.68.47) isn't in the same LAN with br0, so it sends ARP through eth0. But eth0 attached br0, it can't connect protocol stack, eth0 will send ARP response to br0, then br0 find mac does not match, so it drop it. so protocol stack does not get ARP response.

You can change the ip of br0 to 10.74.68.x/24. it will be ok.