Provider networks

A provider (external) network bridges instances to physical network infrastructure that provides layer-3 services. In most cases, provider networks implement layer-2 segmentation using VLAN IDs. A provider network maps to a provider bridge on each compute node that supports launching instances on the provider network. You can create more than one provider bridge, each one requiring a unique name and underlying physical network interface to prevent switching loops. Provider networks and bridges can use arbitrary names, but each mapping must reference valid provider network and bridge names. Each provider bridge can contain one flat (untagged) network and up to the maximum number of vlan (tagged) networks that the physical network infrastructure supports, typically around 4000.

Creating a provider network involves several commands at the host, OVS, and Networking service levels that yield a series of operations at the OVN level to create the virtual network components. The following example creates a flat provider network provider using the provider bridge br-provider and binds a subnet to it.

Create a provider network

  1. On each compute node, create the provider bridge, map the provider network to it, and add the underlying physical or logical (typically a bond) network interface to it.

    # ovs-vsctl --may-exist add-br br-provider -- set bridge br-provider \
      protocols=OpenFlow13
    # ovs-vsctl set open . external-ids:ovn-bridge-mappings=provider:br-provider
    # ovs-vsctl --may-exist add-port br-provider INTERFACE_NAME
    

    Replace INTERFACE_NAME with the name of the underlying network interface.

    Note

    These commands provide no output if successful.

  2. On the controller node, source the administrative project credentials.

  3. On the controller node, to enable this chassis to host gateway routers for external connectivity, set ovn-cms-options to enable-chassis-as-gw.

    # ovs-vsctl set open . external-ids:ovn-cms-options="enable-chassis-as-gw"
    

    Note

    This command provide no output if successful.

  4. On the controller node, create the provider network in the Networking service. In this case, instances and routers in other projects can use the network.

    $ openstack network create --external --share \
      --provider-physical-network provider --provider-network-type flat \
      provider
    +---------------------------+--------------------------------------+
    | Field                     | Value                                |
    +---------------------------+--------------------------------------+
    | admin_state_up            | UP                                   |
    | availability_zone_hints   |                                      |
    | availability_zones        | nova                                 |
    | created_at                | 2016-06-15 15:50:37+00:00            |
    | description               |                                      |
    | id                        | 0243277b-4aa8-46d8-9e10-5c9ad5e01521 |
    | ipv4_address_scope        | None                                 |
    | ipv6_address_scope        | None                                 |
    | is_default                | False                                |
    | mtu                       | 1500                                 |
    | name                      | provider                             |
    | project_id                | b1ebf33664df402693f729090cfab861     |
    | provider:network_type     | flat                                 |
    | provider:physical_network | provider                             |
    | provider:segmentation_id  | None                                 |
    | qos_policy_id             | None                                 |
    | router:external           | External                             |
    | shared                    | True                                 |
    | status                    | ACTIVE                               |
    | subnets                   | 32a61337-c5a3-448a-a1e7-c11d6f062c21 |
    | tags                      | []                                   |
    | updated_at                | 2016-06-15 15:50:37+00:00            |
    +---------------------------+--------------------------------------+
    

    Note

    The value of --provider-physical-network must refer to the provider network name in the mapping.

OVN operations

The OVN mechanism driver and OVN perform the following operations during creation of a provider network.

  1. The mechanism driver translates the network into a logical switch in the OVN northbound database.

     _uuid               : 98edf19f-2dbc-4182-af9b-79cafa4794b6
     acls                : []
     external_ids        : {"neutron:network_name"=provider}
     load_balancer       : []
     name                : "neutron-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
     ports               : [92ee7c2f-cd22-4cac-a9d9-68a374dc7b17]
    
    .. note::
    
       The ``neutron:network_name`` field in ``external_ids`` contains
       the network name and ``name`` contains the network UUID.
    
  2. In addition, because the provider network is handled by a separate bridge, the following logical port is created in the OVN northbound database.

    _uuid               : 92ee7c2f-cd22-4cac-a9d9-68a374dc7b17
    addresses           : [unknown]
    enabled             : []
    external_ids        : {}
    name                : "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
    options             : {network_name=provider}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : localnet
    up                  : false
    
  3. The OVN northbound service translates these objects into datapath bindings, port bindings, and the appropriate multicast groups in the OVN southbound database.

    • Datapath bindings

      _uuid               : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      external_ids        : {logical-switch="98edf19f-2dbc-4182-af9b-79cafa4794b6"}
      tunnel_key          : 109
      
    • Port bindings

      _uuid               : 8427506e-46b5-41e5-a71b-a94a6859e773
      chassis             : []
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      logical_port        : "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"
      mac                 : [unknown]
      options             : {network_name=provider}
      parent_port         : []
      tag                 : []
      tunnel_key          : 1
      type                : localnet
      
    • Logical flows

      Datapath: f1f0981f-a206-4fac-b3a1-dc2030c9909f  Pipeline: ingress
        table= 0(  ls_in_port_sec_l2), priority=  100, match=(eth.src[40]),
          action=(drop;)
        table= 0(  ls_in_port_sec_l2), priority=  100, match=(vlan.present),
          action=(drop;)
        table= 0(  ls_in_port_sec_l2), priority=   50,
          match=(inport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(next;)
        table= 1(  ls_in_port_sec_ip), priority=    0, match=(1),
          action=(next;)
        table= 2(  ls_in_port_sec_nd), priority=    0, match=(1),
          action=(next;)
        table= 3(      ls_in_pre_acl), priority=    0, match=(1),
          action=(next;)
        table= 4(       ls_in_pre_lb), priority=    0, match=(1),
          action=(next;)
        table= 5( ls_in_pre_stateful), priority=  100, match=(reg0[0] == 1),
          action=(ct_next;)
        table= 5( ls_in_pre_stateful), priority=    0, match=(1),
          action=(next;)
        table= 6(          ls_in_acl), priority=    0, match=(1),
          action=(next;)
        table= 7(           ls_in_lb), priority=    0, match=(1),
          action=(next;)
        table= 8(     ls_in_stateful), priority=  100, match=(reg0[1] == 1),
          action=(ct_commit; next;)
        table= 8(     ls_in_stateful), priority=  100, match=(reg0[2] == 1),
          action=(ct_lb;)
        table= 8(     ls_in_stateful), priority=    0, match=(1),
          action=(next;)
        table= 9(      ls_in_arp_rsp), priority=  100,
          match=(inport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(next;)
        table= 9(      ls_in_arp_rsp), priority=    0, match=(1),
          action=(next;)
        table=10(      ls_in_l2_lkup), priority=  100, match=(eth.mcast),
          action=(outport = "_MC_flood"; output;)
        table=10(      ls_in_l2_lkup), priority=    0, match=(1),
          action=(outport = "_MC_unknown"; output;)
      Datapath: f1f0981f-a206-4fac-b3a1-dc2030c9909f  Pipeline: egress
        table= 0(      ls_out_pre_lb), priority=    0, match=(1),
          action=(next;)
        table= 1(     ls_out_pre_acl), priority=    0, match=(1),
          action=(next;)
        table= 2(ls_out_pre_stateful), priority=  100, match=(reg0[0] == 1),
          action=(ct_next;)
        table= 2(ls_out_pre_stateful), priority=    0, match=(1),
          action=(next;)
        table= 3(          ls_out_lb), priority=    0, match=(1),
          action=(next;)
        table= 4(         ls_out_acl), priority=    0, match=(1),
          action=(next;)
        table= 5(    ls_out_stateful), priority=  100, match=(reg0[1] == 1),
          action=(ct_commit; next;)
        table= 5(    ls_out_stateful), priority=  100, match=(reg0[2] == 1),
          action=(ct_lb;)
        table= 5(    ls_out_stateful), priority=    0, match=(1),
          action=(next;)
        table= 6( ls_out_port_sec_ip), priority=    0, match=(1),
          action=(next;)
        table= 7( ls_out_port_sec_l2), priority=  100, match=(eth.mcast),
          action=(output;)
        table= 7( ls_out_port_sec_l2), priority=   50,
          match=(outport == "provnet-e4abf6df-f8cf-49fd-85d4-3ea399f4d645"),
          action=(output;)
      
    • Multicast groups

      _uuid               : 0102f08d-c658-4d0a-a18a-ec8adcaddf4f
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      name                : _MC_unknown
      ports               : [8427506e-46b5-41e5-a71b-a94a6859e773]
      tunnel_key          : 65534
      
      _uuid               : fbc38e51-ac71-4c57-a405-e6066e4c101e
      datapath            : f1f0981f-a206-4fac-b3a1-dc2030c9909f
      name                : _MC_flood
      ports               : [8427506e-46b5-41e5-a71b-a94a6859e773]
      tunnel_key          : 65535
      

Create a subnet on the provider network

The provider network requires at least one subnet that contains the IP address allocation available for instances, default gateway IP address, and metadata such as name resolution.

  1. On the controller node, create a subnet bound to the provider network provider.

    $ openstack subnet create --network provider --subnet-range \
      203.0.113.0/24 --allocation-pool start=203.0.113.101,end=203.0.113.250 \
      --dns-nameserver 8.8.8.8,8.8.4.4 --gateway 203.0.113.1 provider-v4
      +-------------------+--------------------------------------+
      | Field             | Value                                |
      +-------------------+--------------------------------------+
      | allocation_pools  | 203.0.113.101-203.0.113.250          |
      | cidr              | 203.0.113.0/24                       |
      | created_at        | 2016-06-15 15:50:45+00:00            |
      | description       |                                      |
      | dns_nameservers   | 8.8.8.8, 8.8.4.4                     |
      | enable_dhcp       | True                                 |
      | gateway_ip        | 203.0.113.1                          |
      | host_routes       |                                      |
      | id                | 32a61337-c5a3-448a-a1e7-c11d6f062c21 |
      | ip_version        | 4                                    |
      | ipv6_address_mode | None                                 |
      | ipv6_ra_mode      | None                                 |
      | name              | provider-v4                          |
      | network_id        | 0243277b-4aa8-46d8-9e10-5c9ad5e01521 |
      | project_id        | b1ebf33664df402693f729090cfab861     |
      | subnetpool_id     | None                                 |
      | updated_at        | 2016-06-15 15:50:45+00:00            |
      +-------------------+--------------------------------------+
    

If using DHCP to manage instance IP addresses, adding a subnet causes a series of operations in the Networking service and OVN.

  • The Networking service schedules the network on appropriate number of DHCP agents. The example environment contains three DHCP agents.

  • Each DHCP agent spawns a network namespace with a dnsmasq process using an IP address from the subnet allocation.

  • The OVN mechanism driver creates a logical switch port object in the OVN northbound database for each dnsmasq process.

OVN operations

The OVN mechanism driver and OVN perform the following operations during creation of a subnet on the provider network.

  1. If the subnet uses DHCP for IP address management, create logical ports ports for each DHCP agent serving the subnet and bind them to the logical switch. In this example, the subnet contains two DHCP agents.

    _uuid               : 5e144ab9-3e08-4910-b936-869bbbf254c8
    addresses           : ["fa:16:3e:57:f9:ca 203.0.113.101"]
    enabled             : true
    external_ids        : {"neutron:port_name"=""}
    name                : "6ab052c2-7b75-4463-b34f-fd3426f61787"
    options             : {}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : ""
    up                  : true
    
    _uuid               : 38cf8b52-47c4-4e93-be8d-06bf71f6a7c9
    addresses           : ["fa:16:3e:e0:eb:6d 203.0.113.102"]
    enabled             : true
    external_ids        : {"neutron:port_name"=""}
    name                : "94aee636-2394-48bc-b407-8224ab6bb1ab"
    options             : {}
    parent_name         : []
    port_security       : []
    tag                 : []
    type                : ""
    up                  : true
    
    _uuid               : 924500c4-8580-4d5f-a7ad-8769f6e58ff5
    acls                : []
    external_ids        : {"neutron:network_name"=provider}
    load_balancer       : []
    name                : "neutron-670efade-7cd0-4d87-8a04-27f366eb8941"
    ports               : [38cf8b52-47c4-4e93-be8d-06bf71f6a7c9,
                           5e144ab9-3e08-4910-b936-869bbbf254c8,
                           a576b812-9c3e-4cfb-9752-5d8500b3adf9]
    
  2. The OVN northbound service creates port bindings for these logical ports and adds them to the appropriate multicast group.

    • Port bindings

      _uuid               : 030024f4-61c3-4807-859b-07727447c427
      chassis             : fc5ab9e7-bc28-40e8-ad52-2949358cc088
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      logical_port        : "6ab052c2-7b75-4463-b34f-fd3426f61787"
      mac                 : ["fa:16:3e:57:f9:ca 203.0.113.101"]
      options             : {}
      parent_port         : []
      tag                 : []
      tunnel_key          : 2
      type                : ""
      
      _uuid               : cc5bcd19-bcae-4e29-8cee-3ec8a8a75d46
      chassis             : 6a9d0619-8818-41e6-abef-2f3d9a597c03
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      logical_port        : "94aee636-2394-48bc-b407-8224ab6bb1ab"
      mac                 : ["fa:16:3e:e0:eb:6d 203.0.113.102"]
      options             : {}
      parent_port         : []
      tag                 : []
      tunnel_key          : 3
      type                : ""
      
    • Multicast groups

      _uuid               : 39b32ccd-fa49-4046-9527-13318842461e
      datapath            : bd0ab2b3-4cf4-4289-9529-ef430f6a89e6
      name                : _MC_flood
      ports               : [030024f4-61c3-4807-859b-07727447c427,
                             904c3108-234d-41c0-b93c-116b7e352a75,
                             cc5bcd19-bcae-4e29-8cee-3ec8a8a75d46]
      tunnel_key          : 65535
      
  3. The OVN northbound service translates the logical ports into additional logical flows in the OVN southbound database.

    Datapath: bd0ab2b3-4cf4-4289-9529-ef430f6a89e6  Pipeline: ingress
      table= 0(  ls_in_port_sec_l2), priority=   50,
        match=(inport == "94aee636-2394-48bc-b407-8224ab6bb1ab"),
        action=(next;)
      table= 0(  ls_in_port_sec_l2), priority=   50,
        match=(inport == "6ab052c2-7b75-4463-b34f-fd3426f61787"),
        action=(next;)
      table= 9(      ls_in_arp_rsp), priority=   50,
        match=(arp.tpa == 203.0.113.101 && arp.op == 1),
        action=(eth.dst = eth.src; eth.src = fa:16:3e:57:f9:ca;
                arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
                arp.sha = fa:16:3e:57:f9:ca; arp.tpa = arp.spa;
                arp.spa = 203.0.113.101; outport = inport; inport = "";
                /* Allow sending out inport. */ output;)
      table= 9(      ls_in_arp_rsp), priority=   50,
        match=(arp.tpa == 203.0.113.102 && arp.op == 1),
        action=(eth.dst = eth.src; eth.src = fa:16:3e:e0:eb:6d;
                arp.op = 2; /* ARP reply */ arp.tha = arp.sha;
                arp.sha = fa:16:3e:e0:eb:6d; arp.tpa = arp.spa;
                arp.spa = 203.0.113.102; outport = inport;
                inport = ""; /* Allow sending out inport. */ output;)
      table=10(      ls_in_l2_lkup), priority=   50,
        match=(eth.dst == fa:16:3e:57:f9:ca),
        action=(outport = "6ab052c2-7b75-4463-b34f-fd3426f61787"; output;)
      table=10(      ls_in_l2_lkup), priority=   50,
        match=(eth.dst == fa:16:3e:e0:eb:6d),
        action=(outport = "94aee636-2394-48bc-b407-8224ab6bb1ab"; output;)
    Datapath: bd0ab2b3-4cf4-4289-9529-ef430f6a89e6  Pipeline: egress
      table= 7( ls_out_port_sec_l2), priority=   50,
        match=(outport == "6ab052c2-7b75-4463-b34f-fd3426f61787"),
        action=(output;)
      table= 7( ls_out_port_sec_l2), priority=   50,
        match=(outport == "94aee636-2394-48bc-b407-8224ab6bb1ab"),
        action=(output;)
    
  4. For each compute node without a DHCP agent on the subnet:

    • The OVN controller service translates the logical flows into flows on the integration bridge br-int.

      cookie=0x0, duration=22.303s, table=32, n_packets=0, n_bytes=0,
          idle_age=22, priority=100,reg7=0xffff,metadata=0x4
          actions=load:0x4->NXM_NX_TUN_ID[0..23],
              set_field:0xffff/0xffffffff->tun_metadata0,
              move:NXM_NX_REG6[0..14]->NXM_NX_TUN_METADATA0[16..30],
              output:5,output:4,resubmit(,33)
      
  5. For each compute node with a DHCP agent on a subnet:

    • Creation of a DHCP network namespace adds two virtual switch ports. The first port connects the DHCP agent with dnsmasq process to the integration bridge and the second port patches the integration bridge to the provider bridge br-provider.

      # ovs-ofctl show br-int
      OFPT_FEATURES_REPLY (xid=0x2): dpid:000022024a1dc045
      n_tables:254, n_buffers:256
      capabilities: FLOW_STATS TABLE_STATS PORT_STATS QUEUE_STATS ARP_MATCH_IP
      actions: output enqueue set_vlan_vid set_vlan_pcp strip_vlan mod_dl_src mod_dl_dst mod_nw_src mod_nw_dst mod_nw_tos mod_tp_src mod_tp_dst
       7(tap6ab052c2-7b): addr:00:00:00:00:10:7f
           config:     PORT_DOWN
           state:      LINK_DOWN
           speed: 0 Mbps now, 0 Mbps max
       8(patch-br-int-to): addr:6a:8c:30:3f:d7:dd
          config:     0
          state:      0
          speed: 0 Mbps now, 0 Mbps max
      
      # ovs-ofctl -O OpenFlow13 show br-provider
      OFPT_FEATURES_REPLY (OF1.3) (xid=0x2): dpid:0000080027137c4a
      n_tables:254, n_buffers:256
      capabilities: FLOW_STATS TABLE_STATS PORT_STATS GROUP_STATS QUEUE_STATS
      OFPST_PORT_DESC reply (OF1.3) (xid=0x3):
       1(patch-provnet-0): addr:fa:42:c5:3f:d7:6f
           config:     0
           state:      0
           speed: 0 Mbps now, 0 Mbps max
      
    • The OVN controller service translates these logical flows into flows on the integration bridge.

      cookie=0x0, duration=17.731s, table=0, n_packets=3, n_bytes=258,
          idle_age=16, priority=100,in_port=7
          actions=load:0x2->NXM_NX_REG5[],load:0x4->OXM_OF_METADATA[],
              load:0x2->NXM_NX_REG6[],resubmit(,16)
      cookie=0x0, duration=17.730s, table=0, n_packets=15, n_bytes=954,
          idle_age=2, priority=100,in_port=8,vlan_tci=0x0000/0x1000
          actions=load:0x1->NXM_NX_REG5[],load:0x4->OXM_OF_METADATA[],
              load:0x1->NXM_NX_REG6[],resubmit(,16)
      cookie=0x0, duration=17.730s, table=0, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,in_port=8,dl_vlan=0
          actions=strip_vlan,load:0x1->NXM_NX_REG5[],
              load:0x4->OXM_OF_METADATA[],load:0x1->NXM_NX_REG6[],
              resubmit(,16)
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,metadata=0x4,
              dl_src=01:00:00:00:00:00/01:00:00:00:00:00
          actions=drop
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,metadata=0x4,vlan_tci=0x1000/0x1000
          actions=drop
      cookie=0x0, duration=17.732s, table=16, n_packets=3, n_bytes=258,
          idle_age=16, priority=50,reg6=0x2,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=17.732s, table=16, n_packets=0, n_bytes=0,
          idle_age=17, priority=50,reg6=0x3,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=17.732s, table=16, n_packets=15, n_bytes=954,
          idle_age=2, priority=50,reg6=0x1,metadata=0x4 actions=resubmit(,17)
      cookie=0x0, duration=21.714s, table=17, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,18)
      cookie=0x0, duration=21.714s, table=18, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,19)
      cookie=0x0, duration=21.714s, table=19, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,20)
      cookie=0x0, duration=21.714s, table=20, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,21)
      cookie=0x0, duration=21.714s, table=21, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=22,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=21, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=22,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=21, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,22)
      cookie=0x0, duration=21.714s, table=22, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,23)
      cookie=0x0, duration=21.714s, table=23, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,24)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=25,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=25,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,25)
      cookie=0x0, duration=21.714s, table=24, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,25)
      cookie=0x0, duration=21.714s, table=24, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,25)
      cookie=0x0, duration=21.714s, table=25, n_packets=15, n_bytes=954,
          idle_age=6, priority=100,reg6=0x1,metadata=0x4 actions=resubmit(,26)
      cookie=0x0, duration=21.714s, table=25, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,arp,metadata=0x4,
              arp_tpa=203.0.113.101,arp_op=1
          actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],
              mod_dl_src:fa:16:3e:f9:5d:f3,load:0x2->NXM_OF_ARP_OP[],
              move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],
              load:0xfa163ef95df3->NXM_NX_ARP_SHA[],
              move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],
              load:0xc0a81264->NXM_OF_ARP_SPA[],
              move:NXM_NX_REG6[]->NXM_NX_REG7[],
              load:0->NXM_NX_REG6[],load:0->NXM_OF_IN_PORT[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=25, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,arp,metadata=0x4,
              arp_tpa=203.0.113.102,arp_op=1
          actions=move:NXM_OF_ETH_SRC[]->NXM_OF_ETH_DST[],
              mod_dl_src:fa:16:3e:f0:a5:9f,
              load:0x2->NXM_OF_ARP_OP[],
              move:NXM_NX_ARP_SHA[]->NXM_NX_ARP_THA[],
              load:0xfa163ef0a59f->NXM_NX_ARP_SHA[],
              move:NXM_OF_ARP_SPA[]->NXM_OF_ARP_TPA[],
              load:0xc0a81265->NXM_OF_ARP_SPA[],
              move:NXM_NX_REG6[]->NXM_NX_REG7[],
              load:0->NXM_NX_REG6[],load:0->NXM_OF_IN_PORT[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=25, n_packets=3, n_bytes=258,
          idle_age=20, priority=0,metadata=0x4 actions=resubmit(,26)
      cookie=0x0, duration=21.714s, table=26, n_packets=18, n_bytes=1212,
          idle_age=6, priority=100,metadata=0x4,
              dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
          actions=load:0xffff->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,metadata=0x4,dl_dst=fa:16:3e:f0:a5:9f
          actions=load:0x3->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,metadata=0x4,dl_dst=fa:16:3e:f9:5d:f3
          actions=load:0x2->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=21.714s, table=26, n_packets=0, n_bytes=0,
          idle_age=21, priority=0,metadata=0x4
          actions=load:0xfffe->NXM_NX_REG7[],resubmit(,32)
      cookie=0x0, duration=17.731s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x2,metadata=0x4
          actions=load:0x2->NXM_NX_REG5[],resubmit(,34)
      cookie=0x0, duration=118.126s, table=33, n_packets=0, n_bytes=0,
          idle_age=118, hard_age=17, priority=100,reg7=0xfffe,metadata=0x4
          actions=load:0x1->NXM_NX_REG5[],load:0x1->NXM_NX_REG7[],
              resubmit(,34),load:0xfffe->NXM_NX_REG7[]
      cookie=0x0, duration=118.126s, table=33, n_packets=18, n_bytes=1212,
          idle_age=2, hard_age=17, priority=100,reg7=0xffff,metadata=0x4
          actions=load:0x2->NXM_NX_REG5[],load:0x2->NXM_NX_REG7[],
              resubmit(,34),load:0x1->NXM_NX_REG5[],load:0x1->NXM_NX_REG7[],
              resubmit(,34),load:0xffff->NXM_NX_REG7[]
      cookie=0x0, duration=17.730s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x1,metadata=0x4
          actions=load:0x1->NXM_NX_REG5[],resubmit(,34)
      cookie=0x0, duration=17.697s, table=33, n_packets=0, n_bytes=0,
          idle_age=17, priority=100,reg7=0x3,metadata=0x4
          actions=load:0x1->NXM_NX_REG7[],resubmit(,33)
      cookie=0x0, duration=17.731s, table=34, n_packets=3, n_bytes=258,
          idle_age=16, priority=100,reg6=0x2,reg7=0x2,metadata=0x4
          actions=drop
      cookie=0x0, duration=17.730s, table=34, n_packets=15, n_bytes=954,
          idle_age=2, priority=100,reg6=0x1,reg7=0x1,metadata=0x4
          actions=drop
      cookie=0x0, duration=21.714s, table=48, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,49)
      cookie=0x0, duration=21.714s, table=49, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,50)
      cookie=0x0, duration=21.714s, table=50, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=51,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=50, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x1/0x1,metadata=0x4
          actions=ct(table=51,zone=NXM_NX_REG5[0..15])
      cookie=0x0, duration=21.714s, table=50, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,51)
      cookie=0x0, duration=21.714s, table=51, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,52)
      cookie=0x0, duration=21.714s, table=52, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,53)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=54,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x4/0x4,metadata=0x4
          actions=ct(table=54,zone=NXM_NX_REG5[0..15],nat)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ipv6,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,54)
      cookie=0x0, duration=21.714s, table=53, n_packets=0, n_bytes=0,
          idle_age=21, priority=100,ip,reg0=0x2/0x2,metadata=0x4
          actions=ct(commit,zone=NXM_NX_REG5[0..15]),resubmit(,54)
      cookie=0x0, duration=21.714s, table=53, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,54)
      cookie=0x0, duration=21.714s, table=54, n_packets=18, n_bytes=1212,
          idle_age=6, priority=0,metadata=0x4 actions=resubmit(,55)
      cookie=0x0, duration=21.714s, table=55, n_packets=18, n_bytes=1212,
          idle_age=6, priority=100,metadata=0x4,
              dl_dst=01:00:00:00:00:00/01:00:00:00:00:00
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x3,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x2,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.714s, table=55, n_packets=0, n_bytes=0,
          idle_age=21, priority=50,reg7=0x1,metadata=0x4
          actions=resubmit(,64)
      cookie=0x0, duration=21.712s, table=64, n_packets=15, n_bytes=954,
          idle_age=6, priority=100,reg7=0x3,metadata=0x4 actions=output:7
      cookie=0x0, duration=21.711s, table=64, n_packets=3, n_bytes=258,
          idle_age=20, priority=100,reg7=0x1,metadata=0x4 actions=output:8