NSX Edge / Transport node TEP networking scenarios

    During time and different NSX-T versions, different options were available from the perspective of Edge/ESXi Transport node TEP (Tunnel EndPoint) networking aspect, which gives multiple options for someone to fulfil even the most demanding scenarios in this area. Some of them gives more flexibility or simplicity, but ultimate goal for functional SDN is always satisfied.

    One VMware article gives in summary overview what you can use, plan, and I found it very useful in several occasions as a reminder how something in TEP/VLAN area could be achieved - LINK

    In summary, mentioned KB gives following options from TEP networking perspective, comparing Edge nodes and ESXi transport nodes:

  • Edge TEP and ESXi host Transport Node TEP can be configured on the same VLAN in the following configurations:
- Edge VM TEP interface connected to a portgroup on an ESXi host not prepared for NSX

- Edge VM TEP interface connected to a portgroup on a switch not used by NSX, on an ESXi host prepared for NSX

- Edge VM TEP interface connected to a logical switch on a vDS7 with NSX-T 3.1.0 or above

- Edge VM TEP interface connected to a logical switch on a NVDS with NSX-T 3.1.0 or above
  • Edge TEP and ESXi host Transport Node TEP must be on separate VLANs in the following configurations:
- Edge VM TEP interface connected to a vDS portgroup where that vDS is used by NSX-T

- Edge VM TEP interface connected to a logical switch on vDS/NVDS prior to NSX-T 3.1.0

    Now, let's map this to different conclusions on how/when/what is available for implementation inside SDN environment. Couple of assumptions that I will make at this point:

  • NSX-T > 3.1 version being used - which is totally reasonable, having in mind that NSX-V is EoS/EoL + VMware NSX 4.x is already out in the moment of this writing,
  • vDS 7 instead of N-VDS - because of fact it's definitely direction where NSX is going.

I - Consolidated cluster - Management + Compute nodes inside DC on the same hardware - use case in most SMB or mid-Enterprise DC environments

  • Edge/ESXi TN same TEP VLAN option - probably most efficient / one security policy for TEP subnet:
- ESXi TNs (Compute) connected to vDS 7 inside vSphere - MTU changed on vDS to >=1700 (standard requirement)
- Edge nodes connected to NSX segments - there is need for 1 or more segments creation for Edge uplinks (trunks, with all or controlled VLAN propagation) + 1 or more segments creation for T0 uplink North-South connectivity to some ToR (BGP, OSPF, static...), or
- Edge node TEP interface connected to a portgroup on a vDS switch NOT used by NSX, on an ESXi host prepared for NSX.

  • Edge/ESXi TN separate TEP VLAN option - different security policy for TEP subnet / must be used in case:
- ESXi TNs connected to vDS 7 inside vSphere - MTU changed on vDS to >=1700
- Edge nodes connected vDS portgroup used by ESXi TNs - there is need for 1 or more vDS portgroups creation for Edge uplinks (trunks, with all or controlled VLAN propagation) + 1 or more NSX segments creation for T0 uplink North-South connectivity to some ToR (BGP, OSPF, static...)

II - Separated cluster - Management + Compute nodes inside DC on the different hardware - use case in Enterprise DC environments

  • Edge/ESXi TN same TEP VLAN option - one security policy for TEP subnet:
- ESXi TNs connected to vDS 7 inside vSphere - MTU changed on vDS to >=1700
- Edge nodes connected to NSX segments - there is need for 1 or more segments creation for Edge uplinks (trunks, with all or controlled VLAN propagation) + 1 or more segments creation for T0 uplink North-South connectivity to some ToR (BGP, OSPF, static...), or
- Edge node TEP interface connected to a portgroup on an ESXi host NOT prepared for NSX (ie Management cluster), or
- Edge node TEP interface connected to a portgroup on a vDS switch NOT used by NSX, on an ESXi host prepared for NSX.

  • Edge/ESXi TN separate TEP VLAN option - different security policy for TEP subnet / must be used in case:
- ESXi TNs connected to vDS 7 inside vSphere - MTU changed on vDS to >=1700
- Edge nodes connected to vDS portgroup AND that same vDS being used by ESXi TNs - there is need for 1 or more vDS portgroups creation for Edge uplinks (trunks, with all or controlled VLAN propagation) + 1 or more NSX segments creation for T0 uplink North-South connectivity to some ToR (BGP, OSPF, static...)


    Hope this will clarify different use cases and scecarios covering TEP networking from Transport node and Edge perspective 👍


Comments

Post a Comment

Popular posts from this blog

NSX ALB LetsEncrypt with DNS-01 challenge - BIND example

    In some of my previous posts, mentioned here LetsEncrypt script integration and Script parameter usage , I explained how useful it can be for NSX advanced load balancing solutions to utilise this kind of approach for free and automatic certificate manipulation, especially in environments with large number of web services inside. This approach utilises HTTP-01 based challenge with LetsEncrypt systems and L7 HTTP/S virtual services on NSX ALB side.     Now, from some time ago there is enhancement in this area, developed by official Avi Networks devops page, in terms of using DNS-01 challenge also. I tried on-prem option using Bind DNS as server which works very well.     Steps are pretty much similar to HTTP-01 option, which can be summarised as following:      - Create L7 virtual service with publicly available FQDN - certificates resolved can be in both options RSA or ECDSA, as configured during creation;        - Download required DNS-01 challenge script HERE      - Useful help
Read more

VMware SD WAN - multiple locations - LAN IP address space overlapping with NAT

Image
     Different scenarios are possible in terms of routing, NAT-ing and IP overlapping setups using VMware Velocloud SD WAN technology in Customer environments.      Recently I had an PoC with my Customer for the on-prem option with this VMware solution, where different use cases were interesting to show and demonstrate - one of them is something I would like to share and it relates to possibility of LAN-side NAT on Edge (branch) locations, with purpose to have IP overlapped on these setups. Next picture is showing typical Hub&Spoke setup where it can be possible to make this type of configuration: Picture 1. VMware SD WAN lab on-prem environment     Basically thing which needs to be accomplished is appropriate NAT solution for LANs on every branch Edge which are and needs to be the same (192.168.1.0/24 in this example) - as it is shown on Picture 1.      Honestly speaking, NAT is not one of so powerful things inside Velocloud SD WAN solution, if you compares it to traditional netwo
Read more

NSX ALB routing - multiple floating IP and BGP setup

Image
     Recently, I had very interesting scenario around NSX ALB (ex Avi Networks) setup with multiple networks, NAT's and no-NAT's, but more important routing requirement inside Customer environment. As you are aware of - NSX ALB Service engines have multiple NICs - to be more accurate there are 1 management + 9 data interfaces, which can be used with different configurations depending on actual needs and infrastructure. In my specific case, there were following assumptions which were successfully deployed across virtual service configuration: - external network (from NSX ALB perspective) - based on Cisco ACI SDN solution, where basically different L3-outs (specific ACI setup) for multiple NSX ALB needs were configured directly on Cisco platform. For this purpose, we will introduce VRF named XYZ, specifically created for connections mentioned above; - there is a need for multiple floating IP + BGP config in place on NSX ALB SE's, which can be found on this link  Default Gatew
Read more