NSX-T - North/South Edge uplink connection options and scenarios

    In this, a little bit longer post, I'm going to explain a couple typical use case scenarios regarding different options used inside NSX-T environment for connection options on Edge side, regarding TEP and North/South traffic options. Every environment is special use case, but hope you will find here summary options which you can use for successful deployment and design planning. First, couple of assumptions that I made here:

- vSphere environment is v7.x

- vDS distributed switch is in place - this dramatically can simplify NSX-T design and implementation because of NSX support inside vSphere 7. vSphere ESXi is pre-requisite for this and if you have such kind of infra, unless you have some special reason - N-vDS is not necessary at all

- Basically, we are speaking here about post NSX-T v2.5 era where, similar to bare metal Edge nodes, Edge VM form factor also supports same vDS/N-VDS for overlay and external traffic - definitely no more "three N-VDS per Edge VM design", unless you really, really need it

- NICs per physical server - minimum 2 - ideally they should be at least 10Gbps, but for demo and labbing there is no problem going even with 1Gbps only. Also, depending on desired design someone can utilize even higher number of NIC ports, which of course will give you plenty of options where to dedicate different types of traffic (ie management, vMotion, vSAN, iSCSI etc.)

-It's worth mentioning that regarding stateful and stateless services interaction with edge nodes - T0 A/A or A/S design - you can choose where to deploy them in multi-tier setup (T0 and T1s). For example - you can configure T0s in A/A (this article is assuming A/A in it's scenarios) setup and T1 in A/S because you need some of the statefull services (ie NAT, VPN etc.). Also, for highly scalable environments, different Edge clusters hosting only T0 or T1 routing instances are possible.

 

SCENARIO 1 - Single upstream router / redundant UPLINKS

    Next picture shows this type of scenario - in this case with dual uplinks, which can be Active/Active using ECMP or Active/Standby, using appropriate BGP config on T0 routing instance inside Edge cluster:

Relevant overview config, from NSX-T and Cisco IOS perspective, is also shown, with BFD in place as advanced mechanism for link failure detection.

 

SCENARIO 2 - Dual upstream router / single UPLINK per Edge

    This setup involves redundancy at the ToR level. T0 gives option in A/A setup for inter SR BGP routing support on it. Prefer whole routing through just one ToR instance, or using both as A/A for subset of networks - all scenarios are possible using appropriate T0 BGP config. Inter SR routing exchange helps in scenario where ToR routing boxes have same information regarding rest of network infrastructure, inside their routing tables.

Relevant config, with short scenario description is available.

 

SCENARIO 3 - Dual upstream router / dual UPLINK per Edge

    If possible - there is also an option to jump with total 4 uplinks toward physical ToRs. This gives all the nice redundancy options from Scenario 2 plus high throughput by using ECMP and all active paths toward ToR boxes.








Comments

Post a Comment

Popular posts from this blog

NSX ALB LetsEncrypt with DNS-01 challenge - BIND example

    In some of my previous posts, mentioned here LetsEncrypt script integration and Script parameter usage , I explained how useful it can be for NSX advanced load balancing solutions to utilise this kind of approach for free and automatic certificate manipulation, especially in environments with large number of web services inside. This approach utilises HTTP-01 based challenge with LetsEncrypt systems and L7 HTTP/S virtual services on NSX ALB side.     Now, from some time ago there is enhancement in this area, developed by official Avi Networks devops page, in terms of using DNS-01 challenge also. I tried on-prem option using Bind DNS as server which works very well.     Steps are pretty much similar to HTTP-01 option, which can be summarised as following:      - Create L7 virtual service with publicly available FQDN - certificates resolved can be in both options RSA or ECDSA, as configured during creation;        - Download required DNS-01 challenge script HERE      - Useful help
Read more

VMware SD WAN - multiple locations - LAN IP address space overlapping with NAT

Image
     Different scenarios are possible in terms of routing, NAT-ing and IP overlapping setups using VMware Velocloud SD WAN technology in Customer environments.      Recently I had an PoC with my Customer for the on-prem option with this VMware solution, where different use cases were interesting to show and demonstrate - one of them is something I would like to share and it relates to possibility of LAN-side NAT on Edge (branch) locations, with purpose to have IP overlapped on these setups. Next picture is showing typical Hub&Spoke setup where it can be possible to make this type of configuration: Picture 1. VMware SD WAN lab on-prem environment     Basically thing which needs to be accomplished is appropriate NAT solution for LANs on every branch Edge which are and needs to be the same (192.168.1.0/24 in this example) - as it is shown on Picture 1.      Honestly speaking, NAT is not one of so powerful things inside Velocloud SD WAN solution, if you compares it to traditional netwo
Read more

NSX ALB routing - multiple floating IP and BGP setup

Image
     Recently, I had very interesting scenario around NSX ALB (ex Avi Networks) setup with multiple networks, NAT's and no-NAT's, but more important routing requirement inside Customer environment. As you are aware of - NSX ALB Service engines have multiple NICs - to be more accurate there are 1 management + 9 data interfaces, which can be used with different configurations depending on actual needs and infrastructure. In my specific case, there were following assumptions which were successfully deployed across virtual service configuration: - external network (from NSX ALB perspective) - based on Cisco ACI SDN solution, where basically different L3-outs (specific ACI setup) for multiple NSX ALB needs were configured directly on Cisco platform. For this purpose, we will introduce VRF named XYZ, specifically created for connections mentioned above; - there is a need for multiple floating IP + BGP config in place on NSX ALB SE's, which can be found on this link  Default Gatew
Read more