A solution to a problem I had

It's been a while since I wrote something, primarily because of some eye surgery I had. I'm still recovering, but I'm back to writing here. It's time.

Let me get started on AWS Cloud WANs new feature, Service Insertion. It allows you to define either an east-west or north-south inspection rule for your traffic. Under the hood, it will handle the routing to and from your inspection segment. I will talk about one particular use case, which we use for our egress traffic. Or, to quote AWS themselves

Service insertion allows you to steer same-segment or cross-segment traffic using network functions deployed in VPCs or on-premises networks attached to Cloud WAN. Network functions can be third-party network or security appliances such as NGFW, IDS, IPS appliances or native AWS network firewall or Gateway Load Balancer services

AWS Doco

My problem

We run a CloudWAN network that spans multiple regions. We have numerous CNEs deployed. However, we want to have a limited number of egress points to keep costs under control. We also have partners who like to whitelist our egress IPs. Naturally, we also have latency requirements.

Now, the Core Network Edge (CNE) in each location maintains a table of known routes, which is created from attachments and static routes. These routes are fed into segments based on the core network policy, which outlines the rules for sharing them. So, you might say, this static route goes in this segment, and this segment shares its routes with another.

When making routing decisions, the system first checks if the destination is local to that region. If it is not local, one of the shortest paths is selected. The system does not take physical distance or latency into consideration (it is BGP after all) and considers that CloudWAN CNEs are fully meshed. As a result, in a network where each CNE is just one hop away from the others, it is essentially random, which egress a non-local CNE will be chosen. Fun, right?

So tell me more about Service Insertion

OK, service insertion has the following elements.

• Network Function Group (NFG) - A collection of attachments. For example, all your egress VPC attachments. Or maybe attachments to your inspection VPCs
• Attachment Policy - You define which tags will be used to match attachments to the NFG
• Segment action - Here, you define the 'send-to' or 'send-via' action per segment. And you can define something called edge-override. This is what I was after.

I think you can already see where this is going, so for now, I'll spell it out in beautiful Cloudformation JSON.

{
  "network-function-groups": [
      {
        "name": "egress",
        "description": "Egress NFG",
        "require-attachment-acceptance": false
      }
    ],
}

First you're gonna want to define a NFG, it's up to you if you want to manually accept attachments or not.

{
  "rule-number": 92,
  "description": "attach egress vpc to nfg",
  "condition-logic": "and",
  "conditions": [
    {
      "type": "tag-value",
      "operator": "equals",
      "key": "segment",
      "value": "egress"
    },
    {
      "type": "tag-value",
      "operator": "equals",
      "key": "env",
      "value": "prd"
    }
  ],
  "action": {
    "add-to-network-function-group": "egress"
  }
}

Now we define the attachment policy. Here we're saying, if the attachment has a tag of 'segment' with a value of 'egress' and 'env' with a value of 'prd', then add it to the NFG 'egress'. Simple right?

{
  "action": "send-to",
  "segment": "production",
  "via": {
    "network-function-groups": [
      "egress"
    ],
    "with-edge-overrides": [
      {
        "edge-sets": [
          [
            "ap-southeast-1"
          ]
        ],
        "use-edge-location": "ap-southeast-2"
      },
      {
        "edge-sets": [
          [
            "us-west-1"
          ]
        ],
        "use-edge-location": "us-east-1"
      }
    ]
  }
},

Finally, we define the segment action. Here we're saying, if the traffic is in the 'production' segment, send it to the NFG 'egress'. But, if the traffic is coming from 'ap-southeast-1', then send it to 'ap-southeast-2'. And if it's coming from 'us-west-1', then send it to 'us-east-1'. This is the edge-override.

From here you can see you could, for example, have different egress setups per segment. Maybe do something different with dev traffic. Lots of possibilities.

What's next?

It's been a while so I need to focus here again. From this I will follow up with

• AWS Hyrbid networking
• SDN options in AWS
• AWS Tunnel-less Connect (I'm excited about this one, finally no more GRE tunnels)