Published on

AWS Global Networking: Connectivity Spaghetti

  • avatar
    Kris Gillespie

Let's make a connection


Realistically, if you're looking into Cloud WAN, it's probably because you figured out

  • Multi regional TGWs are a pain
  • You have some more scarce resources you want to make available. Probably in a data center or two.

I'm guessing the later. Consider the following


In my chats, this is pretty common. One primary region and a secondary. There are multiple reasons why the secondary, these days it's less and less common as DR (Disaster Recovery) but it's still possible. Still, the idea I want to get across here is, you're happily doing your thing in AWS, got multiple workload VPCs humming along and in the back, you have some connectivity back to your office/Data Center.

So far, so normal. But let's say you want to go to more regions. Add one more. Two more. Now you need to start considering something like this


Now, even this is doable but you have a few problems. One, routing between TGWs is static. So, change anything behind the TGW and you need to add/remove routes. Think lambdas modifying route tables. And even, imagine trying to diagram this kind of setup. Generally speaking, the harder it is to draw, the harder it is to reason and the more likely the complexity will eventually come for you.

Add more regions, add more TGWs, add more spaghetti. Then you start thinking about how can you transit/trunk this yourself. Down this path lays madness. (Or you start hunting some other solutions, like Aviatrix and their ilk)

Let's consider the DC/Office connectivity as well. You now have a mix of static and dynamic routing in your estate. The TGWs is going to propogate the prefixes it learns from the Direct Connects. Depending on your setup, the DC/Office will get all/subset of prefixes back. Specifically what is known on the regional TGW.

So, manual work or lambdas editing route tables. Hmmmmm.

So, CloudWAN?


Everything is dynamic. As you add and remove VPCs, even down the DC, the various route tables are updated. Of course you still (for now) need some TGWs to hook into the Direct Connect Gateways, but otherwise you are leaning on CloudWAN. Add another region? Easy, one more CNE, define which segments you want and you're ready. Attach the VPCs and you're 95% done.

What's next?

We are making solid progress. We've got the following still upcoming:

  • Connect Attachments - Fun with GRE
  • Inspection

Plus another series on Cloud Networking in general.

  • The state of IPv6
  • What is the cost of abstraction in relation to networking
  • How to limit the blast radius
  • Multi Cloud