TL;DR 整理給自己進考場前看過一遍的筆記。啊有順利考過了，按下 Submit 鍵就馬上知道結果真的很刺激。
- Design and implement hybrid IT network architectures at scale
- Design and implement AWS networks
- Automate AWS tasks
- Configure network integration with application services
- Design and implement for security and compliance
- Manage, optimize, and troubleshoot the network
General Best Practices
- Highly available, fault-tolerant network connections are key to a well-architected system.
Articles and White Papers
- Multiple Data Center HA Network Connectivity
- Leverage multiple dynamically routed, rather than statically routed, connections to AWS. This will allow remote connections to fail over automatically between redundant connections. Dynamic routing also enables remote connections to automatically leverage available preferred routes, if applicable, to the on-premises network.
- When selecting AWS Direct Connect network service providers, consider a dual-vendor approach, if financially feasible, to ensure private network diversity.
- Leverage more specific BGP route advertisements or BGP AS-path prepending to ensure AWS uses the most efficient routes to send traffic to your remote data centers.
- e.g., both CGW advertise a summary route or both CGW advertise the same IP CIDR range with different AS combinations, so when one is down, traffic will go through the other route
- Provision sufficient network capacity to ensure that the failure of one network connection does not overwhelm and degrade redundant connections.
- How do I connect multiple VPCs within the same AWS Region?
- VPC peering
- VPCs Connected with AWS Direct Connect
- Transit VPC
- DX Gateway 實際操作示範
- On-prem 如何透過 Ubuntu instance 解析 Route 53 private hosted zone
- CloudHub solution
- A simple hub-and-spoke model that you can use with or without a VPC
- AWS VPN CloudHub leverages an Amazon VPC virtual private gateway with multiple gateways, each using unique BGP autonomous system numbers (ASNs). Your gateways advertise the appropriate routes (BGP prefixes) over their VPN connections. These routing advertisements are received and readvertised to each BGP peer so that each site can send data to and receive data from the other sites. The remote network prefixes for each spoke must have unique ASNs, and the sites must not have overlapping IP ranges. Each site can also send and receive data from the VPC as if they were using a standard VPN connection.
- CloudHub will not allow VPC to VPC transitive traffic
Accessing VPCs in other regions
- Create DX Gateway in any region + private VIF (link)
- Public VIF + VPN to a VPC in the remote region (link)
- Inter-region VPC Peering
- Jumbo Frames (MTU 9001)
- check/set the MTU via
- Intra-region VPC peering supports Jumbo Frame (link)
- Inter-region VPC peering does not support Jumbo Frame
- Once the internet is involved, maximum MTU is 1500
- DX support Jumbo Frames since October, 2018 (link)
- Only applicable for private VIF
- Path MTU Discovery using
Don't Fragment flag in the IP headers
- ICMP message:
Destination Unreachable: Fragmentation Needed and Don't Fragment was Set (Type 3, Code 4)
- VPN -> over the internet -> MTU 1500
- Guide for Network Bandwidth
- EC2 to S3 - 25 Gbps
- EC2 to EC2 - 5 Gbps for single-flow traffic
- EC2 to EC2 (Cluster Placement Group) - 10 Gbps for single-flow traffic
- Best Practices for Configuring Network Interfaces
- You can’t detach the primary (eth0) interface
- ENI could be moved between subnets in an AZ
- If you attach two or more network interfaces from the same subnet to an instance, you may encounter networking issues such as asymmetric routing. If possible, use a secondary private IPv4 address on the primary network interface instead.
- Scenarios for Network Interfaces (e.g. Management Network, the second ENI must reside in another subnet with a more strict security group.)
- Placement groups
- Cluster: low-latency
- Partition: groups of instances in one partition do not share the underlying hardware with groups of instances in different partitions
- Spread: reduce correlated failures
- We recommend that you launch the number of instances that you need in the placement group in a single launch request and that you use the same instance type for all instances in the placement group. If you try to add more instances to the placement group later, or if you try to launch more than one instance type in the placement group, you increase your chances of getting an insufficient capacity error.
- If you stop an instance in a placement group and then start it again, it still runs in the placement group. However, the start fails if there isn’t enough capacity for the instance.
- If you receive a capacity error when launching an instance in a placement group that already has running instances, stop and start all of the instances in the placement group, and try the launch again. Restarting the instances may migrate them to hardware that has capacity for all the requested instances.
- Enhanced Networking
- Enhanced networking uses single root I/O virtualization (SR-IOV) to provide high-performance networking capabilities on supported instance types. SR-IOV is a method of device virtualization that provides higher I/O performance and lower CPU utilization when compared to traditional virtualized network interfaces. Enhanced networking provides higher bandwidth, higher packet per second (PPS) performance, and consistently lower inter-instance latencies.
- Elastic Network Adapters (ENA)
- Up to 100 Gbps
- Using the latest ENA-enabled AMIs on current-generation EC2 instances
- Check through
modinfo ena and
ethtool -i eth0
- Intel 82599 Virtual Function (VF) interface
- Stop/start an instance -> public IP gets renewed (link)
- Restart/reboot an instance -> public IP retains
- HVM -> hardware-assisted VM
- PV -> Para-virtualization (cannot take advantage of hardware)
- Gateway VPC Endpoints
- Route table configuration
- Prefix list ID could be used in security group but not in a NACL
- Intra-region only
- IPv4 only
- Endpoint connections cannot be extended out of a VPC.
- DNS resolution must be enabled
- Interface VPC Endpoints
- Security group concept
- Enable private DNS for the endpoint (enabled by default)
- TCP, IPV4 only
- Intra-region only
- Endpoint policy could be attached
- VPCs can be between
/28, using the
- 5 addresses reserved per subnet (
- IGW attachment could be denied from Organizations (link)
- VPC does not support multicast or broadcast (link)
- Route priority - longest prefix match
- DHCP Option Set
- 可以設定自己的 domain name, NTP or DNS servers etc.
- 如果用了新的 DHCP option set，當舊的 DHCP lease 到期，就會自動使用新的 (every few hours)
- DX FAQ
- How do I establish an AWS VPN over an AWS Direct Connect connection?
- Secure communication
- VPN on public VIF with CGW IP advertised
- How can I configure VPN as a backup for my AWS Direct Connect connection?
- VPN as a backup
- Use the same VGW and advertise the same prefix
- 50 VIFs for dedicated 1Gbps and 10Gbps connections. AWS DX Partner-provided Hosted Connections will have a single VIF.
- DX types
- Hosted VIFs - shared bandwidth, cross account
- Hosted connections - dedicated bandwidth, purchased from an APN partner, sub-1G connection possible, only one VIF can be created (VLAN is assigned on creation by the partner)
- DX requirement
- Your network must use single-mode fiber with a 1000BASE-LX (1310nm) transceiver for 1 gigabit Ethernet or a 10GBASE-LR (1310nm) transceiver for 10 gigabit Ethernet.
- Auto-negotiation for the port must be disabled. Port speed and full-duplex mode must be configured manually.
- 802.1Q VLAN encapsulation must be supported across the entire connection, including intermediate devices.
- VLAN is required for each VIF. AWS DX router will forward the traffic to corresponding VIF based on the VLAN ID.
- BGP and BGP MD5 authentication.
- (Optional) You can configure Bidirectional Forwarding Detection (BFD) on your network. Asynchronous BFD is automatically enabled for AWS Direct Connect virtual interfaces, but does not take effect until you configure it on your router.
- How do I enable BFD for my DX connection?
- The AWS BFD liveness detection minimum interval is 300 ms and the multiplier is three.
- Otherwise, by default, BGP waits for three keep-alive failures of 90 seconds.
- LAG requirement
- Of the same bandwidth
- Terminate at the same endpoint
- At most 4 connections
- VIF prerequisites
- 100 prefixes announced over private VIF
- Local preference BGP communities tags are evaluated before any
AS_PATH attribute (link)
- IPSec and ESP
- IP Protocol 50 for ESP
- UDP port 500 for IPSec
- UDP port 4500 for NAT-T
- The VGW uses BGP over TCP port 179 (the standard BGP port)
- AWS VPN supports IKEv2, 4-byte ASN, AES 256-bit encryption, SHA-2 hashing, additional DH group, custom private ASN for the Amazon side of a BGP session (link)
- Route Tables and VPN Route Priority
- Longest prefix match applies
- Local route > Static route towards IGW, VGW, ENI… > BGP propagated routes from DX > Static routes for VPN > BGP propagated route from VPN
- That said, a more specific route is the only way to make AWS prefer a VPN over a DX connection.
- VPN CloudHub
- 所有不同 branches 的 office 可以透過 AWS VPN 分別建立連線至 VGW，然後透過 VGW 來互相溝通 (spoke and hub)
- How do I troubleshoot BGP connection issues over VPN?
- Check the underlying VPN connection
- IKE or IPsec failing? Are UDP port 500, 4500 and IP 50 protocol allowed?
- Check the BGP configuration on your customer gateway device
- Debugs and packet captures
- Check if the BGP session is going from established to idle states
- More than 100 routes are being advertised -> summarize
AS_PATH prepending to achieve Active/Passive scenario
- Higher MED makes path less preferred
- VRF approach when the same CGW is used to connect to multiple VPCs (link)
- Route 53 Resolver
- On-prem to VPC -> Inbound endpoint
- VPC to on-prem -> Outbound endpoint (by forwarding rule)
- Simple AD forwards DNS requests to the IP address of the Amazon-provided DNS servers for your VPC. These DNS servers will resolve names configured in your Route 53 private hosted zones. By pointing your on-premises computers to your Simple AD, you can now resolve DNS requests to the private hosted zone.
- White-Label Name Servers
- 例如說用 ns1.example.com 取代 ns-2048.awsdns-64.com
- A delegation set = a group of four NS
- Choosing Between Signed URLs and Signed Cookies
- Use signed URLs
- RTMP (as cookies aren’t supported)
- To restrict access to individual files
- Client that does not support cookies
- Use signed cookies
- To provide access to multiple restricted files
- To prevent from the change of current URLs
- Use IP access control group as a virtual firewall
- Minimum MTU = 1200
- 2 private and 1 public subnet will be created
- Workspaces uses Active Directory, options: Microsoft AD/AD connector/Simple AD.
- AWS Config is a fully managed service that provides you with an AWS resource inventory, configuration history, and configuration change notifications to enable security and governance. With AWS Config you can discover existing AWS resources, export a complete inventory of your AWS resources with all configuration details, and determine how a resource was configured at any point in time. These capabilities enable compliance auditing, security analysis, resource change tracking, and troubleshooting.
- It could trigger a Lambda function and notifications via SNS