Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for SNAT Fixed Port Ranges #3175

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

ethan-gallant
Copy link

@ethan-gallant ethan-gallant commented Jan 17, 2025

What type of PR is this?
feature
improvement

Which issue does this PR fix?:
#3174

What does this PR do / Why do we need it?:
This PR adds functionality to exclude specific port ranges from randomized SNAT when using --randomize-fully. This is essential for maintaining compatibility with protocols that require predictable NAT behavior while still benefiting from randomized SNAT for general traffic. Several critical protocols like STUN, ICE, UDP hole punching, and cryptocurrency wallet P2P discovery rely on predictable port mapping to function correctly.

Testing done on this change:
Manually tested on EKS with STUN

With changes (AWS_VPC_K8S_CNI_SNAT_FIXED_PORTS=3478-3479,19302):

Test STUN in a pod with pystun3

snat-test3:~# pystun3 -d -H stun.l.google.com -P 19302 -p 3478
DEBUG:pystun3:Do Test1
DEBUG:pystun3:sendto: ('stun.l.google.com', 19302)
DEBUG:pystun3:recvfrom: ('74.125.250.129', 19302)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'node-ip-redacted', 'ExternalPort': 3478, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test2
DEBUG:pystun3:sendto: ('stun.l.google.com', 19302)
DEBUG:pystun3:recvfrom: ('74.125.250.129', 19302)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'node-ip-redacted', 'ExternalPort': 3478, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
NAT Type: Full Cone
External IP: 44.204.207.113
External Port: 3478

Run TCPDump on the node shows the expected predictable source port

[root@ip-10-42-68-158 /]# tcpdump -vv -n 'udp and (portrange 3478-3479 or port 19302)'
tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
05:36:04.436513 IP (tos 0x0, ttl 126, id 16822, offset 0, flags [DF], proto UDP (17), length 48)
    10.42.68.158.stun > 74.125.250.129.19302: [bad udp cksum 0x93f4 -> 0x92ab!] UDP, length 20
05:36:04.438142 IP (tos 0x0, ttl 58, id 0, offset 0, flags [none], proto UDP (17), length 60)
    74.125.250.129.19302 > 10.42.68.158.stun: [udp sum ok] UDP, length 32
05:36:04.445352 IP (tos 0x0, ttl 126, id 16823, offset 0, flags [DF], proto UDP (17), length 56)
    10.42.68.158.stun > 74.125.250.129.19302: [bad udp cksum 0x93fc -> 0xbed8!] UDP, length 28
05:36:04.446970 IP (tos 0x0, ttl 58, id 0, offset 0, flags [none], proto UDP (17), length 60)
    74.125.250.129.19302 > 10.42.68.158.stun: [udp sum ok] UDP, length 32

Without changes:

Test STUN ports in a pod with pystun3

snat-test3:~# pystun3 -d -H stun.l.google.com -P 19302 -p 3478
DEBUG:pystun3:Do Test1
DEBUG:pystun3:sendto: ('stun.l.google.com', 19302)
DEBUG:pystun3:recvfrom: ('74.125.250.129', 19302)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'rnode-ip-redacted, 'ExternalPort': 5943, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
DEBUG:pystun3:Do Test2
DEBUG:pystun3:sendto: ('stun.l.google.com', 19302)
DEBUG:pystun3:recvfrom: ('74.125.250.129', 19302)
DEBUG:pystun3:Result: {'Resp': True, 'ExternalIP': 'node-ip-redacted', 'ExternalPort': 5943, 'SourceIP': None, 'SourcePort': None, 'ChangedIP': None, 'ChangedPort': None}
NAT Type: Full Cone
External IP: redacted-pub-ip
External Port: 5943
snat-test3:~#

TCP Dump on the node shows a randomized source port

tcpdump: listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
05:25:37.023566 IP (tos 0x0, ttl 126, id 4270, offset 0, flags [DF], proto UDP (17), length 48)
    10.42.68.158.65051 > 74.125.250.129.19302: [bad udp cksum 0x93f4 -> 0x27ff!] UDP, length 20
05:25:37.025056 IP (tos 0x80, ttl 48, id 0, offset 0, flags [none], proto UDP (17), length 60)
    74.125.250.129.19302 > 10.42.68.158.65051: [udp sum ok] UDP, length 32
05:25:37.039780 IP (tos 0x0, ttl 126, id 4274, offset 0, flags [DF], proto UDP (17), length 56)
    10.42.68.158.65051 > 74.125.250.129.19302: [bad udp cksum 0x93fc -> 0x1d48!] UDP, length 28
05:25:37.041096 IP (tos 0x80, ttl 48, id 0, offset 0, flags [none], proto UDP (17), length 60)
    74.125.250.129.19302 > 10.42.68.158.65051: [udp sum ok] UDP, length 32

Will this PR introduce any new dependencies?:
No, this feature will be implemented using existing iptables functionality.

Will this break upgrades or downgrades? Has updating a running cluster been tested?:
No, this change should not break upgrades or downgrades. The default behavior (full randomization) will remain unchanged unless explicitly configured otherwise.

Does this change require updates to the CNI daemonset config files to work?:
Yes, the CNI config will need to be updated to specify excluded port ranges. This can be done via kubectl patch.

Does this PR introduce any user-facing change?:
Yes

Added ability to configure fixed port ranges to be excluded from randomized SNAT, enabling compatibility with protocols that require predictable NAT behavior like STUN, ICE, UDP hole punching, and cryptocurrency wallet P2P discovery.```

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@ethan-gallant ethan-gallant requested a review from a team as a code owner January 17, 2025 05:56
@ethan-gallant ethan-gallant changed the title Support SNAT Fixed Port Ranges Add Support for SNAT Fixed Port Ranges Jan 17, 2025
@ethan-gallant ethan-gallant force-pushed the support-snat-fixed-port-ranges branch 2 times, most recently from 68c50d1 to 9ed428a Compare January 17, 2025 06:04
@ethan-gallant ethan-gallant force-pushed the support-snat-fixed-port-ranges branch from 9ed428a to 9e5f67a Compare January 18, 2025 20:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant