IPv6 Hop-by-Hop & Destination Option #56

rsivakolundu · 2018-10-03T16:03:38Z

IPv6 Hop-by-Hop and Destination Option for INT Header and INT Metadata transport

This reverts commit 511841e.

mhira1 · 2018-10-03T17:58:26Z

telemetry/specs/INT.mdk

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 
+
+Nxt Hdr: 8-bit selector. Identifies the type of header immediately following the 
+Hop-by-Hop or Desitnation Options header. Uses the same values as the IPv4 Protocol


Typo: Desitnation -> Destination

mhira1 · 2018-10-03T17:58:54Z

telemetry/specs/INT.mdk

+Hop-by-Hop or Desitnation Options header. Uses the same values as the IPv4 Protocol
+Field [IANA-PN]
+
+HDR Ext Len: 8-bit unsigned integer. Length of the Hop-by-Hop or Desitnation


Same typo: Desitnation -> Destination

mhira1 · 2018-10-03T18:24:37Z

telemetry/specs/INT.mdk

+|               Payload + Padding (L4/ESP/….)                   |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+
+


We need to add some text to clarify the following regarding INT within IPv6:

What is the use-case for INT as IPv6 destination option? INT has its own destination header type, meant for the INT sink. In general, INT sink and IPv6 destination may not be the same.

How does an INT switch handle the padding following INT data? Each hop inserts Hop-ML worth of metadata which is a multiple of 4 bytes, but not necessarily a multiple of 8 bytes. What happens if each hop is inserting an odd multiple of 4 bytes which is not a multiple of 8 bytes (say 20B). In such a case, at hop 1, we need 4 bytes of padding. Hop 2 can remove the padding inserted by hop 1 and comply. Hop 3 can insert 4 bytes padding again. Or do we say that each hop adds 4B of padding if HopML is an odd multiple of 4B? That would be wasteful.

Regardless of what we do for Reporting physical and logical port ID in telemetry metadata #2, when HopML is an odd multiple of 4, each hop needs to push metadata at the top of the stack and do some manipulation (add/remove padding) at end of the stack. So INT behavior is different here. In other encapsulations, each hop simply inserts at the head of the metadata stack.

Let us discuss the padding issue in person.

mhira1 · 2018-10-03T18:25:36Z

telemetry/specs/INT.mdk

 Geneve options to be defined for INT Headers.

+### INT over IPv6


Should we add INT over IPv6 after the other three encaps? Specially because the text in the paragraph is referring to scenarios where "INT over VXLAN or Geneve is not helpful"

I followed what was done earlier. TCP/UDP was listed first and it referenced encaps. I just stuck to that. I am fine with changing the order.

I may be woefully out of date on IPv6 extension header behavior, but regarding the option '"INT over IPv6" - INT Headers are carried in the IPv6 packets as Hop-by-Hop option.', I had thought that switches in practice have to punt packets with an IPv6 Hop-by-Hop extension header to the slow path, e.g. software forwarding on a general purpose CPU.

I did a quick search and found that RFC 7045 (published Dec 2013) says this in Section 2.2 "Hop-by-Hop Options":

The IPv6 Hop-by-Hop Options header SHOULD be processed by
intermediate forwarding nodes as described in [RFC2460]. However, it
is to be expected that high-performance routers will either ignore it
or assign packets containing it to a slow processing path. Designers
planning to use a hop-by-hop option need to be aware of this likely
behaviour.

Is there really a desire to put INT data into a header that will likely result in slow path processing in the network?

jklr · 2018-10-03T18:56:28Z

@rsivakolundu @mhira1 @mickeyspiegel We can merge this change first, create a v1.x cut, advance INT.mdk to version 2 and merge the two other changes. One thing we need to discuss is where to merge the other future transportations (SRv6, GRE) to. INT v1.x or 2.x, or both?

mickeyspiegel · 2018-10-04T03:01:20Z

telemetry/specs/INT.mdk

+.                                                               .  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+  |
+|  Variable Opt Data (INT Data) |         Padding               |  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+<-+ 


This alignment is not correct. This needs to be either:

Option Type and Opt Data Len is in this position, no Reserved (MBZ) field, and no padding, and the option has an alignment constraint of 4n+2.

The option has an alignment constraint of 4n, so there are two bytes of padding after HDR Ext Len, then Option Type and Opt Data Len starting at byte 4, then two bytes of Reserved (MBZ), then INT data.

IOAM did it this way. I am okay with your suggestion. I will close with IOAM folks on a separate thread.

I think IOAM did my second bullet. They did not explicitly mention the 4n alignment constraint, but they should have. That is the only way that what they defined works.

I need to propose to ask for only one code point and replace Reserved (MBZ) with 1 byte reserved and 1 byte IOAM Type.

mickeyspiegel · 2018-10-04T03:10:21Z

telemetry/specs/INT.mdk

+Option Type: 8-bit identifier of the type of option.
+
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_HOP_BY_HOP_OPTION_IPV6.
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_DESTINATION_OPTION_IPV6.


Looking at the IANA registry, there are a total of 32 code points of which 17 have already been allocated. The registration procedure is IESG Approval, IETF Review or Standards Action. IOAM is asking for 4 code points, which seems unlikely. The chances for INT to get any code points are not high.

I see two options:

Go with the single experimental use hop-by-hop-options codepoint (0b11110), and split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of INT type.

Wait for IOAM to get a codepoint and use that. Split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of IOAM type. Assign relatively high IOAM type codepoints for INT hop-by-hop option and INT destination option.

I see another problem with the corresponding IETF IOAM IPv6 draft. The text says that "a router MUST drop packets which contain extension headers carrying IOAM data-fields", to "ensure that the IOAM data does not unintentionally get forwarded outside the IOAM domain." However, they asked for an Option Type codepoint starting with "00", which means when the option type is unrecognized, "skip over this option and continue processing the header". If the text is correct, then they should ask for any of the other codepoint prefixes "01" (discard the packet), "10" (discard and send ICMP parameter problem, code 2, back to the packet's source address), or "11" (discard and send ICMP only if the packet's destination address was not a multicast address).

I will close loop with IETF and address this comment.

Whatever we do, two codepoints will not fly. At a minimum we would have to go with TBD_IANA_INT_OPTION_IPV6 (not distinguishing between INT hop-by-hop and INT destination), which would later get resolved to either experimental hop-by-hop options codeopint or whatever IOAM has assigned. If we go with IOAM then the INT Type values might need to be shifted to avoid conflicts.

I also wonder if we should use xxx or yyy for the first 3 bits as well given the other open issue I stated above.

mickeyspiegel · 2018-10-04T03:13:43Z

telemetry/specs/INT.mdk

+Variable Opt Data: INT Header and Metadata, multiple of four octets in length.
+
+Padding: 16-bit pad. Needed to ensure that the variable length of the complete 
+Hop-by-Hop or Destination Options Header is an integer multiple of 8 octets long.


I don't think the padding is added explicitly. If there are multiple options in the same extension header, then padding is only inserted to meet the next option's alignment constraint. The padding to 8 bytes is done at the end, after all options have been added to the extension header.

What I suggested for IOAM is that the Hop ML must be even so that padding is not required. However, I did not consider that the INT header is included. For v1 and with the second padding option described above, if Hop ML is even no padding would be required. For v2 due to the odd number of octets before the per hop data, this would not work and padding would have to be added in some cases.

For v2, it looks like we have 4 octets shim header and 12 octets fixed part of INT header before the variable metadata. There are two options with respect to padding:

Go with 4n alignment. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" above would be inserted in front of this option, and if Hop ML is even then there would be an additional 4 octets of padding after this option. If Hop ML is odd, then the additional 4 octets of padding after this option would only be present at even hops, so each hop either strips the incoming 4 octets of padding after this option, or adds 4 octets of padding after this option. If there are other hop-by-hop options present, padding may differ both before and after this option.

Go with 8n alignment and require that Hop ML must be even. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" in front of this option would be 6 octets rather than 2 octets. No padding would be added after this option. If there are other hop-by-hop options present, then the padding may differ both before and after this option, but it would not need to be adjusted at each hop.

IOAM has a similar issue, but the header size is different so the 4n alignment implications are different. When IOAM incremental trace is the only option present in the hop-by-hop options, and if hop ML is even then there would be no padding after this option. 8n alignment would not work since the total size of the fixed headers is not a multiple of 8 octets.

rsivakolundu · 2018-10-05T22:54:57Z

I will follow-up on this issue and get back to you guys. This is the first time I am hearing it. Stay tuned.

…

-Ramesh

On Thu, Oct 4, 2018 at 5:22 PM Andy Fingerhut ***@***.***> wrote: ***@***.**** commented on this pull request. ------------------------------ In telemetry/specs/INT.mdk <#56 (comment)>: > Geneve options to be defined for INT Headers. +### INT over IPv6 I may be woefully out of date on IPv6 extension header behavior, but regarding the option '"INT over IPv6" - INT Headers are carried in the IPv6 packets as Hop-by-Hop option.', I had thought that switches in practice have to punt packets with an IPv6 Hop-by-Hop extension header to the slow path, e.g. software forwarding on a general purpose CPU. I did a quick search and found that RFC 7045 (published Dec 2013) says this in Section 2.2 "Hop-by-Hop Options": The IPv6 Hop-by-Hop Options header SHOULD be processed by intermediate forwarding nodes as described in [RFC2460]. However, it is to be expected that high-performance routers will either ignore it or assign packets containing it to a slow processing path. Designers planning to use a hop-by-hop option need to be aware of this likely behaviour. Is there really a desire to put INT data into a header that will likely result in slow path processing in the network? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#56 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/APOTlh71iKSIC-fuU5V8dPInExCcGAQpks5uhqZWgaJpZM4XGWt3> .

telemetry/specs/INT.mdk

mickeyspiegel · 2020-01-15T01:01:46Z

telemetry/specs/INT.mdk

+Option Type: 8-bit identifier of the type of option.
+
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_HOP_BY_HOP_OPTION_IPV6.
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_DESTINATION_OPTION_IPV6.


I see two options:

Go with the single experimental use hop-by-hop-options codepoint (0b11110), and split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of INT type.

Wait for IOAM to get a codepoint and use that. Split the "Reserved (MBZ)" field into 8 bits reserved followed by 8 bits of IOAM type. Assign relatively high IOAM type codepoints for INT hop-by-hop option and INT destination option.

mickeyspiegel · 2020-01-15T01:25:47Z

telemetry/specs/INT.mdk

+Variable Opt Data: INT Header and Metadata, multiple of four octets in length.
+
+Padding: 16-bit pad. Needed to ensure that the variable length of the complete 
+Hop-by-Hop or Destination Options Header is an integer multiple of 8 octets long.


For v2, it looks like we have 4 octets shim header and 12 octets fixed part of INT header before the variable metadata. There are two options with respect to padding:

Go with 4n alignment. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" above would be inserted in front of this option, and if Hop ML is even then there would be an additional 4 octets of padding after this option. If Hop ML is odd, then the additional 4 octets of padding after this option would only be present at even hops, so each hop either strips the incoming 4 octets of padding after this option, or adds 4 octets of padding after this option. If there are other hop-by-hop options present, padding may differ both before and after this option.

Go with 8n alignment and require that Hop ML must be even. When INT is the only option present in the hop-by-hop options, then the "Padding|(MBZ)" in front of this option would be 6 octets rather than 2 octets. No padding would be added after this option. If there are other hop-by-hop options present, then the padding may differ both before and after this option, but it would not need to be adjusted at each hop.

IOAM has a similar issue, but the header size is different so the 4n alignment implications are different. When IOAM incremental trace is the only option present in the hop-by-hop options, and if hop ML is even then there would be no padding after this option. 8n alignment would not work since the total size of the fixed headers is not a multiple of 8 octets.

mickeyspiegel · 2020-01-15T01:37:51Z

telemetry/specs/INT.mdk

+Option Type: 8-bit identifier of the type of option.
+
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_HOP_BY_HOP_OPTION_IPV6.
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_DESTINATION_OPTION_IPV6.


I see another problem with the corresponding IETF IOAM IPv6 draft. The text says that "a router MUST drop packets which contain extension headers carrying IOAM data-fields", to "ensure that the IOAM data does not unintentionally get forwarded outside the IOAM domain." However, they asked for an Option Type codepoint starting with "00", which means when the option type is unrecognized, "skip over this option and continue processing the header". If the text is correct, then they should ask for any of the other codepoint prefixes "01" (discard the packet), "10" (discard and send ICMP parameter problem, code 2, back to the packet's source address), or "11" (discard and send ICMP only if the packet's destination address was not a multicast address).

mickeyspiegel · 2020-01-15T01:40:53Z

telemetry/specs/INT.mdk

+Field [IANA-PN]
+
+HDR Ext Len: 8-bit unsigned integer. Length of the Hop-by-Hop or Destination
+Options header in 8-octet units, not including the first 8 octets.


Above it only says Hop-by-Hop Options, not Destination Options. We can go either way, but it should be consistent.

telemetry/specs/INT.mdk

mickeyspiegel

In addition to the specific comments, we need to figure out what to say about padding before we finalize.

mickeyspiegel · 2020-02-13T02:46:08Z

telemetry/specs/INT.mdk

+* Reserved (MBZ): 16 bit field, must be filled with zeroes upon transmission and ignored upon 
+reception.
+
+* Type: This field indicates the type of INT Metadata Header and Metadata following.


nit: The figure above labels the field as "INT TYPE". We should make this consistent, I guess with "INT Type"?

mickeyspiegel · 2020-02-13T02:46:17Z

telemetry/specs/INT.mdk

 option, in octets.

-Reserved (MBZ): 16 bit field, must be filled with zeroes upon transmission and ignored upon reception.
+* Reserved (MBZ): 16 bit field, must be filled with zeroes upon transmission and ignored upon 


8 bit field

mickeyspiegel · 2020-02-13T02:57:38Z

telemetry/specs/INT.mdk

+Option Type: 8-bit identifier of the type of option.
+
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_HOP_BY_HOP_OPTION_IPV6.
+  001xxxxxx 8-bit identifier of the type of option. xxxxxx=TBD_IANA_INT_DESTINATION_OPTION_IPV6.


Whatever we do, two codepoints will not fly. At a minimum we would have to go with TBD_IANA_INT_OPTION_IPV6 (not distinguishing between INT hop-by-hop and INT destination), which would later get resolved to either experimental hop-by-hop options codeopint or whatever IOAM has assigned. If we go with IOAM then the INT Type values might need to be shifted to avoid conflicts.

I also wonder if we should use xxx or yyy for the first 3 bits as well given the other open issue I stated above.

rsivakolundu and others added 6 commits June 21, 2018 15:03

Update INT.mdk

3771848

Update INT.mdk

10355fd

Update INT.mdk

fe346e4

Revert "Slides from 2018-05-17 P4 Memory Abstraction Use Cases"

d4972e5

This reverts commit 511841e.

Update INT.mdk

703fd49

Update INT.mdk

cc70169

mhira1 reviewed Oct 3, 2018

View reviewed changes

mhira1 requested review from mickeyspiegel and jklr October 3, 2018 18:26

mickeyspiegel suggested changes Oct 4, 2018

View reviewed changes

rsivakolundu added 3 commits October 3, 2018 21:43

Update INT.mdk

800c506

Update INT.mdk

7456957

Update INT.mdk

c5c1a52

rsivakolundu added 2 commits December 6, 2019 10:40

Update INT.mdk

9e75a1f

Update INT.mdk

06a5ea0

mickeyspiegel suggested changes Jan 15, 2020

View reviewed changes

rsivakolundu added 3 commits January 16, 2020 14:22

Update INT.mdk

4638a0f

Update INT.mdk

4601746

Merge branch 'master' into IPv6-HbyH-Option

5ff83a8

mickeyspiegel suggested changes Feb 13, 2020

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

IPv6 Hop-by-Hop & Destination Option #56

IPv6 Hop-by-Hop & Destination Option #56

rsivakolundu commented Oct 3, 2018

mhira1 Oct 3, 2018

rsivakolundu Feb 11, 2020

mhira1 Oct 3, 2018

rsivakolundu Feb 11, 2020

mhira1 Oct 3, 2018

rsivakolundu Oct 4, 2018

mhira1 Oct 3, 2018

rsivakolundu Oct 4, 2018

jafingerhut Oct 5, 2018

jklr commented Oct 3, 2018 •

edited

Loading

mickeyspiegel Oct 4, 2018

rsivakolundu Oct 4, 2018

mickeyspiegel Oct 4, 2018

mickeyspiegel Oct 4, 2018

rsivakolundu Oct 4, 2018

mickeyspiegel Jan 15, 2020

mickeyspiegel Jan 15, 2020

rsivakolundu Jan 16, 2020

mickeyspiegel Feb 13, 2020

mickeyspiegel Oct 4, 2018

mickeyspiegel Jan 15, 2020

rsivakolundu commented Oct 5, 2018 via email

mickeyspiegel Jan 15, 2020

mickeyspiegel Jan 15, 2020

mickeyspiegel Jan 15, 2020

mickeyspiegel Jan 15, 2020

rsivakolundu Feb 11, 2020

mickeyspiegel left a comment

mickeyspiegel Feb 13, 2020

mickeyspiegel Feb 13, 2020

mickeyspiegel Feb 13, 2020

		\| Payload + Padding (L4/ESP/….) \|
		+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

		Geneve options to be defined for INT Headers.

		### INT over IPv6

IPv6 Hop-by-Hop & Destination Option #56

Are you sure you want to change the base?

IPv6 Hop-by-Hop & Destination Option #56

Conversation

rsivakolundu commented Oct 3, 2018

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklr commented Oct 3, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

rsivakolundu commented Oct 5, 2018 via email

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mickeyspiegel left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jklr commented Oct 3, 2018 •

edited

Loading