Skip to content

Commit

Permalink
Renames k8sgateway refs to kgateway
Browse files Browse the repository at this point in the history
Signed-off-by: Daneyon Hansen <[email protected]>
  • Loading branch information
danehans committed Jan 30, 2025
1 parent 28911a2 commit e0fb98f
Showing 1 changed file with 13 additions and 13 deletions.
26 changes: 13 additions & 13 deletions docs/enhancements/10411.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# EP-10411: Gateway API Inference Extension Support

* Issue: [#10411](https://github.com/k8sgateway/k8sgateway/issues/10411)
* Issue: [#10411](https://github.com/kgateway-dev/kgateway/issues/10411)

## Background

Expand All @@ -12,7 +12,7 @@ The following list defines goals for this EP.

* Provide initial GIE support allowing for easy experimentation of advanced LLM traffic routing via the Endpoint Selector (ES), GIE's reference extension implementation.
* Allow users to enable/disable this feature.
* Implement GIE as a k8sgateway plugin.
* Implement GIE as a kgateway plugin.
* Add [InferencePool](https://github.com/kubernetes-sigs/gateway-api-inference-extension/blob/main/api/v1alpha1/inferencepool_types.go) as a supported HTTPRoute backend reference.
* Provide the ability to manage the GIE deployment.
* Provide e2e testing of this feature.
Expand All @@ -23,26 +23,26 @@ The following list defines goals for this EP.
The following list defines non-goals for this EP.

* Run production traffic using this feature.
* Provide k8sgateway-specific GIE extensions.
* Support non-GIE traffic routing functionality that may be achieved through integration with k8sgateway-specific APIs.
* Provide kgateway-specific GIE extensions.
* Support non-GIE traffic routing functionality that may be achieved through integration with kgateway-specific APIs.
* Provide stats for the initial GIE implementation since it lacks a metrics endpoint.
* Secure the gRPC connection between Gateway and GIE implementations.
* Support k8sgateway upgrades when this feature is enabled.
* Support kgateway upgrades when this feature is enabled.

## Implementation Details

The following sections describe implementation details for this EP.

### Configuration

* Update the [configuration](https://github.com/k8sgateway/k8sgateway/blob/main/install/helm/gloo/generate/values.go) API to enable/disable this feature.
* Update Helm charts to install/uninstall k8sgateway with this feature based on user-provided configuration.
* Update the [configuration](https://github.com/kgateway-dev/kgateway/blob/main/install/helm/gloo/generate/values.go) API to enable/disable this feature.
* Update Helm charts to install/uninstall kgateway with this feature based on user-provided configuration.

__Note:__ Existing Gateway API support, e.g. CRDs, controllers, etc. is required.

### Plugin

* Add GIE as a supported [plugin](https://github.com/k8sgateway/k8sgateway/tree/main/projects/gateway2/extensions2/plugins). The plugin will manage [Endpoints](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/endpoint/endpoint.html) based on the InferencePool resource specification. The Gateway implementation, e.g. Envoy proxy, will forward matching requests using the [External Processing Filter](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_proc/v3/ext_proc.proto#external-processing-filter-proto) to the ES deployment. The ES is responsible for processing the request, selecting an Endpoint, and returning the selected Endpoint to Envoy for routing.
* Add GIE as a supported [plugin](https://github.com/kgateway-dev/kgateway/tree/main/projects/gateway2/extensions2/plugins). The plugin will manage [Endpoints](https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/endpoint/endpoint.html) based on the InferencePool resource specification. The Gateway implementation, e.g. Envoy proxy, will forward matching requests using the [External Processing Filter](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_proc/v3/ext_proc.proto#external-processing-filter-proto) to the ES deployment. The ES is responsible for processing the request, selecting an Endpoint, and returning the selected Endpoint to Envoy for routing.

### Controllers

Expand All @@ -52,20 +52,20 @@ The following sections describe implementation details for this EP.

### Deployer

* Update the [deployer](https://github.com/k8sgateway/k8sgateway/tree/main/projects/gateway2/deployer) to manage the required ES resources, e.g. Deployment.
* Update the [deployer](https://github.com/kgateway-dev/kgateway/tree/main/projects/gateway2/deployer) to manage the required ES resources, e.g. Deployment.

### Translator and Proxy Syncer

* Add InferencePool as a supported HTTPRoute backend reference.
* Update the [translator](https://github.com/k8sgateway/k8sgateway/tree/main/projects/gateway2/translator) package to handle InferencePool references from the HTTPRoute type.
* Enhance the [proxy_syncer](https://github.com/k8sgateway/k8sgateway/tree/main/projects/gateway2/proxy_syncer) to translate the InferencePool custom resource into a Gloo Upstream and sync with the proxy client. When an HTTPRoute references an InferencePool, ensure the Envoy ext_proc filter is attached or the cluster references the ES cluster.
* Update the [translator](https://github.com/kgateway-dev/kgateway/tree/main/projects/gateway2/translator) package to handle InferencePool references from the HTTPRoute type.
* Enhance the [proxy_syncer](https://github.com/kgateway-dev/kgateway/tree/main/projects/gateway2/proxy_syncer) to translate the InferencePool custom resource into a Gloo Upstream and sync with the proxy client. When an HTTPRoute references an InferencePool, ensure the Envoy ext_proc filter is attached or the cluster references the ES cluster.

### Reporting

* Update the [reporter](https://github.com/k8sgateway/k8sgateway/tree/main/projects/gateway2/reports) package to support status reporting, e.g. `ResolvedRefs=true` when HTTPRoute references an InferencePool.
* Update the [reporter](https://github.com/kgateway-dev/kgateway/tree/main/projects/gateway2/reports) package to support status reporting, e.g. `ResolvedRefs=true` when HTTPRoute references an InferencePool.

__Note:__ InferencePool status is currently undefined.

## Open Questions

1. ~Is a new plugin type required or can an existing type be utilized, e.g. UpstreamPlugin?~ A new plugin type will be created specific to GIE.
1. ~~Is a new plugin type required or can an existing type be utilized, e.g. UpstreamPlugin?~~ A new plugin type will be created specific to GIE.

0 comments on commit e0fb98f

Please sign in to comment.