Add model server protocol proposal

[liu-cong]

This is adapted from the initial doc

I didn't include the ORCA load reporting section as it's not currently required by the inference extension, though parallel efforts are happening. The intention is to keep the scope of this protocol small and expand in the future if needed.

Much of the protocol items are "SHOULD" instead of "MUST" given the following reasons:

• Existing model servers already have chosen different implementation. E.g., different metric names. While we would like to drive unification, we don't want to dictate.
• Emerging features like LoRA serving has yet to evolve to a more common understanding. Currently different model server have very different configurations for LoRA serving, for example. It's premature to come to a well defined contract.

I see this as a very initial effort to define the contract, and is an evolving process to monitor industry trends and drive more unification.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: liu-cong
Once this PR has been reviewed and has the lgtm label, please assign ahg-g for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ahg-g]

@smarterclayton

[ahg-g]

@smarterclayton @robscott I think we need to explicitly define two "contracts":

Contract 1 is the one between the gateway and the endpoint picker extension; this contract is defined by the InferenceModel/InferencePool APIs.

Contract 2 is the one between the reference implementation of the endpoint picker and the model server.

For each contract, we need separately establish versioning and define conformance tests and my view is that we should not make them dependent on each other.

The InferencePool API could have a way to communicate to the endpoint picker which protocol to use (e.g., as part of the extension config API), but for the most part it should be part of some raw parameters config (e.g., a map or a generic reference to a CRD) that the extension understands and knowns how to interpret.