feasibility-research(runtime): Support kubernetes as the backend vendor

[gaocegege]

• feat(CLI): Support context #92
• feat(CLI): Support envd-server in conext #939
  • feat(CLI): envd create command #957
• feat(lang): Support data and code integration in envd-server runner #530
• feat(CLI): Support SSH(attach) command #101
• feat: Port forwarding in envd-server runner #958

[gaocegege]

Actually, I am not very familiar with Kubernetes CRI, I need some time to learn it.

[gaocegege]

	mountOption[len(mountOptionsStr)] = mount.Mount{
		Type:   mount.TypeBind,
		Source: buildContext,
		Target: base,
	}

envd/pkg/docker/docker.go

Line 232 in 8e1e498

mountOption[len(mountOptionsStr)] = mount.Mount{

We mount the build context into the docker container now, it is impossible if we use kubernetes. Maybe we need a new rule in starlark to deal with source code

[gaocegege]

	Exec(ctx context.Context, cname string, cmd []string) error
	Destroy(ctx context.Context, name string) (string, error)
	List(ctx context.Context) ([]types.Container, error)
	// GPUEnabled returns true if nvidia container runtime exists in docker daemon.
	GPUEnabled(ctx context.Context) (bool, error)
	// Start creates the container for the given tag and container name.
	StartEnvd(ctx context.Context, tag, name, buildContext string,
		gpuEnabled bool, g ir.Graph, timeout time.Duration, mountOptionsStr []string) (string, string, error)

These funcs in the docker interface should be abstracted to a new interface runtime. Both docker and k8s runtime need to implement.

[knight42]

working dir mount

I think we could not mount local dir to container in k8s cluster directly, how about we spawn a rsyncd inside the container, and sync working dir by using rsync?

We cannot forward port to localhost (or maybe we can via kubectl port forward?)

This module https://pkg.go.dev/k8s.io/client-go/tools/portforward might be helpful.

[gaocegege]

I think we could not mount local dir to container in k8s cluster directly, how about we spawn a rsyncd inside the container, and sync working dir by using rsync?

Yep we may need https://syncthing.net/

[gaocegege]

#dibs
/assign

[LeonardAukea]

working dir mount

I think we could not mount local dir to container in k8s cluster directly, how about we spawn a rsyncd inside the container, and sync working dir by using rsync?

We cannot forward port to localhost (or maybe we can via kubectl port forward?)

This module https://pkg.go.dev/k8s.io/client-go/tools/portforward might be helpful.

Maybe this is relevant to the discussion. @gaocegege It also utilises syncthing https://github.com/ksync/ksync/blob/master/docs/architecture.md

[gaocegege]

working dir mount

I think we could not mount local dir to container in k8s cluster directly, how about we spawn a rsyncd inside the container, and sync working dir by using rsync?

We cannot forward port to localhost (or maybe we can via kubectl port forward?)

This module https://pkg.go.dev/k8s.io/client-go/tools/portforward might be helpful.

Maybe this is relevant to the discussion. @gaocegege It also utilises syncthing https://github.com/ksync/ksync/blob/master/docs/architecture.md

Thanks for the info! it seems like a clean wrapper of sycthing.