Skip to content
This repository has been archived by the owner on Nov 30, 2021. It is now read-only.

feat(ingress) Experimental Native Ingress #732

Closed
wants to merge 9 commits into from
1 change: 1 addition & 0 deletions charts/workflow/requirements.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,7 @@ dependencies:
version: <registry-token-refresher-tag>
repository: https://charts.deis.com/registry-token-refresher
- name: router
condition: !global.experimental_native_engress
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "engress"

version: <router-tag>
repository: https://charts.deis.com/router
- name: workflow-manager
Expand Down
10 changes: 10 additions & 0 deletions charts/workflow/values.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,12 @@ global:
host_port: 5555
# Prefix for the imagepull secret created when using private registry
secret_prefix: "private-registry"
# Experimental feature to toggle using kubernetes ingress instead of the Deis router.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Although we always used to, we should avoid using "Deis" as shorthand for "Deis Workflow" these days. Maybe reword this section like this?

# Experimental feature to use Kubernetes ingress instead of Workflow's deis-router.
#
# Valid values are:
# - true: deis-router will not be deployed. Workflow will not be usable until a Kubernetes ingress controller is installed.
# - false: deis-router will be deployed (default).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

indeed, it confuses people

#
# Valid values are:
# - true: The Deis router will NOT be deployed. Inherently workflow will not be usable until a Kubernetes ingress controller is installed.
# - false: The default mode, and the default behavior of Deis workflow.
experimental_native_ingress: false


s3:
Expand Down Expand Up @@ -107,6 +113,10 @@ controller:
# disabled - turns off open registration
# admin_only - allows for registration by an admin only.
registration_mode: "admin_only"
# The public resolvable hostname to build your cluster with.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say "publicly resolvable here" or "resolvable public hostname" here.

Copy link
Contributor

@krancour krancour Mar 10, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Even with the following line clarifying it, I somehow still feel thrown off by the phrase "to build your cluster with."

Can we maybe replace this and the following sentence with something like:

A resolvable domain or subdomain name. All applications created by the controller will, by default, be subdomains of this. When experimental native ingress is enabled, these subdomains are used in defining host-based routing rules. If, for instance, platform_domain were "example.com", given a application "foobar", the controller will create a rule to route foo.example.com to the appropriate backend.

^ And that's totally imperfect. Feel free to iterate on that if you can improve it further.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed to "publicly resolvable" to keep it simple

If we want to elaborate more on this later we can.

#
# This will be the hostname that is used to build endpoints such as "deis.$HOSTNAME"
platform_domain: ""

database:
# The username and password to be used by the on-cluster database.
Expand Down
1 change: 1 addition & 0 deletions mkdocs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@ pages:
- Configuring Postgres: installing-workflow/configuring-postgres.md
- Configuring the Registry: installing-workflow/configuring-registry.md
- Chart Provenance: installing-workflow/chart-provenance.md
- Experimental Native Ingress: installing-workflow/experimental-native-ingress.md
- Users:
- Command Line Interface: users/cli.md
- Users and Registration: users/registration.md
Expand Down
80 changes: 80 additions & 0 deletions src/installing-workflow/experimental-native-ingress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# Experimental Native Ingress
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm wondering if this page repeats too much documentation from the "normal" doc for installing without this experimental feature. Do we want to consider a lesser page that just highlights what's different when you install with this feature flag set? I really don't know what's best here. I'm just asking the question.

Copy link
Contributor Author

@krisnova krisnova Mar 14, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I already considered hooking into the existing page, I think it makes sense to have a clear change in flow for an entire section, than to try to add a bunch of conditional logic to an existing flow. I visualize the flows like this:

Separate and clear

         ----
----- <        > ------
         ----

Together and conditional

---- <>-<>-<>-<> ---- 

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does "separate and clear" become a maintenance burden if / as additional flags for major / experimental features get added. I'm afraid of committing to the notion of a page per permutation.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the wise thing to do here is to let problems present themselves instead of trying to be too defensive and predict the future. If it becomes a problem, we can always change. The concern seems trivial to be honest, and this an experimental feature after all, shouldn't we be experimenting to see what works best?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it's a trivial concern. We generally accept the practice of avoiding DRY exceptions in code. imo, docs shouldn't be any different. In either case, a DRY exception represents an on-going maintenance burden.

Though I don't think it's a trivial concern, I'm not going to belabor it either. You're doing the legwork on this, so I defer to your judgement.


## Install Deis Workflow (With experimental native ingress support)

Now that Helm is installed and the repository has been added, install Workflow with a native ingress by running:

```
$ helm install deis/workflow --namespace deis --set global.experimental_native_ingress=true,controller.platform_domain=deis.com
```

Where `global.hostname` is a **required** parameter that is traditionally not required for Workflow. In this example we are using `deis.com` for `$hostname`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we using global.hostname or controller.platform_domain? The example disagrees with this sentence.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

controller.platform_domain. It was changed elsewhere but not here.



Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

there's a few extra unneeded newlines here inbetween paragraphs. Mind removing them?


Helm will install a variety of Kubernetes resources in the `deis` namespace.
Wait for the pods that Helm launched to be ready. Monitor their status by running:

```
$ kubectl --namespace=deis get pods
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add -w?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kubernetes will have the pod information stored very quickly, and I think adding a -w adds an implicit requirement on noting ^C and that sounds like adding a lot for a small gain.

```

You should also notice that a Kubernetes ingress has been installed on your cluster. You can view it by running:
Copy link
Member

@mboersma mboersma Mar 9, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"a Kubernetes ingress controller has been installed"? Or is just "ingress" the right terminology here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"ingress" is the proper terminology here, as @kris-nova is referring to ingress resources that are part of the chart and define routing rules for the controller, grafana, etc.

That being said, "a Kubernetes ingress," might need to change because I do think there's more than one installed as part of the platform itself.


```
$ kubectl get ingress --namespace deis
```

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

extra newline here


Depending on the order in which the Workflow components initialize, some pods may restart. This is common during the
installation: if a component's dependencies are not yet available, that component will exit and Kubernetes will
automatically restart it.

Here, it can be seen that the controller, builder and registry all took a few loops before they were able to start:
Copy link
Member

@bacongobbler bacongobbler Mar 16, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

took a few loops waiting for minio*

extra context is good :)


```
$ kubectl --namespace=deis get pods
NAME READY STATUS RESTARTS AGE
deis-builder-hy3xv 1/1 Running 5 5m
deis-controller-g3cu8 1/1 Running 5 5m
deis-database-rad1o 1/1 Running 0 5m
deis-logger-fluentd-1v8uk 1/1 Running 0 5m
deis-logger-fluentd-esm60 1/1 Running 0 5m
deis-logger-sm8b3 1/1 Running 0 5m
deis-minio-4ww3t 1/1 Running 0 5m
deis-registry-asozo 1/1 Running 1 5m
deis-workflow-manager-68nu6 1/1 Running 0 5m
```

## Install a Kubernetes Ingress Controller

Now that Workflow has been deployed with the `global.exerpimental_native_ingress` flag set to `true`, we will need a Kubernetes ingress controller in place to begin routing traffic.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "exerpimental"


Here is an example of how to use [traefik](https://traefik.io/) as an ingress controller for Workflow. Of course, you are welcome to use any controller you wish.

```
$ helm install stable/traefik --name deis-ingress-001 --namespace kube-system
```

## Configure DNS

The experimental ingress feature requires a user to set up a hostname, and assumes the `deis.$host` convention.

We need to point the `deis.$host` record to the public IP address of your ingress controller. You can get the public IP using the following command.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't it actually be *.$host that resolves to their ingress controller?


```
$ kubectl get svc deis-ingress-001 --namespace kube-system
NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE
deis-ingress-001 10.23.253.220 104.154.159.184 80:30231/TCP,443:32264/TCP 19m
```

If we were using `deis.com` as a hostname we would need to create the following A DNS record.

| Name | Type | Value |
| ----------------- |:-------------:| ---------------:|
| deis.deis.com | A | 104.154.159.184 |


Once all of the pods are in the `READY` state, and `deis.$host` resolves to the external IP found above Workflow is up an running!

After installing Workflow, [register a user and deploy an application](../quickstart/deploy-an-app.md).
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since it's experimental, should we provide a link in the docs to github issue reporting? Or encourage users to share their stories of success with different ingress controllers by submitting docs PRs as HOWTOs?

Not necessary, just thinking...

4 changes: 4 additions & 0 deletions src/installing-workflow/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -33,6 +33,10 @@ More rigorous installations would benefit from using outside sources for the fol
* [Redis](../managing-workflow/platform-logging.md#configuring-off-cluster-redis) - Such as AWS Elasticache
* [InfluxDB](../managing-workflow/platform-monitoring.md#configuring-off-cluster-influxdb) and [Grafana](../managing-workflow/platform-monitoring.md#off-cluster-grafana)

#### (Experimental) Kubernetes Native Ingress

Workflow now offers [experimental native ingress](experimental-native-ingress.md) that will allow users to take advantage of native Kubernetes ingress with their cluster. Users will be able to use and define any compatible Kubernetes ingress controller. Feel free to start following along with the [experimental native ingress](experimental-native-ingress.md) guide now.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe make this paragraph a bit more direct?

Workflow now offers experimental native ingress to take advantage of native Kubernetes routing. Any compatible Kubernetes ingress controller can be used in place of Workflow's nginx-based deis-router. Follow this guide to enable experimental native ingress.


## Add the Deis Chart Repository
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add an extra newline here between the paragraph and the new heading? Just a small pet peeve of mine :)


The Deis Chart Repository contains everything needed to install Deis Workflow onto a Kubernetes cluster, with a single `helm install deis/workflow --namespace deis` command.
Expand Down
34 changes: 26 additions & 8 deletions src/quickstart/deploy-an-app.md
Original file line number Diff line number Diff line change
@@ -1,26 +1,44 @@
## Register an Admin User
## Determine your host and hostname values
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For consistency, I think we capitalize all the words here except for "and."


The first user to register against Deis Workflow will automatically be given administrative privileges.
For the rest of this example we will refer to a special variables called `$hostname`. Please choose one of the two methods for building your `$hostname`.

If you installed Deis on GKE or AWS, Deis automatically creates a load balancer for the cluster. To get the IP of this load balancer, run `kubectl --namespace=deis describe svc deis-router`.
#### Option 1: Standard Installation

For a standard installation you can build the hostname using public IP address and a wildcard DNS solution. Instead of setting up DNS records, this example will use `nip.io`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe:

"For a standard installation that includes deis-router, you can build the hostname using its public IP address and a wildcard DNS record."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is another spot where "build the hostname" sounds awkward to me.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I went ahead and

s/build the hostname/calculate the hostname value/g`

..if that helps?


If your router IP is `1.1.1.1`, its `$hostname` will be `1.1.1.1.nip.io`. You can find your IP address by running:

```
kubectl --namespace=deis describe svc deis-router
```

If you do not have an load balancer IP, the router automatically forwards traffic from a kubernetes node to the router. In this case, use the IP of a kubernetes node and the node
port that routes to port 80 on the controller.

Deis requires a wildcard DNS record to dynamically map app names to the router. Instead of setting up DNS records, this example will use `nip.io`. If your router IP is `1.1.1.1`, its url will be `1.1.1.1.nip.io`. The URL of the controller component will be `deis.1.1.1.1.nip.io`.
Deis requires a wildcard DNS record to dynamically map app names to the router.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Deis" -> "Deis Workflow"


#### Option 2: Experimental Native Ingress Installation

Use the controller url to register a user in the cluster.
In this example, the user should already have DNS set up pointing to their known host. The `$hostname` value can be build by appending `deis.` to the value set in `global.exerpimental_native_ingress`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should say "prepending" right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Typo: "exerpimental"

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... to the value set in global.exerpimental_native_ingress.

Should that be controller.platform_domain?


**$hostname**: deis.com

## Register an Admin User

The first user to register against Deis Workflow will automatically be given administrative privileges.

Use the controller `$hostname` to register a user in the cluster.

```
$ deis register http://deis.104.197.125.75.nip.io
$ deis register http://$hostname
username: admin
password:
password (confirm):
email: [email protected]
Registered admin
Logged in as admin
$ deis whoami
You are admin at http://deis.104.197.125.75.nip.io
You are admin at http://$hostname
```

You have now registered your first user and you are ready to deploy an application.
Expand Down Expand Up @@ -50,7 +68,7 @@ Let's use the CLI to tell the platform to deploy an application and then use cur
```
$ deis pull deis/example-go -a proper-barbecue
Creating build... done
$ curl http://proper-barbecue.104.197.125.75.nip.io
$ curl http://proper-barbecue.$hostname
Powered by Deis
```

Expand Down