VOL-3958 support for ingress in helm charts
Change-Id: I8c53016debb11760dd8ae39e8662ba0489d705a9
diff --git a/README.md b/README.md
index c1af707..2be66b4 100644
--- a/README.md
+++ b/README.md
@@ -146,7 +146,7 @@
> ```
If you add a different number of ONOS you need also to tell the `ofagent` to connect to all of them
-by adding it to the `voltha-stack` command. The following is an example for 3 ONOS instances.
+by adding it to the `voltha-stack` command. The following is an example for 3 ONOS instances.
```shell
helm upgrade --install --create-namespace \
-n voltha1 voltha1 onf/voltha-stack \
@@ -273,6 +273,106 @@
helm upgrade --install --create-namespace -n voltha1 voltha1 onf/voltha-stack --set global.stack_name=voltha1 --set voltha_infra_name=voltha-infra --set voltha_infra_namespace=infra
```
+### Using an ingress controller
+
+A process to expose the VOLTHA API external to the Kubernetes cluster was described
+using the `port-forward` option from the `kubectl` command line tool. This mechanism,
+while convenient, is not recommended for a production deployment. For a production
+deployment a [Kubernetes Ingress Controller](https://kubernetes.io/docs/concepts/services-networking/ingress/) is recommended.
+
+There are many choices when deploying an ingress controller including open source and
+commercial options. This document does not recommend or require a specific
+ingress controller, but it is important to note that VOLTHA has only been tested
+using the [NGINX Ingress Controller](https://kubernetes.github.io/ingress-nginx/)
+and thus examples for Kubernetes manifests are offered in that context.
+
+To deploy the NGINX ingress controller please see the [instructions](https://kubernetes.github.io/ingress-nginx/deploy/) on the NGINX site. For instructions on deploying
+an ingress controller when using a kind cluster, as is common for VOLTHA development,
+please see the [instructions](https://kind.sigs.k8s.io/docs/user/ingress/) on the
+kind site.
+
+#### VOLTHA Helm Charts
+
+The VOLTHA Helm charts were updated to deploy Ingress resources for the etcd
+and VOLTHA API services. By default these Ingress resources are disabled and
+not deployed.
+
+By default the Ingress resources are defined without a `host` option. This
+is convenient when running a single stack but is not sufficient when
+running multiple stacks and enabling external access to both stacks through
+the ingress controller.
+
+When using multiple stacks and an Ingress controller virtual hosts will be
+required to differentiate the stacks and direct API requests to the correct
+service. If `--set voltha.ingress.enableVirtualHosts=true` is used when
+installing the voltha stack then a virtual host will be defined using
+`<stack-name>.voltha.local`. In order to use this virtual host this virtual
+host name will need to resolve to the IP address of your Ingress controller.
+
+To include the Ingress resources when deploying VOLTHA the following
+values need to be set:
+
+##### When Installing Infra
+
+`--set etcd.ingress.enabled=true`
+
+##### When Installing the VOLTHA Stack
+
+`--set voltha.ingress.enabled=true`
+
+#### Usage with `voltctl`
+
+Once the Ingress resources are defined `voltctl` should be able to access the
+VOLTHA deployment without having to run `kubectl port-forward` commands for
+each specific service. The configuration to `voltctl` to utilize the ingress
+controller can be specified either via the command line options or via the
+voltctl configuration file (default `~/.volt/config`).
+
+The important settings are
+
+**NOTE:** _Note hostname and port will vary depending on how you deploy as
+well as how the Ingress controller is accessed from outside the cluster._
+
+- `--server`/`server` - the value of the external IP address of the ingress
+controller and the port on which it is listening (_ex:_ `localhost:443` or
+`voltha1.voltha.com:30474`).
+
+- `--kvstore`/`kvstore` - the value of the external IP address of the ingress
+controller and the point on which it is listening (_ex:_ `localhost:443`)
+
+- `--tls`/`tls.useTls` - indicates that TLS should be used when communicating
+through the ingress controller, which is required for the NGINX ingress controller
+(_ex:_ `true`)
+
+_CLI example_:
+
+```bash
+voltctl --server=voltha1.voltha.com:443 --kvstore=localhost:443 --tls version
+```
+
+_`voltctl` configuration file example:_
+
+```yaml
+server: voltha1.example.com:443
+kafka: localhost:443
+kvstore: localhost:443
+tls:
+ useTls: true
+ caCert: ""
+ cert: ""
+ key: ""
+ verify: false
+grpc:
+ timeout: 5m0s
+ maxCallRecvMsgSize: 4M
+kvstoreconfig:
+ timeout: 5s
+```
+
+#### Ingress Controller References
+
+1. [Guide to setting up ingress on a kind cluster.](https://kind.sigs.k8s.io/docs/user/ingress/)
+
## Post installation
Ok, now I have VOLTHA installed and everything is running.
@@ -409,9 +509,9 @@
### Develop with latest code
-The voltha-infra and voltha-stack charts pull the referenced version of every component, if your charts are up to date
-that will be the latest released one.
-If for your development and testing you'd like to have all the `master` of each component, independently if that has
+The voltha-infra and voltha-stack charts pull the referenced version of every component, if your charts are up to date
+that will be the latest released one.
+If for your development and testing you'd like to have all the `master` of each component, independently if that has
been officially tagged and release there is a provided `dev-values.yaml`. You can use it for the a `voltha-stack` like so:
```shell
helm upgrade --install --create-namespace \
@@ -421,8 +521,8 @@
--set voltha_infra_namespace=infra \
-f examples/dev-values.yaml
```
-After the feature you are working on is released by modifying the `VERSION` file and removing `-dev` you
-can remove the `dev-values.yaml` file from your helm command.
+After the feature you are working on is released by modifying the `VERSION` file and removing `-dev` you
+can remove the `dev-values.yaml` file from your helm command.
### Test changes to a chart
diff --git a/voltha-infra/Chart.yaml b/voltha-infra/Chart.yaml
index b739443..02b9783 100644
--- a/voltha-infra/Chart.yaml
+++ b/voltha-infra/Chart.yaml
@@ -29,7 +29,7 @@
name: voltha-infra
appVersion: "2.8-dev"
-version: 0.2.0-dev
+version: 0.2.0
dependencies:
- name: onos-classic
diff --git a/voltha-infra/templates/ingress_etcd.yaml b/voltha-infra/templates/ingress_etcd.yaml
new file mode 100644
index 0000000..9fa1bee
--- /dev/null
+++ b/voltha-infra/templates/ingress_etcd.yaml
@@ -0,0 +1,64 @@
+# Copyright 2021-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+{{- if .Values.etcd.ingress.enabled -}}
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+ name: "{{ .Release.Name }}-etcd"
+ namespace: {{ .Release.Namespace }}
+ {{- with .Values.etcd.labels }}
+ labels:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+ {{- with .Values.etcd.ingress.annotations }}
+ annotations:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+spec:
+ {{- if .Values.etcd.ingress.tls }}
+ tls:
+ {{- range .Values.etcd.ingress.tls }}
+ - hosts:
+ {{- range .hosts }}
+ - {{ . | quote }}
+ {{- end }}
+ secretName: {{ .secretName }}
+ {{- end }}
+ {{- end }}
+ rules:
+ {{- range .Values.etcd.ingress.hosts }}
+ {{- if $.Values.etcd.ingress.enableVirtualHosts }}
+ - host: '{{ $.Release.Name }}.local'
+ http:
+ {{- else }}
+ {{- if .host }}
+ - host: {{ .host | quote }}
+ http:
+ {{- else }}
+ - http:
+ {{- end }}
+ {{- end }}
+ paths:
+ {{- range .paths }}
+ - path: {{ . }}
+ pathType: Prefix
+ backend:
+ service:
+ name: "{{ $.Release.Name }}-etcd"
+ port:
+ number: 2379
+ {{- end }}
+ {{- end }}
+{{- end }}
diff --git a/voltha-infra/values.yaml b/voltha-infra/values.yaml
index 113e1c4..576507b 100644
--- a/voltha-infra/values.yaml
+++ b/voltha-infra/values.yaml
@@ -50,6 +50,18 @@
etcd:
enabled: true
+ ingress:
+ enabled: false
+ annotations:
+ kubernetes.io/ingress.class: "nginx"
+ nginx.ingress.kubernetes.io/ssl-redirect: "true"
+ nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
+ nginx.ingress.kubernetes.io/grpc-backend: "true"
+ enableVirtualHosts: false
+ hosts:
+ - host:
+ paths:
+ - "/etcdserverpb.KV/"
auth:
rbac:
enabled: false
diff --git a/voltha-stack/Chart.yaml b/voltha-stack/Chart.yaml
index 6d0f1fe..27f9143 100644
--- a/voltha-stack/Chart.yaml
+++ b/voltha-stack/Chart.yaml
@@ -28,12 +28,12 @@
name: voltha-stack
appVersion: "2.8-dev"
-version: 0.2.0-dev
+version: 0.2.0
dependencies:
- name: voltha
repository: file://../voltha
- version: 2.8.3
+ version: 2.9.0
condition: voltha.enabled
- name: voltha-adapter-openonu
repository: file://../voltha-adapter-openonu
diff --git a/voltha-stack/values.yaml b/voltha-stack/values.yaml
index 0574d6c..dea3bad 100644
--- a/voltha-stack/values.yaml
+++ b/voltha-stack/values.yaml
@@ -25,6 +25,9 @@
voltha:
enabled: true
+ ingress:
+ enabled: false
+ enableVirtualHosts: false
rw_core:
kv_store_data_prefix: 'service/voltha/{{ .Release.Name }}_{{ .Values.global.stack_name }}'
topics:
diff --git a/voltha/Chart.yaml b/voltha/Chart.yaml
index 074d271..7a7e54f 100644
--- a/voltha/Chart.yaml
+++ b/voltha/Chart.yaml
@@ -14,7 +14,7 @@
---
apiVersion: "v1"
name: "voltha"
-version: "2.9.0-dev"
+version: "2.9.0"
description: "A Helm chart for Voltha based on K8S resources in Voltha project"
keywords:
- "onf"
diff --git a/voltha/templates/_helpers.tpl b/voltha/templates/_helpers.tpl
index bf89b45..0fd902b 100644
--- a/voltha/templates/_helpers.tpl
+++ b/voltha/templates/_helpers.tpl
@@ -21,3 +21,9 @@
{{- $fullname := default (printf "%s-%s" .Release.Name $name) .Values.fullNameOverride -}}
{{- $fullname | trunc 63 | trimSuffix "-" -}}
{{- end -}}
+{{/* Create a default fully qualified virtual hostname. We truncate at 63 chars because . . . */}}
+{{- define "virtual-hostname" -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- $fullname := default (printf "%s.%s.local" .Values.global.stack_name $name) .Values.fullNameOverride -}}
+{{- $fullname | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
diff --git a/voltha/templates/ingress.yaml b/voltha/templates/ingress.yaml
new file mode 100644
index 0000000..69d161c
--- /dev/null
+++ b/voltha/templates/ingress.yaml
@@ -0,0 +1,64 @@
+# Copyright 2021-present Open Networking Foundation
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+{{- if .Values.ingress.enabled -}}
+---
+apiVersion: networking.k8s.io/v1
+kind: Ingress
+metadata:
+ name: {{ template "fullname" . }}
+ namespace: {{ .Release.Namespace }}
+ {{- with .Values.labels }}
+ labels:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+ {{- with .Values.ingress.annotations }}
+ annotations:
+ {{- toYaml . | nindent 4 }}
+ {{- end }}
+spec:
+ {{- if .Values.ingress.tls }}
+ tls:
+ {{- range .Values.ingress.tls }}
+ - hosts:
+ {{- range .hosts }}
+ - {{ . | quote }}
+ {{- end }}
+ secretName: {{ .secretName }}
+ {{- end }}
+ {{- end }}
+ rules:
+ {{- range .Values.ingress.hosts }}
+ {{- if $.Values.ingress.enableVirtualHosts }}
+ - host: '{{ template "virtual-hostname" $ }}'
+ http:
+ {{- else }}
+ {{- if .host }}
+ - host: {{ .host | quote }}
+ http:
+ {{- else }}
+ - http:
+ {{- end }}
+ {{- end }}
+ paths:
+ {{- range .paths }}
+ - path: {{ . }}
+ pathType: Prefix
+ backend:
+ service:
+ name: "{{ template "fullname" $ }}-api"
+ port:
+ number: 55555
+ {{- end }}
+ {{- end }}
+{{- end }}
diff --git a/voltha/values.yaml b/voltha/values.yaml
index 433a67e..e7e5879 100644
--- a/voltha/values.yaml
+++ b/voltha/values.yaml
@@ -99,3 +99,17 @@
repository: '{{ .Values.global.image_org }}voltha-rw-core'
tag: '{{- if hasKey .Values.global "image_tag" }}{{- if .Values.global.image_tag }}{{ .Values.global.image_tag }}{{- else }}2.7.0{{- end }}{{- else }}2.7.0{{- end }}'
pullPolicy: '{{ .Values.global.image_pullPolicy }}'
+
+ingress:
+ enabled: false
+ annotations:
+ kubernetes.io/ingress.class: "nginx"
+ nginx.ingress.kubernetes.io/ssl-redirect: "true"
+ nginx.ingress.kubernetes.io/backend-protocol: "GRPC"
+ nginx.ingress.kubernetes.io/grpc-backend: "true"
+ enableVirtualHosts: false
+ hosts:
+ - host:
+ paths:
+ - "/voltha.VolthaService/"
+ tls: []