[VOL-3387] Removing the TYPE parameter from kind-voltha
- Adding NUM_OF_WORKER_NODES to specify the kind cluster size
- Adding NUM_OF_KAFKA to scale up and down the kafka cluster
- Adding NUM_OF_ETCD to scale up and down the etcd cluster
Change-Id: I0359cd1e6d9b90b8d1d7590084805cd47236b9cf
diff --git a/.gitignore b/.gitignore
index e941ef0..9e0c7ec 100644
--- a/.gitignore
+++ b/.gitignore
@@ -3,14 +3,11 @@
onos-apps
pkg
src
-install.log
*.swp
*.log
logger/*
-voltha-debug-dump-minimal*
voltha-system-tests
voltha-debug-dump-*
-minimal-env.sh
-full-env.sh
+*-env.sh
.DS_Store
kube-config-*
diff --git a/Makefile b/Makefile
index e22fe44..ee3d79c 100644
--- a/Makefile
+++ b/Makefile
@@ -15,7 +15,7 @@
shellcheck --version
shellcheck $(SHELL_FILES)
-YAML_FILES ?= $(shell find . -type f \( -name '*.yaml' -o -name '*.yml' -o -name '*.cfg' \) -print )
+YAML_FILES ?= $(shell find . -type f \( -name '*.yaml' -o -name '*.yml' \) -print )
yamllint: ## lint check YAML files with yamllint
yamllint --version
yamllint \
diff --git a/README.md b/README.md
index 9c73f9b..d7a2db2 100644
--- a/README.md
+++ b/README.md
@@ -17,28 +17,18 @@
_NOTE: If you are deploying a KinD Kubernetes cluster using the `voltha`
script, you must also also have Docker installed_
-## INSTALL TYPE
-The `voltha` script can install two variations or types of a VOLTHA deployment:
-**minimal** or **full**. The difference is characterized in the following
-table:
+## NAMING YOUR CLUSTER
-| RESOURCE | MINIMAL | FULL |
-| ----------------------- | ------------- | ------------------------- |
-| K8s Control Plane Nodes | 1 | 1 |
-| K8s Workers | 2 | 3 |
-| EtcdOperator Components | Operator only | Operator, Backup, Restore |
-| EtcdCluster | 1 Member | 3 Members |
+The `voltha` script can be used to manage multiple clusters at the same time,
+thus it's important to always name your cluster.
-Throughout this `README.md` file deployment and configuration files are
-referenced in the form **$TYPE-cluster.cfg** and **$TYPE-values.yaml**.
-Depending on which type of deloyment you wish to install replace **$TYPE**
-with either **minimal** or **full**. If you set the environment variable `TYPE`
-to the desired deployment type, example below, then the commands can be
-executed via a simply copy and paste to your command line.
```bash
-export TYPE=minimal
+export NAME=minimal
```
+Throughout this `README.md` file you'll find multiple references to `$NAME`
+and that can always be replaced with the name your cluster (the default value is `local`).
+
## TL;DR
OK, if you really don't care how it starts and you just want it started. After
cloning the repository and making sure you have the prerequisites installed,
@@ -79,8 +69,7 @@
| OPTION | DEFAULT | DESCRIPTION |
| --------------------------------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------ |
-| `TYPE` | minimal | `minimal` or `full` and determines number of cluster nodes and etcd cluster members |
-| `NAME` | TYPE | Name of the KinD Cluster to be created |
+| `NAME` | `minimal` | Name of the KinD Cluster to be created |
| `DEPLOY_K8S` | yes | Should the KinD Kubernetes cluster be deployed? |
| `JUST_K8S` | no | Should just the KinD Kubernetes cluster be depoyed? (i.e. no VOLTHA) |
| `SCHEDULE_ON_CONTROL_NODES` | no | Untaint the control plane (master) K8s nodes so that PODs may be scheduled on them |
@@ -152,6 +141,15 @@
| `ELASTICSEARCH_PORT` | dynamic | (advanced) Override dynamic port selection for port forward for elasticsearch |
| `KIBANA_PORT` | dynamic | (advanced) Override dynamic port selection for port forward for kibana |
+### Managing the cluster(s) size
+The `voltha` script supports configuration for cluster of different sizes.
+For example you can setup an HA cluster with:
+
+```bash
+NUM_OF_WORKER_NODES=3 NUM_OF_ONOS=3 NUM_OF_KAFKA=3 NUM_OF_ETCD=3 NUM_OF_ATOMIX=3 ./voltha up
+```
+As usual one can add as many other options, for example `CONFIG_SADIS=y`.
+
### Custom Namespaces
Separate namespaces can be specified for various components
@@ -211,18 +209,19 @@
### Controlling VOLTHA with an ONOS cluster
To provide HA, resilinecy and failover ONOS can be configured in cluster mode.
-A 3 node ONOS deployment for example can be achieved via the `full` configuration:
+A 3 node ONOS deployment for example can be achieved via:
```bash
-TYPE=full WITH_ONOS=classic NUM_OF_ONOS=3 NUM_OF_ATOMIX=3 ./voltha up
+NUM_OF_WORKER_NODES=3 WITH_ONOS=classic NUM_OF_ONOS=3 NUM_OF_ATOMIX=3 ./voltha up
```
-Please note the `TYPE=full` that deploys a 3 worker nodes for k8s and `WITH_ONOS=classic` that uses the
+Please note the above command deploys a 3 worker nodes for k8s and `WITH_ONOS=classic` that uses the
cluster-enabled [helm chart](https://charts.onosproject.org) for ONOS (version 2.2).
The `NUM_OF_ONOS=3 NUM_OF_ATOMIX=3` flags set the number of Atomix nodes and ONOS nodes.
+
As usual one can add as many other options, for example `CONFIG_SADIS=y`.
## GENERATED CONFIGURATION
When the voltha script is run it generates a file that contains the
-configuration settings. This file will be named `$TYPE-env.sh`. The user can
+configuration settings. This file will be named `$NAME-env.sh`. The user can
`source` this file to set the configuration as well as establish key environment
variables in order to access VOLTHA, including:
@@ -235,7 +234,7 @@
After `voltha up` is run, it is useful to source this file.
## QUICK CHECK
-After source the `$TYPE-env.sh` file, you should be able to access VOLTHA via
+After source the `$NAME-env.sh` file, you should be able to access VOLTHA via
the control application `voltctl`. To validate this you can use the following
command
```bash
@@ -267,8 +266,8 @@
## TROUBLESHOOTING
When VOLTHA is installed the install log is written to the file
-`install-$TYPE.log`. If the install appears stalled or is not completing
+`install-$NAME.log`. If the install appears stalled or is not completing
consulting this file may indicate the reason.
-Similarly, when VOLTHA is uninstalled `down-$TYPE.log` is written and should
+Similarly, when VOLTHA is uninstalled `down-$NAME.log` is written and should
be consulted in the event of an error or unexpected result.
diff --git a/full-cluster.cfg b/full-cluster.cfg
deleted file mode 100644
index 6808558..0000000
--- a/full-cluster.cfg
+++ /dev/null
@@ -1,22 +0,0 @@
----
-# Copyright 2019 Ciena Corporation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-kind: Cluster
-apiVersion: kind.sigs.k8s.io/v1alpha3
-nodes:
- - role: control-plane
- - role: worker
- - role: worker
- - role: worker
diff --git a/minimal-cluster.cfg b/minimal-cluster.cfg
deleted file mode 100644
index 127a19a..0000000
--- a/minimal-cluster.cfg
+++ /dev/null
@@ -1,22 +0,0 @@
----
-# Copyright 2019 Ciena Corporation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-kind: Cluster
-apiVersion: kind.sigs.k8s.io/v1alpha3
-nodes:
- - role: control-plane
- extraPortMappings:
- - role: worker
- - role: worker
diff --git a/minimal-values.yaml b/minimal-values.yaml
deleted file mode 100644
index 3fb1e3b..0000000
--- a/minimal-values.yaml
+++ /dev/null
@@ -1,317 +0,0 @@
----
-# Copyright 2019 Ciena Corporation
-#
-# Licensed under the Apache License, Version 2.0 (the "License");
-# you may not use this file except in compliance with the License.
-# You may obtain a copy of the License at
-#
-# http://www.apache.org/licenses/LICENSE-2.0
-#
-# Unless required by applicable law or agreed to in writing, software
-# distributed under the License is distributed on an "AS IS" BASIS,
-# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-# See the License for the specific language governing permissions and
-# limitations under the License.
-
-# IMAGE SELECTION
-# ---------------
-# By default no defaults are set and the image tags specified in the helm
-# charts will be used. If you would like to use the images generated from the
-# HEAD of the "master" git branches, then uncomment this below block and
-# set "<VALUE>" to "master". Similarly, if you want to use the images
-# generated from the HEAD of the "voltha-2.1" git branches, then uncomment
-# this block and replace "<VALUE>" with "voltha-2.1".
-defaults:
- image_tag: master
- image_pullPolicy: Always
-
-private_etcd_cluster: false
-private_kafka_cluster: false
-
-# OPENONU-ADAPTER IMPLEMENTATION
-# ------------------------------
-# There are currently two implementations of OPENONU-ADAPTER: the original
-# written in Python and the reimplementation written in Go. The variable
-# `use_openonu_adapter_go` can be used to optionally use the new Go
-# implementation. This is done by setting that value to `true`.
-#
-# Along with this change you will also may need to set the Docker image
-# information below (search for START_OPENONU_ADAPTER_GO).
-use_openonu_adapter_go: false
-
-images:
- onos:
- repository: voltha/voltha-onos
- # IMAGE_SELECTION
- # ---------------
- # The helm chart used to deploy ONOS is the public ONOS helm chart so,
- # there is a need to specify the exact image repository and image tag.
- # If you would like to use the "master", "voltha-2.1", or other image
- # just replace the "tag" value below.
- tag: master
-
-# IMAGE_SELECTION
-# ---------------
-# Below are a list of all the images utilized by kind-voltha. This list is
-# provided as a conveinence if you would like to override on a per image
-# basis. If you are using the defaults, master, or voltha-2.1 branch there
-# is no need to utilize this list.
-# adapter_open_olt:
-# repository: voltha/voltha-openolt-adapter
-# tag: 2.1.1
-# adapter_open_onu:
-# repository: voltha/voltha-openonu-adapter
-# tag: 2.1.0
-# adapter_simulated_olt:
-# repository: voltha/voltha-adapter-simulated-olt
-# tag: 2.1.1
-# adapter_simulated_onu:
-# repository: voltha/voltha-adapter-simulated-onu
-# tag: 2.1.1
-# bbsim:
-# repository: voltha/bbsim
-# tag: master
-# ofagent:
-# repository: voltha/voltha-ofagent-go
-# tag: master
-# rw_core:
-# repository: voltha/voltha-rw-core
-# tag: 2.1.1
-# START_OPENONU_ADAPTER_GO - Uncomment the following block to use the
-# Go implementation of the openonu-adapter
-# adapter_open_onu_go:
-# repository: voltha-openonu-adapter-go
-# tag: master
-# END_OPENONU_ADAPTER_GO
-
-etcd:
- replicas: 1
- extraEnv:
- - name: ETCD_AUTO_COMPACTION_RETENTION
- value: "1"
- - name: ETCD_SNAPSHOT_COUNT
- value: "100000"
- - name: ETCD_HEARTBEAT_INTERVAL
- value: "100"
- - name: ETCD_ELECTION_TIMEOUT
- value: "1000"
- - name: ETCD_MAX_SNAPSHOTS
- value: "5"
- - name: ETCD_QUOTA_BACKEND_BYTES
- value: "0"
- - name: ETCD_MAX_REQUEST_BYTES
- value: "1572864"
- - name: ETCD_GRPC_KEEPALIVE_MIN_TIME
- value: "5s"
- - name: ETCD_GRPC_KEEPALIVE_TIMEOUT
- value: "5s"
- - name: ETCD_DEBUG
- value: "false"
-
-kafka:
- zookeeper:
- replicaCount: 1
- persistence:
- enabled: false
- replicas: 1
- persistence:
- enabled: false
- envOverrides:
- KAFKA_LOG4J_ROOT_LOGLEVEL: ERROR
- KAFKA_TOOLS_LOG4J_LOGLEVEL: ERROR
- # yamllint disable-line rule:line-length
- KAFKA_LOG4J.LOGGERS: "kafka.zookeeper=ERROR,state.change.logger=ERROR,kafka=ERROR,kafka.controller=ERROR"
- KAFKA_LOG_RETENTION_HOURS: 1
- KAFKA_DEFAULT_REPLICATION_FACTOR: 1
- KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
-
-# voltha:
-# fullNameOverride: voltha
-
-# open-olt:
-# fullNameOverride: voltha-adapter-openolt
-
-# open-onu:
-# fullNameOverride: voltha-adapter-openonu
-
-onos:
- image:
- repository: voltha/voltha-onos
- tag: master
- pullPolicy: Always
- onos_env:
- - name: POD_IP
- valueFrom:
- fieldRef:
- fieldPath: status.podIP
- - name: NAMESPACE
- valueFrom:
- fieldRef:
- fieldPath: metadata.namespace
- - name: ONOS_APPS
- value: "drivers,openflow-base,hostprovider"
- # No persistent volume in Atomix to have clean state for each re-deploy of ONOS
- atomix:
- persistence:
- enabled: false
-
-# Customization for BBSIM
-bbsim:
- nni: 1
- pon: 1
- onu: 1
- sadisFormat: "att"
- kafkaEventTopic: "bbsim"
- enableEvents: false
-
-# Customization for BBSIM SADIS Servier
-sadis:
- config:
- bbsim_sadis_server:
- sleep_time: 5s
-
-# START EFK Setup to push voltha logs
-# elasticstack config
-# ref: https://github.com/elastic/helm-charts/tree/7.7.0/elasticsearch
-elasticsearch:
- replicas: 1
- minimumMasterNodes: 1
- # set cpu and memory configuration
- resources:
- requests:
- cpu: "400m"
- memory: "1Gi"
- limits:
- cpu: "1000m"
- memory: "2Gi"
- # setup persistence volume.By default persistence volumeclaim is disabled
- volumeClaimTemplate:
- accessModes: ["ReadWriteOnce"]
- resources:
- requests:
- storage: 5Gi
- persistence:
- enabled: false
- # setup cluster health status as yellow
- clusterHealthCheckParams: "wait_for_status=yellow&timeout=1s"
-
-# kibana config
-# ref: https://github.com/elastic/helm-charts/tree/7.7.0/kibana
-kibana:
- elasticsearchHosts: "http://elasticsearch-master:9200"
-
-# fluentd-elasticsearch config
-# ref: https://github.com/kiwigrid/helm-charts/tree/master/charts/fluentd-elasticsearch
-fluentd-elasticsearch:
- elasticsearch:
- # set elasticsearch host
- hosts: ["elasticsearch-master:9200"]
- sslVerify: false
-
-# SHOCK THE MONKEY OR LET LOSE THE DOGS OF WAR
-# The VOLTHA charts have support for adding extra labels to deployments and
-# pods. These extra labels can be used to integrate with other utilities
-# such as kube-monkey to add a bit of chaos to the cluster. Below are some
-# settings that can be uncommented to opt-in VOLTHA deployments/pods to
-# kube-monkey. For example, if you want ALL deployments and pods to opt-in
-# then uncomment the `extra_deployment_labels` and `extra_pod_label` blocks.
-# If you want to be more selected then comment the blocks that pertain to the
-# targets you care about.
-
-# extra_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# extra_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# rw_core_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# rw_core_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# ofagent_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# ofagent_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# openolt_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# openolt_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# openonu_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# openonu_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# simolt_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# simolt_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# simonu_deployment_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
-#
-# simonu_pod_labels:
-# kube-monkey/enabled: enabled
-# kube-monkey/identifier: monkey-victim
-# kube-monkey/mtbf: 1
-# kube-monkey/kill-mode: fixed
-# kube-monkey/kill-value: 1
diff --git a/full-values.yaml b/values.yaml
similarity index 99%
rename from full-values.yaml
rename to values.yaml
index 23598d7..d58069b 100644
--- a/full-values.yaml
+++ b/values.yaml
@@ -85,7 +85,6 @@
# END_OPENONU_ADAPTER_GO
etcd:
- replicas: 3
extraEnv:
- name: ETCD_AUTO_COMPACTION_RETENTION
value: "1"
@@ -110,10 +109,8 @@
kafka:
zookeeper:
- replicaCount: 3
persistence:
enabled: false
- replicas: 3
persistence:
enabled: false
envOverrides:
diff --git a/voltha b/voltha
index f5c4403..cc81802 100755
--- a/voltha
+++ b/voltha
@@ -65,8 +65,7 @@
CIVIS="$(tput civis)"
fi
-TYPE=${TYPE:-minimal}
-NAME=${NAME:-$TYPE}
+NAME=${NAME:-minimal}
ENABLE_ONOS_EXTRANEOUS_RULES=${ENABLE_ONOS_EXTRANEOUS_RULES:-no}
INFRA_NS=${INFRA_NS:-default}
@@ -137,7 +136,11 @@
EXTRA_HELM_INSTALL_ARGS=${EXTRA_HELM_INSTALL_ARGS:-}
INTERNAL_EXTRA_HELM_INSTALL_ARGS=
NUM_OF_BBSIM=${NUM_OF_BBSIM:-1}
+NUM_OF_WORKER_NODES=${NUM_OF_WORKER_NODES:-2}
+NUM_OF_CONTROLLER_NODES=${NUM_OF_CONTROLLER_NODES:-1}
NUM_OF_OPENONU=${NUM_OF_OPENONU:-1}
+NUM_OF_KAFKA=${NUM_OF_KAFKA:-1}
+NUM_OF_ETCD=${NUM_OF_ETCD:-1}
MAX_NUM_OF_BBSIM=10
MAX_NUM_OF_OPENONU=10
LEGACY_BBSIM_INDEX=${LEGACY_BBSIM_INDEX:-no}
@@ -151,33 +154,6 @@
BBSIM_LABEL="-l app=bbsim"
NO_LABEL=
-# check number (range) of bbsim, max bbsim must not exceed 10 instances!
-# note: instances will be numbered from 0 to 9
-if [ "$1" == "up" ]; then
- if [ "$NUM_OF_BBSIM" -lt 1 ]; then
- >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of BBSIM instances. NUM_OF_BBSIM (${NUM_OF_BBSIM}) is less than 1${NORMAL}"
- exit 1
- fi
- if [ "$NUM_OF_BBSIM" -gt "$MAX_NUM_OF_BBSIM" ]; then
- >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of BBSIM instances. NUM_OF_BBSIM (${NUM_OF_BBSIM}) is greater than $MAX_NUM_OF_BBSIM${NORMAL}"
- exit 1
- fi
-
- if [ "$NUM_OF_OPENONU" -lt 1 ]; then
- >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of OPENONU instances. NUM_OF_OPENONU (${NUM_OF_OPENONU}) is less than 1${NORMAL}"
- exit 1
- fi
- if [ "$NUM_OF_OPENONU" -gt "$MAX_NUM_OF_OPENONU" ]; then
- >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of OPENONU instances. NUM_OF_OPENONU (${NUM_OF_OPENONU}) is greater than $MAX_NUM_OF_OPENONU${NORMAL}"
- exit 1
- fi
-fi
-# Verify TYPE setting
-if [ "$(echo ":minimal:full:" | grep -ic ":$TYPE:")" -eq 0 ]; then
- >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid \$TYPE value of '$TYPE'. Should be 'minimal' or 'full'${NORMAL}"
- exit 1
-fi
-
# checks to see if a given WORD is in the given LIST of words
function is_in() {
local WORD LIST
@@ -304,7 +280,6 @@
ALL_OPTIONS="\
NAME \
- TYPE \
$ALL_YES_NO \
$OPT_YES_NO \
WAIT_TIMEOUT \
@@ -316,6 +291,10 @@
BBSIM_SADIS_SERVER_CHART \
BBSIM_SADIS_SERVER_CHART_VERSION \
NUM_OF_BBSIM \
+ NUM_OF_WORKER_NODES \
+ NUM_OF_CONTROLLER_NODES \
+ NUM_OF_KAFKA \
+ NUM_OF_ETCD \
ELASTICSEARCH_CHART \
ELASTICSEARCH_CHART_VERSION \
KIBANA_CHART \
@@ -380,6 +359,59 @@
eval "$VAR"="$(normalize_yes_no "$VAR")"
done
+# check number (range) of bbsim, max bbsim must not exceed 10 instances!
+# note: instances will be numbered from 0 to 9
+if [ "$1" == "up" ]; then
+ if [ "$NUM_OF_BBSIM" -lt 1 ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of BBSIM instances. NUM_OF_BBSIM (${NUM_OF_BBSIM}) is less than 1${NORMAL}"
+ exit 1
+ fi
+ if [ "$NUM_OF_BBSIM" -gt "$MAX_NUM_OF_BBSIM" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of BBSIM instances. NUM_OF_BBSIM (${NUM_OF_BBSIM}) is greater than $MAX_NUM_OF_BBSIM${NORMAL}"
+ exit 1
+ fi
+
+ if [ "$NUM_OF_OPENONU" -lt 1 ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of OPENONU instances. NUM_OF_OPENONU (${NUM_OF_OPENONU}) is less than 1${NORMAL}"
+ exit 1
+ fi
+ if [ "$NUM_OF_OPENONU" -gt "$MAX_NUM_OF_OPENONU" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of OPENONU instances. NUM_OF_OPENONU (${NUM_OF_OPENONU}) is greater than $MAX_NUM_OF_OPENONU${NORMAL}"
+ exit 1
+ fi
+
+ # check that NUM_OF_KAFKA, NUM_OF_ONOS, NUM_OF_ATOMIX, NUM_OF_ETCD is:
+ # <= NUM_OF_WORKER_NODES + 1 if SCHEDULE_ON_CONTROL_NODES == y
+ # <= NUM_OF_WORKER_NODES if SCHEDULE_ON_CONTROL_NODES == n
+ SCHEDULABLE_NODES=$NUM_OF_WORKER_NODES
+
+ if [ "$SCHEDULE_ON_CONTROL_NODES" == "yes" ]; then
+ SCHEDULABLE_NODES=$((NUM_OF_CONTROLLER_NODES+NUM_OF_WORKER_NODES))
+ fi
+
+ NODES="SCHEDULE_ON_CONTROL_NODES: $SCHEDULE_ON_CONTROL_NODES, SCHEDULABLE_NODES: $SCHEDULABLE_NODES, NUM_OF_CONTROLLER_NODES: $NUM_OF_CONTROLLER_NODES, NUM_OF_WORKER_NODES: $NUM_OF_WORKER_NODES"
+
+ if [ ! "$NUM_OF_KAFKA" -le "$SCHEDULABLE_NODES" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of KAFKA replicas. NUM_OF_KAFKA (${NUM_OF_KAFKA}) is greater than the available nodes ($NODES)${NORMAL}"
+ exit 1
+ fi
+
+ if [ ! "$NUM_OF_ETCD" -le "$SCHEDULABLE_NODES" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of ETCD replicas. NUM_OF_ETCD (${NUM_OF_ETCD}) is greater than the available nodes ($NODES)${NORMAL}"
+ exit 1
+ fi
+
+ if [ ! "$NUM_OF_ATOMIX" -le "$SCHEDULABLE_NODES" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of ATOMIX replicas. NUM_OF_ATOMIX (${NUM_OF_ATOMIX}) is greater than the available nodes ($NODES)${NORMAL}"
+ exit 1
+ fi
+
+ if [ ! "$NUM_OF_ONOS" -le "$SCHEDULABLE_NODES" ]; then
+ >&2 echo -e "${RED}${BOLD}${ERROR}ERROR:${NORMAL}${RED} Invalid setting of ONOS replicas. NUM_OF_ONOS (${NUM_OF_ONOS}) is greater than the available nodes ($NODES)${NORMAL}"
+ exit 1
+ fi
+fi
+
# normalize non-truth value options
if is_in "$WITH_KAFKA" "e,external"; then
WITH_KAFKA="external"
@@ -1218,7 +1250,7 @@
_HELM_ARGS="--create-namespace"
fi
fi
- CMD=("helm $_HELM_COMMAND --debug --dry-run -f $NAME-values.yaml $CHART_ARGS $INTERNAL_EXTRA_HELM_INSTALL_ARGS $EXTRA_HELM_INSTALL_ARGS --set defaults.log_level=$VOLTHA_LOG_LEVEL --namespace $NAMESPACE $_HELM_ARGS $CHART_VERSION $EXTRA_HELM_FLAGS $_HELM_NAME_ARG $CHART")
+ CMD=("helm $_HELM_COMMAND --debug --dry-run -f values.yaml $CHART_ARGS $INTERNAL_EXTRA_HELM_INSTALL_ARGS $EXTRA_HELM_INSTALL_ARGS --set defaults.log_level=$VOLTHA_LOG_LEVEL --namespace $NAMESPACE $_HELM_ARGS $CHART_VERSION $EXTRA_HELM_FLAGS $_HELM_NAME_ARG $CHART")
${CMD[*]} 2>/dev/null | awk 'PRINT==1 {print}; /^USER-SUPPLIED VALUES:/ {PRINT = 1}; /^$/ {PRINT = 0}'
}
@@ -1276,13 +1308,13 @@
VALUES_FILE="$(mktemp)"
if [ "$FILTER" == "-" ]; then
- cp "$NAME-values.yaml" "$VALUES_FILE"
+ cp "values.yaml" "$VALUES_FILE"
elif [ "${FILTER:0:1}" == "+" ]; then
- cp "$NAME-values.yaml" "$VALUES_FILE"
- yq r - "${FILTER:1}" <"$NAME-values.yaml" >>"$VALUES_FILE"
+ cp "values.yaml" "$VALUES_FILE"
+ yq r - "${FILTER:1}" <"values.yaml" >>"$VALUES_FILE"
cat "$VALUES_FILE" >>"$LOG"
else
- yq r - "$FILTER" <"$NAME-values.yaml" >"$VALUES_FILE"
+ yq r - "$FILTER" <"values.yaml" >"$VALUES_FILE"
fi
if [ "$WITH_PPROF" == "yes" ]; then
PPROF_ARG="--set profiler.enabled=true"
@@ -1317,7 +1349,7 @@
espin "$INDENT" "$VERIFIED"
}
-echo "INSTALL TYPE: $TYPE" >> "$LOG"
+echo "INSTALL NAME: $NAME" >> "$LOG"
STIME="$(date +%s)"
if [ "$INSTALL_KUBECTL" == "no" ]; then
@@ -1514,22 +1546,28 @@
sspin
if [ "$HAVE" -eq 0 ]; then
espin "$NOT_VERIFIED"
- bspin - "Verify cluster configuration"
- if [ ! -r "./$NAME-cluster.cfg" ]; then
- espin - "$NOT_VERIFIED"
- bspin - "Download cluster configuration: $TYPE-cluster.cfg to $NAME-cluster.cfg $DOWNLOAD"
- ERR_OUT="$(mktemp)"
- if ! (set -x; curl --fail -o "./$NAME-cluster.cfg" -sSL "https://raw.githubusercontent.com/opencord/kind-voltha/$VK_RELEASE/$TYPE-cluster.cfg" >>"$LOG" 2>>"$ERR_OUT") >>"$LOG" 2>&1; then
- espin - "$THEX"
- echo -e "${RED}${BOLD}${ERROR}ERROR: $NAME-cluster.cfg${NORMAL}${RED} - $(cat "$ERR_OUT")${NORMAL}"
- echo "ERROR: $(cat "$ERR_OUT")" >>"$LOG"
- rm -rf "$ERR_OUT" "./$NAME-cluster.cfg"
- exit 1
- fi
- rm -rf "$ERR_OUT"
- else
- espin - "$VERIFIED"
+ bspin - "Generating cluster configuration"
+
+ touch "$NAME-cluster.cfg"
+
+ yq w -i "$NAME-cluster.cfg" kind Cluster
+ yq w -i "$NAME-cluster.cfg" apiVersion "kind.sigs.k8s.io/v1alpha3"
+
+ if [ ! "$NUM_OF_CONTROLLER_NODES" -eq 0 ]; then
+ for instance in $(seq 1 "$NUM_OF_CONTROLLER_NODES"); do
+ yq w -i "$NAME-cluster.cfg" "nodes[+].role" "control-plane"
+ done
fi
+
+ if [ ! "$NUM_OF_WORKER_NODES" -eq 0 ]; then
+ for instance in $(seq 1 "$NUM_OF_WORKER_NODES"); do
+ yq w -i "$NAME-cluster.cfg" "nodes[+].role" worker
+ done
+ fi
+ espin - "$VERIFIED"
+
+ cat "$NAME-cluster.cfg" >> "$LOG" 2>&1
+
kind create cluster --name "voltha-$NAME" --config "$NAME-cluster.cfg"
else
espin "$VERIFIED"
@@ -1550,9 +1588,14 @@
kube-proxy-.* \
kube-scheduler-voltha-$NAME-control-plane"
- EXPECT="$(test "$TYPE" == "minimal" && echo "12" || echo "14")"
+ EXPECT="8"
+ if [ ! "$NUM_OF_WORKER_NODES" -eq 0 ]; then
+ NUM=$((2*NUM_OF_WORKER_NODES)) # kindnet and kube-proxy are deployed on each node
+ EXPECT=$((EXPECT+NUM))
+ fi
wait_for_pods - "kube-system" "$EXPECT" "includes" "Waiting for system PODs to start" "$NO_LABEL" "$P"
fi
+
if [ "$WITH_TIMINGS" == "yes" ]; then
NOW="$(date +%s)"
printtime $((NOW - STIME))
@@ -1692,16 +1735,16 @@
EOV
STIME="$(date +%s)"
-bspin "Verify Helm values file: $NAME-values.yaml"
-if [ ! -r "./$NAME-values.yaml" ]; then
+bspin "Verify Helm values file: values.yaml"
+if [ ! -r "./values.yaml" ]; then
espin "$NOT_VERIFIED"
- bspin - "Download Helm values file: $TYPE-values.yaml to $NAME-values.yaml $DOWNLOAD"
+ bspin - "Download Helm values file: values.yaml to values.yaml $DOWNLOAD"
ERR_OUT="$(mktemp)"
- if ! (set -x; curl --fail -o "./$NAME-values.yaml" -sSL "https://raw.githubusercontent.com/opencord/kind-voltha/$VK_RELEASE/$TYPE-values.yaml" >>"$LOG" 2>"$ERR_OUT") >>"$LOG" 2>&1; then
+ if ! (set -x; curl --fail -o "./values.yaml" -sSL "https://raw.githubusercontent.com/opencord/kind-voltha/$VK_RELEASE/values.yaml" >>"$LOG" 2>"$ERR_OUT") >>"$LOG" 2>&1; then
espin - "$THEX"
- echo -e "${RED}${BOLD}${ERROR}ERROR: $NAME-values.yaml${NORMAL}${RED} - $(cat "$ERR_OUT")${NORMAL}"
+ echo -e "${RED}${BOLD}${ERROR}ERROR: values.yaml${NORMAL}${RED} - $(cat "$ERR_OUT")${NORMAL}"
echo "ERROR: $(cat "$ERR_OUT")" >>"$LOG"
- rm -rf "$ERR_OUT" "./$NAME-values.yaml"
+ rm -rf "$ERR_OUT" "./values.yaml"
exit 1
fi
rm -rf "$ERR_OUT"
@@ -1997,10 +2040,11 @@
if is_in "$WITH_ETCD" "yes,external"; then
STIME="$(date +%s)"
bspin "Verify external ETCD cluster $OLD_KEY"
- EXPECT="$(yq r - "etcd.replicas" <"$NAME-values.yaml")"
+ EXPECT=$NUM_OF_ETCD
if [ "$HELM_USE_UPGRADE" == "yes" ] || [ "$(helm_is_deployed "$INFRA_NS" "^etcd\$")" -ne 1 ]; then
espin "$NOT_VERIFIED"
- helm_install - "$INFRA_NS" etcd "$ETCD_CHART" "$ETCD_CHART_VERSION" etcd "$_HELM_DESC external ETCD cluster"
+ INTERNAL_EXTRA_HELM_INSTALL_ARGS="--set replicas=$NUM_OF_ETCD" helm_install - "$INFRA_NS" etcd "$ETCD_CHART" "$ETCD_CHART_VERSION" etcd "$_HELM_DESC external ETCD cluster"
+ INTERNAL_EXTRA_HELM_INSTALL_ARGS=
else
espin "$VERIFIED"
fi
@@ -2013,24 +2057,23 @@
if is_in "$WITH_KAFKA" "yes,external"; then
_TMP="$(mktemp -u)"
- _COUNT="$(test "$TYPE" == "minimal" && echo "1" || echo "3")"
cat << EOC > "$_TMP"
configurationOverrides:
- "default.replication.factor": $_COUNT
- "offsets.topic.replication.factor": $_COUNT
+ "default.replication.factor": $NUM_OF_KAFKA
+ "offsets.topic.replication.factor": $NUM_OF_KAFKA
"log.retention.hours": 4
"log.message.timestamp.type": "LogAppendTime"
persistence:
enabled: false
zookeeper:
- replicaCount: 1
+ replicaCount: $NUM_OF_KAFKA
persistence:
enabled: false
-replicas: $_COUNT
+replicas: $NUM_OF_KAFKA
EOC
STIME="$(date +%s)"
bspin "Verify external Kafka cluster $OLD_KEY"
- EXPECT="$(test "$TYPE" == "minimal" && echo "2" || echo "4")"
+ EXPECT=$((NUM_OF_KAFKA*2))
if [ "$HELM_USE_UPGRADE" == "yes" ] || [ "$(helm_is_deployed "$INFRA_NS" "^kafka\$")" -ne 1 ]; then
espin "$NOT_VERIFIED"
INTERNAL_EXTRA_HELM_INSTALL_ARGS+=" -f $_TMP"