VOL-1295: Deploy a read/write Voltha core pair to each Kubernetes node
Change-Id: I47e402c5c3cd91a0ef735d3f7cfd6c48d9053e0c
diff --git a/k8s/README.md b/k8s/README.md
new file mode 100644
index 0000000..9d9a36f
--- /dev/null
+++ b/k8s/README.md
@@ -0,0 +1,38 @@
+## How to deploy read/write core pairs on Kubernetes
+
+The current technique installs a separate rw-core deployment to each Kubernetes node, where each deployment consists of a pair (replicas = 2) of co-located rw-cores. Co-location is enforced by making use of the Kubernetes nodeSelector constraint applied at the pod spec level.
+
+In order for node selection to work, a label must be applied to each node. There is a set of built-in node labels that comes with Kubernetes out of the box, one of which is kubernetes.io/hostname. This label can be used to constrain the deployment of a core pair to a node with a specific hostname. Another approach is to take greater control and create new node labels.
+
+The following discussion assumes installation of the voltha-k8s-playground (https://github.com/ciena/voltha-k8s-playground) which configures three Kubernetes nodes named k8s1, k8s2, and k8s3.
+
+Create a "nodename" label for each Kubernetes node:
+```
+kubectl label nodes k8s1 nodename=k8s1
+kubectl label nodes k8s2 nodename=k8s2
+kubectl label nodes k8s3 nodename=k8s3
+```
+
+Verify that the labels have been applied:
+```
+kubectl get nodes --show-labels
+NAME STATUS ROLES AGE VERSION LABELS
+k8s1 Ready master,node 4h v1.9.5 ...,kubernetes.io/hostname=k8s1,nodename=k8s1
+k8s2 Ready node 4h v1.9.5 ...,kubernetes.io/hostname=k8s2,nodename=k8s2
+k8s3 Ready node 4h v1.9.5 ...,kubernetes.io/hostname=k8s3,nodename=k8s3
+```
+
+
+Ensure that a nodeSelector section appears in the deployment's pod spec (such a section should already exist in each manifest):
+```
+ ...
+ nodeSelector:
+ nodename: k8s1
+```
+
+Once the labels have been applied, deploy the 3 core pairs:
+```
+kubectl apply -f k8s/rw-core-pair1.yml
+kubectl apply -f k8s/rw-core-pair2.yml
+kubectl apply -f k8s/rw-core-pair3.yml
+```
diff --git a/k8s/rw-core-pair1.yml b/k8s/rw-core-pair1.yml
new file mode 100644
index 0000000..bae5092
--- /dev/null
+++ b/k8s/rw-core-pair1.yml
@@ -0,0 +1,80 @@
+# Copyright 2018 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: Service
+metadata:
+ name: rw-core1
+ namespace: voltha
+spec:
+ clusterIP: None
+ ports:
+ - name: grpc
+ port: 50057
+ targetPort: 50057
+ selector:
+ app: rw-core
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: rw-core1
+ namespace: voltha
+spec:
+ replicas: 2
+ selector:
+ matchLabels:
+ app: rw-core
+ template:
+ metadata:
+ labels:
+ app: rw-core
+ annotations:
+ cni: "calico"
+ spec:
+ containers:
+ - name: voltha
+ image: voltha-rw-core
+ env:
+ - name: NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ args:
+ - "/app/rw_core"
+ - "-kv_store_type=etcd"
+ - "-kv_store_host=etcd.$(NAMESPACE).svc.cluster.local"
+ - "-kv_store_port=2379"
+ - "-grpc_host=$(POD_IP)"
+ - "-grpc_port=50057"
+ - "-banner=true"
+ - "-kafka_adapter_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_adapter_port=9092"
+ - "-kafka_cluster_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_cluster_port=9092"
+ - "-rw_core_topic=rwcore"
+ - "-log_level=0"
+ ports:
+ - containerPort: 50057
+ name: grpc-port
+ imagePullPolicy: IfNotPresent
+ nodeSelector:
+ nodename: k8s1
+
+
+
diff --git a/k8s/rw-core-pair2.yml b/k8s/rw-core-pair2.yml
new file mode 100644
index 0000000..133f263
--- /dev/null
+++ b/k8s/rw-core-pair2.yml
@@ -0,0 +1,80 @@
+# Copyright 2018 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: Service
+metadata:
+ name: rw-core2
+ namespace: voltha
+spec:
+ clusterIP: None
+ ports:
+ - name: grpc
+ port: 50057
+ targetPort: 50057
+ selector:
+ app: rw-core
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: rw-core2
+ namespace: voltha
+spec:
+ replicas: 2
+ selector:
+ matchLabels:
+ app: rw-core
+ template:
+ metadata:
+ labels:
+ app: rw-core
+ annotations:
+ cni: "calico"
+ spec:
+ containers:
+ - name: voltha
+ image: voltha-rw-core
+ env:
+ - name: NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ args:
+ - "/app/rw_core"
+ - "-kv_store_type=etcd"
+ - "-kv_store_host=etcd.$(NAMESPACE).svc.cluster.local"
+ - "-kv_store_port=2379"
+ - "-grpc_host=$(POD_IP)"
+ - "-grpc_port=50057"
+ - "-banner=true"
+ - "-kafka_adapter_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_adapter_port=9092"
+ - "-kafka_cluster_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_cluster_port=9092"
+ - "-rw_core_topic=rwcore"
+ - "-log_level=0"
+ ports:
+ - containerPort: 50057
+ name: grpc-port
+ imagePullPolicy: IfNotPresent
+ nodeSelector:
+ nodename: k8s2
+
+
+
diff --git a/k8s/rw-core-pair3.yml b/k8s/rw-core-pair3.yml
new file mode 100644
index 0000000..9879f35
--- /dev/null
+++ b/k8s/rw-core-pair3.yml
@@ -0,0 +1,80 @@
+# Copyright 2018 the original author or authors.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: v1
+kind: Service
+metadata:
+ name: rw-core3
+ namespace: voltha
+spec:
+ clusterIP: None
+ ports:
+ - name: grpc
+ port: 50057
+ targetPort: 50057
+ selector:
+ app: rw-core
+---
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+ name: rw-core3
+ namespace: voltha
+spec:
+ replicas: 2
+ selector:
+ matchLabels:
+ app: rw-core
+ template:
+ metadata:
+ labels:
+ app: rw-core
+ annotations:
+ cni: "calico"
+ spec:
+ containers:
+ - name: voltha
+ image: voltha-rw-core
+ env:
+ - name: NAMESPACE
+ valueFrom:
+ fieldRef:
+ fieldPath: metadata.namespace
+ - name: POD_IP
+ valueFrom:
+ fieldRef:
+ fieldPath: status.podIP
+ args:
+ - "/app/rw_core"
+ - "-kv_store_type=etcd"
+ - "-kv_store_host=etcd.$(NAMESPACE).svc.cluster.local"
+ - "-kv_store_port=2379"
+ - "-grpc_host=$(POD_IP)"
+ - "-grpc_port=50057"
+ - "-banner=true"
+ - "-kafka_adapter_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_adapter_port=9092"
+ - "-kafka_cluster_host=kafka.$(NAMESPACE).svc.cluster.local"
+ - "-kafka_cluster_port=9092"
+ - "-rw_core_topic=rwcore"
+ - "-log_level=0"
+ ports:
+ - containerPort: 50057
+ name: grpc-port
+ imagePullPolicy: IfNotPresent
+ nodeSelector:
+ nodename: k8s3
+
+
+