blob: 615c5388093c5f03cdd1266bcf0969b22f111e1e [file] [log] [blame]
hwchiu20f35d92019-11-05 05:22:58 +00001#Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15*** Settings ***
hwchiu85695932019-12-18 08:05:25 +000016Documentation Provide the function to perform system related test
hwchiu20f35d92019-11-05 05:22:58 +000017Library OperatingSystem
hwchiu85695932019-12-18 08:05:25 +000018Resource ../../libraries/k8s.robot
hwchiu20f35d92019-11-05 05:22:58 +000019
20*** Variables ***
21${timeout} 60s
hwchiu85695932019-12-18 08:05:25 +000022${desired_ETCD_cluster_size} 3
23${minimal_ETCD_cluster_size} 2
hwchiu20f35d92019-11-05 05:22:58 +000024${namespace} voltha
hwchiu74380812020-01-08 18:05:18 +080025${ETCD_resources} etcdclusters.etcd.database.coreos.com
26${ETCD_name} voltha-etcd-cluster
27${ETCD_pod_label_key} etcd_cluster
hwchiu85695932019-12-18 08:05:25 +000028${common_pod_label_key} app
29${rwcore_pod_label_value} rw-core
30${ofagent_pod_label_value} ofagent
31${adapter_openolt_pod_label_value} adapter-open-olt
hwchiu20f35d92019-11-05 05:22:58 +000032
33*** Test Cases ***
hwchiu85695932019-12-18 08:05:25 +000034Scale Down ETCD Cluster
35 [Documentation] Scale Down the ETCD cluster to minimal size, skip test if current cluster size < 3
36 [Tags] scaledown ETCDdown
37 ${current_size}= Get ETCD Running Size voltha
38 Pass Execution If '${current_size}' != '${desired_ETCD_cluster_size}'
hwchiu20f35d92019-11-05 05:22:58 +000039 ... 'Skip the test if the cluster size smaller than minimal size 3'
40 # The minimal cluster size after scale down
hwchiu85695932019-12-18 08:05:25 +000041 # based on https://github.com/ETCD-io/ETCD/blob/master/Documentation/faq.md#what-is-failure-tolerance
42 Scale ETCD ${namespace} ${minimal_ETCD_cluster_size}
43 Wait Until Keyword Succeeds ${timeout} 2s Validate ETCD Size ${namespace} ${minimal_ETCD_cluster_size}
hwchiu20f35d92019-11-05 05:22:58 +000044
hwchiu85695932019-12-18 08:05:25 +000045Scale Up ETCD Cluster
46 [Documentation] Recover the ETCD cluster by scaling up its size
47 [Tags] scaleup ETCDup
48 ${current_size}= Get ETCD Running Size voltha
49 Pass Execution If '${current_size}' != '${minimal_ETCD_cluster_size}'
hwchiu20f35d92019-11-05 05:22:58 +000050 ... 'Skip the test if the cluster size smaller than minimal size 3'
hwchiu85695932019-12-18 08:05:25 +000051 Scale ETCD ${namespace} ${desired_ETCD_cluster_size}
52 # We scale up the size to 3, the recommended size of ETCD cluster.
53 Wait Until Keyword Succeeds ${timeout} 2s Validate ETCD Size ${namespace} ${desired_ETCD_cluster_size}
54
55ETCD Failure Test
56 [Documentation] Failure Scenario Test: ETCD Crash
57 [Tags] FailureTest
58 Delete K8s Pods By Label ${namespace} ${ETCD_pod_label_key} ${ETCD_name}
59 Wait Until Keyword Succeeds ${timeout} 2s
60 ... Pods Does Not Exist By Label ${namespace} ${ETCD_pod_label_key} ${ETCD_name}
61 Wait Until Keyword Succeeds ${timeout} 2s
62 ... Pods Does Not Ready By Label ${namespace} ${common_pod_label_key} ${rwcore_pod_label_value}
63 Wait Until Keyword Succeeds ${timeout} 2s
64 ... Pods Does Not Ready By Label ${namespace} ${common_pod_label_key} ${ofagent_pod_label_value}
65 Wait Until Keyword Succeeds ${timeout} 2s
66 ... Pods Does Not Ready By Label ${namespace} ${common_pod_label_key} ${adapter_openolt_pod_label_value}
hwchiu20f35d92019-11-05 05:22:58 +000067
68*** Keywords ***
hwchiu85695932019-12-18 08:05:25 +000069Get ETCD Running Size
hwchiu20f35d92019-11-05 05:22:58 +000070 [Arguments] ${namespace}
hwchiu85695932019-12-18 08:05:25 +000071 [Documentation] Get the number of running ETCD nodes
hwchiu20f35d92019-11-05 05:22:58 +000072 ${rc} ${size}= Run and Return Rc and Output
hwchiu85695932019-12-18 08:05:25 +000073 ... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.size}'
hwchiu20f35d92019-11-05 05:22:58 +000074 Should Be Equal As Integers ${rc} 0
75 [Return] ${size}
76
hwchiu85695932019-12-18 08:05:25 +000077Scale ETCD
hwchiu20f35d92019-11-05 05:22:58 +000078 [Arguments] ${namespace} ${size}
hwchiu85695932019-12-18 08:05:25 +000079 [Documentation] Scale down the number of ETCD pod
hwchiu20f35d92019-11-05 05:22:58 +000080 ${rc}= Run and Return Rc
hwchiu85695932019-12-18 08:05:25 +000081 ... kubectl -n ${namespace} patch ${ETCD_resources} ${ETCD_name} --type='merge' -p '{"spec":{"size":${size}}}'
hwchiu20f35d92019-11-05 05:22:58 +000082 Should Be Equal As Integers ${rc} 0
83
hwchiu85695932019-12-18 08:05:25 +000084Validate ETCD Size
85 [Arguments] ${namespace} ${ETCD_cluster_size}
86 [Documentation] Scale down the number of ETCD pod
hwchiu20f35d92019-11-05 05:22:58 +000087 ${rc} ${size}= Run and Return Rc and Output
hwchiu85695932019-12-18 08:05:25 +000088 ... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.size}'
89 Should Be Equal As Integers ${size} ${ETCD_cluster_size}