blob: 74f8ad5174380df61ea06d9eae5ad66ac53ed82d [file] [log] [blame]
hwchiu20f35d92019-11-05 05:22:58 +00001#Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15*** Settings ***
hwchiu85695932019-12-18 08:05:25 +000016Documentation Provide the function to perform system related test
hwchiuab524f02020-02-03 23:24:19 +000017Suite Setup Common Test Suite Setup
18Test Setup Setup
19Test Teardown Teardown
hwchiuab524f02020-02-03 23:24:19 +000020Resource ../../libraries/onos.robot
21Resource ../../libraries/voltctl.robot
22Resource ../../libraries/voltha.robot
23Resource ../../libraries/utils.robot
hwchiu85695932019-12-18 08:05:25 +000024Resource ../../libraries/k8s.robot
hwchiuab524f02020-02-03 23:24:19 +000025Resource ../../variables/variables.robot
hwchiu20f35d92019-11-05 05:22:58 +000026
27*** Variables ***
hwchiu58af72d2020-01-14 00:50:35 +000028${timeout} 120s
Zack Williamsa8fe75a2020-01-10 14:25:27 -070029${desired_ETCD_cluster_size} 3
30${minimal_ETCD_cluster_size} 2
31${namespace} voltha
32${ETCD_resources} etcdclusters.etcd.database.coreos.com
33${ETCD_name} voltha-etcd-cluster
hwchiu74380812020-01-08 18:05:18 +080034${ETCD_pod_label_key} etcd_cluster
hwchiu85695932019-12-18 08:05:25 +000035${common_pod_label_key} app
36${rwcore_pod_label_value} rw-core
37${ofagent_pod_label_value} ofagent
38${adapter_openolt_pod_label_value} adapter-open-olt
hwchiu20f35d92019-11-05 05:22:58 +000039
40*** Test Cases ***
hwchiuab524f02020-02-03 23:24:19 +000041ECTD Scale Test
42 [Documentation] Perform the sanity test if some ETCD endpoints crash
43 [Tags] functional bbsim
hwchiu3d9a0982020-02-06 00:19:19 +000044 [Setup] Run Keywords Announce Message START TEST SanityTest
45 ... AND Setup
46 [Teardown] Run Keywords Collect Logs
47 ... AND Announce Message END TEST SanityTest
48 ... AND Teardown Suite
Zack Williamsa8fe75a2020-01-10 14:25:27 -070049 ${current_size}= Get ETCD Running Size voltha
hwchiu85695932019-12-18 08:05:25 +000050 Pass Execution If '${current_size}' != '${desired_ETCD_cluster_size}'
hwchiu20f35d92019-11-05 05:22:58 +000051 ... 'Skip the test if the cluster size smaller than minimal size 3'
52 # The minimal cluster size after scale down
hwchiu85695932019-12-18 08:05:25 +000053 # based on https://github.com/ETCD-io/ETCD/blob/master/Documentation/faq.md#what-is-failure-tolerance
54 Scale ETCD ${namespace} ${minimal_ETCD_cluster_size}
hwchiu58af72d2020-01-14 00:50:35 +000055 Wait Until Keyword Succeeds ${timeout} 2s
Zack Williamsa8fe75a2020-01-10 14:25:27 -070056 ... Validate ETCD Size ${namespace} ${minimal_ETCD_cluster_size}
57 Wait Until Keyword Succeeds ${timeout} 2s
58 ... Check Expected Running Pods Number By Label ${namespace}
59 ... ${ETCD_pod_label_key} ${ETCD_name} 2
hwchiuab524f02020-02-03 23:24:19 +000060 # Perform the sanity-test
61 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
hwchiu85695932019-12-18 08:05:25 +000062 # We scale up the size to 3, the recommended size of ETCD cluster.
hwchiuab524f02020-02-03 23:24:19 +000063 Scale ETCD ${namespace} ${desired_ETCD_cluster_size}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070064 Wait Until Keyword Succeeds ${timeout} 2s
65 ... Validate ETCD Size ${namespace} ${desired_ETCD_cluster_size}
hwchiu85695932019-12-18 08:05:25 +000066
67ETCD Failure Test
68 [Documentation] Failure Scenario Test: ETCD Crash
69 [Tags] FailureTest
70 Delete K8s Pods By Label ${namespace} ${ETCD_pod_label_key} ${ETCD_name}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070071 Wait Until Keyword Succeeds ${timeout} 2s
72 ... Pods Do Not Exist By Label ${namespace} ${ETCD_pod_label_key} ${ETCD_name}
73 Wait Until Keyword Succeeds ${timeout} 2s
74 ... Pods Are Ready By Label ${namespace} ${common_pod_label_key} ${rwcore_pod_label_value}
75 Wait Until Keyword Succeeds ${timeout} 2s
76 ... Pods Are Ready By Label ${namespace} ${common_pod_label_key} ${ofagent_pod_label_value}
77 Wait Until Keyword Succeeds ${timeout} 2s
78 ... Pods Are Ready By Label ${namespace} ${common_pod_label_key} ${adapter_openolt_pod_label_value}
hwchiu20f35d92019-11-05 05:22:58 +000079
80*** Keywords ***
hwchiu85695932019-12-18 08:05:25 +000081Get ETCD Running Size
hwchiu20f35d92019-11-05 05:22:58 +000082 [Arguments] ${namespace}
hwchiu85695932019-12-18 08:05:25 +000083 [Documentation] Get the number of running ETCD nodes
hwchiu20f35d92019-11-05 05:22:58 +000084 ${rc} ${size}= Run and Return Rc and Output
hwchiu85695932019-12-18 08:05:25 +000085 ... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.size}'
hwchiu20f35d92019-11-05 05:22:58 +000086 Should Be Equal As Integers ${rc} 0
87 [Return] ${size}
88
hwchiu85695932019-12-18 08:05:25 +000089Scale ETCD
hwchiu20f35d92019-11-05 05:22:58 +000090 [Arguments] ${namespace} ${size}
hwchiu85695932019-12-18 08:05:25 +000091 [Documentation] Scale down the number of ETCD pod
hwchiu20f35d92019-11-05 05:22:58 +000092 ${rc}= Run and Return Rc
hwchiu85695932019-12-18 08:05:25 +000093 ... kubectl -n ${namespace} patch ${ETCD_resources} ${ETCD_name} --type='merge' -p '{"spec":{"size":${size}}}'
hwchiu20f35d92019-11-05 05:22:58 +000094 Should Be Equal As Integers ${rc} 0
95
hwchiu85695932019-12-18 08:05:25 +000096Validate ETCD Size
97 [Arguments] ${namespace} ${ETCD_cluster_size}
98 [Documentation] Scale down the number of ETCD pod
hwchiu20f35d92019-11-05 05:22:58 +000099 ${rc} ${size}= Run and Return Rc and Output
hwchiu85695932019-12-18 08:05:25 +0000100 ... kubectl -n ${namespace} get ${ETCD_resources} ${ETCD_name} -o jsonpath='{.status.size}'
101 Should Be Equal As Integers ${size} ${ETCD_cluster_size}