Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 1 | # Copyright 2017-present Open Networking Foundation |
| 2 | # |
| 3 | # Licensed under the Apache License, Version 2.0 (the "License"); |
| 4 | # you may not use this file except in compliance with the License. |
| 5 | # You may obtain a copy of the License at |
| 6 | # |
| 7 | # http://www.apache.org/licenses/LICENSE-2.0 |
| 8 | # |
| 9 | # Unless required by applicable law or agreed to in writing, software |
| 10 | # distributed under the License is distributed on an "AS IS" BASIS, |
| 11 | # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. |
| 12 | # See the License for the specific language governing permissions and |
| 13 | # limitations under the License. |
| 14 | |
| 15 | |
| 16 | *** Settings *** |
| 17 | Documentation Test suite to validate K8s in the experimental ControlKube Scenario |
| 18 | Suite Setup Setup |
| 19 | Library OperatingSystem |
| 20 | Library ../cord-api/Framework/utils/onosUtils.py |
| 21 | Library ../cord-api/Framework/utils/utils.py |
| 22 | Resource ../cord-api/Framework/utils/utils.robot |
| 23 | |
| 24 | *** Variables *** |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 25 | ${deployment} physical |
| 26 | @{physical_nodes} node1 node2 node3 |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 27 | @{minikube_nodes} minikube |
| 28 | ${resources_file} ${CURDIR}/data/k8resources.json |
| 29 | |
| 30 | |
| 31 | *** Test Cases *** |
| 32 | Validate K8 Nodes |
| 33 | [Documentation] Validates that all nodes that are running in the K8 are healthy |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 34 | [Tags] nodes |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 35 | ${nodes}= Run kubectl get nodes -o json |
| 36 | Log ${nodes} |
| 37 | @{nodes}= Get Names ${nodes} |
| 38 | ##set nodes based on deployment |
| 39 | @{nodes_expected}= Set Variable If '${deployment}' == 'physical' ${physical_nodes} ${minikube_nodes} |
| 40 | Log ${nodes_expected} |
| 41 | #validates that all expected nodes to be running |
| 42 | : FOR ${i} IN @{nodes_expected} |
| 43 | \ List Should Contain Value ${nodes} ${i} |
| 44 | : FOR ${i} IN @{nodes} |
| 45 | \ ${status}= Run kubectl get nodes ${i} -o jsonpath='{.status.conditions[?(@.type=="Ready")].status}' |
| 46 | \ ${outofdisk}= Run kubectl get nodes ${i} -o jsonpath='{.status.conditions[?(@.type=="OutOfDisk")].status}' |
| 47 | \ ${memorypressure}= Run kubectl get nodes ${i} -o jsonpath='{.status.conditions[?(@.type=="MemoryPressure")].status}' |
| 48 | \ ${diskpressure}= Run kubectl get nodes ${i} -o jsonpath='{.status.conditions[?(@.type=="DiskPressure")].status}' |
| 49 | \ Should Be Equal ${status} True |
| 50 | \ Should Be Equal ${outofdisk} False |
| 51 | \ Should Be Equal ${memorypressure} False |
| 52 | \ Should Be Equal ${diskpressure} False |
| 53 | |
| 54 | Validate XOS Core Containers |
| 55 | [Documentation] Validates that all xos core containers that are running in the K8 Pods are healthy |
| 56 | [Tags] xos-core |
| 57 | @{core_container_names}= Run Keyword and Continue on Failure Validate Pods xos- |
| 58 | #validates that all expected containers to be running are in one of the pods inspected above |
| 59 | : FOR ${i} IN @{core_containers} |
| 60 | \ Run Keyword and Continue on Failure List Should Contain Value ${core_container_names} ${i} |
| 61 | |
| 62 | Validate RCord-Lite Containers |
| 63 | [Documentation] Validates that all rcord-lite containers that are running in the K8 Pods are healthy |
| 64 | [Tags] rcord-lite |
| 65 | @{rcord_container_names}= Run Keyword and Continue on Failure Validate Pods rcord-lite- |
| 66 | #validates that all expected containers to be running are in one of the pods inspected above |
| 67 | : FOR ${i} IN @{rcord_lite_containers} |
| 68 | \ Run Keyword and Continue on Failure List Should Contain Value ${rcord_container_names} ${i} |
| 69 | |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 70 | Validate Voltha Containers |
| 71 | [Documentation] Validates that all voltha containers that are running in the K8 Pods are healthy |
| 72 | [Tags] voltha |
| 73 | @{voltha_container_names}= Run Keyword and Continue on Failure Validate Pods voltha- |
| 74 | #validates that all expected containers to be running are in one of the pods inspected above |
| 75 | : FOR ${i} IN @{voltha_containers} |
| 76 | \ Run Keyword and Continue on Failure List Should Contain Value ${voltha_container_names} ${i} |
| 77 | |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 78 | Validate ONOS-Fabric Containers |
| 79 | [Documentation] Validates that all onos-fabric containers that are running in the K8 Pods are healthy |
| 80 | [Tags] onos-fabric |
| 81 | @{onos_container_names}= Run Keyword and Continue on Failure Validate Pods onos-fabric |
| 82 | #validates that all expected containers to be running are in one of the pods inspected above |
| 83 | : FOR ${i} IN @{onos_fabric_containers} |
| 84 | \ Run Keyword and Continue on Failure List Should Contain Value ${onos_container_names} ${i} |
| 85 | |
| 86 | Validate XOS Core Deployments |
| 87 | [Documentation] Validates that all xos-core deployments successfully rolled out and available |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 88 | [Tags] xos-core |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 89 | Validate Deployments xos- ${core_deployments} |
| 90 | |
| 91 | Validate RCord-Lite Deployments |
| 92 | [Documentation] Validates that all rcord-lite deployments successfully rolled out and available |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 93 | [Tags] rcord-lite |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 94 | Validate Deployments rcord-lite- ${rcord_lite_deployments} |
| 95 | |
| 96 | Validate ONOS-Fabric Deployments |
| 97 | [Documentation] Validates that all onos-fabric deployments successfully rolled out and available |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 98 | [Tags] onos-fabric |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 99 | Validate Deployments onos-fabric ${onos_fabric_deployments} |
| 100 | |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 101 | Validate Voltha Deployments |
| 102 | [Documentation] Validates that all voltha deployments successfully rolled out and available |
| 103 | [Tags] voltha |
| 104 | Validate Deployments voltha ${voltha_deployments} |
| 105 | |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 106 | Validate XOS Core Services |
| 107 | [Documentation] Validates that all expected xos-core services that are running in K8s |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 108 | [Tags] xos-core |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 109 | ${services}= Run kubectl get services -o json |
| 110 | Log ${services} |
| 111 | @{services}= Get Names ${services} |
| 112 | #validates that all expected services are running |
| 113 | : FOR ${i} IN @{xos_core_services} |
| 114 | \ Run Keyword and Continue on Failure List Should Contain Value ${services} ${i} |
| 115 | ##check for appropriate ports for each service and match with values in helm charts |
| 116 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].port}' |
| 117 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].targetPort}' |
| 118 | |
| 119 | Validate ONOS-Fabric Services |
| 120 | [Documentation] Validates that all expected onos-fabric services that are running in K8s |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 121 | [Tags] onos-fabric |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 122 | ${services}= Run kubectl get services -o json |
| 123 | Log ${services} |
| 124 | @{services}= Get Names ${services} |
| 125 | #validates that all expected services are running |
| 126 | : FOR ${i} IN @{onos_fabric_services} |
| 127 | \ Run Keyword and Continue on Failure List Should Contain Value ${services} ${i} |
| 128 | ##check for appropriate ports for each service and match with values in helm charts |
| 129 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].port}' |
| 130 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].targetPort}' |
| 131 | |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 132 | Validate Voltha Services |
| 133 | [Documentation] Validates that all expected voltha services that are running in K8s |
| 134 | [Tags] voltha |
| 135 | ${services}= Run kubectl get services -o json |
| 136 | Log ${services} |
| 137 | @{services}= Get Names ${services} |
| 138 | #validates that all expected services are running |
| 139 | : FOR ${i} IN @{voltha_services} |
| 140 | \ Run Keyword and Continue on Failure List Should Contain Value ${services} ${i} |
| 141 | ##check for appropriate ports for each service and match with values in helm charts |
| 142 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].port}' |
| 143 | ## eg. kubectl get svc xos-core -o jsonpath='{.spec.ports[?(@.name=="secure")].targetPort}' |
| 144 | |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 145 | *** Keywords *** |
| 146 | Setup |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 147 | Log Parsing the datafile for Kubernetes resources for each helm chart |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 148 | @{core_containers}= Create List |
| 149 | @{rcord_lite_containers}= Create List |
| 150 | @{onos_fabric_containers}= Create List |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 151 | @{voltha_containers}= Create List |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 152 | @{core_deployments}= Create List |
| 153 | @{rcord_lite_deployments}= Create List |
| 154 | @{onos_fabric_deployments}= Create List |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 155 | @{voltha_deployments}= Create List |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 156 | @{xos_core_services}= Create List |
| 157 | @{onos_fabric_services}= Create List |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 158 | @{voltha_services}= Create List |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 159 | ${xos-core_containers} utils.jsonToList ${resources_file} xos-core-containers |
| 160 | : FOR ${container} IN @{xos-core_containers} |
| 161 | \ Append To List ${core_containers} ${container} |
| 162 | ${rcord_containers} utils.jsonToList ${resources_file} rcord-lite-containers |
| 163 | : FOR ${container} IN @{rcord_containers} |
| 164 | \ Append To List ${rcord_lite_containers} ${container} |
| 165 | ${onosfabric_containers} utils.jsonToList ${resources_file} onos-fabric-containers |
| 166 | : FOR ${container} IN @{onosfabric_containers} |
| 167 | \ Append To List ${onos_fabric_containers} ${container} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 168 | ${volthaContainers} utils.jsonToList ${resources_file} voltha-containers |
| 169 | : FOR ${container} IN @{volthaContainers} |
| 170 | \ Append To List ${voltha_containers} ${container} |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 171 | ${xos-core_deployments} utils.jsonToList ${resources_file} xos-core-deployments |
| 172 | : FOR ${deployment} IN @{xos-core_containers} |
| 173 | \ Append To List ${core_deployments} ${deployment} |
| 174 | ${rcord_deployments} utils.jsonToList ${resources_file} rcord-lite-deployments |
| 175 | : FOR ${deployment} IN @{rcord_deployments} |
| 176 | \ Append To List ${rcord_lite_deployments} ${deployment} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 177 | ${onosfabric_deployments} utils.jsonToList ${resources_file} onos-fabric-deployments |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 178 | : FOR ${deployment} IN @{onosfabric_deployments} |
| 179 | \ Append To List ${onos_fabric_deployments} ${deployment} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 180 | ${volthaDeployments} utils.jsonToList ${resources_file} voltha-deployments |
| 181 | : FOR ${deployment} IN @{volthaDeployments} |
| 182 | \ Append To List ${voltha_deployments} ${deployment} |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 183 | ${core_services} utils.jsonToList ${resources_file} xos-core-services |
| 184 | : FOR ${service} IN @{core_services} |
| 185 | \ Append To List ${xos_core_services} ${service} |
| 186 | ${onos_services} utils.jsonToList ${resources_file} onos-fabric-services |
| 187 | : FOR ${service} IN @{onos_services} |
| 188 | \ Append To List ${onos_fabric_services} ${service} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 189 | ${volthaServices} utils.jsonToList ${resources_file} voltha-services |
| 190 | : FOR ${service} IN @{volthaServices} |
| 191 | \ Append To List ${voltha_services} ${service} |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 192 | Set Suite Variable @{core_containers} |
| 193 | Set Suite Variable @{rcord_lite_containers} |
| 194 | Set Suite Variable @{onos_fabric_containers} |
| 195 | Set Suite Variable @{core_deployments} |
| 196 | Set Suite Variable @{rcord_lite_deployments} |
| 197 | Set Suite Variable @{onos_fabric_deployments} |
| 198 | Set Suite Variable @{xos_core_services} |
| 199 | Set Suite Variable @{onos_fabric_services} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 200 | Set Suite Variable @{voltha_containers} |
| 201 | Set Suite Variable @{voltha_services} |
| 202 | Set Suite Variable @{voltha_deployments} |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 203 | |
| 204 | Get Names |
| 205 | [Documentation] Gets names of K8 resources running |
| 206 | [Arguments] ${output} |
| 207 | @{names}= Create List |
| 208 | ${output}= To JSON ${output} |
| 209 | ${len}= Get Length ${output} |
| 210 | ${length}= Get Length ${output['items']} |
| 211 | : FOR ${INDEX} IN RANGE 0 ${length} |
| 212 | \ ${item}= Get From List ${output['items']} ${INDEX} |
| 213 | \ ${metadata}= Get From Dictionary ${item} metadata |
| 214 | \ ${name}= Get From Dictionary ${metadata} name |
| 215 | \ Append To List ${names} ${name} |
| 216 | [Return] @{names} |
| 217 | |
| 218 | Validate Pods |
| 219 | [Arguments] ${component} |
| 220 | @{container_names}= Create List |
| 221 | ${pods}= Run kubectl get pods -o json |
| 222 | Log ${pods} |
| 223 | ${pods}= To JSON ${pods} |
| 224 | ${len}= Get Length ${pods} |
| 225 | ${length}= Get Length ${pods['items']} |
| 226 | : FOR ${INDEX} IN RANGE 0 ${length} |
| 227 | \ ${item}= Get From List ${pods['items']} ${INDEX} |
| 228 | \ ${metadata}= Get From Dictionary ${item} metadata |
| 229 | \ ${name}= Get From Dictionary ${metadata} name |
| 230 | \ Continue For Loop If '${component}' not in '''${name}''' |
Kailash Khalasi | 615940e | 2018-06-18 10:29:16 -0700 | [diff] [blame] | 231 | \ Continue For Loop If 'tosca-loader' in '''${name}''' |
Kailash Khalasi | d314adb | 2018-07-23 11:42:45 -0700 | [diff] [blame] | 232 | \ Continue For Loop If '-test' in '''${name}''' |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 233 | \ ${status}= Get From Dictionary ${item} status |
| 234 | \ ${containerStatuses}= Get From Dictionary ${status} containerStatuses |
| 235 | \ Log ${containerStatuses} |
| 236 | \ ${cstatus}= Get From List ${containerStatuses} 0 |
| 237 | \ Log ${cstatus} |
| 238 | \ ${restartCount}= Get From Dictionary ${cstatus} restartCount |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 239 | \ Run Keyword and Continue On Failure Should Be Equal As Integers ${restartCount} 0 |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 240 | \ ${container_name}= Get From Dictionary ${cstatus} name |
| 241 | \ ${state}= Get From Dictionary ${cstatus} state |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 242 | \ Run Keyword and Continue On Failure Should Contain ${state} running |
| 243 | \ Run Keyword and Continue On Failure Should Not Contain ${state} stopped |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 244 | \ Log ${state} |
| 245 | \ Append To List ${container_names} ${container_name} |
| 246 | [Return] ${container_names} |
| 247 | |
| 248 | Validate Deployments |
| 249 | [Arguments] ${component} ${expected_deployments} |
| 250 | ${deployments}= Run kubectl get deployments -o json |
| 251 | @{deployments}= Get Names ${deployments} |
| 252 | : FOR ${i} IN @{deployments} |
| 253 | \ Continue For Loop If '${component}' not in '''${i}''' |
| 254 | \ ${rollout_status}= Run kubectl rollout status deployment/${i} |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 255 | \ Run Keyword and Continue On Failure Should Be Equal ${rollout_status} deployment "${i}" successfully rolled out |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 256 | \ ##validate replication sets |
| 257 | \ ${desired}= Run kubectl get deployments ${i} -o jsonpath='{.status.replicas}' |
| 258 | \ ${available}= Run kubectl get deployments ${i} -o jsonpath='{.status.availableReplicas}' |
Kailash Khalasi | 55c65a7 | 2018-06-05 11:28:20 -0700 | [diff] [blame] | 259 | \ Run Keyword and Continue On Failure Should Be Equal ${desired} ${available} |
Kailash Khalasi | 2713931 | 2018-05-29 08:43:50 -0700 | [diff] [blame] | 260 | #validates that all expected deployments to exist |
| 261 | : FOR ${i} IN @{expected_deployments} |
Kailash Khalasi | d314adb | 2018-07-23 11:42:45 -0700 | [diff] [blame] | 262 | \ Run Keyword and Continue On Failure List Should Contain Value ${deployments} ${i} |