blob: 89c5fcac968e2e995cce4595f2145029ff170950 [file] [log] [blame]
David Bainbridge117d23e2019-09-30 20:37:51 +00001# Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
David Bainbridge117d23e2019-09-30 20:37:51 +000014# voltctl common functions
15
16*** Settings ***
17Documentation Library for various utilities
18Library SSHLibrary
David Bainbridge117d23e2019-09-30 20:37:51 +000019Library String
20Library DateTime
21Library Process
22Library Collections
23Library RequestsLibrary
24Library OperatingSystem
25
26*** Keywords ***
27Lookup Service IP
28 [Arguments] ${namespace} ${name}
David Bainbridgef81cd642019-11-20 00:14:47 +000029 [Documentation] Uses kubectl to resolve a service name to an IP
Gilles Depatie675a2062019-10-22 12:44:42 -040030 ${rc} ${ip}= Run and Return Rc and Output
31 ... kubectl get svc -n ${namespace} ${name} -o jsonpath={.spec.clusterIP}
David Bainbridge117d23e2019-09-30 20:37:51 +000032 Should Be Equal as Integers ${rc} 0
33 [Return] ${ip}
34
35Lookup Service PORT
36 [Arguments] ${namespace} ${name}
David Bainbridgef81cd642019-11-20 00:14:47 +000037 [Documentation] Uses kubectl to resolve a service name to an PORT
Gilles Depatie675a2062019-10-22 12:44:42 -040038 ${rc} ${port}= Run and Return Rc and Output
39 ... kubectl get svc -n ${namespace} ${name} -o jsonpath={.spec.ports[0].port}
David Bainbridge117d23e2019-09-30 20:37:51 +000040 Should Be Equal as Integers ${rc} 0
41 [Return] ${port}
suraj gourd64356b2019-11-07 13:26:20 +000042
43Restart Pod
44 [Arguments] ${namespace} ${name}
Matteo Scandolo6f24ea92021-04-29 11:55:50 -070045 [Documentation] *DEPRECATED* Use Restart Pod By Label instead
suraj gour067451d2019-11-13 11:20:13 +000046 ${rc} ${restart_pod_name}= Run and Return Rc and Output
47 ... kubectl get pods -n ${namespace} | grep ${name} | awk 'NR==1{print $1}'
suraj gourd64356b2019-11-07 13:26:20 +000048 Log ${restart_pod_name}
suraj gour067451d2019-11-13 11:20:13 +000049 Should Not Be Empty ${restart_pod_name} Unable to parse pod name
Zack Williamsa8fe75a2020-01-10 14:25:27 -070050 ${rc} ${output}= Run and Return Rc and Output
51 ... kubectl delete pod ${restart_pod_name} -n ${namespace} --grace-period=0 --force
suraj gourd64356b2019-11-07 13:26:20 +000052 Log ${output}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +000053
Matteo Scandolo6f24ea92021-04-29 11:55:50 -070054Restart Pod By Label
55 [Arguments] ${namespace} ${label_key} ${label_value}
56 [Documentation] Uses kubectl to force delete pod
57 ${rc} ${restart_pod_name}= Run and Return Rc and Output
58 ... kubectl get pods -n ${namespace} -l ${label_key}=${label_value} --no-headers | awk '{print $1}'
59 Log ${restart_pod_name}
60 Should Not Be Empty ${restart_pod_name} Unable to parse pod name
61 ${rc} ${output}= Run and Return Rc and Output
62 ... kubectl delete pod ${restart_pod_name} -n ${namespace} --grace-period=0 --force
63 Log ${output}
64
Scott Baker60e570d2020-02-02 22:10:13 -080065Exec Pod
66 [Arguments] ${namespace} ${name} ${command}
67 [Documentation] Uses kubectl to execute a command in the pod and return the output
68 ${rc} ${exec_pod_name}= Run and Return Rc and Output
Andrea Campanella1b25ac52020-09-09 14:34:31 +020069 ... kubectl -n ${namespace} get pods -l app=${name} -o name
Scott Baker60e570d2020-02-02 22:10:13 -080070 Log ${exec_pod_name}
71 Should Not Be Empty ${exec_pod_name} Unable to parse pod name
72 ${rc} ${output}= Run and Return Rc and Output
73 ... kubectl exec -i ${exec_pod_name} -n ${namespace} -- ${command}
74 Log ${output}
75 [return] ${output}
76
Holger Hildebrandt23147742020-11-16 10:13:21 +000077Exec Pod In Kube
TorstenThiemefe7099e2021-01-29 08:41:04 +000078 [Arguments] ${namespace} ${name} ${command} ${grep}=${EMPTY}
Holger Hildebrandt23147742020-11-16 10:13:21 +000079 [Documentation] Uses kubectl to execute a command in the pod and return the output
TorstenThiemefe7099e2021-01-29 08:41:04 +000080 ${rc} ${exec_pod_name}= Run Keyword If '${grep}'=='${EMPTY}'
81 ... Run and Return Rc and Output kubectl -n ${namespace} get pods -l app.kubernetes.io/name=${name} -o name
82 ... ELSE Run and Return Rc and Output
83 ... kubectl -n ${namespace} get pods -l app.kubernetes.io/name=${name} -o name \| grep ${grep}
Hardik Windlassec86c232021-02-02 13:56:12 +000084 Log ${exec_pod_name}
85 Should Not Be Empty ${exec_pod_name} Unable to parse pod name
86 ${rc} ${output}= Run and Return Rc and Output
87 ... kubectl exec -i ${exec_pod_name} -n ${namespace} -- ${command}
88 Log ${output}
89 [return] ${output}
Matteo Scandoloa80b4732020-09-04 13:51:10 -070090
91Exec Pod And Return Output And RC
92 [Arguments] ${namespace} ${name} ${command}
93 [Documentation] Uses kubectl to execute a command in the pod and return the output
94 ${rc} ${exec_pod_name}= Run and Return Rc and Output
95 ... kubectl get pods -n ${namespace} | grep ${name} | awk 'NR==1{print $1}'
Holger Hildebrandt23147742020-11-16 10:13:21 +000096 Log ${exec_pod_name}
97 Should Not Be Empty ${exec_pod_name} Unable to parse pod name
98 ${rc} ${output}= Run and Return Rc and Output
99 ... kubectl exec -i ${exec_pod_name} -n ${namespace} -- ${command}
100 Log ${output}
Matteo Scandoloa80b4732020-09-04 13:51:10 -0700101 [return] ${output} ${rc}
Holger Hildebrandt23147742020-11-16 10:13:21 +0000102
Scott Baker60e570d2020-02-02 22:10:13 -0800103Exec Pod Separate Stderr
104 [Arguments] ${namespace} ${name} ${command}
105 [Documentation] Uses kubectl to execute a command in the pod and return the stderr and stdout
106 ${rc} ${exec_pod_name}= Run and Return Rc and Output
Andrea Campanella1b25ac52020-09-09 14:34:31 +0200107 ... kubectl -n ${namespace} get pods -l app=${name} -o name
Scott Baker60e570d2020-02-02 22:10:13 -0800108 Log ${exec_pod_name}
109 Should Not Be Empty ${exec_pod_name} Unable to parse pod name
110 @{args}= Split String ${command}
111 ${result}= Run Process
112 ... kubectl exec -i ${exec_pod_name} -n ${namespace} -- @{args}
113 ${stdout}= Set Variable ${result.stdout}
114 ${stderr}= Set Variable ${result.stderr}
115 Log ${stdout}
116 Log ${stderr}
117 [return] ${stdout} ${stderr}
118
Scott Baker0478bab2020-04-10 17:19:57 -0700119Copy File To Pod
Andrea Campanella0aa21d62021-07-22 10:44:32 +0200120 [Arguments] ${namespace} ${label} ${src} ${dest}
Scott Baker0478bab2020-04-10 17:19:57 -0700121 [Documentation] Uses kubectl to copy a file to a pod
122 ${rc} ${exec_pod_name}= Run and Return Rc and Output
Andrea Campanella0aa21d62021-07-22 10:44:32 +0200123 ... kubectl get pods -n ${namespace} -l ${label} --no-headers | awk 'NR==1{print $1}'
Scott Baker0478bab2020-04-10 17:19:57 -0700124 Log ${exec_pod_name}
125 Should Not Be Empty ${exec_pod_name} Unable to parse pod name
126 ${rc} ${output}= Run and Return Rc and Output
127 ... kubectl cp -n ${namespace} ${src} ${exec_pod_name}:${dest}
128 Log ${output}
129 [return] ${output}
130
Scott Baker60e570d2020-02-02 22:10:13 -0800131Apply Kubernetes Resources
132 [Arguments] ${resource_yaml} ${namespace}
133 [Documentation] Use kubectl to create resources given a yaml file
134 ${rc} Run and Return Rc
135 ... kubectl apply -n ${namespace} -f ${resource_yaml}
136 Should Be Equal as Integers ${rc} 0
137
Scott Baker0478bab2020-04-10 17:19:57 -0700138Delete Kubernetes Resources
139 [Arguments] ${resource_yaml} ${namespace}
140 [Documentation] Use kubectl to delete resources given a yaml file
141 ${rc} Run and Return Rc
142 ... kubectl delete -n ${namespace} -f ${resource_yaml}
143 Should Be Equal as Integers ${rc} 0
144
suraj gour1ecfae92019-12-20 15:11:40 +0000145Validate Pod Status
146 [Arguments] ${pod_name} ${namespace} ${expectedStatus}
147 [Documentation] To run the kubectl command and check the status of the given pod matches the expected status
Andy Bavierb63f6d22020-03-12 15:34:37 -0700148 ${length}= Run kubectl get pod -n ${namespace} -o name | wc -l
149 ${matched}= Set Variable False
150 FOR ${index} IN RANGE ${length}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700151 ${currentPodName}= Run
152 ... kubectl get pod -n ${namespace} -o=jsonpath="{.items[${index}].status.containerStatuses[0].name}"
suraj gour1ecfae92019-12-20 15:11:40 +0000153 Log Required Pod : ${pod_name}
154 Log Current Pod: ${currentPodName}
Andy Bavierb63f6d22020-03-12 15:34:37 -0700155 ${matched}= Set Variable If '${currentPodName}'=='${pod_name}' True False
156 Exit For Loop If ${matched}
suraj gour1ecfae92019-12-20 15:11:40 +0000157 END
Andy Bavierb63f6d22020-03-12 15:34:37 -0700158 Should Be True ${matched} No pod ${podname} found
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700159 ${currentStatusofPod}= Run
160 ... kubectl get pod -n ${namespace} -o=jsonpath="{.items[${index}].status.phase}"
suraj gour1ecfae92019-12-20 15:11:40 +0000161 Log ${currentStatusofPod}
162 Should Contain ${currentStatusofPod} ${expectedStatus}
163
Matteo Scandoloa80b4732020-09-04 13:51:10 -0700164Get Pod Name By Label
165 [Arguments] ${namespace} ${label_key} ${label_value}
166 [Documentation] Return a pod name from a given label
167 ${rc} ${pod_name}= Run and Return Rc and Output
168 ... kubectl get pods -n ${namespace} -l ${label_key}=${label_value} --no-headers | awk '{print $1}'
169 Should Not Be Empty ${pod_name} Pod not found
170 [return] ${pod_name}
171
Hung-Wei Chiu2bee4d42020-04-24 11:31:50 -0700172Validate Pods Status By Label
173 [Arguments] ${namespace} ${label_key} ${label_value} ${expectedStatus}
174 [Documentation] To run the kubectl command and check the status of all pods filter
175 ... by label matche the expected status
176 ${command}= Catenate
177 ... kubectl -n ${namespace} get pods -l ${label_key}=${label_value}
178 ... -o=jsonpath="{.items[?(.status.phase=='${expectedStatus}')].status.phase}"
179 ${pods_status}= Run ${command}
180 Should Not Be Equal ${pods_status} ${EMPTY} Can't filter out Pods with exptected status ${expectedStatus}
181
Andrea Campanella4c404632020-08-26 14:41:36 +0200182Validate Pods Status By Name
183 [Arguments] ${namespace} ${name} ${expectedStatus}
184 [Documentation] To run the kubectl command and check the status of all pods filter
185 ... by label matche the expected status
186 ${command}= Catenate
187 ... kubectl -n ${namespace} get pods ${name}
188 ... -o=jsonpath="{.status.phase}"
189 ${pods_status}= Run ${command}
190 Should Not Be Equal ${pods_status} ${EMPTY} Can't filter out Pods with exptected status ${expectedStatus}
191
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000192Verify All Voltha Pods For Any Error Logs
193 [Arguments] ${datetime}
194 [Documentation] This keyword checks for the error occurence in the voltha pods
195 &{errorPodDict} Create Dictionary
Matteo Scandolo142e6272020-04-29 17:36:59 -0700196 &{containerDict} Get Container Dictionary voltha
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000197 FOR ${podName} IN @{PODLIST1}
198 ${containerName} Get From Dictionary ${containerDict} ${podName}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700199 ${rc} ${logOutput} Run And Return Rc And Output
200 ... kubectl logs --timestamps -n voltha --since-time=${datetime} ${containerName}
201 Run Keyword And Ignore Error
202 ... Run Keyword If '${logOutput}'=='${EMPTY}'
203 ... Run Keywords Log No Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000204 ... AND Continue For Loop
205 ${errorDict} Check For Error Logs in Pod Type1 Given the Log Output ${logOutput}
206 ${returnStatusFlagList} Get Dictionary Keys ${errorDict}
207 ${returnStatusFlag} Get From List ${returnStatusFlagList} 0
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700208 Run Keyword And Ignore Error
209 ... Run Keyword If '${returnStatusFlag}'=='Nologfound'
210 ... Run Keywords Log No Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000211 ... AND Continue For Loop
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700212 Run Keyword And Ignore Error
213 ... Run Keyword If '${returnStatusFlag}'=='UnexpectedErrorfound'
214 ... Run Keywords Log Unexpected Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000215 ... AND Set to Dictionary ${errorPodDict} ${podName} ${errorDict}
216 END
217 FOR ${podName} IN @{PODLIST2}
218 ${containerName} Get From Dictionary ${containerDict} ${podName}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700219 ${rc} ${logOutput} Run And Return Rc And Output
220 ... kubectl logs --timestamps -n voltha --since-time=${datetime} ${containerName}
221 Run Keyword And Ignore Error
222 ... Run Keyword If '${logOutput}'=='${EMPTY}'
223 ... Run Keywords Log No Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000224 ... AND Continue For Loop
225 ${errorDict} Check For Error Logs in Pod Type2 Given the Log Output ${logOutput}
226 ${returnStatusFlagList} Get Dictionary Keys ${errorDict}
227 ${returnStatusFlag} Get From List ${returnStatusFlagList} 0
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700228 Run Keyword And Ignore Error
229 ... Run Keyword If '${returnStatusFlag}'=='Nologfound'
230 ... Run Keywords Log No Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000231 ... AND Continue For Loop
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700232 Run Keyword And Ignore Error
233 ... Run Keyword If '${returnStatusFlag}'=='UnexpectedErrorfound'
234 ... Run Keywords Log Unexpected Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000235 ... AND Set to Dictionary ${errorPodDict} ${podName} ${errorDict}
236 END
237 Print to Console Error Statement logged in the following pods : ${errorPodDict}
238 [Return] ${errorPodDict}
239
240Check For Error Logs in Pod Type1 Given the Log Output
241 [Arguments] ${logOutput} ${logLevel}=error ${errorMessage}=${EMPTY}
242 [Documentation] Checks for error message in the particular list of pods
243 Log ${logOutput}
244 ${linesContainingLog} = Get Lines Matching Regexp ${logOutput} .*\s\${logLevel}.* partial_match=true
245 ${is_exec_status} ${output} Run Keyword And Ignore Error Should Be Empty ${linesContainingLog}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700246 ${returnStatusFlag} Set Variable If '${is_exec_status}'=='PASS'
247 ... Nologfound '${is_exec_status}'=='FAIL' Errorlogfound
248 ${linesContainingError} = Get Lines Matching Regexp
249 ... ${logOutput} .*\s\${logLevel}.*${errorMessage} partial_match=true
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000250 ${is_exec_status} ${output} Run Keyword And Ignore Error Should Be Empty ${linesContainingError}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700251 ${returnStatusFlag} Set Variable If '${is_exec_status}'=='PASS'
252 ... UnexpectedErrorfound '${is_exec_status}'=='FAIL' MatchingErrorlogfound
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000253 Log {linesContainingError}
254 &{errorDict} Create Dictionary ${returnStatusFlag} ${linesContainingLog}
255 [Return] ${errorDict}
256
257Check For Error Logs in Pod Type2 Given the Log Output
258 [Arguments] ${logOutput} ${logLevel}=warn ${errorMessage}=${EMPTY}
259 [Documentation] Checks for error message in the particular set of pods
260 Log ${logOutput}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700261 ${linesContainingLog} = Get Lines Matching Regexp
262 ... ${logOutput} .*?\s.*level.*${logLevel}.* partial_match=true
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000263 ${is_exec_status} ${output} Run Keyword And Ignore Error Should Be Empty ${linesContainingLog}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700264 ${returnStatusFlag} Set Variable If '${is_exec_status}'=='PASS'
265 ... Nologfound '${is_exec_status}'=='FAIL' Errorlogfound
266 ${linesContainingError} = Get Lines Matching Regexp
267 ... ${logOutput} .*?\s.*level.*${logLevel}.*msg.*${errorMessage} partial_match=true
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000268 ${is_exec_status} ${output} Run Keyword And Ignore Error Should Be Empty ${linesContainingError}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700269 ${returnStatusFlag} Set Variable If '${is_exec_status}'=='PASS'
270 ... UnexpectedErrorfound '${is_exec_status}'=='FAIL' MatchingErrorlogfound
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000271 Log {linesContainingError}
272 &{errorDict} Create Dictionary ${returnStatusFlag} ${linesContainingLog}
273 [Return] ${errorDict}
274
275Get Container Dictionary
Matteo Scandolo142e6272020-04-29 17:36:59 -0700276 [Arguments] ${namespace}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000277 [Documentation] Creates a mapping for pod name and container name and returns the same
278 &{containerDict} Create Dictionary
279 ${containerName} Set Variable ${EMPTY}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700280 ${podName} Run kubectl get deployment -n ${namespace} | awk 'NR>1 {print $1}'
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000281 @{podNameList}= Split To Lines ${podName}
282 Append To List ${podNameList} voltha-etcd-cluster voltha-kafka voltha-ro-core voltha-zookeeper
283 Log ${podNameList}
284 #Creatiing dictionary to correspond pod name and container name
285 FOR ${pod} IN @{podNameList}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700286 ${containerName} Run kubectl get pod -n ${namespace} | grep ${pod} | awk '{print $1}'
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000287 &{containerDict} Set To Dictionary ${containerDict} ${pod} ${containerName}
288 END
289 Log ${containerDict}
290 [Return] ${containerDict}
291
292Validate Error For Given Pods
293 [Arguments] ${datetime} ${podDict}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700294 [Documentation]
295 ... This keyword is used to get the list of pods if there is any unexpected error
296 ... in a particular pod(s) given the time-${datetime} from which the log needs to
297 ... be analysed and the dictionary of pods and the error in the dictionary format
298 ... ${podDict] .
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000299 ...
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700300 ... Usage: ${returnStatusFlag} Validate Error For Given Pods ${datetime} ${podDict}
301 ...
302 ... Arguments:
303 ...
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000304 ... ${datetime} = time from which the log needs to be taken
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700305 ... ${podDict} = Key-value pair of the pod name and the error msg
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000306 ...
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700307 ... Example: ${podDict} = Set Dictionary ${podDict} radius sample error message.
308 ...
309 ... In case the radius pod log has any other error than the expected
310 ... error, then the podname will be returned
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000311 ${podList} = Get Dictionary Keys ${podDict}
312 FOR ${podName} IN @{podList}
313 ${containerName} Get From Dictionary ${containerDict} ${podName}
314 ${expectedError} Get From Dictionary ${podDict} ${podName}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700315 ${rc} ${logOutput} Run And Return Rc And Output
316 ... kubectl logs --timestamps -n voltha --since-time=${datetime} ${containerName}
317 Run Keyword And Ignore Error
318 ... Run Keyword If '${logOutput}'=='${EMPTY}'
319 ... Run Keywords Log No Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000320 ... AND Continue For Loop
321 ${returnStatusFlag} Check For Error Logs in Pod Type1 Given the Log Output ${logOutput}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700322 Run Keyword And Ignore Error
323 ... Run Keyword If '${returnStatusFlag}'=='Nologfound'
324 ... Run Keywords Log No Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000325 ... AND Continue For Loop
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700326 Run Keyword And Ignore Error
327 ... Run Keyword If '${returnStatusFlag}'=='UnexpectedErrorfound'
328 ... Run Keywords Log Unexpected Error Log found in pod ${podName}
Gayathri.Selvan61fbcdb2019-11-14 05:08:19 +0000329 ... AND Append To List ${errorPodList} ${podName}
330 END
331 [Return] ${errorPodList}
332
David Bainbridgef81cd642019-11-20 00:14:47 +0000333Delete K8s Pod
334 [Arguments] ${namespace} ${name}
335 [Documentation] Uses kubectl to delete a named POD
336 ${rc} Run and Return Rc
337 ... kubectl delete -n ${namespace} pod/${name}
338 Should Be Equal as Integers ${rc} 0
339
hwchiu85695932019-12-18 08:05:25 +0000340Delete K8s Pods By Label
341 [Arguments] ${namespace} ${key} ${value}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700342 [Documentation] Uses kubectl to delete a PODs, filtering by label
hwchiu85695932019-12-18 08:05:25 +0000343 ${rc}= Run and Return Rc
344 ... kubectl -n ${namespace} delete pods -l${key}=${value}
345 Should Be Equal as Integers ${rc} 0
346
Andrea Campanella4c404632020-08-26 14:41:36 +0200347Delete K8s Pods By Name
348 [Arguments] ${namespace} ${value}
349 [Documentation] Uses kubectl to delete a PODs, filtering by label
350 ${rc}= Run and Return Rc
351 ... kubectl -n ${namespace} delete pods ${value}
352 Should Be Equal as Integers ${rc} 0
353
David Bainbridgef81cd642019-11-20 00:14:47 +0000354Scale K8s Deployment
355 [Arguments] ${namespace} ${name} ${count}
356 [Documentation] Uses kubectl to scale a named deployment
357 ${rc} Run and Return Rc
358 ... kubectl scale --replicas=${count} -n ${namespace} deploy/${name}
359 Should Be Equal as Integers ${rc} 0
360
Andrea Campanella3dcce272021-01-15 16:04:47 +0100361Get K8s Deployment by Pod Label
362 [Arguments] ${namespace} ${key} ${value}
363 [Documentation] Uses kubectl to scale a deployment given the app name of the pod
364 ${rc} ${name} Run And Return Rc And Output
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000365 ... kubectl describe rs -n ${namespace} -l ${key}=${value} | grep "Controlled By" | awk -F'/' '{print $2}' | awk 'FNR == 1'
Andrea Campanella3dcce272021-01-15 16:04:47 +0100366 Should Be Equal as Integers ${rc} 0
367 [Return] ${name}
368
369Scale K8s Deployment by Pod Label
370 [Arguments] ${namespace} ${key} ${value} ${count}
371 [Documentation] Uses kubectl to scale a deployment given the app name of the pod
372 ${name} Get K8s Deployment by Pod Label ${namespace} ${key} ${value}
373 Scale K8s Deployment ${namespace} ${name} ${count}
374
David Bainbridgef81cd642019-11-20 00:14:47 +0000375Pod Exists
376 [Arguments] ${namespace} ${name}
377 [Documentation] Succeeds it the named POD exists
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700378 ${rc} ${count} Run and Return Rc
379 ... kubectl get -n ${namespace} pod -o json | jq -r ".items[].metadata.name" | grep ${name}
Andy Bavierb63f6d22020-03-12 15:34:37 -0700380 Should Be True ${count}>0 Pod ${name} not found
David Bainbridgef81cd642019-11-20 00:14:47 +0000381
382Pod Does Not Exist
383 [Arguments] ${namespace} ${name}
384 [Documentation] Succeeds if the named POD does not exist
385 ${rc} ${count} Run and Return Rc And Output
386 ... kubectl get -n ${namespace} pod -o json | jq -r ".items[].metadata.name" | grep -c ${name}
387 Should Be Equal As Integers ${count} 0
Andy Bavierb63f6d22020-03-12 15:34:37 -0700388 Should Be True ${count}==0 Pod ${name} exists but should not
David Bainbridgef81cd642019-11-20 00:14:47 +0000389
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000390Wait For Pods Not Exist
391 [Arguments] ${namespace} ${list_names}
392 [Documentation] Checks the passed PODs are no longer existing
TorstenThiemefe7099e2021-01-29 08:41:04 +0000393 FOR ${pod_name} IN @{list_names}
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000394 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 3s
TorstenThiemefe7099e2021-01-29 08:41:04 +0000395 ... Pod Does Not Exist ${namespace} ${pod_name}
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000396 END
397
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700398Pods Do Not Exist By Label
hwchiu85695932019-12-18 08:05:25 +0000399 [Arguments] ${namespace} ${key} ${value}
400 [Documentation] Succeeds if the named POD does not exist
401 ${rc} ${count} Run and Return Rc And Output
402 ... kubectl get -n ${namespace} pod -l${key}=${value} -o json | jq -r ".items[].metadata.name" | wc -l
403 Should Be Equal As Integers ${count} 0
Andy Bavierb63f6d22020-03-12 15:34:37 -0700404 Should Be True ${count}==0 Pod with label ${key}=${value} exists but should not
hwchiu85695932019-12-18 08:05:25 +0000405
David Bainbridgef81cd642019-11-20 00:14:47 +0000406Get Available Deployment Replicas
407 [Arguments] ${namespace} ${name}
408 [Documentation] Succeeds if the named POD exists and has a ready count > 0
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700409 ${rc} ${count} Run and Return Rc and Output
410 ... kubectl get -n ${namespace} deploy/${name} -o jsonpath='{.status.availableReplicas}'
David Bainbridgef81cd642019-11-20 00:14:47 +0000411 ${result}= Run Keyword If '${count}' == '' Set Variable 0
412 ... ELSE Set Variable ${count}
413 [Return] ${result}
414
Andrea Campanella3dcce272021-01-15 16:04:47 +0100415Check Expected Available Deployment Replicas By Pod Label
416 [Arguments] ${namespace} ${key} ${value} ${expected}
417 [Documentation] Succeeds if the named deployment has the expected number of available replicas
418 ${name} Get K8s Deployment by Pod Label ${namespace} ${key} ${value}
419 Check Expected Available Deployment Replicas ${namespace} ${name} ${expected}
420
David Bainbridgef81cd642019-11-20 00:14:47 +0000421Check Expected Available Deployment Replicas
422 [Arguments] ${namespace} ${name} ${expected}
423 [Documentation] Succeeds if the named deployment has the expected number of available replicas
424 ${count}= Get Available Deployment Replicas ${namespace} ${name}
425 Should Be Equal As Integers ${expected} ${count}
426
427Get Deployment Replica Count
428 [Arguments] ${namespace} ${name}
429 [Documentation] Uses kubectl to fetch the number of configured replicas on a deployment
430 ${rc} ${value} Run and Return Rc and Output
431 ... kubectl -n ${namespace} get deploy/${name} -o 'jsonpath={.status.replicas}'
432 Should Be Equal as Integers ${rc} 0
433 ${replicas}= Run Keyword If '${value}' == '' Set Variable 0
434 ... ELSE Set Variable ${value}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700435 [Return] ${replicas}
David Bainbridgef81cd642019-11-20 00:14:47 +0000436
437Does Deployment Have Replicas
438 [Arguments] ${namespace} ${name} ${expected_count}
439 [Documentation] Uses kubectl to fetch the number of configured replicas on a deployment
440 ${rc} ${value} Run and Return Rc and Output
441 ... kubectl -n ${namespace} get deploy/${name} -o 'jsonpath={.status.replicas}'
442 Should Be Equal as Integers ${rc} 0
443 ${replicas}= Run Keyword If '${value}' == '' Set Variable 0
444 ... ELSE Set Variable ${value}
445 Should be Equal as Integers ${replicas} ${expected_count}
hwchiu85695932019-12-18 08:05:25 +0000446
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700447Pods Are Ready By Label
hwchiu85695932019-12-18 08:05:25 +0000448 [Arguments] ${namespace} ${key} ${value}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700449 [Documentation] Check that all pods with a label are ready
450 ${output}= Run
451 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o=jsonpath="{.items[].status.containerStatuses[].ready}"
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000452 Should Not Contain ${output} false
453
454Wait For Pods Ready
455 [Arguments] ${namespace} ${list_apps}
456 [Documentation] Checks the passed PODs are ready
TorstenThiemefe7099e2021-01-29 08:41:04 +0000457 FOR ${app_name} IN @{list_apps}
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000458 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 3s
TorstenThiemefe7099e2021-01-29 08:41:04 +0000459 ... Pods Are Ready By Label ${namespace} app ${app_name}
TorstenThiemed2cd91d2020-07-28 07:13:44 +0000460 END
Gayathri.Selvan49398962020-01-13 07:19:12 +0000461
hwchiu58af72d2020-01-14 00:50:35 +0000462Check Expected Running Pods Number By Label
463 [Arguments] ${namespace} ${key} ${value} ${number}
464 [Documentation] Succeeds if the desired pod has expected number replicas
465 ${rc} ${count} Run and Return Rc and Output
466 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o json | jq -r ".items[].status.phase" | wc -l
467 Should Be Equal as Integers ${count} ${number}
Andrea Campanella962fe832020-09-21 10:41:47 +0200468
469Get Number of Running Pods Number By Label
470 [Arguments] ${namespace} ${key} ${value}
471 [Documentation] Returns the number of pods for a given label
472 ${rc} ${count} Run and Return Rc and Output
473 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o name | wc -l
474 [Return] ${count}
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000475
476Get Pod Restart Count
477 [Arguments] ${namespace} ${name}
478 [Documentation] Returns the restart count for the given Pod
479 ${rc} ${count}= Run and Return Rc and Output
480 ... kubectl get pods -n ${namespace} | grep ${name} | awk 'NR==1{print $4}'
481 [Return] ${count}
482
483Verify ONOS Pod Restart
484 [Arguments] ${restarted}=True
485 [Documentation] Verifies if any of the given ONOS instances restarted
486 ${num_onos}= Wait Until Keyword Succeeds 20s 5s Get Number of Running Pods Number By Label default
487 ... app onos-onos-classic
488 FOR ${I} IN RANGE 0 ${num_onos}
489 ${onos_pod}= Catenate SEPARATOR=- onos-onos-classic ${I}
490 ${count}= Get Pod Restart Count default ${onos_pod}
491 Run Keyword If ${restarted}
492 ... Should Not Be Equal As Integers ${count} 0 ONOS Pod ${onos_pod} Not Restarted
493 ... ELSE
494 ... Should Be Equal As Integers ${count} 0 ONOS Pod ${onos_pod} Restarted
495 END
496
497Deploy Pod New Image
498 [Arguments] ${namespace} ${deployment} ${container} ${image}
499 [Documentation] Deploys the Pod given image
500 ${rc} Run and Return Rc
501 ... kubectl -n ${namespace} set image deployment/${deployment} ${container}=${image}
502 Should Be Equal as Integers ${rc} 0
503
504Verify Pod Image
505 [Arguments] ${namespace} ${key} ${value} ${image}
506 [Documentation] Verifies the Pod Image
507 ${output}= Run
Hardik Windlass08451302021-03-09 12:14:36 +0000508 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o=jsonpath="{.items[*].spec.containers[*].image}"
509 Should Be Equal '${output}' '${image}'
Hardik Windlassdc2610f2021-03-09 07:33:51 +0000510
511Get Pod Image And App Version And Helm Chart By Label
512 [Arguments] ${namespace} ${key} ${value}
513 [Documentation] Retrieves Pod Image and, App and Helm Chart Version details
514 ${image}= Run
515 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o=jsonpath="{.items[*].spec.containers[*].image}"
516 ${cmd}= Catenate SEPARATOR=
517 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o=
518 ... jsonpath="{.items[*].metadata.labels.\\app\\.kubernetes\\.io\\/version}"
519 ${app_version}= Run ${cmd}
520 ${helm_chart}= Run
521 ... kubectl -n ${namespace} get pods -l ${key}=${value} -o=jsonpath="{.items[*].metadata.labels.\\helm\\.sh\\/chart}"
522 [Return] ${image} ${app_version} ${helm_chart}