blob: f943b4c2a7703ee73f86d2f81ae39f6877e2e4b0 [file] [log] [blame]
TorstenThiemed4f48962020-12-08 12:17:19 +00001# Copyright 2020 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
TorstenThieme1619db22020-04-03 12:01:15 +000015*** Settings ***
TorstenThieme109683b2020-09-24 12:35:41 +000016Documentation Test states of ONU Go adapter with ATT workflows only (not for DT/TT workflow!)
TorstenThieme1619db22020-04-03 12:01:15 +000017Suite Setup Setup Suite
18Suite Teardown Teardown Suite
19Test Setup Setup
20Test Teardown Teardown
21Library Collections
22Library String
23Library OperatingSystem
24Library XML
25Library RequestsLibrary
26Library ../../libraries/DependencyLibrary.py
27Resource ../../libraries/onos.robot
28Resource ../../libraries/voltctl.robot
29Resource ../../libraries/voltha.robot
30Resource ../../libraries/utils.robot
31Resource ../../libraries/k8s.robot
32Resource ../../variables/variables.robot
Matteo Scandolo5d9efae2020-12-14 09:37:41 -100033Resource ../../libraries/Voltha_ONUUtilities.robot
TorstenThieme1619db22020-04-03 12:01:15 +000034
35*** Variables ***
TorstenThiemed4f48962020-12-08 12:17:19 +000036${timeout} 60s
TorstenThieme1619db22020-04-03 12:01:15 +000037${of_id} 0
38${logical_id} 0
39${has_dataplane} True
40${external_libs} True
41${teardown_device} True
42${scripts} ../../scripts
43# Per-test logging on failure is turned off by default; set this variable to enable
44${container_log_dir} ${None}
TorstenThiemee0e5bba2020-08-05 11:20:13 +000045# state to test variable, can be passed via the command line too, valid values: 1-6
46# 1 -> activating-onu
47# 2 -> starting-openomci
48# 3 -> discovery-mibsync-complete
49# 4 -> initial-mib-downloaded
50# 5 -> tech-profile-config-download-success
51# 6 -> omci-flows-pushed
TorstenThieme87cd6202020-09-09 10:01:28 +000052# example: -v state2test:5
TorstenThieme66c91a82020-10-19 13:37:53 +000053# example: -v state2test:omci-flows-pushed
TorstenThieme1619db22020-04-03 12:01:15 +000054${state2test} 6
TorstenThiemee0e5bba2020-08-05 11:20:13 +000055# test mode variable, can be passed via the command line too, valid values: SingleState, Up2State, SingleStateTime
TorstenThieme87cd6202020-09-09 10:01:28 +000056# example: -v testmode:SingleStateTime
TorstenThieme1619db22020-04-03 12:01:15 +000057${testmode} SingleState
TorstenThieme87cd6202020-09-09 10:01:28 +000058# used tech profile, can be passed via the command line too, valid values: default (=1T1GEM), 1T4GEM, 1T8GEM
59# example: -v techprofile:1T4GEM
TorstenThiemee0e5bba2020-08-05 11:20:13 +000060${techprofile} default
TorstenThiemee0e5bba2020-08-05 11:20:13 +000061# flag debugmode is used, if true timeout calculation various, can be passed via the command line too
TorstenThieme87cd6202020-09-09 10:01:28 +000062# example: -v debugmode:True
TorstenThieme401af432020-06-11 15:53:53 +000063${debugmode} False
TorstenThiemee0e5bba2020-08-05 11:20:13 +000064# logging flag to enable Collect Logs, can be passed via the command line too
TorstenThieme87cd6202020-09-09 10:01:28 +000065# example: -v logging:True
TorstenThieme5e324e42020-07-27 09:36:16 +000066${logging} False
TorstenThieme66c91a82020-10-19 13:37:53 +000067# if True execution will be paused before clean up, only use in case of manual testing, do not use in ci pipeline!
TorstenThieme87cd6202020-09-09 10:01:28 +000068# example: -v pausebeforecleanup:True
TorstenThiemeb41007d2020-06-22 12:14:12 +000069${pausebeforecleanup} False
TorstenThiemed4f48962020-12-08 12:17:19 +000070# if True some outputs to console are done during running tests e.g. long duration flow test
71# example: -v print2console:True
72${print2console} False
TorstenThiemee0e5bba2020-08-05 11:20:13 +000073${data_dir} ../data
74
TorstenThieme1619db22020-04-03 12:01:15 +000075
76*** Test Cases ***
77ONU State Test
78 [Documentation] Validates the ONU Go adapter states
TorstenThieme66c91a82020-10-19 13:37:53 +000079 [Tags] sanityOnuGo StateTestOnuGo
TorstenThieme1619db22020-04-03 12:01:15 +000080 [Setup] Run Keywords Start Logging ONUStateTest
81 ... AND Setup Test
82 Run Keyword If ${has_dataplane} Clean Up Linux
TorstenThiemefd7e7ba2020-11-06 13:57:57 +000083 FOR ${I} IN RANGE 0 ${num_olts}
84 #get olt serial number
85 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
86 #validate olt states
87 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
88 Enable Device ${olt_device_id}
89 END
TorstenThieme94e4ae42020-05-18 13:01:42 +000090 ${timeStart} = Get Current Date
91 Set Global Variable ${timeStart}
TorstenThieme1619db22020-04-03 12:01:15 +000092 Run Keyword If "${testmode}"=="SingleState" Do ONU Single State Test
93 ... ELSE IF "${testmode}"=="Up2State" Do ONU Up To State Test
TorstenThieme94e4ae42020-05-18 13:01:42 +000094 ... ELSE IF "${testmode}"=="SingleStateTime" Do ONU Single State Test Time
TorstenThieme1619db22020-04-03 12:01:15 +000095 ... ELSE Fail The testmode (${testmode}) is not valid!
TorstenThieme5e324e42020-07-27 09:36:16 +000096 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
TorstenThieme1619db22020-04-03 12:01:15 +000097 ... AND Stop Logging ONUStateTest
98
TorstenThiemee0e5bba2020-08-05 11:20:13 +000099Check Loaded Tech Profile
100 [Documentation] Validates the loaded Tech Profile
101 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
102 ... Check will be executed only the reached ONU state is 5 (tech-profile-config-download-success) or higher
TorstenThieme66c91a82020-10-19 13:37:53 +0000103 [Tags] functionalOnuGo CheckTechProfileOnuGo
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000104 [Setup] Start Logging ONUCheckTechProfile
TorstenThieme66c91a82020-10-19 13:37:53 +0000105 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
106 ... Do Check Tech Profile
TorstenThieme5a205ba2020-09-29 08:08:29 +0000107 ... ELSE Pass Execution ${skip_message} skipped
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000108 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
109 ... AND Stop Logging ONUCheckTechProfile
110
TorstenThieme401af432020-06-11 15:53:53 +0000111Onu Port Check
TorstenThieme87cd6202020-09-09 10:01:28 +0000112 [Documentation] Validates that all the UNI ports show up in ONOS
TorstenThieme00958682020-06-19 11:29:31 +0000113 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
TorstenThieme66c91a82020-10-19 13:37:53 +0000114 [Tags] functionalOnuGo PortTestOnuGo
TorstenThieme401af432020-06-11 15:53:53 +0000115 [Setup] Start Logging ONUPortTest
TorstenThiemed4f48962020-12-08 12:17:19 +0000116 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
117 ... Do Onu Port Check
118 ... ELSE Pass Execution ${skip_message} skipped
TorstenThieme5e324e42020-07-27 09:36:16 +0000119 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
TorstenThieme401af432020-06-11 15:53:53 +0000120 ... AND Stop Logging ONUPortTest
121
TorstenThieme17756ea2020-11-11 14:09:47 +0000122Onu Etcd Data Check
123 [Documentation] Validates ONU data stored in ETCD
124 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
125 [Tags] functionalOnuGo EtcdDataOnuGo
126 [Setup] Start Logging ONUEtcdDataTest
127 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
128 ... Do Onu Etcd Data Check
129 ... ELSE Pass Execution ${skip_message} skipped
130 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
131 ... AND Stop Logging ONUEtcdDataTest
132
TorstenThieme109683b2020-09-24 12:35:41 +0000133Onu Flow Check
134 [Documentation] Validates the onu flows in ONOS and Voltha
135 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
mpagenkoc2e7ba52020-11-12 13:50:57 +0000136 [Tags] functionalOnuGo FlowTestOnuGo
TorstenThieme109683b2020-09-24 12:35:41 +0000137 [Setup] Start Logging ONUFlowTest
TorstenThieme66c91a82020-10-19 13:37:53 +0000138 Run Keyword If '${onu_state}'=='omci-flows-pushed' Do Onu Flow Check
TorstenThieme5a205ba2020-09-29 08:08:29 +0000139 ... ELSE Pass Execution ${skip_message} skipped
TorstenThieme109683b2020-09-24 12:35:41 +0000140 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
141 ... AND Stop Logging ONUFlowTest
142
TorstenThiemee3841a02020-09-23 08:20:43 +0000143Disable Enable Onu Device
144 [Documentation] Disables/enables ONU Device and check states
145 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
TorstenThieme66c91a82020-10-19 13:37:53 +0000146 [Tags] functionalOnuGo DisableEnableOnuGo
TorstenThiemee3841a02020-09-23 08:20:43 +0000147 [Setup] Start Logging DisableEnableONUDevice
TorstenThieme66c91a82020-10-19 13:37:53 +0000148 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
149 ... Do Disable Enable Onu Test
TorstenThieme5a205ba2020-09-29 08:08:29 +0000150 ... ELSE Pass Execution ${skip_message} skipped
TorstenThiemee3841a02020-09-23 08:20:43 +0000151 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
152 ... AND Stop Logging DisableEnableONUDevice
153
TorstenThieme015d5c02020-09-11 09:53:34 +0000154Reconcile Onu Device
155 [Documentation] Reconciles ONU Device and check state
156 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
TorstenThieme66c91a82020-10-19 13:37:53 +0000157 [Tags] functionalOnuGo ReconcileOnuGo
TorstenThieme015d5c02020-09-11 09:53:34 +0000158 [Setup] Start Logging ReconcileONUDevice
TorstenThieme66c91a82020-10-19 13:37:53 +0000159 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
160 ... Do Reconcile Onu Device
TorstenThieme5a205ba2020-09-29 08:08:29 +0000161 ... ELSE Pass Execution ${skip_message} skipped
TorstenThieme015d5c02020-09-11 09:53:34 +0000162 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
163 ... AND Stop Logging ReconcileONUDevice
164
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000165Power Off Power On Onu Device
166 [Documentation] Power off and Power on of all ONU Devices and check state
167 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
TorstenThieme66c91a82020-10-19 13:37:53 +0000168 [Tags] functionalOnuGo PowerOffPowerOnOnuGo
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000169 [Setup] Start Logging PowerOffPowerOnONUDevice
TorstenThieme66c91a82020-10-19 13:37:53 +0000170 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
171 ... Do Power Off Power On Onu Device
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000172 ... ELSE Pass Execution ${skip_message} skipped
173 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
174 ... AND Stop Logging PowerOffPowerOnONUDevice
175
TorstenThieme1fbe8082020-10-21 13:27:59 +0000176Soft Reboot Onu Device
177 [Documentation] Reboots softly all ONU Devices and check state
178 ... Assuming that ONU State Test was executed where all the ONUs are reached the expected state!
179 [Tags] functionalOnuGo SoftRebootOnuGo
180 [Setup] Start Logging SoftRebootONUDevice
181 Run Keyword If '${onu_state}'=='tech-profile-config-download-success' or '${onu_state}'=='omci-flows-pushed'
182 ... Do Soft Reboot Onu Device
183 ... ELSE Pass Execution ${skip_message} skipped
184 [Teardown] Run Keywords Run Keyword If ${logging} Collect Logs
185 ... AND Stop Logging SoftRebootONUDevice
186
TorstenThieme1619db22020-04-03 12:01:15 +0000187*** Keywords ***
188Setup Suite
189 [Documentation] Set up the test suite
TorstenThieme87cd6202020-09-09 10:01:28 +0000190 ${LogInfo}= Catenate
191 ... \r\nPassed arguments:
TorstenThieme66c91a82020-10-19 13:37:53 +0000192 ... state2test:${state2test}, testmode:${testmode}, techprofile:${techprofile},
TorstenThieme015d5c02020-09-11 09:53:34 +0000193 ... debugmode:${debugmode}, logging:${logging}, pausebeforecleanup:${pausebeforecleanup},
TorstenThiemed4f48962020-12-08 12:17:19 +0000194 ... print2console:${print2console}
TorstenThieme87cd6202020-09-09 10:01:28 +0000195 Log ${LogInfo} console=yes
TorstenThieme1619db22020-04-03 12:01:15 +0000196 Common Test Suite Setup
TorstenThieme5a205ba2020-09-29 08:08:29 +0000197 # prepare skip message in yellow for console log
198 ${skip}= Evaluate "\\033[33mSKIP\\033[0m"
199 ${skipped}= Evaluate "\\033[33m${SPACE*14} ===> Test case above was skipped! <=== ${SPACE*15}\\033[0m"
200 ${skip_message} Catenate ${skipped} | ${skip} |
201 Set Suite Variable ${skip_message}
TorstenThiemed4f48962020-12-08 12:17:19 +0000202 Set Suite Variable ${all_onu_timeout} ${timeout}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000203 Run Keyword If ${num_all_onus}>4 Calculate Timeout
TorstenThieme1488d772020-09-28 10:52:43 +0000204 ${techprofile}= Set Variable If "${techprofile}"=="1T1GEM" default ${techprofile}
TorstenThieme73850052020-12-10 12:43:28 +0000205 Set Suite Variable ${techprofile}
TorstenThieme87cd6202020-09-09 10:01:28 +0000206 Run Keyword If "${techprofile}"=="default" Log To Console \nTechProfile:default (1T1GEM)
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000207 ... ELSE IF "${techprofile}"=="1T4GEM" Set Tech Profile 1T4GEM
208 ... ELSE IF "${techprofile}"=="1T8GEM" Set Tech Profile 1T8GEM
209 ... ELSE Fail The TechProfile (${techprofile}) is not valid!
210 ${onos_ssh_connection} Open ONOS SSH Connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
211 Set Suite Variable ${onos_ssh_connection}
TorstenThieme66c91a82020-10-19 13:37:53 +0000212 # map the passed onu state to reached and make it visible for test suite
213 ${admin_state} ${oper_status} ${connect_status} ${onu_state_nb} ${onu_state}=
214 ... Map State ${state2test}
215 Set Suite Variable ${admin_state}
216 Set Suite Variable ${oper_status}
217 Set Suite Variable ${connect_status}
218 Set Suite Variable ${onu_state_nb}
219 Set Suite Variable ${onu_state}
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000220 # delete etcd MIB Template Data
221 Delete MIB Template Data
TorstenThieme1619db22020-04-03 12:01:15 +0000222
TorstenThiemeb41007d2020-06-22 12:14:12 +0000223Teardown Suite
224 [Documentation] Replaces the Suite Teardown in utils.robot.
225 ... Cleans up and checks all ONU ports disabled in ONOS.
226 ... Furthermore gives the possibility to pause the execution.
227 Run Keyword If ${pausebeforecleanup} Import Library Dialogs
228 Run Keyword If ${pausebeforecleanup} Pause Execution Press OK to continue with clean up!
TorstenThieme87cd6202020-09-09 10:01:28 +0000229 Run Keyword If ${pausebeforecleanup} Log Teardown will be continued... console=yes
TorstenThiemeb41007d2020-06-22 12:14:12 +0000230 Run Keyword If ${teardown_device} Delete All Devices and Verify
TorstenThiemefcafba02020-11-11 16:18:50 +0000231 Validate Onu Data In Etcd 0
TorstenThiemed4f48962020-12-08 12:17:19 +0000232 # Re-open ssh connection to onos since no keep alive is implemented in SSH library
233 Close ONOS SSH Connection ${onos_ssh_connection}
234 ${onos_ssh_connection} Open ONOS SSH Connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
235 Set Suite Variable ${onos_ssh_connection}
236 Wait for Ports in ONOS for all OLTs ${onos_ssh_connection} 0 BBSM ${timeout}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000237 Close ONOS SSH Connection ${onos_ssh_connection}
TorstenThieme87cd6202020-09-09 10:01:28 +0000238 Remove Tech Profile
TorstenThiemeb41007d2020-06-22 12:14:12 +0000239
TorstenThieme1619db22020-04-03 12:01:15 +0000240Setup Test
241 [Documentation] Pre-test Setup
242 #test for empty device list
243 Test Empty Device List
244 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
Suchitra Vemurib7253a52020-07-14 22:35:17 -0700245 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
TorstenThieme1619db22020-04-03 12:01:15 +0000246 Run Keyword If ${has_dataplane} Sleep 60s
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000247 # Create a list of olt ids (logical and device_id)
248 ${olt_ids} Create List
249 FOR ${I} IN RANGE 0 ${num_olts}
250 #create/preprovision device
251 ${olt_device_id}= Create Device ${list_olts}[${I}][ip] ${OLT_PORT}
252 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
253 #validate olt states
254 Wait Until Keyword Succeeds ${timeout} 5s
255 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id}
256 Sleep 5s
257 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
258 ${olt} Create Dictionary device_id ${olt_device_id} logical_id ${logical_id}
259 ... of_id ${of_id} sn ${olt_serial_number}
260 Append To List ${olt_ids} ${olt}
261 END
262 Set Global Variable ${olt_ids}
TorstenThieme1619db22020-04-03 12:01:15 +0000263
264Calculate Timeout
265 [Documentation] Calculates the timeout regarding num-onus in case of more than 4 onus
TorstenThiemed4f48962020-12-08 12:17:19 +0000266 ${all_onu_timeout} Fetch From Left ${all_onu_timeout} s
267 ${all_onu_timeout}= evaluate ${all_onu_timeout}+((${num_all_onus}-4)*10)
268 ${all_onu_timeout}= Set Variable If (not ${debugmode}) and (${all_onu_timeout}>300)
269 ... 300 ${all_onu_timeout}
270 ${all_onu_timeout}= Catenate SEPARATOR= ${all_onu_timeout} s
271 Set Suite Variable ${all_onu_timeout}
TorstenThieme1619db22020-04-03 12:01:15 +0000272
273Do ONU Up To State Test
274 [Documentation] This keyword performs Up2State Test
275 ... All states up to the passed have to be checked
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000276 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme1619db22020-04-03 12:01:15 +0000277 ${src}= Set Variable ${hosts.src[${I}]}
278 ${dst}= Set Variable ${hosts.dst[${I}]}
TorstenThieme66c91a82020-10-19 13:37:53 +0000279 Run Keyword If ${onu_state_nb}>=1
TorstenThiemed4f48962020-12-08 12:17:19 +0000280 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000281 ... Validate Device ENABLED ACTIVATING REACHABLE
282 ... ${src['onu']} onu=True onu_reason=activating-onu
TorstenThieme66c91a82020-10-19 13:37:53 +0000283 Run Keyword If ${onu_state_nb}>=2
TorstenThiemed4f48962020-12-08 12:17:19 +0000284 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000285 ... Validate Device ENABLED ACTIVATING REACHABLE
286 ... ${src['onu']} onu=True onu_reason=starting-openomci
TorstenThieme66c91a82020-10-19 13:37:53 +0000287 Run Keyword If ${onu_state_nb}>=3
TorstenThiemed4f48962020-12-08 12:17:19 +0000288 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000289 ... Validate Device ENABLED ACTIVATING REACHABLE
290 ... ${src['onu']} onu=True onu_reason=discovery-mibsync-complete
TorstenThieme66c91a82020-10-19 13:37:53 +0000291 Run Keyword If ${onu_state_nb}>=4
TorstenThiemed4f48962020-12-08 12:17:19 +0000292 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000293 ... Validate Device ENABLED ACTIVE REACHABLE
294 ... ${src['onu']} onu=True onu_reason=initial-mib-downloaded
TorstenThieme66c91a82020-10-19 13:37:53 +0000295 Run Keyword If ${onu_state_nb}>=5
TorstenThiemed4f48962020-12-08 12:17:19 +0000296 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000297 ... Validate Device ENABLED ACTIVE REACHABLE
298 ... ${src['onu']} onu=True onu_reason=tech-profile-config-download-success
TorstenThieme66c91a82020-10-19 13:37:53 +0000299 Run Keyword If ${onu_state_nb}>=6
TorstenThiemed4f48962020-12-08 12:17:19 +0000300 ... Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme1619db22020-04-03 12:01:15 +0000301 ... Validate Device ENABLED ACTIVE REACHABLE
302 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
303 END
304
305Do ONU Single State Test
306 [Documentation] This keyword performs SingleState Test
307 ... Only the passed state has to be checked
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000308 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme1619db22020-04-03 12:01:15 +0000309 ${src}= Set Variable ${hosts.src[${I}]}
310 ${dst}= Set Variable ${hosts.dst[${I}]}
TorstenThiemed4f48962020-12-08 12:17:19 +0000311 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme015d5c02020-09-11 09:53:34 +0000312 ... Validate Device ${admin_state} ${oper_status} ${connect_status}
313 ... ${src['onu']} onu=True onu_reason=${onu_state}
TorstenThieme1619db22020-04-03 12:01:15 +0000314 END
315
TorstenThieme94e4ae42020-05-18 13:01:42 +0000316Do ONU Single State Test Time
317 [Documentation] This keyword performs SingleState Test with calculate running time
318 ... Only the passed state has to be checked and the duration each single onu adapter needed
319 ... will be calculated and printed out
TorstenThieme17becfc2020-10-08 09:46:27 +0000320 #${ListfinishedONUs} Create List
321 #Set Global Variable ${ListfinishedONUs}
TorstenThieme94e4ae42020-05-18 13:01:42 +0000322 Create File ONU_Startup_Time.txt This file contains the startup times of all ONUs.
323 ${list_onus} Create List
TorstenThieme401af432020-06-11 15:53:53 +0000324 Build ONU SN List ${list_onus}
TorstenThiemed4f48962020-12-08 12:17:19 +0000325 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${all_onu_timeout} 50ms
TorstenThieme9949b172020-06-16 10:00:15 +0000326 ... Validate ONU Devices MIB State With Duration
TorstenThiemed4f48962020-12-08 12:17:19 +0000327 ... ${onu_state} ${list_onus} ${timeStart} print2console=${print2console}
TorstenThieme401af432020-06-11 15:53:53 +0000328 ... output_file=ONU_Startup_Time.txt
TorstenThieme94e4ae42020-05-18 13:01:42 +0000329
TorstenThieme1619db22020-04-03 12:01:15 +0000330Do Onu Port Check
TorstenThieme00958682020-06-19 11:29:31 +0000331 [Documentation] Check that all the UNI ports show up in ONOS
TorstenThiemed4f48962020-12-08 12:17:19 +0000332 Wait for Ports in ONOS for all OLTs ${onos_ssh_connection} ${num_all_onus} BBSM
TorstenThieme9949b172020-06-16 10:00:15 +0000333
TorstenThieme17756ea2020-11-11 14:09:47 +0000334Do Onu Etcd Data Check
335 [Documentation] Check Onu data stored in etcd
336 Validate Onu Data In Etcd
337
TorstenThieme109683b2020-09-24 12:35:41 +0000338Do Onu Flow Check
TorstenThieme52ef8392020-11-10 13:42:26 +0000339 [Documentation] This keyword iterate all OLTs and performs Do Onu Flow Checks Per OLT
340 # Check and store vlan rules
341 ${firstvlanrules}= Run Keyword And Continue On Failure Validate Vlan Rules In Etcd
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000342 FOR ${J} IN RANGE 0 ${num_olts}
343 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
344 ${onu_count}= Set Variable ${list_olts}[${J}][onucount]
345 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
346 ... ${olt_serial_number}
347 Set Global Variable ${of_id}
TorstenThiemed4f48962020-12-08 12:17:19 +0000348 # Verify Default Meter in ONOS (valid only for ATT)
349 Do Onu Subscriber Add Per OLT ${of_id} ${olt_serial_number} ${onu_count}
350 END
351 FOR ${J} IN RANGE 0 ${num_olts}
352 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
353 ${onu_count}= Set Variable ${list_olts}[${J}][onucount]
354 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
355 ... ${olt_serial_number}
356 Set Global Variable ${of_id}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000357 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
358 ... Get NNI Port in ONOS ${of_id}
359 Set Global Variable ${nni_port}
360 # Verify Default Meter in ONOS (valid only for ATT)
TorstenThiemed4f48962020-12-08 12:17:19 +0000361 Do Onu Flow Check Per OLT ${of_id} ${nni_port} ${olt_serial_number} ${onu_count}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000362 END
TorstenThieme52ef8392020-11-10 13:42:26 +0000363 #log flows for verification
364 ${flowsresult}= Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT} flows -s
365 log ${flowsresult}
366 #check for previous state is kept (normally omci-flows-pushed)
367 Sleep 10s
368 Run Keyword And Continue On Failure Do Current State Test All Onus ${state2test}
369 ${secondvlanrules}= Run Keyword And Continue On Failure Validate Vlan Rules In Etcd nbofcookieslice=3
370 ... prevvlanrules=${firstvlanrules}
371 FOR ${J} IN RANGE 0 ${num_olts}
372 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
373 ${onu_count}= Set Variable ${list_olts}[${J}][onucount]
374 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
375 ... ${olt_serial_number}
376 Set Global Variable ${of_id}
377 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
378 ... Get NNI Port in ONOS ${of_id}
379 Set Global Variable ${nni_port}
380 # Verify Default Meter in ONOS (valid only for ATT)
381 Do Onu Subscriber Remove Per OLT ${of_id} ${nni_port} ${olt_serial_number} ${onu_count}
382 END
383 #check for previous state is kept (normally omci-flows-pushed)
384 Sleep 10s
TorstenThiemed4f48962020-12-08 12:17:19 +0000385 Run Keyword If ${print2console} Log \r\nStart State Test All Onus. console=yes
TorstenThieme52ef8392020-11-10 13:42:26 +0000386 Run Keyword And Continue On Failure Do Current State Test All Onus ${state2test}
TorstenThiemed4f48962020-12-08 12:17:19 +0000387 Run Keyword If ${print2console} Log \r\nFinished State Test All Onus. console=yes
TorstenThieme52ef8392020-11-10 13:42:26 +0000388 Run Keyword And Continue On Failure Validate Vlan Rules In Etcd prevvlanrules=${firstvlanrules}
389 ... setvidequal=True
TorstenThiemed4f48962020-12-08 12:17:19 +0000390 # Re-open ssh connection to onos since no keep alive is implemented in SSH library
391 Close ONOS SSH Connection ${onos_ssh_connection}
392 ${onos_ssh_connection} Open ONOS SSH Connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
393 Set Suite Variable ${onos_ssh_connection}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000394
TorstenThiemed4f48962020-12-08 12:17:19 +0000395Do Onu Subscriber Add Per OLT
396 [Documentation] Add Subscriber per OLT
397 [Arguments] ${of_id} ${olt_serial_number} ${num_onus}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000398 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme109683b2020-09-24 12:35:41 +0000399 ${src}= Set Variable ${hosts.src[${I}]}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000400 Continue For Loop If "${olt_serial_number}"!="${src['olt']}"
TorstenThieme109683b2020-09-24 12:35:41 +0000401 ${onu_device_id}= Get Device ID From SN ${src['onu']}
402 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
403 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
404 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
405 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
406 ... volt-add-subscriber-access ${of_id} ${onu_port}
TorstenThiemed4f48962020-12-08 12:17:19 +0000407 Run Keyword If ${print2console} Log \r\n[${I}] volt-add-subscriber-access ${of_id} ${onu_port}.
408 ... console=yes
409 END
410
411Do Onu Flow Check Per OLT
412 [Documentation] Checks all ONU flows show up in ONOS and Voltha
413 [Arguments] ${of_id} ${nni_port} ${olt_serial_number} ${num_onus}
414 FOR ${I} IN RANGE 0 ${num_all_onus}
415 ${src}= Set Variable ${hosts.src[${I}]}
416 Continue For Loop If "${olt_serial_number}"!="${src['olt']}"
417 ${onu_device_id}= Get Device ID From SN ${src['onu']}
418 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
419 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
TorstenThieme109683b2020-09-24 12:35:41 +0000420 # Verify subscriber access flows are added for the ONU port
421 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
422 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
423 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
TorstenThiemed4f48962020-12-08 12:17:19 +0000424 ${logoutput} Catenate \r\n[${I}] Verify Subscriber Access Flows Added For
425 ... ONU ${of_id} ${onu_port} ${src['c_tag']} ${src['s_tag']}.
426 Run Keyword If ${print2console} Log ${logoutput} console=yes
TorstenThieme109683b2020-09-24 12:35:41 +0000427 END
TorstenThieme52ef8392020-11-10 13:42:26 +0000428
429Do Onu Subscriber Remove Per OLT
430 [Documentation] Removes per OLT subscribers in ONOS and Voltha
431 [Arguments] ${of_id} ${nni_port} ${olt_serial_number} ${num_onus}
432 FOR ${I} IN RANGE 0 ${num_all_onus}
433 ${src}= Set Variable ${hosts.src[${I}]}
434 ${dst}= Set Variable ${hosts.dst[${I}]}
435 Continue For Loop If "${olt_serial_number}"!="${src['olt']}"
436 ${onu_device_id}= Get Device ID From SN ${src['onu']}
437 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
438 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
439 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
440 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
441 ... volt-remove-subscriber-access ${of_id} ${onu_port}
TorstenThiemed4f48962020-12-08 12:17:19 +0000442 Run Keyword If ${print2console} Log \r\n[${I}] volt-remove-subscriber-access ${of_id} ${onu_port}.
443 ... console=yes
TorstenThieme52ef8392020-11-10 13:42:26 +0000444 END
TorstenThieme109683b2020-09-24 12:35:41 +0000445
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000446Set Tech Profile
447 [Documentation] This keyword set the passed TechProfile for the test
448 [Arguments] ${TechProfile}
TorstenThieme87cd6202020-09-09 10:01:28 +0000449 Log To Console \nTechProfile:${TechProfile}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000450 ${namespace}= Set Variable default
Andrea Campanella60dde302020-09-09 18:42:56 +0200451 ${podname}= Set Variable etcd
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000452 ${src}= Set Variable ${data_dir}/TechProfile-${TechProfile}.json
453 ${dest}= Set Variable /tmp/flexpod.json
454 ${command} Catenate
TorstenThiemea0221942020-12-09 11:35:03 +0000455 ... /bin/sh -c 'cat ${dest} | ETCDCTL_API=3 etcdctl put service/voltha/technology_profiles/XGS-PON/64'
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000456 Copy File To Pod ${namespace} ${podname} ${src} ${dest}
Holger Hildebrandt23147742020-11-16 10:13:21 +0000457 Exec Pod In Kube ${namespace} ${podname} ${command}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000458 ${commandget} Catenate
TorstenThiemea0221942020-12-09 11:35:03 +0000459 ... /bin/sh -c 'ETCDCTL_API=3 etcdctl get --prefix service/voltha/technology_profiles/XGS-PON/64'
Holger Hildebrandt23147742020-11-16 10:13:21 +0000460 Exec Pod In Kube ${namespace} ${podname} ${commandget}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000461
462Remove Tech Profile
463 [Documentation] This keyword removes TechProfile
TorstenThieme015d5c02020-09-11 09:53:34 +0000464 Log To Console \nTechProfile:${TechProfile}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000465 ${namespace}= Set Variable default
Andrea Campanella60dde302020-09-09 18:42:56 +0200466 ${podname}= Set Variable etcd
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000467 ${command} Catenate
TorstenThiemea0221942020-12-09 11:35:03 +0000468 ... /bin/sh -c 'ETCDCTL_API=3 etcdctl del --prefix service/voltha/technology_profiles/XGS-PON/64'
Holger Hildebrandt23147742020-11-16 10:13:21 +0000469 Exec Pod In Kube ${namespace} ${podname} ${command}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000470 ${commandget} Catenate
TorstenThiemea0221942020-12-09 11:35:03 +0000471 ... /bin/sh -c 'ETCDCTL_API=3 etcdctl get --prefix service/voltha/technology_profiles/XGS-PON/64'
Holger Hildebrandt23147742020-11-16 10:13:21 +0000472 Exec Pod In Kube ${namespace} ${podname} ${commandget}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000473
474Do Check Tech Profile
475 [Documentation] This keyword checks the loaded TechProfile
476 ${namespace}= Set Variable default
Andrea Campanella60dde302020-09-09 18:42:56 +0200477 ${podname}= Set Variable etcd
TorstenThiemed4f48962020-12-08 12:17:19 +0000478 ${stackname}= Get Stack Name
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000479 ${commandget} Catenate
TorstenThiemed4f48962020-12-08 12:17:19 +0000480 ... /bin/sh -c 'ETCDCTL_API=3 etcdctl get --prefix service/${stackname}/technology_profiles/XGS-PON/64'
Holger Hildebrandt23147742020-11-16 10:13:21 +0000481 ${result}= Exec Pod In Kube ${namespace} ${podname} ${commandget}
TorstenThiemee0e5bba2020-08-05 11:20:13 +0000482 ${num_gem_ports}= Set Variable 1
483 ${num_gem_ports}= Set Variable If
484 ... "${techprofile}"=="default" 1
485 ... "${techprofile}"=="1T4GEM" 4
486 ... "${techprofile}"=="1T8GEM" 8
487 @{resultList} Split String ${result} separator=,
488 ${num_of_count_matches}= Get Match Count ${resultList} "num_gem_ports": ${num_gem_ports}
489 ... whitespace_insensitive=True
TorstenThiemea0221942020-12-09 11:35:03 +0000490 ${num_of_expected_matches}= Evaluate ${num_all_onus}
TorstenThiemea0221942020-12-09 11:35:03 +0000491 Should Be Equal As Integers ${num_of_expected_matches} ${num_of_count_matches}
492 ... TechProfile (${TechProfile}) not loaded correctly:${num_of_count_matches} of ${num_of_expected_matches}
TorstenThieme015d5c02020-09-11 09:53:34 +0000493
494Do Disable Enable Onu Test
495 [Documentation] This keyword disables/enables all onus and checks the states.
TorstenThieme1fbe8082020-10-21 13:27:59 +0000496 [Arguments] ${state2check}=${state2test} ${checkstatebeforedisable}=True
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000497 ... ${state2checkafterdisable}=tech-profile-config-delete-success
TorstenThieme1fbe8082020-10-21 13:27:59 +0000498 Run Keyword If ${checkstatebeforedisable} Do Current State Test All Onus ${state2check}
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000499 Do Disable Onu Device
TorstenThiemed4f48962020-12-08 12:17:19 +0000500 ${alternative_onu_reason}= Set Variable If
501 ... '${state2checkafterdisable}'=='tech-profile-config-delete-success' omci-flows-deleted
502 ... '${state2checkafterdisable}'=='omci-admin-lock' tech-profile-config-delete-success ${EMPTY}
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000503 Do Current State Test All Onus ${state2checkafterdisable} alternativeonustate=${alternative_onu_reason}
TorstenThieme17becfc2020-10-08 09:46:27 +0000504 Log Ports
505 #check no port is enabled in ONOS
TorstenThiemed4f48962020-12-08 12:17:19 +0000506 Wait for Ports in ONOS for all OLTs ${onos_ssh_connection} 0 BBSM
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000507 Do Enable Onu Device
TorstenThieme17becfc2020-10-08 09:46:27 +0000508 Do Current State Test All Onus ${state2check}
509 Log Ports onlyenabled=True
510 #check that all the UNI ports show up in ONOS again
TorstenThiemed4f48962020-12-08 12:17:19 +0000511 Wait for Ports in ONOS for all OLTs ${onos_ssh_connection} ${num_all_onus} BBSM
TorstenThieme015d5c02020-09-11 09:53:34 +0000512
513Do Reconcile Onu Device
514 [Documentation] This keyword reconciles ONU device and check the state afterwards.
515 ... Following steps will be executed:
516 ... - restart openonu adaptor
517 ... - check openonu adaptor is ready again
518 ... - check previous state is kept
519 ... - ONU-Disable
520 ... - wait some seconds
521 ... - check for state omci-admin-lock
522 ... - ONU-Enable
523 ... - wait some seconds
524 ... - check for state onu-reenabled
525 ... - port check
TorstenThieme015d5c02020-09-11 09:53:34 +0000526 ${list_openonu_apps} Create List adapter-open-onu
527 ${namespace}= Set Variable voltha
528 ${adaptorname}= Set Variable open-onu
TorstenThieme015d5c02020-09-11 09:53:34 +0000529 Kill Adaptor ${namespace} ${adaptorname}
530 Sleep 5s
531 Wait For Pods Ready ${namespace} ${list_openonu_apps}
532 Do Disable Enable Onu Test
TorstenThiemed4f48962020-12-08 12:17:19 +0000533 Do Onu Port Check
TorstenThieme015d5c02020-09-11 09:53:34 +0000534
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000535Do Power Off Power On Onu Device
536 [Documentation] This keyword power off/on all onus and checks the states.
537 Do Power Off ONU Device
538 Sleep 5s
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000539 #Do Current State Test All Onus stopping-openomci
540 Do Current State Test All Onus tech-profile-config-delete-success
541 ... ENABLED DISCOVERED UNREACHABLE alternativeonustate=omci-flows-deleted
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000542 Do Power On ONU Device
543 Do Current State Test All Onus ${state2test}
544
TorstenThieme1fbe8082020-10-21 13:27:59 +0000545Do Soft Reboot Onu Device
546 [Documentation] This keyword reboots softly all onus and checks the states.
547 ${namespace}= Set Variable voltha
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000548 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme1fbe8082020-10-21 13:27:59 +0000549 ${src}= Set Variable ${hosts.src[${I}]}
550 ${onu_device_id}= Get Device ID From SN ${src['onu']}
TorstenThiemea0b11602020-10-30 08:39:24 +0000551 Reboot ONU ${onu_device_id} False
TorstenThieme1fbe8082020-10-21 13:27:59 +0000552 END
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000553 Run Keyword Unless ${has_dataplane} Do Current State Test All Onus tech-profile-config-delete-success
554 ... ENABLED DISCOVERED REACHABLE alternativeonustate=omci-flows-deleted
TorstenThieme73850052020-12-10 12:43:28 +0000555 Sleep 5s
TorstenThiemea0b11602020-10-30 08:39:24 +0000556 Run Keyword Unless ${has_dataplane} Do Disable Enable Onu Test checkstatebeforedisable=False
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000557 ... state2checkafterdisable=omci-admin-lock
TorstenThiemea0b11602020-10-30 08:39:24 +0000558 Run Keyword If ${has_dataplane} Do Current State Test All Onus omci-flows-pushed
TorstenThiemed4f48962020-12-08 12:17:19 +0000559 Do Onu Port Check
TorstenThieme1fbe8082020-10-21 13:27:59 +0000560
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000561Do Disable Onu Device
562 [Documentation] This keyword disables all onus.
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000563 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme17becfc2020-10-08 09:46:27 +0000564 ${src}= Set Variable ${hosts.src[${I}]}
565 ${onu_device_id}= Get Device ID From SN ${src['onu']}
566 Disable Device ${onu_device_id}
567 Wait Until Keyword Succeeds 20s 2s Test Devices Disabled in VOLTHA Id=${onu_device_id}
568 END
569
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000570Do Enable Onu Device
571 [Documentation] This keyword enables all onus.
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000572 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme17becfc2020-10-08 09:46:27 +0000573 ${src}= Set Variable ${hosts.src[${I}]}
574 ${onu_device_id}= Get Device ID From SN ${src['onu']}
575 Enable Device ${onu_device_id}
576 END
577
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000578Do Power Off ONU Device
579 [Documentation] This keyword power off all onus.
580 ${namespace}= Set Variable voltha
TorstenThiemefd7e7ba2020-11-06 13:57:57 +0000581 FOR ${I} IN RANGE 0 ${num_all_onus}
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000582 ${src}= Set Variable ${hosts.src[${I}]}
Holger Hildebrandt23147742020-11-16 10:13:21 +0000583 ${result}= Exec Pod In Kube ${namespace} bbsim bbsimctl onu shutdown ${src['onu']}
TorstenThieme5ca1dd12020-10-16 08:42:16 +0000584 Should Contain ${result} successfully msg=Can not shutdown ${src['onu']} values=False
585 END