blob: 93a5b956814d7f961d3ec1adb8f15273248f466b [file] [log] [blame]
Gayathri.Selvan283a63c2020-01-23 04:09:18 +00001# Copyright 2017 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000014
15*** Settings ***
HungWei Chiu87edb1f2020-01-28 00:10:58 +000016Documentation Test various failure scenarios
HungWei Chiu1408fb92020-03-03 19:43:30 -050017Suite Setup Setup Suite
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000018Test Setup Setup
19Test Teardown Teardown
20#Suite Teardown Teardown Suite
21Library Collections
22Library String
23Library OperatingSystem
24Library XML
25Library RequestsLibrary
26Library ../../libraries/DependencyLibrary.py
27Resource ../../libraries/onos.robot
28Resource ../../libraries/voltctl.robot
29Resource ../../libraries/voltha.robot
30Resource ../../libraries/utils.robot
31Resource ../../libraries/k8s.robot
32Resource ../../variables/variables.robot
HungWei Chiu87edb1f2020-01-28 00:10:58 +000033Resource ../../libraries/power_switch.robot
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000034
35*** Variables ***
36${POD_NAME} flex-ocp-cord
37${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
38${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
39#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
40${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
41${HELM_CHARTS_DIR} ~/helm-charts
42${VOLTHA_POD_NUM} 8
43${NAMESPACE} voltha
Suchitra Vemurica1c5d32020-04-09 18:45:29 -070044${DEFAULTSPACE} default
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000045# For below variable value, using deployment name as using grep for
46# parsing radius pod name, we can also use full radius pod name
47${RESTART_POD_NAME} radius
48${timeout} 60s
49${of_id} 0
50${logical_id} 0
Suchitra Vemuri109feb12020-01-23 16:25:02 -080051${has_dataplane} True
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000052${teardown_device} False
53${scripts} ../../scripts
54
Andy Bavierabeba262020-02-07 16:22:16 -070055# Per-test logging on failure is turned off by default; set this variable to enable
56${container_log_dir} ${None}
57
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000058*** Test Cases ***
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080059Verify ONU after rebooting physically
60 [Documentation] Test the ONU functionality by physically turning on/off ONU.
61 ... Prerequisite : Subscriber are authenticated/DHCP/pingable state
62 ... Test case runs only on the PODs that are configured with PowerSwitch that
63 ... controls the power off/on ONUs/OLT remotely (simulating a physical reboot)
64 ... VOL-2634
65 [Tags] functional PowerSwitch
Andy Baviere187eda2020-04-20 15:00:02 -070066 [Setup] Start Logging ONUreboot_PowerSwitch
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080067 [Teardown] Run Keywords Collect Logs
Suchitra Vemuri9a6dd6d2020-02-28 17:46:26 -080068 ... AND Stop Logging ONUreboot_PowerSwitch
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080069 ... AND Delete All Devices and Verify
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080070 # Add OLT device
71 setup
72 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
73 Run Keyword If ${has_dataplane} Clean Up Linux
74 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
75 Power Switch Connection Suite ${web_power_switch.ip} ${web_power_switch.user} ${web_power_switch.password}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080076 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080077 ${src}= Set Variable ${hosts.src[${I}]}
78 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080079 ${of_id}= Get ofID From OLT List ${src['olt']}
Hardik Windlass0560f802020-05-06 23:06:53 +053080 ${onu_device_id}= Get Device ID From SN ${src['onu']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080081 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
82 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
83 Disable Switch Outlet ${src['power_switch_port']}
84 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
85 ... Wait Until Keyword Succeeds 60s 2s
86 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
87 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hardik Windlass7c801e62020-05-04 19:33:21 +053088 # Remove Subscriber Access (To replicate ATT workflow)
89 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
90 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080091
92 Enable Switch Outlet ${src['power_switch_port']}
93 # Check ONU port is Enabled in ONOS
94 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -070095 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080096 # Verify EAPOL flows are added for the ONU port
97 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -080098 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080099 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700100 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800101 ... ENABLED ACTIVE REACHABLE
102 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlass7c801e62020-05-04 19:33:21 +0530103 Run Keyword If ${has_dataplane} Clean Up Linux
104 # Perform Authentication
105 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
106 ... /tmp/wpa ${src['dp_iface_name']} log
107 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
108 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
109 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
110 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
111 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
112 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
113 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
114 ... volt-add-subscriber-access ${of_id} ${onu_port}
115 # Verify that no pending flows exist for the ONU port
116 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
117 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530118 # Verify subscriber access flows are added for the ONU port
119 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
120 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
121 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530122 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
123 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800124 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530125 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
126 ... ${dst['container_name']}
127 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
128 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
129 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800130 Run Keyword And Ignore Error Collect Logs
131 END
Suchitra Vemurid7096132020-02-21 14:50:47 -0800132 # Deleting OLT after tests completes independently (as this test doesn't not run on each POD)
Suchitra Vemuri0656bc22020-03-21 19:07:49 -0700133 #Run Keyword If ${has_dataplane} Delete Device and Verify
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800134
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700135Verify OLT after rebooting physically
136 [Documentation] Test the physical reboot of the OLT
137 ... Prerequisite : Subscriber are authenticated/DHCP/pingable state
138 ... Test performs a physical reboot, performs "reboot" from the OLT CLI
139 ... VOL-1956
140 [Tags] functional PhysicalOLTReboot
Andy Baviere187eda2020-04-20 15:00:02 -0700141 [Setup] Start Logging PhysicalOLTReboot
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700142 [Teardown] Run Keywords Collect Logs
143 ... AND Stop Logging PhysicalOLTReboot
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800144 ... AND Delete All Devices and Verify
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700145 # Add OLT device
146 setup
147 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
148 Run Keyword If ${has_dataplane} Clean Up Linux
149 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
150 # Reboot the OLT from the OLT CLI
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800151 FOR ${I} IN RANGE 0 ${olt_count}
152 ${olt_user}= Get From Dictionary ${list_olts}[${I}] user
153 ${olt_pass}= Get From Dictionary ${list_olts}[${I}] pass
154 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
155 ${olt_serial_number}= Get From Dictionary ${list_olts}[${I}] sn
156 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
157 Run Keyword If ${has_dataplane} Login And Run Command On Remote System
158 ... sudo reboot ${olt_ssh_ip} ${olt_user} ${olt_pass} prompt=#
159 END
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700160 Run Keyword And Ignore Error Collect Logs
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800161 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700162 ${src}= Set Variable ${hosts.src[${I}]}
163 ${dst}= Set Variable ${hosts.dst[${I}]}
164 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
165 ... Wait Until Keyword Succeeds 60s 2s
166 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
167 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
168 END
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800169 # Wait for the OLTs to come back up
170 FOR ${I} IN RANGE 0 ${olt_count}
171 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
172 ${olt_user}= Get From Dictionary ${list_olts}[${I}] user
173 ${olt_pass}= Get From Dictionary ${list_olts}[${I}] pass
174 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
175 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
176 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
177 ... Check Remote System Reachability True ${olt_ssh_ip}
178 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
179 ... Validate OLT Device ENABLED ACTIVE
180 ... REACHABLE ${olt_serial_number}
181 END
Suchitra Vemuri64db9652020-03-22 21:11:13 -0700182 # Waiting extra time for the ONUs to come up
183 Sleep 60s
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700184 Run Keyword And Ignore Error Collect Logs
185 Run Keyword If ${has_dataplane} Clean Up Linux
186 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
187 # Deleting OLT after test completes
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800188 #Run Keyword If ${has_dataplane} Delete All Devices and Verify
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700189
Suchitra Vemuri05180cb2020-03-26 13:13:27 -0700190Verify restart openolt-adapter container after subscriber provisioning
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800191 [Documentation] Restart openolt-adapter container after VOLTHA is operational.
192 ... Prerequisite : ONUs are authenticated and pingable.
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800193 [Tags] functional VOL-1958 Restart-OpenOlt released
Andy Baviere187eda2020-04-20 15:00:02 -0700194 [Setup] Start Logging Restart-OpenOlt
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800195 [Teardown] Run Keywords Collect Logs
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800196 ... AND Stop Logging Restart-OpenOlt
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800197 # Add OLT device
198 setup
199 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
200 Run Keyword If ${has_dataplane} Clean Up Linux
201 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
202 ${waitforRestart} Set Variable 120s
203 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
204 Log ${podStatusOutput}
205 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530206 ${podName} Set Variable adapter-open-olt
207 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
208 Sleep 5s
209 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
210 ... app ${podName} Running
Suchitra Vemuri312bfe02020-03-16 16:11:34 -0700211 # Wait for 1min after openolt adapter is restarted
212 Sleep 60s
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800213 Run Keyword If ${has_dataplane} Clean Up Linux
214 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Suchitra Vemurid7096132020-02-21 14:50:47 -0800215 Run Keyword and Ignore Error Collect Logs
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800216 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
217 Log ${podStatusOutput}
218 ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
219 Should Be Equal As Strings ${countAfterRestart} ${countBforRestart}
220 Log to console Pod ${podName} restarted and sanity checks passed successfully
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800221
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800222Check OLT/ONU Authentication After Radius Pod Restart
223 [Documentation] After radius restart, triggers reassociation, checks status and
224 ... authentication, validates dhcp and ping. Note : wpa reassociate works only when
225 ... wpa supplicant is running in background hence it is recommended to remove
226 ... teardown from previous test or uncomment 'Teardown None'.
227 ... Assuming that test1 was executed where all the ONUs are authenticated/DHCP/pingable
228 [Tags] functional RadiusRestart released
Andy Baviere187eda2020-04-20 15:00:02 -0700229 [Setup] Start Logging RadiusRestart
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800230 [Teardown] Run Keywords Collect Logs
231 ... AND Stop Logging RadiusRestart
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800232 ${waitforRestart} Set Variable 120s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530233 ${podName} Set Variable radius
234 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${DEFAULTSPACE} app ${podName}
235 Sleep 5s
236 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pods Status By Label ${DEFAULTSPACE}
237 ... app ${podName} Running
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800238 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800239 ${src}= Set Variable ${hosts.src[${I}]}
240 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800241 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800242 ${onu_device_id}= Get Device ID From SN ${src['onu']}
243 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s
244 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
245 Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800246 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800247 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
248 ... Validate Authentication After Reassociate True ${src['dp_iface_name']}
249 ... ${src['ip']} ${src['user']} ${src['pass']}
250 ... ${src['container_type']} ${src['container_name']}
ubuntu6b6e7d42020-03-02 12:35:42 -0800251 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800252 ... ${ONOS_SSH_PORT} ${onu_port}
253 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
254 ... Validate DHCP and Ping True True ${src['dp_iface_name']}
255 ... ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
256 ... ${src['ip']} ${src['user']} ${src['pass']}
257 ... ${src['container_type']} ${src['container_name']}
258 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']}
259 ... ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
260 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
ubuntu6b6e7d42020-03-02 12:35:42 -0800261 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800262 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
263 Run Keyword and Ignore Error Collect Logs
264 END
265
Suchitra Vemuri05180cb2020-03-26 13:13:27 -0700266Verify openolt adapter restart before subscriber provisioning
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800267 [Documentation] Deploys an device instance and waits for it to authenticate. After
268 ... authentication is successful the rw-core deployment is scaled to 0 instances to
269 ... simulate a POD crash. The test then scales the rw-core back to a single instance
270 ... and configures ONOS for access. The test succeeds if the device is able to
271 ... complete the DHCP sequence.
Hung-Wei Chiud3576762020-03-25 18:06:26 -0700272 [Tags] functional olt-adapter-restart
Andy Baviere187eda2020-04-20 15:00:02 -0700273 [Setup] Start Logging OltAdapterRestart
Suchitra Vemuria0c086f2020-03-29 19:14:13 -0700274 #... AND Clear All Devices Then Create New Device
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800275 [Teardown] Run Keywords Collect Logs
276 ... AND Stop Logging OltAdapterRestart
Hung-Wei Chiud3576762020-03-25 18:06:26 -0700277 # Add OLT and perform sanity test
Suchitra Vemuri59ad2772020-03-26 22:27:27 -0700278 #setup
Suchitra Vemuri91537c82020-03-29 15:03:26 -0700279 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuri59ad2772020-03-26 22:27:27 -0700280 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800281 #Set Global Variable ${of_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800282
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800283 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800284 ${src}= Set Variable ${hosts.src[${I}]}
285 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800286 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800287 ${onu_device_id}= Get Device ID From SN ${src['onu']}
288 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
289 ... ${of_id}
290
291 # Bring up the device and verify it authenticates
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700292 Wait Until Keyword Succeeds 360s 5s Validate Device ENABLED ACTIVE REACHABLE
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800293 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
ubuntu6b6e7d42020-03-02 12:35:42 -0800294 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800295 ... ${ONOS_SSH_PORT} ${onu_port}
Andy Bavier84834d42020-02-25 13:49:50 -0700296 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
297 ... /tmp/wpa ${src['dp_iface_name']} log
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800298 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
299 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700300 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
ubuntu6b6e7d42020-03-02 12:35:42 -0800301 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800302 ... ${ONOS_SSH_PORT} ${onu_port}
303 END
304 # Scale down the open OLT adapter deployment to 0 PODs and once confirmed, scale it back to 1
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530305 Scale K8s Deployment voltha open-olt-voltha-adapter-openolt 0
306 Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist voltha open-olt-voltha-adapter-openolt
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800307 # Scale up the open OLT adapter deployment and make sure both it and the ofagent deployment are back
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530308 Scale K8s Deployment voltha open-olt-voltha-adapter-openolt 1
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800309 Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530310 ... Check Expected Available Deployment Replicas voltha open-olt-voltha-adapter-openolt 1
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800311
312 # Ensure the device is available in ONOS, this represents system connectivity being restored
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800313 FOR ${I} IN RANGE 0 ${olt_count}
314 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
315 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
316 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
317 ... ${olt_serial_number}
318 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
319 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
320 END
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800321
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800322 FOR ${I} IN RANGE 0 ${num_all_onus}
323 ${src}= Set Variable ${hosts.src[${I}]}
324 ${dst}= Set Variable ${hosts.dst[${I}]}
325 ${of_id}= Get ofID From OLT List ${src['olt']}
326 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
327 ... ${of_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800328 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
ubuntu6b6e7d42020-03-02 12:35:42 -0800329 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800330 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530331 # Verify subscriber access flows are added for the ONU port
332 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
333 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
334 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800335 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
336 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
337 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
338 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
339 ... ${dst['container_name']}
340 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
ubuntu6b6e7d42020-03-02 12:35:42 -0800341 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800342 END
343
hwchiu14c05102020-03-27 00:44:30 +0000344Verify restart ofagent container after subscriber is provisioned
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000345 [Documentation] Restart ofagent container after VOLTHA is operational.
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000346 ... Prerequisite : ONUs are authenticated and pingable.
Hardik Windlass7a846662020-03-19 20:42:59 +0530347 [Tags] functional VOL-2409 ofagentRestart
Andy Baviere187eda2020-04-20 15:00:02 -0700348 [Setup] Start Logging ofagentRestart
Andy Bavier4a8450e2020-02-04 08:58:37 -0700349 [Teardown] Run Keywords Collect Logs
Andy Bavierabeba262020-02-07 16:22:16 -0700350 ... AND Stop Logging ofagentRestart
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530351 ... AND Scale K8s Deployment ${NAMESPACE} voltha-voltha-ofagent 1
Suchitra Vemuriceb002a2020-03-25 13:36:21 -0700352 # set timeout value
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000353 ${waitforRestart} Set Variable 120s
Andy Bavier4a8450e2020-02-04 08:58:37 -0700354 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000355 Log ${podStatusOutput}
Andy Bavier4a8450e2020-02-04 08:58:37 -0700356 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530357 ${podName} Set Variable ofagent
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530358 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000359 Sleep 60s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530360 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
361 ... app ${podName} Running
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530362 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
363 Run Keyword If ${has_dataplane} Clean Up Linux
364 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Andy Bavier4a8450e2020-02-04 08:58:37 -0700365 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000366 Log ${podStatusOutput}
Andy Bavier4a8450e2020-02-04 08:58:37 -0700367 ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000368 Should Be Equal As Strings ${countAfterRestart} ${countBforRestart}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530369 # Scale Down the Of-Agent Deployment
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530370 Scale K8s Deployment ${NAMESPACE} voltha-voltha-ofagent 0
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530371 Sleep 30s
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800372 FOR ${I} IN RANGE 0 ${num_all_onus}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530373 ${src}= Set Variable ${hosts.src[${I}]}
374 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800375 ${of_id}= Get ofID From OLT List ${src['olt']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530376 Run Keyword and Ignore Error Collect Logs
377 ${onu_device_id}= Get Device ID From SN ${src['onu']}
378 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
379 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
380 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700381 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530382 ... ENABLED ACTIVE REACHABLE
383 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
384 # Check ONU port is Disabled in ONOS
385 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700386 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530387 # Verify EAPOL flows are present for the ONU port
388 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
389 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
390 # Verify ONU in AAA-Users
391 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
392 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
393 # Verify DHCP-Allocations
394 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
395 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530396 # Verify subscriber access flows are added for the ONU port
397 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
398 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
399 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530400 # Verify Ping
401 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Check Ping True
402 ... ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']} ${src['ip']}
403 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
404 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
405 Run Keyword and Ignore Error Collect Logs
406 END
407 # Scale Up the Of-Agent Deployment
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530408 Scale K8s Deployment ${NAMESPACE} voltha-voltha-ofagent 1
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530409 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pod Status ofagent ${NAMESPACE}
410 ... Running
411 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
412 Run Keyword If ${has_dataplane} Clean Up Linux
413 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
414 Log to console Pod ${podName} restarted and sanity checks passed successfully
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000415
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000416Check ONU adapter crash not forcing authentication again
417 [Documentation] After ONU adapter restart, checks wpa log for 'authentication started'
418 ... message count to make sure auth not started again and validates EAP status and ping.
419 ... Assuming that test1 or sanity was executed where all the ONUs are authenticated/DHCP/pingable
hwchiu14c05102020-03-27 00:44:30 +0000420 [Tags] functional ONUAdaptCrash
Andy Baviere187eda2020-04-20 15:00:02 -0700421 [Setup] Start Logging ONUAdaptCrash
Andy Bavierabeba262020-02-07 16:22:16 -0700422 [Teardown] Run Keywords Collect Logs
423 ... AND Stop Logging ONUAdaptCrash
hwchiu14c05102020-03-27 00:44:30 +0000424 # Wait for adapter to resync
425 Sleep 60s
426 # Restart the onu
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530427 ${podName} Set Variable adapter-open-onu
Hung-Wei Chiu2bee4d42020-04-24 11:31:50 -0700428 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
hwchiu14c05102020-03-27 00:44:30 +0000429 # Validate ONU Ports
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800430 FOR ${I} IN RANGE 0 ${num_all_onus}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000431 ${src}= Set Variable ${hosts.src[${I}]}
432 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800433 ${of_id}= Get ofID From OLT List ${src['olt']}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000434 ${onu_device_id}= Get Device ID From SN ${src['onu']}
435 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
436 ... ${of_id}
hwchiu14c05102020-03-27 00:44:30 +0000437 Run Keyword And Continue On Failure Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700438 ... ${src['onu']}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000439 ${output}= Run Keyword If ${has_dataplane} Login And Run Command On Remote System
440 ... wpa_cli status | grep SUCCESS ${src['ip']} ${src['user']} ${src['pass']}
441 ... ${src['container_type']} ${src['container_name']}
442 Run Keyword If ${has_dataplane} Should Contain ${output} SUCCESS
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000443 END
Hung-Wei Chiu2bee4d42020-04-24 11:31:50 -0700444 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
445 ... app ${podName} Running
hwchiu14c05102020-03-27 00:44:30 +0000446 # Wait for adapter to resync
447 Sleep 60s
Andy Bavier8da0e132020-04-13 10:25:16 -0700448 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800449 # Validate OLTs are active in ONOS
450 FOR ${I} IN RANGE 0 ${olt_count}
451 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
452 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
453 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
454 ... ${olt_serial_number}
455 END
hwchiu14c05102020-03-27 00:44:30 +0000456 # Perform all steps in Sanity Test except the subscriber addition
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800457 FOR ${I} IN RANGE 0 ${num_all_onus}
hwchiu14c05102020-03-27 00:44:30 +0000458 ${src}= Set Variable ${hosts.src[${I}]}
459 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800460 ${of_id}= Get ofID From OLT List ${src['olt']}
hwchiu14c05102020-03-27 00:44:30 +0000461 Run Keyword and Ignore Error Collect Logs
462 ${onu_device_id}= Get Device ID From SN ${src['onu']}
463 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
464 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
465 # Check ONU port is Enabled in ONOS
466 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700467 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
hwchiu14c05102020-03-27 00:44:30 +0000468 # Verify EAPOL flows are added for the ONU port
469 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
470 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
471 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700472 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
hwchiu14c05102020-03-27 00:44:30 +0000473 ... ENABLED ACTIVE REACHABLE
474 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
475 # Perform Authentication
476 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
477 ... /tmp/wpa ${src['dp_iface_name']} log
478 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
479 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
480 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
481 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
482 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
483 # Verify that no pending flows exist for the ONU port
484 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
485 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530486 # Verify subscriber access flows are added for the ONU port
487 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
488 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
489 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
hwchiu14c05102020-03-27 00:44:30 +0000490 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
491 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
492 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
493 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
494 ... ${dst['container_name']}
495 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
496 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
497 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
498 Run Keyword and Ignore Error Collect Logs
499 END
500
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000501 Run Keyword and Ignore Error Collect Logs
502
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700503Sanity E2E Test for OLT/ONU on POD With Core Fail and Restart
504 [Documentation] Deploys an device instance and waits for it to authenticate. After
505 ... authentication is successful the rw-core deployment is scaled to 0 instances to
506 ... simulate a POD crash. The test then scales the rw-core back to a single instance
507 ... and configures ONOS for access. The test succeeds if the device is able to
508 ... complete the DHCP sequence.
509 [Tags] functional rwcore-restart
Andy Baviere187eda2020-04-20 15:00:02 -0700510 [Setup] Run Keywords Start Logging RwCoreFailAndRestart
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700511 ... AND Clear All Devices Then Create New Device
512 [Teardown] Run Keywords Collect Logs
513 ... AND Stop Logging RwCoreFailAndRestart
Suchitra Vemuri5d3383f2020-04-03 11:02:26 -0700514 #... AND Delete Device and Verify
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700515 Run Keyword and Ignore Error Collect Logs
516 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800517 FOR ${I} IN RANGE 0 ${olt_count}
518 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
519 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
520 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
521 ... ${olt_serial_number}
522 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
523 ... Get NNI Port in ONOS ${of_id}
524 # Set Global Variable ${nni_port}
525 END
526 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700527 ${src}= Set Variable ${hosts.src[${I}]}
528 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800529 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700530 ${onu_device_id}= Get Device ID From SN ${src['onu']}
531 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
532 ... ${of_id}
533 # Bring up the device and verify it authenticates
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700534 Wait Until Keyword Succeeds 360s 5s Validate Device ENABLED ACTIVE REACHABLE
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700535 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
536 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
537 ... ${ONOS_SSH_PORT} ${onu_port}
538 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
539 ... /tmp/wpa ${src['dp_iface_name']} log
540 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
541 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
542 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
543 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
544 ... ${ONOS_SSH_PORT} ${onu_port}
545 END
546
547 # Scale down the rw-core deployment to 0 PODs and once confirmed, scale it back to 1
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530548 Scale K8s Deployment voltha voltha-voltha-rw-core 0
549 Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist voltha voltha-voltha-rw-core
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700550 # Ensure the ofagent POD goes "not-ready" as expected
551 Wait Until keyword Succeeds ${timeout} 2s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530552 ... Check Expected Available Deployment Replicas voltha voltha-voltha-ofagent 0
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700553 # Scale up the core deployment and make sure both it and the ofagent deployment are back
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530554 Scale K8s Deployment voltha voltha-voltha-rw-core 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700555 Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530556 ... Check Expected Available Deployment Replicas voltha voltha-voltha-rw-core 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700557 Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530558 ... Check Expected Available Deployment Replicas voltha voltha-voltha-ofagent 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700559 # For some reason scaling down and up the POD behind a service causes the port forward to stop working,
560 # so restart the port forwarding for the API service
David Bainbridge07321ff2020-06-12 13:55:42 -0700561 Restart VOLTHA Port Forward voltha-api
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700562 # Ensure that the ofagent pod is up and ready and the device is available in ONOS, this
563 # represents system connectivity being restored
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800564 FOR ${I} IN RANGE 0 ${olt_count}
565 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
566 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
567 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
568 ... ${olt_serial_number}
569 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
570 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
571 END
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700572
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800573 FOR ${I} IN RANGE 0 ${num_all_onus}
Andy Bavier52094622020-05-12 15:54:24 -0700574 ${src}= Set Variable ${hosts.src[${I}]}
575 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800576 ${of_id}= Get ofID From OLT List ${src['olt']}
Andy Bavier52094622020-05-12 15:54:24 -0700577 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
578 ... ${of_id}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700579 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
580 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
581 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530582 # Verify subscriber access flows are added for the ONU port
583 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
584 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
585 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700586 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
587 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
588 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
589 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
590 ... ${dst['container_name']}
591 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
592 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
593 END
594
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700595Verify OLT Soft Reboot
596 [Documentation] Test soft reboot of the OLT using voltctl command
597 [Tags] VOL-2745 OLTSoftReboot functional
598 [Setup] Start Logging OLTSoftReboot
599 [Teardown] Run Keywords Collect Logs
600 ... AND Stop Logging OLTSoftReboot
Suchitra Vemuri6282ba12020-08-13 12:50:54 -0700601 #Delete All Devices and Verify
602 #Setup
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700603 ## Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
Suchitra Vemuri6282ba12020-08-13 12:50:54 -0700604 #Run Keyword If ${has_dataplane} Clean Up Linux
605 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800606 FOR ${I} IN RANGE 0 ${olt_count}
607 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
608 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
609 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
610 ... Validate OLT Device ENABLED ACTIVE
611 ... REACHABLE ${olt_serial_number}
612 # Reboot the OLT using "voltctl device reboot" command
613 Reboot Device ${olt_device_id}
614 END
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700615 Run Keyword And Ignore Error Collect Logs
616 #Verify that ping fails
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800617 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700618 ${src}= Set Variable ${hosts.src[${I}]}
619 ${dst}= Set Variable ${hosts.dst[${I}]}
620 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
621 ... Wait Until Keyword Succeeds 60s 2s
622 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
623 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
624 END
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700625 # Check OLT states
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800626 FOR ${I} IN RANGE 0 ${olt_count}
627 ${olt_serial_number}= Get From Dictionary ${list_olts}[${I}] sn
628 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
629 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
630 # Wait for the OLT to come back up
631 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
632 ... Check Remote System Reachability True ${olt_ssh_ip}
633 # Check OLT states
634 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
635 ... Validate OLT Device ENABLED ACTIVE
636 ... REACHABLE ${olt_serial_number}
637 END
638 # Waiting extra time for the ONUs to come up
639 Sleep 60s
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700640 Run Keyword And Ignore Error Collect Logs
641 #Check after reboot that ONUs are active, authenticated/DHCP/pingable
642 Run Keyword If ${has_dataplane} Clean Up Linux
643 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
644
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700645Verify restart ofagent container before subscriber is provisioned
646 [Documentation] Restart ofagent container before subscriber is provisioned.
647 [Tags] functional VOL-2962 ofagentRestart2
648 [Setup] Start Logging ofagentRestart2
649 [Teardown] Run Keywords Collect Logs
Suchitra Vemuri8130be82020-09-15 16:47:58 -0700650 ... AND Stop Logging ofagentRestart2
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700651 ... AND Scale K8s Deployment ${NAMESPACE} voltha-voltha-ofagent 1
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800652 Delete All Devices And Verify
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700653 setup
654 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800655 FOR ${I} IN RANGE 0 ${olt_count}
656 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
657 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
658 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
659 ... ${olt_serial_number}
660 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
661 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
662 END
663 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700664 ${src}= Set Variable ${hosts.src[${I}]}
665 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800666 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700667 ${onu_device_id}= Get Device ID From SN ${src['onu']}
668 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
669 ... ${of_id}
670 # Bring up the device and verify it authenticates
671 Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
672 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
673 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
674 ... ${ONOS_SSH_PORT} ${onu_port}
675 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
676 ... /tmp/wpa ${src['dp_iface_name']} log
677 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
678 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
679 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
680 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
681 ... ${ONOS_SSH_PORT} ${onu_port}
682 END
683 # Restart POD ofagent
684 ${waitforRestart} Set Variable 120s
685 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
686 Log ${podStatusOutput}
687 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
688 ${podName} Set Variable ofagent
689 Restart Pod ${NAMESPACE} ${podName}
690 Sleep 60s
691 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pod Status ofagent ${NAMESPACE}
692 ... Running
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800693 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700694 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
695 #Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
696 #... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
697 ${src}= Set Variable ${hosts.src[${I}]}
698 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800699 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700700 Run Keyword and Ignore Error Collect Logs
701 ${onu_device_id}= Get Device ID From SN ${src['onu']}
702 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
703 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
704 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
705 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
706 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
707 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
708 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
709 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
710 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
711 ... ${dst['container_name']}
712 # Verify DHCP-Allocations
713 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
714 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
715 Run Keyword and Ignore Error Collect Logs
716 END
717
Hema123aff52020-05-26 15:41:47 +0530718Verify ONU Soft Reboot
719 [Documentation] Test soft reboot of the ONU using voltctl command
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800720 [Tags] VOL-1957 ONUSoftReboot functional notready
Hema123aff52020-05-26 15:41:47 +0530721 [Setup] Start Logging ONUSoftReboot
722 #... AND Setup
Andy Bavier4a8450e2020-02-04 08:58:37 -0700723 [Teardown] Run Keywords Collect Logs
Hema123aff52020-05-26 15:41:47 +0530724 ... AND Stop Logging ONUSoftReboot
725 #... AND Delete Device and Verify
726 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
727 #Run Keyword If ${has_dataplane} Clean Up Linux
728 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
729 #Reboot the ONU and verify that ping fails
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800730 FOR ${I} IN RANGE 0 ${num_all_onus}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000731 ${src}= Set Variable ${hosts.src[${I}]}
732 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800733 ${of_id}= Get ofID From OLT List ${src['olt']}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000734 ${onu_device_id}= Get Device ID From SN ${src['onu']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800735 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
736 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Hema123aff52020-05-26 15:41:47 +0530737 Reboot Device ${onu_device_id}
738 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
739 ... Wait Until Keyword Succeeds 60s 2s
740 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
741 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
742 # Remove Subscriber Access (To replicate ATT workflow)
743 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${ONOS_SSH_IP}
744 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800745 Verify ping is succesful except for given device ${num_all_onus} ${onu_device_id}
Hema123aff52020-05-26 15:41:47 +0530746 Sleep 40s
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800747 # Check ONU port is Enabled in ONOS
748 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700749 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800750 # Verify EAPOL flows are added for the ONU port
751 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800752 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800753 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700754 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800755 ... ENABLED ACTIVE REACHABLE
756 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hema123aff52020-05-26 15:41:47 +0530757 Run Keyword If ${has_dataplane} Clean Up Linux
758 # Perform Authentication
759 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
760 ... /tmp/wpa ${src['dp_iface_name']} log
761 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
762 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
763 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
764 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
765 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
766 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
767 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
768 ... volt-add-subscriber-access ${of_id} ${onu_port}
769 # Verify that no pending flows exist for the ONU port
770 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
771 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530772 # Verify subscriber access flows are added for the ONU port
773 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
774 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
775 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hema123aff52020-05-26 15:41:47 +0530776 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
777 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800778 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hema123aff52020-05-26 15:41:47 +0530779 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
780 ... ${dst['container_name']}
781 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
782 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
783 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
784 Run Keyword And Ignore Error Collect Logs
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000785 END
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800786 #Delete All Devices And Verify
787 #Run Keyword If ${has_dataplane} Clean Up Linux
788 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
789
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800790
791*** Keywords ***
792Setup Suite
793 [Documentation] Set up the test suite
794 Common Test Suite Setup
HungWei Chiu1408fb92020-03-03 19:43:30 -0500795 #power_switch.robot needs it to support different vendor's power switch
Hung-Wei Chiufcc6d8e2020-03-10 12:05:11 -0700796 ${switch_type}= Get Variable Value ${web_power_switch.type}
797 Run Keyword If "${switch_type}"!="" Set Global Variable ${powerswitch_type} ${switch_type}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800798
799Clear All Devices Then Create New Device
800 [Documentation] Remove any devices from VOLTHA and ONOS
801 # Remove all devices from voltha and nos
802 Delete All Devices and Verify
803 # Execute normal test Setup Keyword
804 Setup