blob: cfae5ee6a14e5fdb3e6a6e2a74e26b3b2c7bd1c9 [file] [log] [blame]
Gayathri.Selvan283a63c2020-01-23 04:09:18 +00001# Copyright 2017 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000014
15*** Settings ***
HungWei Chiu87edb1f2020-01-28 00:10:58 +000016Documentation Test various failure scenarios
HungWei Chiu1408fb92020-03-03 19:43:30 -050017Suite Setup Setup Suite
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000018Test Setup Setup
19Test Teardown Teardown
TorstenThieme4e2168e2021-06-22 14:01:47 +000020Suite Teardown Teardown Suite
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000021Library Collections
22Library String
23Library OperatingSystem
24Library XML
25Library RequestsLibrary
26Library ../../libraries/DependencyLibrary.py
27Resource ../../libraries/onos.robot
28Resource ../../libraries/voltctl.robot
29Resource ../../libraries/voltha.robot
30Resource ../../libraries/utils.robot
31Resource ../../libraries/k8s.robot
32Resource ../../variables/variables.robot
HungWei Chiu87edb1f2020-01-28 00:10:58 +000033Resource ../../libraries/power_switch.robot
Hardik Windlassc082c422021-03-16 15:44:10 +000034Resource ../../libraries/bbsim.robot
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000035
36*** Variables ***
37${POD_NAME} flex-ocp-cord
38${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
39${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
40#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
41${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
42${HELM_CHARTS_DIR} ~/helm-charts
43${VOLTHA_POD_NUM} 8
44${NAMESPACE} voltha
Matteo Scandoloa4e59522021-08-31 14:56:24 -070045${STACK_NAME} voltha
Suchitra Vemurica1c5d32020-04-09 18:45:29 -070046${DEFAULTSPACE} default
Matteo Scandolo10acf562021-05-03 15:35:16 -070047${INFRA_NAMESPACE} infra
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000048# For below variable value, using deployment name as using grep for
49# parsing radius pod name, we can also use full radius pod name
50${RESTART_POD_NAME} radius
51${timeout} 60s
52${of_id} 0
53${logical_id} 0
Suchitra Vemuri109feb12020-01-23 16:25:02 -080054${has_dataplane} True
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000055${teardown_device} False
56${scripts} ../../scripts
57
Matteo Scandolo7e519fb2021-08-13 11:35:16 -070058${suppressaddsubscriber} True
59
Andy Bavierabeba262020-02-07 16:22:16 -070060# Per-test logging on failure is turned off by default; set this variable to enable
61${container_log_dir} ${None}
62
Matteo Scandoloda1c64b2021-07-20 10:14:49 -070063# logging flag to enable Collect Logs, can be passed via the command line too
64# example: -v logging:False
65${logging} True
66
Gayathri.Selvan283a63c2020-01-23 04:09:18 +000067*** Test Cases ***
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080068Verify ONU after rebooting physically
69 [Documentation] Test the ONU functionality by physically turning on/off ONU.
70 ... Prerequisite : Subscriber are authenticated/DHCP/pingable state
71 ... Test case runs only on the PODs that are configured with PowerSwitch that
72 ... controls the power off/on ONUs/OLT remotely (simulating a physical reboot)
73 ... VOL-2634
74 [Tags] functional PowerSwitch
Andy Baviere187eda2020-04-20 15:00:02 -070075 [Setup] Start Logging ONUreboot_PowerSwitch
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080076 [Teardown] Run Keywords Collect Logs
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080077 ... AND Delete All Devices and Verify
Girish Gowdraf244d5b2021-09-01 17:16:16 -070078 ... AND Run Keyword If ${logging} Collect Logs
79 ... AND Stop Logging ONUreboot_PowerSwitch
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080080 # Add OLT device
81 setup
82 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
83 Run Keyword If ${has_dataplane} Clean Up Linux
84 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
85 Power Switch Connection Suite ${web_power_switch.ip} ${web_power_switch.user} ${web_power_switch.password}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080086 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080087 ${src}= Set Variable ${hosts.src[${I}]}
88 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -080089 ${of_id}= Get ofID From OLT List ${src['olt']}
Hardik Windlass0560f802020-05-06 23:06:53 +053090 ${onu_device_id}= Get Device ID From SN ${src['onu']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080091 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass1ed2eee2021-06-25 09:51:03 +000092 ... Get ONU Port in ONOS ${src['onu']} ${of_id} ${src['uni_id']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -080093 Disable Switch Outlet ${src['power_switch_port']}
94 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
95 ... Wait Until Keyword Succeeds 60s 2s
96 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
97 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hardik Windlass7c801e62020-05-04 19:33:21 +053098 # Remove Subscriber Access (To replicate ATT workflow)
TorstenThieme4e2168e2021-06-22 14:01:47 +000099 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530100 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800101
102 Enable Switch Outlet ${src['power_switch_port']}
103 # Check ONU port is Enabled in ONOS
104 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Hardik Windlass39015672021-07-05 05:48:08 +0000105 ... Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800106 # Verify EAPOL flows are added for the ONU port
107 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700108 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800109 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700110 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800111 ... ENABLED ACTIVE REACHABLE
112 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlass7c801e62020-05-04 19:33:21 +0530113 Run Keyword If ${has_dataplane} Clean Up Linux
114 # Perform Authentication
115 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
116 ... /tmp/wpa ${src['dp_iface_name']} log
117 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
118 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
119 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
120 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
121 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
122 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
TorstenThieme4e2168e2021-06-22 14:01:47 +0000123 ... Execute ONOS CLI Command use single connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530124 ... volt-add-subscriber-access ${of_id} ${onu_port}
125 # Verify that no pending flows exist for the ONU port
126 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
127 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530128 # Verify subscriber access flows are added for the ONU port
129 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
130 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
131 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530132 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
133 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800134 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hardik Windlass7c801e62020-05-04 19:33:21 +0530135 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
136 ... ${dst['container_name']}
137 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
138 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800139 END
Suchitra Vemurid7096132020-02-21 14:50:47 -0800140 # Deleting OLT after tests completes independently (as this test doesn't not run on each POD)
Suchitra Vemuri0656bc22020-03-21 19:07:49 -0700141 #Run Keyword If ${has_dataplane} Delete Device and Verify
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800142
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700143Verify OLT after rebooting physically
144 [Documentation] Test the physical reboot of the OLT
145 ... Prerequisite : Subscriber are authenticated/DHCP/pingable state
146 ... Test performs a physical reboot, performs "reboot" from the OLT CLI
147 ... VOL-1956
148 [Tags] functional PhysicalOLTReboot
Andy Baviere187eda2020-04-20 15:00:02 -0700149 [Setup] Start Logging PhysicalOLTReboot
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700150 [Teardown] Run Keywords Collect Logs
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800151 ... AND Delete All Devices and Verify
Girish Gowdraf244d5b2021-09-01 17:16:16 -0700152 ... AND Run Keyword If ${logging} Collect Logs
153 ... AND Stop Logging PhysicalOLTReboot
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700154 # Add OLT device
155 setup
156 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
157 Run Keyword If ${has_dataplane} Clean Up Linux
158 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
159 # Reboot the OLT from the OLT CLI
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800160 FOR ${I} IN RANGE 0 ${olt_count}
161 ${olt_user}= Get From Dictionary ${list_olts}[${I}] user
162 ${olt_pass}= Get From Dictionary ${list_olts}[${I}] pass
163 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
164 ${olt_serial_number}= Get From Dictionary ${list_olts}[${I}] sn
165 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
166 Run Keyword If ${has_dataplane} Login And Run Command On Remote System
Andrea Campanella5ef88142021-02-02 14:34:50 +0100167 ... reboot ${olt_ssh_ip} ${olt_user} ${olt_pass} prompt=#
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800168 END
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800169 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700170 ${src}= Set Variable ${hosts.src[${I}]}
171 ${dst}= Set Variable ${hosts.dst[${I}]}
172 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
173 ... Wait Until Keyword Succeeds 60s 2s
174 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
175 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
176 END
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800177 # Wait for the OLTs to come back up
178 FOR ${I} IN RANGE 0 ${olt_count}
179 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
180 ${olt_user}= Get From Dictionary ${list_olts}[${I}] user
181 ${olt_pass}= Get From Dictionary ${list_olts}[${I}] pass
182 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
183 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
184 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
185 ... Check Remote System Reachability True ${olt_ssh_ip}
186 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
187 ... Validate OLT Device ENABLED ACTIVE
188 ... REACHABLE ${olt_serial_number}
189 END
Suchitra Vemuri64db9652020-03-22 21:11:13 -0700190 # Waiting extra time for the ONUs to come up
191 Sleep 60s
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700192 Run Keyword If ${has_dataplane} Clean Up Linux
193 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
194 # Deleting OLT after test completes
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800195 #Run Keyword If ${has_dataplane} Delete All Devices and Verify
Suchitra Vemuri784e1ae2020-03-16 18:40:25 -0700196
Suchitra Vemuri05180cb2020-03-26 13:13:27 -0700197Verify restart openolt-adapter container after subscriber provisioning
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800198 [Documentation] Restart openolt-adapter container after VOLTHA is operational.
199 ... Prerequisite : ONUs are authenticated and pingable.
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800200 [Tags] functional VOL-1958 Restart-OpenOlt released
Andy Baviere187eda2020-04-20 15:00:02 -0700201 [Setup] Start Logging Restart-OpenOlt
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800202 [Teardown] Run Keywords Collect Logs
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800203 ... AND Stop Logging Restart-OpenOlt
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800204 # Add OLT device
205 setup
206 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
207 Run Keyword If ${has_dataplane} Clean Up Linux
208 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
209 ${waitforRestart} Set Variable 120s
210 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
211 Log ${podStatusOutput}
212 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Andrea Campanella3dcce272021-01-15 16:04:47 +0100213 ${podName} Set Variable ${OLT_ADAPTER_APP_LABEL}
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530214 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530215 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
216 ... app ${podName} Running
Suchitra Vemuri312bfe02020-03-16 16:11:34 -0700217 # Wait for 1min after openolt adapter is restarted
Hardik Windlass5e214b22021-02-26 10:37:14 +0000218 # TBD: Need for this Sleep
Suchitra Vemuri312bfe02020-03-16 16:11:34 -0700219 Sleep 60s
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800220 Run Keyword If ${has_dataplane} Clean Up Linux
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700221 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
Suchitra Vemuria7dae322020-02-19 22:38:18 -0800222 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
223 Log ${podStatusOutput}
224 ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
225 Should Be Equal As Strings ${countAfterRestart} ${countBforRestart}
226 Log to console Pod ${podName} restarted and sanity checks passed successfully
Suchitra Vemuri60be98b2020-02-19 16:41:21 -0800227
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800228Check OLT/ONU Authentication After Radius Pod Restart
229 [Documentation] After radius restart, triggers reassociation, checks status and
230 ... authentication, validates dhcp and ping. Note : wpa reassociate works only when
231 ... wpa supplicant is running in background hence it is recommended to remove
232 ... teardown from previous test or uncomment 'Teardown None'.
233 ... Assuming that test1 was executed where all the ONUs are authenticated/DHCP/pingable
234 [Tags] functional RadiusRestart released
Andy Baviere187eda2020-04-20 15:00:02 -0700235 [Setup] Start Logging RadiusRestart
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800236 [Teardown] Run Keywords Collect Logs
237 ... AND Stop Logging RadiusRestart
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800238 ${waitforRestart} Set Variable 120s
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530239 ${podName} Set Variable radius
Matteo Scandolo10acf562021-05-03 15:35:16 -0700240 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${INFRA_NAMESPACE} app ${podName}
241 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pods Status By Label ${INFRA_NAMESPACE}
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530242 ... app ${podName} Running
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800243 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800244 ${src}= Set Variable ${hosts.src[${I}]}
245 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800246 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800247 ${onu_device_id}= Get Device ID From SN ${src['onu']}
248 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000249 ... Get ONU Port in ONOS ${src['onu']} ${of_id} ${src['uni_id']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800250 Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700251 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
252 ... ${src['c_tag']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800253 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
254 ... Validate Authentication After Reassociate True ${src['dp_iface_name']}
255 ... ${src['ip']} ${src['user']} ${src['pass']}
256 ... ${src['container_type']} ${src['container_name']}
ubuntu6b6e7d42020-03-02 12:35:42 -0800257 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800258 ... ${ONOS_SSH_PORT} ${onu_port}
259 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
260 ... Validate DHCP and Ping True True ${src['dp_iface_name']}
261 ... ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
262 ... ${src['ip']} ${src['user']} ${src['pass']}
263 ... ${src['container_type']} ${src['container_name']}
264 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']}
265 ... ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
266 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
ubuntu6b6e7d42020-03-02 12:35:42 -0800267 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800268 END
269
Suchitra Vemuri05180cb2020-03-26 13:13:27 -0700270Verify openolt adapter restart before subscriber provisioning
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800271 [Documentation] Deploys an device instance and waits for it to authenticate. After
272 ... authentication is successful the rw-core deployment is scaled to 0 instances to
273 ... simulate a POD crash. The test then scales the rw-core back to a single instance
274 ... and configures ONOS for access. The test succeeds if the device is able to
275 ... complete the DHCP sequence.
Hung-Wei Chiud3576762020-03-25 18:06:26 -0700276 [Tags] functional olt-adapter-restart
Andy Baviere187eda2020-04-20 15:00:02 -0700277 [Setup] Start Logging OltAdapterRestart
Suchitra Vemuria0c086f2020-03-29 19:14:13 -0700278 #... AND Clear All Devices Then Create New Device
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800279 [Teardown] Run Keywords Collect Logs
280 ... AND Stop Logging OltAdapterRestart
Hung-Wei Chiud3576762020-03-25 18:06:26 -0700281 # Add OLT and perform sanity test
Suchitra Vemuri59ad2772020-03-26 22:27:27 -0700282 #setup
Suchitra Vemuri91537c82020-03-29 15:03:26 -0700283 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuri59ad2772020-03-26 22:27:27 -0700284 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800285 #Set Global Variable ${of_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800286
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800287 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800288 ${src}= Set Variable ${hosts.src[${I}]}
289 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800290 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800291 ${onu_device_id}= Get Device ID From SN ${src['onu']}
292 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000293 ... ${of_id} ${src['uni_id']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800294
295 # Bring up the device and verify it authenticates
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700296 Wait Until Keyword Succeeds 360s 5s Validate Device ENABLED ACTIVE REACHABLE
Hardik Windlass8b446492021-06-10 06:25:23 +0000297 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed by_dev_id=True
ubuntu6b6e7d42020-03-02 12:35:42 -0800298 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700299 ... ${ONOS_SSH_PORT} ${of_id} ${onu_port} ${src['c_tag']}
Andy Bavier84834d42020-02-25 13:49:50 -0700300 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
301 ... /tmp/wpa ${src['dp_iface_name']} log
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800302 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
303 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700304 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
ubuntu6b6e7d42020-03-02 12:35:42 -0800305 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800306 ... ${ONOS_SSH_PORT} ${onu_port}
307 END
308 # Scale down the open OLT adapter deployment to 0 PODs and once confirmed, scale it back to 1
Andrea Campanella3dcce272021-01-15 16:04:47 +0100309 Scale K8s Deployment by Pod Label ${NAMESPACE} app ${OLT_ADAPTER_APP_LABEL} 0
310 Wait Until Keyword Succeeds ${timeout} 2s Pods Do Not Exist By Label ${NAMESPACE} app
311 ... ${OLT_ADAPTER_APP_LABEL}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800312 # Scale up the open OLT adapter deployment and make sure both it and the ofagent deployment are back
Andrea Campanella3dcce272021-01-15 16:04:47 +0100313 Scale K8s Deployment by Pod Label ${NAMESPACE} app ${OLT_ADAPTER_APP_LABEL} 1
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800314 Wait Until Keyword Succeeds ${timeout} 2s
Andrea Campanella3dcce272021-01-15 16:04:47 +0100315 ... Check Expected Available Deployment Replicas By Pod Label ${NAMESPACE} app ${OLT_ADAPTER_APP_LABEL} 1
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800316
317 # Ensure the device is available in ONOS, this represents system connectivity being restored
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800318 FOR ${I} IN RANGE 0 ${olt_count}
319 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
320 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
321 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
322 ... ${olt_serial_number}
323 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
324 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
325 END
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800326
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800327 FOR ${I} IN RANGE 0 ${num_all_onus}
328 ${src}= Set Variable ${hosts.src[${I}]}
329 ${dst}= Set Variable ${hosts.dst[${I}]}
330 ${of_id}= Get ofID From OLT List ${src['olt']}
331 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000332 ... ${of_id} ${src['uni_id']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800333 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
TorstenThieme4e2168e2021-06-22 14:01:47 +0000334 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800335 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530336 # Verify subscriber access flows are added for the ONU port
337 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
338 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
339 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800340 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
341 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
342 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
343 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
344 ... ${dst['container_name']}
345 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
ubuntu6b6e7d42020-03-02 12:35:42 -0800346 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800347 END
348
hwchiu14c05102020-03-27 00:44:30 +0000349Verify restart ofagent container after subscriber is provisioned
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000350 [Documentation] Restart ofagent container after VOLTHA is operational.
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000351 ... Prerequisite : ONUs are authenticated and pingable.
Hardik Windlass7a846662020-03-19 20:42:59 +0530352 [Tags] functional VOL-2409 ofagentRestart
Andy Baviere187eda2020-04-20 15:00:02 -0700353 [Setup] Start Logging ofagentRestart
Andy Bavier4a8450e2020-02-04 08:58:37 -0700354 [Teardown] Run Keywords Collect Logs
Andy Bavierabeba262020-02-07 16:22:16 -0700355 ... AND Stop Logging ofagentRestart
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700356 ... AND Scale K8s Deployment ${NAMESPACE} ${STACK_NAME}-voltha-ofagent 1
Suchitra Vemuriceb002a2020-03-25 13:36:21 -0700357 # set timeout value
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000358 ${waitforRestart} Set Variable 120s
Andy Bavier4a8450e2020-02-04 08:58:37 -0700359 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000360 Log ${podStatusOutput}
Andy Bavier4a8450e2020-02-04 08:58:37 -0700361 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530362 ${podName} Set Variable ofagent
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530363 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530364 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
365 ... app ${podName} Running
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530366 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
367 Run Keyword If ${has_dataplane} Clean Up Linux
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700368 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
Andy Bavier4a8450e2020-02-04 08:58:37 -0700369 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000370 Log ${podStatusOutput}
Andy Bavier4a8450e2020-02-04 08:58:37 -0700371 ${countAfterRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000372 Should Be Equal As Strings ${countAfterRestart} ${countBforRestart}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530373 # Scale Down the Of-Agent Deployment
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700374 Scale K8s Deployment ${NAMESPACE} ${STACK_NAME}-voltha-ofagent 0
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530375 Sleep 30s
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800376 FOR ${I} IN RANGE 0 ${num_all_onus}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530377 ${src}= Set Variable ${hosts.src[${I}]}
378 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800379 ${of_id}= Get ofID From OLT List ${src['olt']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530380 ${onu_device_id}= Get Device ID From SN ${src['onu']}
381 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000382 ... Get ONU Port in ONOS ${src['onu']} ${of_id} ${src['uni_id']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530383 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700384 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530385 ... ENABLED ACTIVE REACHABLE
386 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
387 # Check ONU port is Disabled in ONOS
388 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Hardik Windlass39015672021-07-05 05:48:08 +0000389 ... Verify UNI Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']} ${src['uni_id']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530390 # Verify EAPOL flows are present for the ONU port
391 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700392 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
393 ... ${src['c_tag']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530394 # Verify ONU in AAA-Users
395 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
396 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
397 # Verify DHCP-Allocations
398 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
399 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530400 # Verify subscriber access flows are added for the ONU port
401 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
402 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
403 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530404 # Verify Ping
405 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Check Ping True
406 ... ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']} ${src['ip']}
407 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530408 END
409 # Scale Up the Of-Agent Deployment
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700410 Scale K8s Deployment ${NAMESPACE} ${STACK_NAME}-voltha-ofagent 1
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530411 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pod Status ofagent ${NAMESPACE}
412 ... Running
413 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
414 Run Keyword If ${has_dataplane} Clean Up Linux
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700415 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
Hardik Windlass8f2aadc2020-03-16 20:16:39 +0530416 Log to console Pod ${podName} restarted and sanity checks passed successfully
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000417
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000418Check ONU adapter crash not forcing authentication again
419 [Documentation] After ONU adapter restart, checks wpa log for 'authentication started'
420 ... message count to make sure auth not started again and validates EAP status and ping.
421 ... Assuming that test1 or sanity was executed where all the ONUs are authenticated/DHCP/pingable
hwchiu14c05102020-03-27 00:44:30 +0000422 [Tags] functional ONUAdaptCrash
Andy Baviere187eda2020-04-20 15:00:02 -0700423 [Setup] Start Logging ONUAdaptCrash
Andy Bavierabeba262020-02-07 16:22:16 -0700424 [Teardown] Run Keywords Collect Logs
425 ... AND Stop Logging ONUAdaptCrash
hwchiu14c05102020-03-27 00:44:30 +0000426 # Wait for adapter to resync
Hardik Windlass5e214b22021-02-26 10:37:14 +0000427 # TBD: Need for this Sleep
hwchiu14c05102020-03-27 00:44:30 +0000428 Sleep 60s
429 # Restart the onu
Hardik Windlass79b40ff2020-06-11 22:55:47 +0530430 ${podName} Set Variable adapter-open-onu
Hung-Wei Chiu2bee4d42020-04-24 11:31:50 -0700431 Wait Until Keyword Succeeds ${timeout} 15s Delete K8s Pods By Label ${NAMESPACE} app ${podName}
hwchiu14c05102020-03-27 00:44:30 +0000432 # Validate ONU Ports
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800433 FOR ${I} IN RANGE 0 ${num_all_onus}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000434 ${src}= Set Variable ${hosts.src[${I}]}
435 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800436 ${of_id}= Get ofID From OLT List ${src['olt']}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000437 ${onu_device_id}= Get Device ID From SN ${src['onu']}
438 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000439 ... ${of_id} ${src['uni_id']}
Hardik Windlass39015672021-07-05 05:48:08 +0000440 Run Keyword And Continue On Failure Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000441 ... ${src['onu']} ${src['uni_id']}
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000442 ${output}= Run Keyword If ${has_dataplane} Login And Run Command On Remote System
443 ... wpa_cli status | grep SUCCESS ${src['ip']} ${src['user']} ${src['pass']}
444 ... ${src['container_type']} ${src['container_name']}
445 Run Keyword If ${has_dataplane} Should Contain ${output} SUCCESS
Gayathri.Selvanc0b37022020-02-06 05:17:00 +0000446 END
Hung-Wei Chiu2bee4d42020-04-24 11:31:50 -0700447 Wait Until Keyword Succeeds ${timeout} 2s Validate Pods Status By Label ${NAMESPACE}
448 ... app ${podName} Running
hwchiu14c05102020-03-27 00:44:30 +0000449 # Wait for adapter to resync
Hardik Windlass5e214b22021-02-26 10:37:14 +0000450 # TBD: Need for this Sleep
hwchiu14c05102020-03-27 00:44:30 +0000451 Sleep 60s
Andy Bavier8da0e132020-04-13 10:25:16 -0700452 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800453 # Validate OLTs are active in ONOS
454 FOR ${I} IN RANGE 0 ${olt_count}
455 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
456 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
457 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
458 ... ${olt_serial_number}
459 END
hwchiu14c05102020-03-27 00:44:30 +0000460 # Perform all steps in Sanity Test except the subscriber addition
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800461 FOR ${I} IN RANGE 0 ${num_all_onus}
hwchiu14c05102020-03-27 00:44:30 +0000462 ${src}= Set Variable ${hosts.src[${I}]}
463 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800464 ${of_id}= Get ofID From OLT List ${src['olt']}
hwchiu14c05102020-03-27 00:44:30 +0000465 ${onu_device_id}= Get Device ID From SN ${src['onu']}
466 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000467 ... Get ONU Port in ONOS ${src['onu']} ${of_id} ${src['uni_id']}
hwchiu14c05102020-03-27 00:44:30 +0000468 # Check ONU port is Enabled in ONOS
469 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Hardik Windlass39015672021-07-05 05:48:08 +0000470 ... Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']} ${src['uni_id']}
hwchiu14c05102020-03-27 00:44:30 +0000471 # Verify EAPOL flows are added for the ONU port
472 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700473 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
474 ... ${src['c_tag']}
hwchiu14c05102020-03-27 00:44:30 +0000475 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700476 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
hwchiu14c05102020-03-27 00:44:30 +0000477 ... ENABLED ACTIVE REACHABLE
478 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
479 # Perform Authentication
480 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
481 ... /tmp/wpa ${src['dp_iface_name']} log
482 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
483 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
484 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
485 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
486 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
487 # Verify that no pending flows exist for the ONU port
488 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
489 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530490 # Verify subscriber access flows are added for the ONU port
491 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
492 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
493 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
hwchiu14c05102020-03-27 00:44:30 +0000494 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
495 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
496 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
497 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
498 ... ${dst['container_name']}
499 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
500 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
hwchiu14c05102020-03-27 00:44:30 +0000501 END
502
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700503Sanity E2E Test for OLT/ONU on POD With Core Fail and Restart
504 [Documentation] Deploys an device instance and waits for it to authenticate. After
505 ... authentication is successful the rw-core deployment is scaled to 0 instances to
506 ... simulate a POD crash. The test then scales the rw-core back to a single instance
507 ... and configures ONOS for access. The test succeeds if the device is able to
508 ... complete the DHCP sequence.
509 [Tags] functional rwcore-restart
Andy Baviere187eda2020-04-20 15:00:02 -0700510 [Setup] Run Keywords Start Logging RwCoreFailAndRestart
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700511 ... AND Clear All Devices Then Create New Device
512 [Teardown] Run Keywords Collect Logs
513 ... AND Stop Logging RwCoreFailAndRestart
Suchitra Vemuri5d3383f2020-04-03 11:02:26 -0700514 #... AND Delete Device and Verify
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700515 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800516 FOR ${I} IN RANGE 0 ${olt_count}
517 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
518 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
519 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
520 ... ${olt_serial_number}
521 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
522 ... Get NNI Port in ONOS ${of_id}
523 # Set Global Variable ${nni_port}
524 END
525 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700526 ${src}= Set Variable ${hosts.src[${I}]}
527 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800528 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700529 ${onu_device_id}= Get Device ID From SN ${src['onu']}
530 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000531 ... ${of_id} ${src['uni_id']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700532 # Bring up the device and verify it authenticates
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700533 Wait Until Keyword Succeeds 360s 5s Validate Device ENABLED ACTIVE REACHABLE
Hardik Windlass8b446492021-06-10 06:25:23 +0000534 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed by_dev_id=True
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700535 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700536 ... ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700537 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
538 ... /tmp/wpa ${src['dp_iface_name']} log
539 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
540 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
541 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
542 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
543 ... ${ONOS_SSH_PORT} ${onu_port}
544 END
545
546 # Scale down the rw-core deployment to 0 PODs and once confirmed, scale it back to 1
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700547 Scale K8s Deployment ${NAMESPACE} ${STACK_NAME}-voltha-rw-core 0
548 Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist ${NAMESPACE} ${STACK_NAME}-voltha-rw-core
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700549 # Ensure the ofagent POD goes "not-ready" as expected
550 Wait Until keyword Succeeds ${timeout} 2s
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700551 ... Check Expected Available Deployment Replicas ${NAMESPACE} ${STACK_NAME}-voltha-ofagent 0
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700552 # Scale up the core deployment and make sure both it and the ofagent deployment are back
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700553 Scale K8s Deployment ${NAMESPACE} ${STACK_NAME}-voltha-rw-core 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700554 Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700555 ... Check Expected Available Deployment Replicas ${NAMESPACE} ${STACK_NAME}-voltha-rw-core 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700556 Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandoloa4e59522021-08-31 14:56:24 -0700557 ... Check Expected Available Deployment Replicas ${NAMESPACE} ${STACK_NAME}-voltha-ofagent 1
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700558 # For some reason scaling down and up the POD behind a service causes the port forward to stop working,
559 # so restart the port forwarding for the API service
David Bainbridge07321ff2020-06-12 13:55:42 -0700560 Restart VOLTHA Port Forward voltha-api
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700561 # Ensure that the ofagent pod is up and ready and the device is available in ONOS, this
562 # represents system connectivity being restored
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800563 FOR ${I} IN RANGE 0 ${olt_count}
564 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
565 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
566 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
567 ... ${olt_serial_number}
Girish Gowdrae4de3b32021-03-23 22:20:00 -0700568 Wait Until Keyword Succeeds 120s 2s Device Is Available In ONOS
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800569 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
570 END
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700571
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800572 FOR ${I} IN RANGE 0 ${num_all_onus}
Andy Bavier52094622020-05-12 15:54:24 -0700573 ${src}= Set Variable ${hosts.src[${I}]}
574 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800575 ${of_id}= Get ofID From OLT List ${src['olt']}
Andy Bavier52094622020-05-12 15:54:24 -0700576 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000577 ... ${of_id} ${src['uni_id']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700578 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
TorstenThieme4e2168e2021-06-22 14:01:47 +0000579 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700580 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530581 # Verify subscriber access flows are added for the ONU port
582 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
583 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
584 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Suchitra Vemuriebf65322020-04-03 10:09:41 -0700585 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
586 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
587 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
588 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
589 ... ${dst['container_name']}
590 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
591 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
592 END
593
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700594Verify OLT Soft Reboot
595 [Documentation] Test soft reboot of the OLT using voltctl command
596 [Tags] VOL-2745 OLTSoftReboot functional
597 [Setup] Start Logging OLTSoftReboot
598 [Teardown] Run Keywords Collect Logs
599 ... AND Stop Logging OLTSoftReboot
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800600 FOR ${I} IN RANGE 0 ${olt_count}
601 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
602 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
603 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
604 ... Validate OLT Device ENABLED ACTIVE
605 ... REACHABLE ${olt_serial_number}
606 # Reboot the OLT using "voltctl device reboot" command
607 Reboot Device ${olt_device_id}
Andrea Campanella9d8d3a52021-06-17 12:18:23 +0200608 # Wait for the OLT to actually go down
609 Wait Until Keyword Succeeds 360s 5s Validate OLT Device ENABLED UNKNOWN UNREACHABLE
610 ... ${olt_serial_number}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800611 END
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700612 #Verify that ping fails
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800613 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700614 ${src}= Set Variable ${hosts.src[${I}]}
615 ${dst}= Set Variable ${hosts.dst[${I}]}
616 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
617 ... Wait Until Keyword Succeeds 60s 2s
618 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
619 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
620 END
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700621 # Check OLT states
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800622 FOR ${I} IN RANGE 0 ${olt_count}
623 ${olt_serial_number}= Get From Dictionary ${list_olts}[${I}] sn
624 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[${I}] sship
625 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
626 # Wait for the OLT to come back up
627 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
628 ... Check Remote System Reachability True ${olt_ssh_ip}
629 # Check OLT states
630 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
631 ... Validate OLT Device ENABLED ACTIVE
632 ... REACHABLE ${olt_serial_number}
633 END
634 # Waiting extra time for the ONUs to come up
635 Sleep 60s
Suchitra Vemuri19e942f2020-08-11 18:04:28 -0700636 #Check after reboot that ONUs are active, authenticated/DHCP/pingable
637 Run Keyword If ${has_dataplane} Clean Up Linux
638 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
639
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700640Verify restart ofagent container before subscriber is provisioned
641 [Documentation] Restart ofagent container before subscriber is provisioned.
642 [Tags] functional VOL-2962 ofagentRestart2
643 [Setup] Start Logging ofagentRestart2
644 [Teardown] Run Keywords Collect Logs
Suchitra Vemuri8130be82020-09-15 16:47:58 -0700645 ... AND Stop Logging ofagentRestart2
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000646 ... AND Scale K8s Deployment ${NAMESPACE} ${NAMESPACE}-voltha-ofagent 1
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800647 Delete All Devices And Verify
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700648 setup
649 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800650 FOR ${I} IN RANGE 0 ${olt_count}
651 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
652 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
653 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
654 ... ${olt_serial_number}
655 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
656 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
657 END
658 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700659 ${src}= Set Variable ${hosts.src[${I}]}
660 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800661 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700662 ${onu_device_id}= Get Device ID From SN ${src['onu']}
663 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000664 ... ${of_id} ${src['uni_id']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700665 # Bring up the device and verify it authenticates
666 Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
Hardik Windlass8b446492021-06-10 06:25:23 +0000667 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed by_dev_id=True
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700668 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700669 ... ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700670 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
671 ... /tmp/wpa ${src['dp_iface_name']} log
672 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
673 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
674 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
675 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
676 ... ${ONOS_SSH_PORT} ${onu_port}
677 END
678 # Restart POD ofagent
679 ${waitforRestart} Set Variable 120s
680 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
681 Log ${podStatusOutput}
682 ${countBforRestart}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
683 ${podName} Set Variable ofagent
TorstenThieme37165402021-09-03 11:39:40 +0000684 Restart Pod By Label ${NAMESPACE} app ${podName}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700685 Wait Until Keyword Succeeds ${waitforRestart} 2s Validate Pod Status ofagent ${NAMESPACE}
686 ... Running
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800687 FOR ${I} IN RANGE 0 ${num_all_onus}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700688 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
TorstenThieme4e2168e2021-06-22 14:01:47 +0000689 #Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700690 #... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
691 ${src}= Set Variable ${hosts.src[${I}]}
692 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800693 ${of_id}= Get ofID From OLT List ${src['olt']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700694 ${onu_device_id}= Get Device ID From SN ${src['onu']}
695 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Hardik Windlass1ed2eee2021-06-25 09:51:03 +0000696 ... Get ONU Port in ONOS ${src['onu']} ${of_id} ${src['uni_id']}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700697 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
TorstenThieme4e2168e2021-06-22 14:01:47 +0000698 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700699 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
700 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
701 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
702 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
703 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
704 ... ${dst['container_name']}
705 # Verify DHCP-Allocations
706 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
707 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri10e737e2020-08-14 14:59:10 -0700708 END
709
Hardik Windlassc082c422021-03-16 15:44:10 +0000710Verify OLT Grpc Disconnection
711 [Documentation] Restarts OLT Grpc Server and verifies everything works as before without any system disruption.
712 [Tags] functional VOL-3904 restartGrpcServer bbsim
713 [Setup] Start Logging restartGrpcServer
714 [Teardown] Run Keywords Collect Logs
715 ... AND Stop Logging restartGrpcServer
716 Delete All Devices And Verify
717 Setup
718 Run Keyword If ${has_dataplane} Clean Up Linux
719 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
720 FOR ${J} IN RANGE 0 ${num_olts}
721 ${bbsim_rel}= Catenate SEPARATOR= bbsim ${J}
722 ${bbsim_pod}= Get Pod Name By Label ${NAMESPACE} release ${bbsim_rel}
723 Restart Grpc Server ${NAMESPACE} ${bbsim_pod} 5
724 END
725 # Repeat sanity test without subscriber changes
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700726 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
Hardik Windlassc082c422021-03-16 15:44:10 +0000727 # Additional Verification
728 Wait Until Keyword Succeeds ${timeout} 2s Delete All Devices and Verify
729 Setup
730 Run Keyword If ${has_dataplane} Clean Up Linux
731 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
732
Hema123aff52020-05-26 15:41:47 +0530733Verify ONU Soft Reboot
734 [Documentation] Test soft reboot of the ONU using voltctl command
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800735 [Tags] VOL-1957 ONUSoftReboot functional notready
Hema123aff52020-05-26 15:41:47 +0530736 [Setup] Start Logging ONUSoftReboot
737 #... AND Setup
Andy Bavier4a8450e2020-02-04 08:58:37 -0700738 [Teardown] Run Keywords Collect Logs
Hema123aff52020-05-26 15:41:47 +0530739 ... AND Stop Logging ONUSoftReboot
740 #... AND Delete Device and Verify
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800741 FOR ${I} IN RANGE 0 ${num_all_onus}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000742 ${src}= Set Variable ${hosts.src[${I}]}
743 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800744 ${of_id}= Get ofID From OLT List ${src['olt']}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000745 ${onu_device_id}= Get Device ID From SN ${src['onu']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800746 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
747 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Hema123aff52020-05-26 15:41:47 +0530748 Reboot Device ${onu_device_id}
749 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
750 ... Wait Until Keyword Succeeds 60s 2s
751 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
752 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
753 # Remove Subscriber Access (To replicate ATT workflow)
TorstenThieme4e2168e2021-06-22 14:01:47 +0000754 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
Hema123aff52020-05-26 15:41:47 +0530755 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800756 Verify ping is succesful except for given device ${num_all_onus} ${onu_device_id}
Hema123aff52020-05-26 15:41:47 +0530757 Sleep 40s
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800758 # Check ONU port is Enabled in ONOS
759 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Hardik Windlass39015672021-07-05 05:48:08 +0000760 ... Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800761 # Verify EAPOL flows are added for the ONU port
762 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Matteo Scandolo7e519fb2021-08-13 11:35:16 -0700763 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id} ${onu_port}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800764 # Verify ONU state in voltha
Suchitra Vemuricdc32d82020-07-17 12:24:46 -0700765 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s Validate Device
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800766 ... ENABLED ACTIVE REACHABLE
767 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hema123aff52020-05-26 15:41:47 +0530768 Run Keyword If ${has_dataplane} Clean Up Linux
769 # Perform Authentication
770 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
771 ... /tmp/wpa ${src['dp_iface_name']} log
772 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
773 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
774 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
775 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
776 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
777 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
TorstenThieme4e2168e2021-06-22 14:01:47 +0000778 ... Execute ONOS CLI Command use single connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hema123aff52020-05-26 15:41:47 +0530779 ... volt-add-subscriber-access ${of_id} ${onu_port}
780 # Verify that no pending flows exist for the ONU port
781 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
782 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530783 # Verify subscriber access flows are added for the ONU port
784 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
785 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
786 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Hema123aff52020-05-26 15:41:47 +0530787 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
788 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
Suchitra Vemuricd2f64f2020-02-18 18:30:27 -0800789 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Hema123aff52020-05-26 15:41:47 +0530790 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
791 ... ${dst['container_name']}
792 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
793 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000794 END
Suchitra Vemuria6879aa2020-11-03 11:03:11 -0800795
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800796
797*** Keywords ***
798Setup Suite
799 [Documentation] Set up the test suite
800 Common Test Suite Setup
HungWei Chiu1408fb92020-03-03 19:43:30 -0500801 #power_switch.robot needs it to support different vendor's power switch
Hung-Wei Chiufcc6d8e2020-03-10 12:05:11 -0700802 ${switch_type}= Get Variable Value ${web_power_switch.type}
803 Run Keyword If "${switch_type}"!="" Set Global Variable ${powerswitch_type} ${switch_type}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800804
805Clear All Devices Then Create New Device
806 [Documentation] Remove any devices from VOLTHA and ONOS
807 # Remove all devices from voltha and nos
808 Delete All Devices and Verify
809 # Execute normal test Setup Keyword
810 Setup
TorstenThieme4e2168e2021-06-22 14:01:47 +0000811
812Teardown Suite
813 [Documentation] Clean up ONOS SSH connections
814 Close All ONOS SSH Connections