blob: ea5d6948ac098c72fbb6e2c35b4f152972d1b0da [file] [log] [blame]
David Bainbridgef81cd642019-11-20 00:14:47 +00001# Copyright 2017 - present Open Networking Foundation
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -07002#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
Matteo Scandolo1294aeb2019-09-24 16:20:32 -070015# FIXME Can we use the same test against BBSim and Hardware?
16
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070017*** Settings ***
Zack Williamsec53a1b2019-09-16 15:50:52 -070018Documentation Test various end-to-end scenarios
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070019Suite Setup Setup Suite
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070020Test Setup Setup
21Test Teardown Teardown
Andy Bavierba9866b2019-10-11 07:11:53 -070022Suite Teardown Teardown Suite
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070023Library Collections
24Library String
25Library OperatingSystem
26Library XML
27Library RequestsLibrary
Andy Bavierba9866b2019-10-11 07:11:53 -070028Library ../../libraries/DependencyLibrary.py
Suchitra Vemurib8ed2d52019-09-30 13:22:51 -070029Resource ../../libraries/onos.robot
30Resource ../../libraries/voltctl.robot
David Bainbridgef81cd642019-11-20 00:14:47 +000031Resource ../../libraries/voltha.robot
Suchitra Vemurib8ed2d52019-09-30 13:22:51 -070032Resource ../../libraries/utils.robot
Andy Bavierba9866b2019-10-11 07:11:53 -070033Resource ../../libraries/k8s.robot
Suchitra Vemurib8ed2d52019-09-30 13:22:51 -070034Resource ../../variables/variables.robot
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070035
36*** Variables ***
Zack Williamsec53a1b2019-09-16 15:50:52 -070037${POD_NAME} flex-ocp-cord
38${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
39${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
40#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
41${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
42${HELM_CHARTS_DIR} ~/helm-charts
43${VOLTHA_POD_NUM} 8
suraj gourd64356b2019-11-07 13:26:20 +000044${NAMESPACE} voltha
45# For below variable value, using deployment name as using grep for
46# parsing radius pod name, we can also use full radius pod name
47${RESTART_POD_NAME} radius
Andy Bavierba9866b2019-10-11 07:11:53 -070048${timeout} 60s
Zack Williamsec53a1b2019-09-16 15:50:52 -070049${of_id} 0
50${logical_id} 0
Andy Bavierba9866b2019-10-11 07:11:53 -070051${has_dataplane} True
52${external_libs} True
53${teardown_device} False
David Bainbridgef81cd642019-11-20 00:14:47 +000054${scripts} ../../scripts
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070055
56*** Test Cases ***
57Sanity E2E Test for OLT/ONU on POD
58 [Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
59 ... Validate successful authentication/DHCP/E2E ping for the tech profile that is used
Andy Bavierba9866b2019-10-11 07:11:53 -070060 [Tags] sanity test1
Suchitra Vemuri6db89412019-11-14 14:52:54 -080061 [Teardown] NONE
Suchitra Vemuric5295a32019-12-15 20:32:04 -080062 Run Keyword If ${has_dataplane} Clean Up Linux
Suchitra Vemurib89b9ac2019-12-11 21:53:11 -080063 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
Suchitra Vemuri582574b2019-12-10 21:11:54 -080064 Run Keyword and Ignore Error Collect Logs
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -070065
Suchitra Vemuri6db89412019-11-14 14:52:54 -080066Test Disable and Enable ONU
67 [Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
68 ... Assuming that test1 was executed where all the ONUs are authenticated/DHCP/pingable
69 ... Perform disable on the ONUs and validate that the pings do not succeed
70 ... Perform enable on the ONUs and validate that the pings are successful
Suchitra Vemuri582574b2019-12-10 21:11:54 -080071 [Tags] functional DisableEnableONU
Suchitra Vemuri6db89412019-11-14 14:52:54 -080072 [Setup] None
suraj gour7f6d5fe2019-11-29 10:56:35 +000073 [Teardown] None
Suchitra Vemuri6db89412019-11-14 14:52:54 -080074
75 FOR ${I} IN RANGE 0 ${num_onus}
76 ${src}= Set Variable ${hosts.src[${I}]}
77 ${dst}= Set Variable ${hosts.dst[${I}]}
78
79 ${onu_device_id}= Get Device ID From SN ${src['onu']}
80 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
81 ... ${of_id}
82 Disable Device ${onu_device_id}
83 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping False ${dst['dp_iface_ip_qinq']}
84 ... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
85 Enable Device ${onu_device_id}
86 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
87 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
88 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping True ${dst['dp_iface_ip_qinq']}
89 ... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Suchitra Vemuri582574b2019-12-10 21:11:54 -080090 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
91 Run Keyword and Ignore Error Collect Logs
Suchitra Vemuri6db89412019-11-14 14:52:54 -080092 END
Suchitra Vemuri582574b2019-12-10 21:11:54 -080093 Run Keyword and Ignore Error Collect Logs
Suchitra Vemuri6db89412019-11-14 14:52:54 -080094
Suchitra Vemuri5994cd12019-12-17 22:20:55 -080095Test Subscriber Delete and Add
96 [Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
97 ... Assuming that all the ONUs are authenticated/DHCP/pingable
98 ... Delete a subscriber and validate that the pings do not succeed
99 ... Re-add the subscriber and validate that the pings are successful
100 [Tags] functional DisableEnableONU
101 [Setup] None
102 [Teardown] None
103
104 FOR ${I} IN RANGE 0 ${num_onus}
105 ${src}= Set Variable ${hosts.src[${I}]}
106 ${dst}= Set Variable ${hosts.dst[${I}]}
107
108 ${onu_device_id}= Get Device ID From SN ${src['onu']}
109 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
110 ... ${of_id}
111 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
112 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
113 Sleep 10s
114 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping False ${dst['dp_iface_ip_qinq']}
115 ... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
116 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
117 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
118 Sleep 10s
119 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
120 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
121 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping True ${dst['dp_iface_ip_qinq']}
122 ... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
123 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
124 Run Keyword and Ignore Error Collect Logs
125 END
126 Run Keyword and Ignore Error Collect Logs
127
suraj gourd64356b2019-11-07 13:26:20 +0000128Check OLT/ONU Authentication After Radius Pod Restart
suraj gour7f6d5fe2019-11-29 10:56:35 +0000129 [Documentation] After radius restart, triggers reassociation, checks status and
130 ... authentication, validates dhcp and ping. Note : wpa reassociate works only when
131 ... wpa supplicant is running in background hence it is recommended to remove
132 ... teardown from previous test or uncomment 'Teardown None'.
133 ... Assuming that test1 was executed where all the ONUs are authenticated/DHCP/pingable
Suchitra Vemuri582574b2019-12-10 21:11:54 -0800134 [Tags] functional RadiusRestart
suraj gourd5cfdbb2019-12-13 12:44:55 +0000135 [Setup] None
Suchitra Vemuribb00d9d2019-12-16 15:38:38 -0800136 [Teardown] None
suraj gourd64356b2019-11-07 13:26:20 +0000137 Wait Until Keyword Succeeds ${timeout} 15s Restart Pod ${NAMESPACE} ${RESTART_POD_NAME}
138
139 FOR ${I} IN RANGE 0 ${num_onus}
140 ${src}= Set Variable ${hosts.src[${I}]}
141 ${dst}= Set Variable ${hosts.dst[${I}]}
142
143
144 ${onu_device_id}= Get Device ID From SN ${src['onu']}
145 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
146 ... ${of_id}
147 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${k8s_node_ip}
148 ... ${ONOS_SSH_PORT} ${onu_port}
suraj gour7f6d5fe2019-11-29 10:56:35 +0000149 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication After Reassociate
150 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
suraj gourd64356b2019-11-07 13:26:20 +0000151 ... ${src['container_type']} ${src['container_name']}
152 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${k8s_node_ip}
153 ... ${ONOS_SSH_PORT} ${onu_port}
154 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
155 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
156 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
157 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
158 ... ${dst['container_name']}
159 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
160 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri582574b2019-12-10 21:11:54 -0800161 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
162 Run Keyword and Ignore Error Collect Logs
suraj gourd64356b2019-11-07 13:26:20 +0000163 END
Suchitra Vemuri582574b2019-12-10 21:11:54 -0800164 Run Keyword and Ignore Error Collect Logs
suraj gourd64356b2019-11-07 13:26:20 +0000165
suraj gourd5cfdbb2019-12-13 12:44:55 +0000166Check DHCP attempt fails when subscriber is not added
167 [Documentation] Validates when removed subscriber access, DHCP attempt, ping fails and
168 ... when again added subscriber access, DHCP attempt, ping succeeds
169 ... Assuming that test1 or sanity test was executed where all the ONUs are authenticated/DHCP/pingable
Suchitra Vemuribb00d9d2019-12-16 15:38:38 -0800170 [Tags] functional SubsRemoveDHCP
suraj gourd5cfdbb2019-12-13 12:44:55 +0000171 [Setup] None
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800172 [Teardown] None
suraj gourd5cfdbb2019-12-13 12:44:55 +0000173
174 FOR ${I} IN RANGE 0 ${num_onus}
175 ${src}= Set Variable ${hosts.src[${I}]}
176 ${dst}= Set Variable ${hosts.dst[${I}]}
177
178 ${onu_device_id}= Get Device ID From SN ${src['onu']}
179 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
180 ... ${of_id}
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800181 Run Keyword And Ignore Error Login And Run Command On Remote System killall dhclient ${src['ip']}
182 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
183 Run Keyword And Ignore Error Login And Run Command On Remote System ps -ef | grep dhclient ${src['ip']}
184 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000185 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
186 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800187 Sleep 5s
188 Run Keyword And Ignore Error Login And Run Command On Remote System ps -ef | grep dhclient ${src['ip']}
189 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000190 # For releasing IP, also deleting lease file. Hence passing path where DHCP Client program writes the lease
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800191 #Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds ${timeout} 2s
192 #... Send Dhclient Request To Release Assigned IP ${src['dp_iface_name']} ${src['ip']}
193 #... ${src['user']} /var/lib/dhcp ${src['pass']} ${src['container_type']} ${src['container_name']}
suraj gour19ee3a82019-12-16 09:43:14 +0000194 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds ${timeout} 2s
195 ... Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
196 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800197 Run Keyword And Ignore Error Login And Run Command On Remote System ifconfig | grep -A 10 ens ${src['ip']}
198 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000199 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping False
200 ... False ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
201 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
202 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
203 ... ${dst['container_name']}
204 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
205 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
206 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
207 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
208 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
209 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
210 ... ${dst['container_name']}
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800211 Run Keyword and Ignore Error Collect Logs
212 END
213 Run Keyword and Ignore Error Collect Logs
214
215
216Test Disable and Enable ONU scenario for ATT workflow
217 [Documentation] Validates E2E Ping Connectivity and object states for the given scenario:
218 ... Assuming that test1 was executed where all the ONUs are authenticated/DHCP/pingable
219 ... Perform disable on the ONUs, call volt-remove-subscriber and validate that the pings do not succeed
220 ... Perform enable on the ONUs, authentication check, volt-add-subscriber-access and validate that the pings are successful
221 ... VOL-2284
Andy Bavier08ae5852019-12-19 09:12:42 -0700222 [Tags] functional ATT_DisableEnableONU notready
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800223 [Setup] None
224 #[Teardown] None
225
226 FOR ${I} IN RANGE 0 ${num_onus}
227 ${src}= Set Variable ${hosts.src[${I}]}
228 ${dst}= Set Variable ${hosts.dst[${I}]}
229
230 ${onu_device_id}= Get Device ID From SN ${src['onu']}
231 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
232 ... ${of_id}
233 Disable Device ${onu_device_id}
234 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
235 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
236 Sleep 10s
237 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping False ${dst['dp_iface_ip_qinq']}
238 ... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
239 ... ELSE sleep 60s
240 Enable Device ${onu_device_id}
241 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${k8s_node_ip}
242 ... ${ONOS_SSH_PORT} ${onu_port}
243 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication After Reassociate
244 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
245 ... ${src['container_type']} ${src['container_name']}
246 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${k8s_node_ip}
247 ... ${ONOS_SSH_PORT} ${onu_port}
248 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
249 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
250 Sleep 10s
251 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
Andy Bavier08ae5852019-12-19 09:12:42 -0700252 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
Suchitra Vemuri5994cd12019-12-17 22:20:55 -0800253 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
254 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
255 ... ${dst['container_name']}
256 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
257 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
258 #Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s Check Ping True ${dst['dp_iface_ip_qinq']}
259 #... ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
260 Run Keyword and Ignore Error Collect Logs
suraj gourd5cfdbb2019-12-13 12:44:55 +0000261 END
262 Run Keyword and Ignore Error Collect Logs
263
David Bainbridgef81cd642019-11-20 00:14:47 +0000264Sanity E2E Test for OLT/ONU on POD With Core Fail and Restart
265 [Documentation] Deploys an device instance and waits for it to authenticate. After
266 ... authentication is successful the rw-core deployment is scaled to 0 instances to
267 ... simulate a POD crash. The test then scales the rw-core back to a single instance
268 ... and configures ONOS for access. The test succeeds if the device is able to
269 ... complete the DHCP sequence.
270 [Tags] bbsim rwcore-restart
271 [Setup] Clear All Devices Then Create New Device
272 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
273 Set Global Variable ${of_id}
274
275 FOR ${I} IN RANGE 0 ${num_onus}
276 ${src}= Set Variable ${hosts.src[${I}]}
277 ${dst}= Set Variable ${hosts.dst[${I}]}
278 ${onu_device_id}= Get Device ID From SN ${src['onu']}
279 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
280 ... ${of_id}
281
282 # Bring up the device and verify it authenticates
283 Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
284 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed
285 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${k8s_node_ip}
286 ... ${ONOS_SSH_PORT} ${onu_port}
287 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
288 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
289 ... ${src['container_type']} ${src['container_name']}
290 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${k8s_node_ip}
291 ... ${ONOS_SSH_PORT} ${onu_port}
292
293 # Scale down the rw-core deployment to 0 PODs and once confirmed, scale it back to 1
294 Scale K8s Deployment voltha voltha-rw-core 0
295 Wait Until Keyword Succeeds ${timeout} 2s Pod Does Not Exist voltha voltha-rw-core
296 # Ensure the ofagent POD goes "not-ready" as expected
297 Wait Until keyword Succeeds ${timeout} 2s Check Expected Available Deployment Replicas voltha voltha-ofagent 0
298 # Scale up the core deployment and make sure both it and the ofagent deployment are back
299 Scale K8s Deployment voltha voltha-rw-core 1
300 Wait Until Keyword Succeeds ${timeout} 2s Check Expected Available Deployment Replicas voltha voltha-rw-core 1
301 Wait Until Keyword Succeeds ${timeout} 2s Check Expected Available Deployment Replicas voltha voltha-ofagent 1
302
303 # For some reason scaling down and up the POD behind a service causes the port forward to stop working,
304 # so restart the port forwarding for the API service
305 Restart VOLTHA Port Foward voltha-api-minimal
306
307 # Ensure that the ofagent pod is up and ready and the device is available in ONOS, this
308 # represents system connectivity being restored
309 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
310 ... http://karaf:karaf@${k8s_node_ip}:${ONOS_REST_PORT} ${of_id}
311
312 # Add subscriber access and verify that DHCP completes to ensure system is still functioning properly
313 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
314 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
315 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
316 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
317 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
318 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
319 ... ${dst['container_name']}
320 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
321 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
322 END
323
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700324*** Keywords ***
325Setup Suite
Andy Bavier88cd9f62019-11-26 16:22:33 -0700326 [Documentation] Set up the test suite
327 Common Test Suite Setup
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700328
329Setup
Zack Williamsec53a1b2019-09-16 15:50:52 -0700330 [Documentation] Pre-test Setup
Gilles Depatieb5682f82019-10-31 10:39:45 -0400331 #test for empty device list
David Bainbridgef81cd642019-11-20 00:14:47 +0000332 Test Empty Device List
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700333 #create/preprovision device
Suchitra Vemuric59eb272019-09-18 17:59:33 -0700334 ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
335 Set Suite Variable ${olt_device_id}
Gilles Depatieb5682f82019-10-31 10:39:45 -0400336 #validate olt states
337 Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN
338 ... ${EMPTY} ${olt_device_id}
Suchitra Vemuric59eb272019-09-18 17:59:33 -0700339 Enable Device ${olt_device_id}
Gilles Depatieb5682f82019-10-31 10:39:45 -0400340 Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device ENABLED ACTIVE REACHABLE
341 ... ${olt_serial_number}
Suchitra Vemuri00d147d2019-09-13 13:07:32 -0700342 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
343 Set Suite Variable ${logical_id}
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700344
David Bainbridgef81cd642019-11-20 00:14:47 +0000345Delete All Devices and Verify
346 [Documentation] Remove any devices from VOLTHA and ONOS
347
348 # Clear devices from VOLTHA
349 Disable Devices In Voltha Root=true
350 Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
351 Delete Devices In Voltha Root=true
352 Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
353
354 # Clear devices from ONOS
355 Remove All Devices From ONOS
356 ... http://karaf:karaf@${k8s_node_ip}:${ONOS_REST_PORT}
357
358Clear All Devices Then Create New Device
359 [Documentation] Remove any devices from VOLTHA and ONOS
360
361 # Remove all devices from voltha and nos
362 Delete All Devices and Verify
363
364 # Execute normal test Setup Keyword
365 Setup
366
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700367Teardown
368 [Documentation] kills processes and cleans up interfaces on src+dst servers
Suchitra Vemuri582574b2019-12-10 21:11:54 -0800369 Run Keyword If ${has_dataplane} Clean Up Linux
370 Run Keyword If ${external_libs} Log Kubernetes Containers Logs Since Time ${datetime} ${container_list}
371
372Collect Logs
373 [Documentation] Collect Logs from voltha and onos cli for various commands
Suchitra Vemuri1a970a62019-11-26 12:52:16 -0800374 Run Keyword and Ignore Error Get Device List from Voltha
375 Run Keyword and Ignore Error Get Device Output from Voltha ${olt_device_id}
376 Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
Andy Bavierba9866b2019-10-11 07:11:53 -0700377 Run Keyword If ${external_libs} Get ONOS Status ${k8s_node_ip}
Andy Bavierba9866b2019-10-11 07:11:53 -0700378
379Teardown Suite
380 [Documentation] Clean up device if desired
David Bainbridgef81cd642019-11-20 00:14:47 +0000381 Run Keyword If ${teardown_device} Delete All Devices and Verify
Gilles Depatieb5682f82019-10-31 10:39:45 -0400382
Suchitra Vemuri65cd65f2019-08-30 14:39:22 -0700383Clean Up Linux
384 [Documentation] Kill processes and clean up interfaces on src+dst servers
Suchitra Vemuri8a9c3782019-10-23 12:43:01 -0700385 FOR ${I} IN RANGE 0 ${num_onus}
386 ${src}= Set Variable ${hosts.src[${I}]}
387 ${dst}= Set Variable ${hosts.dst[${I}]}
388 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
389 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
390 Run Keyword And Ignore Error Kill Linux Process [d]hclient ${src['ip']}
391 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
392 Run Keyword If '${dst['ip']}' != '${None}' Run Keyword And Ignore Error
393 ... Kill Linux Process [d]hcpd ${dst['ip']} ${dst['user']}
394 ... ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
395 Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
396 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
397 Run Keyword If '${dst['ip']}' != '${None}' Delete Interface on Remote Host
398 ... ${dst['dp_iface_name']}.${src['s_tag']} ${dst['ip']} ${dst['user']} ${dst['pass']}
399 ... ${dst['container_type']} ${dst['container_name']}
400 END
Andy Bavierba9866b2019-10-11 07:11:53 -0700401
402Delete Device and Verify
403 [Documentation] Disable -> Delete devices via voltctl and verify its removed
Andy Bavierba9866b2019-10-11 07:11:53 -0700404 ${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
405 Should Be Equal As Integers ${rc} 0
Gilles Depatieb5682f82019-10-31 10:39:45 -0400406 Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Device DISABLED UNKNOWN REACHABLE
407 ... ${olt_serial_number}
Andy Bavierba9866b2019-10-11 07:11:53 -0700408 ${rc} ${output}= Run and Return Rc and Output ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
409 Should Be Equal As Integers ${rc} 0
410 Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
Suchitra Vemurib89b9ac2019-12-11 21:53:11 -0800411
412Perform Sanity Test
413 [Documentation] This keyword performs Sanity Test Procedure
414 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
415 ... This keyword can be used to call in any other tests where sanity check is required
416 ... and avoids duplication of code.
417
418 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
419 Set Global Variable ${of_id}
420
421 FOR ${I} IN RANGE 0 ${num_onus}
422 ${src}= Set Variable ${hosts.src[${I}]}
423 ${dst}= Set Variable ${hosts.dst[${I}]}
424
425 Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
426 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
427
428 ${onu_device_id}= Get Device ID From SN ${src['onu']}
429 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
430 ... ${of_id}
431 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${k8s_node_ip}
432 ... ${ONOS_SSH_PORT} ${onu_port}
433 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
434 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
435 ... ${src['container_type']} ${src['container_name']}
436 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${k8s_node_ip}
437 ... ${ONOS_SSH_PORT} ${onu_port}
438 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command ${k8s_node_ip}
439 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
440 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
441 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
442 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
443 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
444 ... ${dst['container_name']}
445 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
446 ... Validate Subscriber DHCP Allocation ${k8s_node_ip} ${ONOS_SSH_PORT} ${onu_port}
447 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
448 Run Keyword and Ignore Error Collect Logs
449 END