blob: 8a70340c3ca1576da77afa2a4c6f46bf12f6b345 [file] [log] [blame]
Matteo Scandolo142e6272020-04-29 17:36:59 -07001# Copyright 2017 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15# Tests can be enabled by passing the following tags:
16# - [setup] Creates and enable the OLT devices
17# - [activation] Checks that ONUs are active in VOLTHA and ports discevered in ONOS
18# - [flow-before] Checks that flows are pushed (before subscriber provisioning)
19# - [authentication] Checks that subscribers are correctly authenticated
20# - [provision] Provision the data-plane flows for all the subscribers
21# - [flow-after] Checks that flows are pushed (after subscriber provisioning)
22# - [dhcp] Checks that subscribers have received an IP
23#
24# To run the full test:
25# robot Voltha_Scale_Tests.robot
26#
27# To run only ceratain tests:
28# robot -i activation -i flow-before Voltha_Scale_Tests.robot
29#
30# To exclude only ceratain tests:
31# robot -e -i flow-before Voltha_Scale_Tests.robot
32#
33# Once te test complete you can extrapolate the results by using
34# python extract-times.py
35
36*** Settings ***
37Documentation Collect measurements on VOLTHA performances
38Suite Setup Setup Suite
39#Test Setup Setup
40#Test Teardown Teardown
Matteo Scandolo37bca8d2020-07-31 11:28:40 -070041Suite Teardown Teardown Suite
Matteo Scandolo142e6272020-04-29 17:36:59 -070042Library Collections
43Library String
44Library OperatingSystem
45Library XML
Matteo Scandolo142e6272020-04-29 17:36:59 -070046Library RequestsLibrary
47Library ../../libraries/DependencyLibrary.py
48Resource ../../libraries/onos.robot
49Resource ../../libraries/voltctl.robot
50Resource ../../libraries/voltha.robot
Matteo Scandoloeb26a842020-05-08 10:06:24 -070051Resource ../../libraries/flows.robot
Matteo Scandolo142e6272020-04-29 17:36:59 -070052Resource ../../libraries/k8s.robot
Hardik Windlass513afd12021-02-03 15:19:46 +000053Resource ../../libraries/utils.robot
54Resource ../../libraries/bbsim.robot
Matteo Scandolo142e6272020-04-29 17:36:59 -070055Resource ../../variables/variables.robot
56
57*** Variables ***
58${ONOS_SSH_IP} 127.0.0.1
Matteo Scandoloec73e702021-04-09 11:09:38 -070059${ONOS_SSH_PORT} 30115
Hardik Windlass513afd12021-02-03 15:19:46 +000060${ONOS_REST_IP} 127.0.0.1
Matteo Scandoloec73e702021-04-09 11:09:38 -070061${ONOS_REST_PORT} 30120
Matteo Scandolo142e6272020-04-29 17:36:59 -070062
Hardik Windlass513afd12021-02-03 15:19:46 +000063${BBSIM_REST_IP} 127.0.0.1
64${BBSIM_REST_PORT} 50071
65
66${NAMESPACE} default
67
Matteo Scandolo142e6272020-04-29 17:36:59 -070068# Scale pipeline values
Matteo Scandolo50be75c2020-11-12 11:14:12 -080069${stackId} 1
Matteo Scandolo142e6272020-04-29 17:36:59 -070070${olt} 1
71${pon} 1
72${onu} 1
73
Matteo Scandolo142e6272020-04-29 17:36:59 -070074${enableFlowProvisioning} true
75${enableSubscriberProvisioning} true
76
77${workflow} att
Matteo Scandoloeb26a842020-05-08 10:06:24 -070078${withEapol} false
79${withDhcp} false
80${withIgmp} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +020081# as of now the LLDP flow is always installed
82${withLLDP} true
Matteo Scandolo142e6272020-04-29 17:36:59 -070083
84# Per-test logging on failure is turned off by default; set this variable to enable
85${container_log_dir} ${None}
86
Hardik Windlass513afd12021-02-03 15:19:46 +000087${timeout} 10m
88
Matteo Scandolo142e6272020-04-29 17:36:59 -070089*** Test Cases ***
90
91Create and Enable devices
92 [Documentation] Create and enable the OLTs in VOLTHA
Matteo Scandolo830071d2021-07-01 15:17:27 +020093 [Tags] setup
Matteo Scandolo142e6272020-04-29 17:36:59 -070094 ${olt_device_ids}= Create List
95 FOR ${INDEX} IN RANGE 0 ${olt}
Matteo Scandoloeb26a842020-05-08 10:06:24 -070096 ${olt_device_id}= Create Device bbsim${INDEX} 50060 openolt
Matteo Scandolo142e6272020-04-29 17:36:59 -070097 Enable Device ${olt_device_id}
98 Append To List ${olt_device_ids} ${olt_device_id}
99 END
100
101 Set Suite Variable ${olt_device_ids}
102
Matteo Scandolo7d1a80d2021-04-09 14:30:43 -0700103OLTs in ONOS
104 [Documentation] Check that ONOS recognize the correct number of OLTs
105 [Tags] activation plot-onos-olts
Matteo Scandolo520f77e2021-06-01 16:14:47 -0700106 ${onos_devices}= Compute Device IDs
107 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200108 Wait for Olt in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${deviceId}
Matteo Scandolo520f77e2021-06-01 16:14:47 -0700109 END
Matteo Scandolo7d1a80d2021-04-09 14:30:43 -0700110
Matteo Scandolo142e6272020-04-29 17:36:59 -0700111Onu Activation in VOLTHA
112 [Documentation] Check that all ONUs reach the ACTIVE/ENABLED state in VOLTHA
Matteo Scandolo830071d2021-07-01 15:17:27 +0200113 [Tags] activation plot-voltha-onus
Matteo Scandolo142e6272020-04-29 17:36:59 -0700114 Wait For ONUs In VOLTHA ${total_onus}
115
116Port Discovery in ONOS
117 [Documentation] Check that all the UNI ports show up in ONOS
Matteo Scandolo830071d2021-07-01 15:17:27 +0200118 [Tags] activation plot-onos-ports
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800119 ${onos_devices}= Compute Device IDs
120 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200121 Wait for Ports in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${total_onus_per_olt} ${deviceId} BBSM
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800122 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700123
124Flows validation in VOLTHA before subscriber provisioning
Matteo Scandoloeb26a842020-05-08 10:06:24 -0700125 [Documentation] Check that all the flows has been stored in the logical device
Matteo Scandolo830071d2021-07-01 15:17:27 +0200126 [Tags] flow-before plot-voltha-flows-before
Matteo Scandolo142e6272020-04-29 17:36:59 -0700127 # NOTE fail the test immediately if we're trying to check flows without provisioning them
128 Should Be Equal ${enableFlowProvisioning} true
129 Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200130 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700131
Matteo Scandolo616daab2020-05-13 11:49:24 -0700132Flows validation in VOLTHA Adapters before subscriber provisioning
133 [Documentation] Check that all flows has been store in devices of type openolt
Matteo Scandolo830071d2021-07-01 15:17:27 +0200134 [Tags] flow-before plot-voltha-openolt-flows-before only-me
Matteo Scandolo616daab2020-05-13 11:49:24 -0700135 Should Be Equal ${enableFlowProvisioning} true
136 Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200137 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo616daab2020-05-13 11:49:24 -0700138
Matteo Scandolo142e6272020-04-29 17:36:59 -0700139Flows validation in ONOS before subscriber provisioning
Matteo Scandoloeb26a842020-05-08 10:06:24 -0700140 [Documentation] Check that all the flows has been acknowledged
Matteo Scandolo830071d2021-07-01 15:17:27 +0200141 [Tags] flow-before plot-onos-flows-before
Matteo Scandolo142e6272020-04-29 17:36:59 -0700142 # NOTE fail the test immediately if we're trying to check flows without provisioning them
143 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800144
145 ${onos_devices}= Compute Device IDs
146 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200147 Wait for all flows to in ADDED state ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800148 ... ${deviceId} ${workflow} ${total_onus_per_olt} 1 false
149 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
150 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700151
152Wait for subscribers to be Authenticated
153 [Documentation] Check that all subscribers have successfully authenticated
Matteo Scandolo830071d2021-07-01 15:17:27 +0200154 [Tags] authentication plot-onos-auth
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800155
156 ${onos_devices}= Compute Device IDs
157 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200158 Wait for AAA Authentication ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${total_onus_per_olt} ${deviceId}
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800159 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700160
161Provision subscribers
162 [Documentation] Provision data plane flows for all the subscribers
Matteo Scandolo830071d2021-07-01 15:17:27 +0200163 [Tags] provision
Matteo Scandolo142e6272020-04-29 17:36:59 -0700164 Should Be Equal ${enableSubscriberProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800165 ${onos_devices}= Compute Device IDs
166 FOR ${olt} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200167 Provision all subscribers on device ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${ONOS_SSH_IP} ${ONOS_REST_PORT} ${olt}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700168 END
169
170Flows validation in VOLTHA after subscriber provisioning
171 [Documentation] Check that all the flows has been stored in the logical device
Matteo Scandolo830071d2021-07-01 15:17:27 +0200172 [Tags] flow-after plot-voltha-flows-after
Matteo Scandolo142e6272020-04-29 17:36:59 -0700173 # NOTE fail the test immediately if we're trying to check flows without provisioning them
174 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800175
Matteo Scandolo142e6272020-04-29 17:36:59 -0700176 Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} true
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200177 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700178
Matteo Scandolo616daab2020-05-13 11:49:24 -0700179Flows validation in VOLTHA Adapters after subscriber provisioning
180 [Documentation] Check that all flows has been store in devices of type openolt
Matteo Scandolo830071d2021-07-01 15:17:27 +0200181 [Tags] flow-after plot-voltha-openolt-flows-after only-me
Matteo Scandolo616daab2020-05-13 11:49:24 -0700182 Should Be Equal ${enableFlowProvisioning} true
183 Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} true
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200184 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo616daab2020-05-13 11:49:24 -0700185
Matteo Scandolo142e6272020-04-29 17:36:59 -0700186Flows validation in ONOS after subscriber provisioning
187 [Documentation] Check that all the flows has been acknowledged
Matteo Scandolo830071d2021-07-01 15:17:27 +0200188 [Tags] flow-after plot-onos-flows-after
Matteo Scandolo142e6272020-04-29 17:36:59 -0700189 # NOTE fail the test immediately if we're trying to check flows without provisioning them
190 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800191
192 ${onos_devices}= Compute Device IDs
193 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200194 Wait for all flows to in ADDED state ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800195 ... ${deviceId} ${workflow} ${total_onus_per_olt} 1 true
196 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
197 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700198
199Wait for subscribers to have an IP
200 [Documentation] Check that all subscribers have received a DHCP_ACK
Matteo Scandolo830071d2021-07-01 15:17:27 +0200201 [Tags] dhcp plot-onos-dhcp
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800202 ${onos_devices}= Compute Device IDs
203 FOR ${deviceId} IN @{onos_devices}
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200204 Wait for DHCP Ack ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${total_onus_per_olt} ${workflow} ${deviceId}
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800205 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700206
Hardik Windlass513afd12021-02-03 15:19:46 +0000207Perform Igmp Join
208 [Documentation] Performs Igmp Join for all the ONUs of all the OLTs (based on Rest Endpoint)
209 [Tags] non-critical igmp igmp-join
210 FOR ${INDEX} IN RANGE 0 ${olt}
211 ${bbsim_rel}= Catenate SEPARATOR= bbsim ${INDEX}
212 ${bbsim_rel_local_port}= Evaluate ${BBSIM_REST_PORT}+${INDEX}
213 Create Session ${bbsim_rel} http://${BBSIM_REST_IP}:${bbsim_rel_local_port}
214 ${bbsim_pod}= Get Pod Name By Label ${NAMESPACE} release ${bbsim_rel}
215 ${onu_list}= Get ONUs List ${NAMESPACE} ${bbsim_pod}
216 Perform Igmp Join or Leave Per OLT ${bbsim_rel} ${onu_list} join
217 List Service ${NAMESPACE} ${bbsim_pod}
218 END
219
220Wait for ONUs Join Igmp Group
221 [Documentation] Checks the ONUs Join the IGMP Group
222 ... Note: Currently, it expects all the ONUs on an OLT joined the same group
223 [Tags] non-critical igmp igmp-join igmp-count-verify igmp-join-count-verify
224 ${onos_devices}= Compute Device IDs
225 FOR ${deviceId} IN @{onos_devices}
226 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200227 ... Verify ONUs in Group Count in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${total_onus_per_olt} ${deviceId}
Hardik Windlass513afd12021-02-03 15:19:46 +0000228 END
229
Hardik Windlass513afd12021-02-03 15:19:46 +0000230Perform Igmp Leave
231 [Documentation] Performs Igmp Leave for all the ONUs of all the OLTs (based on Rest Endpoint)
232 [Tags] non-critical igmp igmp-leave
233 FOR ${INDEX} IN RANGE 0 ${olt}
234 ${bbsim_rel}= Catenate SEPARATOR= bbsim ${INDEX}
235 ${bbsim_rel_local_port}= Evaluate ${BBSIM_REST_PORT}+${INDEX}
236 Create Session ${bbsim_rel} http://${BBSIM_REST_IP}:${bbsim_rel_local_port}
237 ${bbsim_pod}= Get Pod Name By Label ${NAMESPACE} release ${bbsim_rel}
238 ${onu_list}= Get ONUs List ${NAMESPACE} ${bbsim_pod}
239 Perform Igmp Join or Leave Per OLT ${bbsim_rel} ${onu_list} leave
240 List Service ${NAMESPACE} ${bbsim_pod}
241 END
242
243Wait for ONUs Leave Igmp Group
244 [Documentation] Checks the ONUs Leave the IGMP Group
245 ... Note: Currently, it expects all the ONUs on an OLT left the same group
246 [Tags] non-critical igmp igmp-leave igmp-count-verify igmp-leave-count-verify
247 ${onos_devices}= Compute Device IDs
248 FOR ${deviceId} IN @{onos_devices}
249 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Andrea Campanellabbfe55f2021-07-02 10:46:01 +0200250 ... Verify Empty Group in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${deviceId}
Hardik Windlass513afd12021-02-03 15:19:46 +0000251 END
252
Matteo Scandolo142e6272020-04-29 17:36:59 -0700253Disable and Delete devices
254 [Documentation] Disable and delete the OLTs in VOLTHA
255 [Tags] non-critical teardown
Matteo Scandolof078ba72021-12-15 15:31:47 -0800256
257 ${rc} ${output}= Run And Return Rc And Output voltctl -c ${VOLTCTL_CONFIG} device list -f Type=openolt -q
258 Should Be Equal As Integers ${rc} 0 Failed to get device list from voltctl: ${output}
259 Log ${output}
260 ${devices}= Split To Lines ${output}
261 Log ${devices}
262
263 FOR ${id} IN @{devices}
264 Disable Device ${id}
265 Delete Device ${id}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700266 END
267
Matteo Scandolo75911092021-11-16 17:05:36 -0800268 ${onos_devices}= Compute Device IDs
269 FOR ${deviceId} IN @{onos_devices}
270 Wait for all flows to be removed ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${deviceId}
271 END
272
Matteo Scandolo142e6272020-04-29 17:36:59 -0700273*** Keywords ***
274Setup Suite
Matteo Scandolo37bca8d2020-07-31 11:28:40 -0700275 [Documentation] Setup test global variables, open an SSH connection to ONOS and starts a timer
Matteo Scandolo142e6272020-04-29 17:36:59 -0700276 Set Suite Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
Matteo Scandolo5899be12020-11-11 15:38:07 -0800277 Set Suite Variable ${VOLTCTL_CONFIG} %{VOLTCONFIG}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700278
279 ${total_onus}= Evaluate ${olt} * ${pon} * ${onu}
280 Set Suite Variable ${total_onus}
281
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800282 ${total_onus_per_olt}= Evaluate ${pon} * ${onu}
283 Set Suite Variable ${total_onus_per_olt}
284
Hardik Windlass513afd12021-02-03 15:19:46 +0000285 ${onos_auth}= Create List karaf karaf
286 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
287 Run Keyword If '${workflow}'=='tt'
288 ... Send File To Onos ${CURDIR}/../../tests/data/onos-igmp.json apps/
289
Matteo Scandolo37bca8d2020-07-31 11:28:40 -0700290Teardown Suite
291 [Documentation] Close the SSH connection to ONOS
TorstenThieme4e2168e2021-06-22 14:01:47 +0000292 Close All ONOS SSH Connections
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800293
294Compute device IDs
295 [Documentation] Creates a list of ONOS device ID based on the test configuration
296 # TODO read ${olt} and ${stackid} from parameters
297 ${base}= Set Variable of:00000a0a0a0a0a
298 ${device_ids}= Create List
299 FOR ${olt_id} IN RANGE 0 ${olt}
300 ${decimal_id}= Catenate SEPARATOR= ${stackid} ${olt_id}
301 ${piece}= Convert To Hex ${decimal_id} length=2 lowercase=yes
302 ${id}= Catenate SEPARATOR= ${base} ${piece}
303 Append To List ${device_ids} ${id}
304 END
305
Hardik Windlass513afd12021-02-03 15:19:46 +0000306 [Return] ${device_ids}
307
308Perform Igmp Join or Leave Per OLT
309 [Documentation] Performs Igmp Join for all the ONUs of an OLT (based on Rest Endpoint)
310 [Arguments] ${bbsim_rel_session} ${onu_list} ${task}
311 FOR ${onu} IN @{onu_list}
312 JoinOrLeave Igmp Rest Based ${bbsim_rel_session} ${onu} ${task} 224.0.0.22
313 END
314