blob: 04163a3f2e8012202e046d32939f24847274032c [file] [log] [blame]
Matteo Scandolo142e6272020-04-29 17:36:59 -07001# Copyright 2017 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15# Tests can be enabled by passing the following tags:
16# - [setup] Creates and enable the OLT devices
17# - [activation] Checks that ONUs are active in VOLTHA and ports discevered in ONOS
18# - [flow-before] Checks that flows are pushed (before subscriber provisioning)
19# - [authentication] Checks that subscribers are correctly authenticated
20# - [provision] Provision the data-plane flows for all the subscribers
21# - [flow-after] Checks that flows are pushed (after subscriber provisioning)
22# - [dhcp] Checks that subscribers have received an IP
23#
24# To run the full test:
25# robot Voltha_Scale_Tests.robot
26#
27# To run only ceratain tests:
28# robot -i activation -i flow-before Voltha_Scale_Tests.robot
29#
30# To exclude only ceratain tests:
31# robot -e -i flow-before Voltha_Scale_Tests.robot
32#
33# Once te test complete you can extrapolate the results by using
34# python extract-times.py
35
36*** Settings ***
37Documentation Collect measurements on VOLTHA performances
38Suite Setup Setup Suite
39#Test Setup Setup
40#Test Teardown Teardown
Matteo Scandolo37bca8d2020-07-31 11:28:40 -070041Suite Teardown Teardown Suite
Matteo Scandolo142e6272020-04-29 17:36:59 -070042Library Collections
43Library String
44Library OperatingSystem
45Library XML
Matteo Scandolo142e6272020-04-29 17:36:59 -070046Library RequestsLibrary
47Library ../../libraries/DependencyLibrary.py
48Resource ../../libraries/onos.robot
49Resource ../../libraries/voltctl.robot
50Resource ../../libraries/voltha.robot
Matteo Scandoloeb26a842020-05-08 10:06:24 -070051Resource ../../libraries/flows.robot
Matteo Scandolo142e6272020-04-29 17:36:59 -070052Resource ../../libraries/k8s.robot
Hardik Windlass513afd12021-02-03 15:19:46 +000053Resource ../../libraries/utils.robot
54Resource ../../libraries/bbsim.robot
Matteo Scandolo142e6272020-04-29 17:36:59 -070055Resource ../../variables/variables.robot
56
57*** Variables ***
58${ONOS_SSH_IP} 127.0.0.1
59${ONOS_SSH_PORT} 8101
Hardik Windlass513afd12021-02-03 15:19:46 +000060${ONOS_REST_IP} 127.0.0.1
Matteo Scandolo96dbe432020-05-28 10:51:57 -070061${ONOS_REST_PORT} 8181
Matteo Scandolo142e6272020-04-29 17:36:59 -070062
Hardik Windlass513afd12021-02-03 15:19:46 +000063${BBSIM_REST_IP} 127.0.0.1
64${BBSIM_REST_PORT} 50071
65
66${NAMESPACE} default
67
Matteo Scandolo142e6272020-04-29 17:36:59 -070068# Scale pipeline values
Matteo Scandolo50be75c2020-11-12 11:14:12 -080069${stackId} 1
Matteo Scandolo142e6272020-04-29 17:36:59 -070070${olt} 1
71${pon} 1
72${onu} 1
73
Matteo Scandolo142e6272020-04-29 17:36:59 -070074${enableFlowProvisioning} true
75${enableSubscriberProvisioning} true
76
77${workflow} att
Matteo Scandoloeb26a842020-05-08 10:06:24 -070078${withEapol} false
79${withDhcp} false
80${withIgmp} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +020081# as of now the LLDP flow is always installed
82${withLLDP} true
Matteo Scandolo142e6272020-04-29 17:36:59 -070083
84# Per-test logging on failure is turned off by default; set this variable to enable
85${container_log_dir} ${None}
86
Hardik Windlass513afd12021-02-03 15:19:46 +000087${timeout} 10m
88
Matteo Scandolo142e6272020-04-29 17:36:59 -070089*** Test Cases ***
90
91Create and Enable devices
92 [Documentation] Create and enable the OLTs in VOLTHA
Andrea Campanella70cf0a72020-05-27 10:55:15 +020093 [Tags] non-critical setup
Matteo Scandolo142e6272020-04-29 17:36:59 -070094 ${olt_device_ids}= Create List
95 FOR ${INDEX} IN RANGE 0 ${olt}
Matteo Scandoloeb26a842020-05-08 10:06:24 -070096 ${olt_device_id}= Create Device bbsim${INDEX} 50060 openolt
Matteo Scandolo142e6272020-04-29 17:36:59 -070097 Enable Device ${olt_device_id}
98 Append To List ${olt_device_ids} ${olt_device_id}
99 END
100
101 Set Suite Variable ${olt_device_ids}
102
103Onu Activation in VOLTHA
104 [Documentation] Check that all ONUs reach the ACTIVE/ENABLED state in VOLTHA
105 [Tags] non-critical activation plot-voltha-onus
106 Wait For ONUs In VOLTHA ${total_onus}
107
108Port Discovery in ONOS
109 [Documentation] Check that all the UNI ports show up in ONOS
110 [Tags] non-critical activation plot-onos-ports
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800111 ${onos_devices}= Compute Device IDs
112 FOR ${deviceId} IN @{onos_devices}
113 Wait for Ports in ONOS ${onos_ssh_connection} ${total_onus_per_olt} ${deviceId} BBSM
114 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700115
116Flows validation in VOLTHA before subscriber provisioning
Matteo Scandoloeb26a842020-05-08 10:06:24 -0700117 [Documentation] Check that all the flows has been stored in the logical device
Matteo Scandolo142e6272020-04-29 17:36:59 -0700118 [Tags] non-critical flow-before plot-voltha-flows-before
119 # NOTE fail the test immediately if we're trying to check flows without provisioning them
120 Should Be Equal ${enableFlowProvisioning} true
121 Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200122 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700123
Matteo Scandolo616daab2020-05-13 11:49:24 -0700124Flows validation in VOLTHA Adapters before subscriber provisioning
125 [Documentation] Check that all flows has been store in devices of type openolt
Matteo Scandolo96dbe432020-05-28 10:51:57 -0700126 [Tags] non-critical flow-before plot-voltha-openolt-flows-before only-me
Matteo Scandolo616daab2020-05-13 11:49:24 -0700127 Should Be Equal ${enableFlowProvisioning} true
128 Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} false
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200129 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo616daab2020-05-13 11:49:24 -0700130
Matteo Scandolo142e6272020-04-29 17:36:59 -0700131Flows validation in ONOS before subscriber provisioning
Matteo Scandoloeb26a842020-05-08 10:06:24 -0700132 [Documentation] Check that all the flows has been acknowledged
Matteo Scandolo142e6272020-04-29 17:36:59 -0700133 [Tags] non-critical flow-before plot-onos-flows-before
134 # NOTE fail the test immediately if we're trying to check flows without provisioning them
135 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800136
137 ${onos_devices}= Compute Device IDs
138 FOR ${deviceId} IN @{onos_devices}
139 Wait for all flows to in ADDED state ${onos_ssh_connection}
140 ... ${deviceId} ${workflow} ${total_onus_per_olt} 1 false
141 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
142 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700143
144Wait for subscribers to be Authenticated
145 [Documentation] Check that all subscribers have successfully authenticated
146 [Tags] non-critical authentication plot-onos-auth
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800147
148 ${onos_devices}= Compute Device IDs
149 FOR ${deviceId} IN @{onos_devices}
150 Wait for AAA Authentication ${onos_ssh_connection} ${total_onus_per_olt} ${deviceId}
151 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700152
153Provision subscribers
154 [Documentation] Provision data plane flows for all the subscribers
155 [Tags] non-critical provision
156 Should Be Equal ${enableSubscriberProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800157 ${onos_devices}= Compute Device IDs
158 FOR ${olt} IN @{onos_devices}
Matteo Scandolo37bca8d2020-07-31 11:28:40 -0700159 Provision all subscribers on device ${onos_ssh_connection} ${ONOS_SSH_IP} ${ONOS_REST_PORT} ${olt}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700160 END
161
162Flows validation in VOLTHA after subscriber provisioning
163 [Documentation] Check that all the flows has been stored in the logical device
164 [Tags] non-critical flow-after plot-voltha-flows-after
165 # NOTE fail the test immediately if we're trying to check flows without provisioning them
166 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800167
Matteo Scandolo142e6272020-04-29 17:36:59 -0700168 Wait for Logical Devices flows ${workflow} ${total_onus} ${olt} true
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200169 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700170
Matteo Scandolo616daab2020-05-13 11:49:24 -0700171Flows validation in VOLTHA Adapters after subscriber provisioning
172 [Documentation] Check that all flows has been store in devices of type openolt
Matteo Scandolo96dbe432020-05-28 10:51:57 -0700173 [Tags] non-critical flow-after plot-voltha-openolt-flows-after only-me
Matteo Scandolo616daab2020-05-13 11:49:24 -0700174 Should Be Equal ${enableFlowProvisioning} true
175 Wait for OpenOLT Devices flows ${workflow} ${total_onus} ${olt} true
Andrea Campanella70cf0a72020-05-27 10:55:15 +0200176 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
Matteo Scandolo616daab2020-05-13 11:49:24 -0700177
Matteo Scandolo142e6272020-04-29 17:36:59 -0700178Flows validation in ONOS after subscriber provisioning
179 [Documentation] Check that all the flows has been acknowledged
180 [Tags] non-critical flow-after plot-onos-flows-after
181 # NOTE fail the test immediately if we're trying to check flows without provisioning them
182 Should Be Equal ${enableFlowProvisioning} true
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800183
184 ${onos_devices}= Compute Device IDs
185 FOR ${deviceId} IN @{onos_devices}
186 Wait for all flows to in ADDED state ${onos_ssh_connection}
187 ... ${deviceId} ${workflow} ${total_onus_per_olt} 1 true
188 ... ${withEapol} ${withDhcp} ${withIgmp} ${withLLDP}
189 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700190
191Wait for subscribers to have an IP
192 [Documentation] Check that all subscribers have received a DHCP_ACK
193 [Tags] non-critical dhcp plot-onos-dhcp
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800194 ${onos_devices}= Compute Device IDs
195 FOR ${deviceId} IN @{onos_devices}
196 Wait for DHCP Ack ${onos_ssh_connection} ${total_onus_per_olt} ${workflow} ${deviceId}
197 END
Matteo Scandolo142e6272020-04-29 17:36:59 -0700198
Hardik Windlass513afd12021-02-03 15:19:46 +0000199Perform Igmp Join
200 [Documentation] Performs Igmp Join for all the ONUs of all the OLTs (based on Rest Endpoint)
201 [Tags] non-critical igmp igmp-join
202 FOR ${INDEX} IN RANGE 0 ${olt}
203 ${bbsim_rel}= Catenate SEPARATOR= bbsim ${INDEX}
204 ${bbsim_rel_local_port}= Evaluate ${BBSIM_REST_PORT}+${INDEX}
205 Create Session ${bbsim_rel} http://${BBSIM_REST_IP}:${bbsim_rel_local_port}
206 ${bbsim_pod}= Get Pod Name By Label ${NAMESPACE} release ${bbsim_rel}
207 ${onu_list}= Get ONUs List ${NAMESPACE} ${bbsim_pod}
208 Perform Igmp Join or Leave Per OLT ${bbsim_rel} ${onu_list} join
209 List Service ${NAMESPACE} ${bbsim_pod}
210 END
211
212Wait for ONUs Join Igmp Group
213 [Documentation] Checks the ONUs Join the IGMP Group
214 ... Note: Currently, it expects all the ONUs on an OLT joined the same group
215 [Tags] non-critical igmp igmp-join igmp-count-verify igmp-join-count-verify
216 ${onos_devices}= Compute Device IDs
217 FOR ${deviceId} IN @{onos_devices}
218 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
219 ... Verify ONUs in Group Count in ONOS ${onos_ssh_connection} ${total_onus_per_olt} ${deviceId}
220 END
221
Hardik Windlass513afd12021-02-03 15:19:46 +0000222Perform Igmp Leave
223 [Documentation] Performs Igmp Leave for all the ONUs of all the OLTs (based on Rest Endpoint)
224 [Tags] non-critical igmp igmp-leave
225 FOR ${INDEX} IN RANGE 0 ${olt}
226 ${bbsim_rel}= Catenate SEPARATOR= bbsim ${INDEX}
227 ${bbsim_rel_local_port}= Evaluate ${BBSIM_REST_PORT}+${INDEX}
228 Create Session ${bbsim_rel} http://${BBSIM_REST_IP}:${bbsim_rel_local_port}
229 ${bbsim_pod}= Get Pod Name By Label ${NAMESPACE} release ${bbsim_rel}
230 ${onu_list}= Get ONUs List ${NAMESPACE} ${bbsim_pod}
231 Perform Igmp Join or Leave Per OLT ${bbsim_rel} ${onu_list} leave
232 List Service ${NAMESPACE} ${bbsim_pod}
233 END
234
235Wait for ONUs Leave Igmp Group
236 [Documentation] Checks the ONUs Leave the IGMP Group
237 ... Note: Currently, it expects all the ONUs on an OLT left the same group
238 [Tags] non-critical igmp igmp-leave igmp-count-verify igmp-leave-count-verify
239 ${onos_devices}= Compute Device IDs
240 FOR ${deviceId} IN @{onos_devices}
241 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
242 ... Verify Empty Group in ONOS ${onos_ssh_connection} ${deviceId}
243 END
244
Matteo Scandolo142e6272020-04-29 17:36:59 -0700245Disable and Delete devices
246 [Documentation] Disable and delete the OLTs in VOLTHA
247 [Tags] non-critical teardown
248 FOR ${olt_device_id} IN @{olt_device_ids}
249 Disable Device ${olt_device_id}
250 Delete Device ${olt_device_id}
Matteo Scandolo5899be12020-11-11 15:38:07 -0800251 Remove Values From List ${olt_device_ids} ${olt_device_id}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700252 END
253
254 Set Suite Variable ${olt_device_ids}
255
256*** Keywords ***
257Setup Suite
Matteo Scandolo37bca8d2020-07-31 11:28:40 -0700258 [Documentation] Setup test global variables, open an SSH connection to ONOS and starts a timer
Matteo Scandolo142e6272020-04-29 17:36:59 -0700259 Set Suite Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
Matteo Scandolo5899be12020-11-11 15:38:07 -0800260 Set Suite Variable ${VOLTCTL_CONFIG} %{VOLTCONFIG}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700261
262 ${total_onus}= Evaluate ${olt} * ${pon} * ${onu}
263 Set Suite Variable ${total_onus}
264
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800265 ${total_onus_per_olt}= Evaluate ${pon} * ${onu}
266 Set Suite Variable ${total_onus_per_olt}
267
Hardik Windlass513afd12021-02-03 15:19:46 +0000268 ${onos_auth}= Create List karaf karaf
269 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
270 Run Keyword If '${workflow}'=='tt'
271 ... Send File To Onos ${CURDIR}/../../tests/data/onos-igmp.json apps/
272
Matteo Scandolo37bca8d2020-07-31 11:28:40 -0700273 ${onos_ssh_connection} Open ONOS SSH Connection ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
274 Set Suite Variable ${onos_ssh_connection}
275
276Teardown Suite
277 [Documentation] Close the SSH connection to ONOS
Matteo Scandolo50be75c2020-11-12 11:14:12 -0800278 Close ONOS SSH Connection ${onos_ssh_connection}
279
280Compute device IDs
281 [Documentation] Creates a list of ONOS device ID based on the test configuration
282 # TODO read ${olt} and ${stackid} from parameters
283 ${base}= Set Variable of:00000a0a0a0a0a
284 ${device_ids}= Create List
285 FOR ${olt_id} IN RANGE 0 ${olt}
286 ${decimal_id}= Catenate SEPARATOR= ${stackid} ${olt_id}
287 ${piece}= Convert To Hex ${decimal_id} length=2 lowercase=yes
288 ${id}= Catenate SEPARATOR= ${base} ${piece}
289 Append To List ${device_ids} ${id}
290 END
291
Hardik Windlass513afd12021-02-03 15:19:46 +0000292 [Return] ${device_ids}
293
294Perform Igmp Join or Leave Per OLT
295 [Documentation] Performs Igmp Join for all the ONUs of an OLT (based on Rest Endpoint)
296 [Arguments] ${bbsim_rel_session} ${onu_list} ${task}
297 FOR ${onu} IN @{onu_list}
298 JoinOrLeave Igmp Rest Based ${bbsim_rel_session} ${onu} ${task} 224.0.0.22
299 END
300