blob: c2f4bbd86647e0890d231f718e559ecb60c19839 [file] [log] [blame]
TorstenThiemed9cc06e2022-11-09 12:12:46 +00001# Copyright 2022 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15*** Settings ***
16Documentation Test of open ONU error cases
17... The test suite validates special error scenarios of openonu_go_adapter
18... -- power off during mib download
19... -- power of during flow configuration
20... -- onu capabilities against configuration requirements priority queues
21... -- onu capabilities against configuration requirements tconts
22... see also VOL-4796
23... In case of kafka pod runs in k8s cluster - kafka has to deploy with following EXTRA_HELM_FLAGS
24... --set externalAccess.enabled=true,
25... --set externalAccess.service.type=NodePort,
26... --set externalAccess.service.nodePorts[0]=${KAFKA_PORT},
27... --set externalAccess.service.domain=${KAFKA_IP}
28... with e.g. service.domain=10.0.02.15 or 127.0.0.1 and service.nodePorts[0]=30201!
29... For voltha-infra prefix kafka. is needed e.g.: --set kafka.externalAccess.enabled=true
30Suite Setup Setup Suite
31Suite Teardown Teardown Suite
32Test Setup Setup
33Test Teardown Teardown
34Library Collections
35Library String
36Library OperatingSystem
37Library XML
38Library RequestsLibrary
39Library ../../libraries/DependencyLibrary.py
40Resource ../../libraries/onos.robot
41Resource ../../libraries/voltctl.robot
42Resource ../../libraries/voltha.robot
43Resource ../../libraries/utils.robot
44Resource ../../libraries/k8s.robot
45Resource ../../libraries/onu_utilities.robot
46Resource ../../libraries/bbsim.robot
47Resource ../../variables/variables.robot
48
49Library kafka_robot.KafkaClient log_level=DEBUG WITH NAME kafka
50Library grpc_robot.VolthaTools WITH NAME volthatools
51
52
53*** Variables ***
54${NAMESPACE} voltha
55${INFRA_NAMESPACE} default
56${timeout} 60s
57${of_id} 0
58${logical_id} 0
59${has_dataplane} True
60${external_libs} True
61${teardown_device} True
62${scripts} ../../scripts
63# Per-test logging on failure is turned off by default; set this variable to enable
64${container_log_dir} ${None}
65
66# logging flag to enable Collect Logs, can be passed via the command line too
67# example: -v logging:True
68${logging} False
69# if True execution will be paused before clean up, only use in case of manual testing, do not use in ci pipeline!
70# example: -v pausebeforecleanup:True
71${pausebeforecleanup} False
72# flag for first test, needed due default timeout in BBSim to mimic OLT reboot of 60 seconds
73${firsttest} True
74# determines the environment workflow: DT, TT or ATT (default)
75# example: -v workflow:DT
76${workflow} ATT
77# KV Store Prefix
78# example: -v kvstoreprefix:voltha/voltha_voltha
79${kvstoreprefix} voltha/voltha_voltha
80# when voltha is running in k8s port forwarding is needed
81# example: -v PORT_FORWARDING:False
82${PORT_FORWARDING} True
83# kafka ip e.g. ip of master host where k8s is running
84# example: -v KAFKA_IP:10.0.2.15
85${KAFKA_IP} 127.0.0.1
86# kafka port: port of kafka nodeport
87# example: -v KAFKA_PORT:30201
88${KAFKA_PORT} 30201
89# kafka service port: service port of kafka nodeport
90# example: -v KAFKA_SVC_PORT:9094
91${KAFKA_SVC_PORT} 9094
92# onu MIB audit interval
93# example: -v ONU_MIB_AUDIT_INTERVAL:50s
94${ONU_MIB_AUDIT_INTERVAL} 60s
95# MDS mitsmatches per ONU
96# example: -v MDS_MISMATCHES_PER_ONU:2
97${MDS_MISMATCHES_PER_ONU} 3
98# if True execution will be paused before clean up, only use in case of manual testing, do not use in ci pipeline!
99# example: -v pausebeforecleanup:True
100${pausebeforecleanup} False
101# if True some outputs to console are done during running tests e.g. long duration flow test
102# example: -v print2console:True
103${print2console} False
104${suppressaddsubscriber} True
105${data_dir} ../data
106
107# flag to choose the subscriber provisioning command type in ONOS
108# TT often provision a single services for a subscriber (eg: hsia, voip, ...) one after the other.
109# if set to True, command used is "volt-add-subscriber-unitag"
110# if set to False, comand used is "volt-add-subscriber-access"
111${unitag_sub} False
112
113*** Test Cases ***
114Verify Power Off During MIB Downloading
115 [Documentation] Validates the ONU Go adapter regarding robustness in case of ONU power-off during MIB downloading
116 [Tags] functional PowerOffWhileMibDownload
117 [Setup] Run Keywords Start Logging PowerOffWhileMibDownload
118 ... AND Setup Test
119 FOR ${I} IN RANGE 0 ${num_olts}
120 #get olt serial number
121 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
122 #validate olt states
123 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
124 Enable Device ${olt_device_id}
125 END
126 Current State Test All Onus starting-openomci
127 Power Off ONU Device ${namespace}
128 Current State Test All Onus stopping-openomci
129 Power On ONU Device ${namespace}
130 Run Keyword If "${workflow}"=="DT" Perform Sanity Test DT
131 ... ELSE IF "${workflow}"=="TT" Perform Sanity Tests TT
132 ... ELSE Perform Sanity Test
133 [Teardown] Run Keywords Printout ONU Serial Number and Device Id print2console=${print2console}
134 ... AND Run Keyword If ${logging} Collect Logs
135 ... AND Delete All Devices and Verify
136 ... AND Delete MIB Template Data ${INFRA_NAMESPACE}
137 ... AND Stop Logging PowerOffWhileMibDownload
138
139Verify Power Off During Flow Configuration
140 [Documentation] Validates the ONU Go adapter regarding robustness in case of ONU power-off during flow configuration
141 ... see VOL-4828
142 [Tags] functional PowerOffWhileFlowConfig
143 [Setup] Run Keywords Start Logging PowerOffWhileFlowConfig
144 ... AND Setup Test
145 FOR ${I} IN RANGE 0 ${num_olts}
146 #get olt serial number
147 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
148 #validate olt states
149 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
150 Enable Device ${olt_device_id}
151 END
152 ${state2test}= Set Variable If
153 ... "${workflow}"=="DT" initial-mib-downloaded
154 ... "${workflow}"=="TT" initial-mib-downloaded
155 ... "${workflow}"=="ATT" omci-flows-pushed
156 ... initial-mib-downloaded
157 Current State Test All Onus ${state2test}
158 FOR ${J} IN RANGE 0 ${num_olts}
159 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
160 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
161 ... ${olt_serial_number}
162 Do Onu Subscriber Add Per OLT ${of_id} ${olt_serial_number} ${print2console}
163 END
164 Power Off ONU Device ${namespace}
165 Current State Test All Onus stopping-openomci
166 Power On ONU Device ${namespace}
167 Run Keyword If "${workflow}"=="DT" Perform Sanity Test DT ${suppressaddsubscriber}
168 ... ELSE IF "${workflow}"=="TT" Perform Sanity Tests TT ${suppressaddsubscriber}
169 ... ELSE Perform Sanity Test ${suppressaddsubscriber}
170 [Teardown] Run Keywords Printout ONU Serial Number and Device Id print2console=${print2console}
171 ... AND Run Keyword If ${logging} Collect Logs
172 ... AND Delete All Devices and Verify
173 ... AND Delete MIB Template Data ${INFRA_NAMESPACE}
174 ... AND Stop Logging PowerOffWhileFlowConfig
175
176Verify ONU capabilities against configuration requirements Priority Queues
177 [Documentation] Validates the ONU capabilities against configuration requirements regarding number of priority queues.
178 ... Require more priority queues than ONU capabilities. Check for correct reason/state of ONU and kafka
179 ... message.
180 ... see VOL-4827
181 [Tags] functional ONUCapabilitiesVsConfigReqPrioQueues
182 [Setup] Run Keywords Start Logging ONUCapabilitiesVsConfigReqPrioQueues
TorstenThieme52ddc9d2023-01-04 12:42:19 +0000183 ... AND Set Tech Profile 1T65GEM-error-case-priority-queues
TorstenThiemed9cc06e2022-11-09 12:12:46 +0000184 ... AND Setup Test
185 kafka.Records Clear
186 FOR ${I} IN RANGE 0 ${num_olts}
187 #get olt serial number
188 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
189 #validate olt states
190 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
191 Enable Device ${olt_device_id}
192 END
193 ${state2test}= Set Variable If
194 ... "${workflow}"=="DT" initial-mib-downloaded
195 ... "${workflow}"=="TT" initial-mib-downloaded
196 ... "${workflow}"=="ATT" stopping-openomci
197 ... initial-mib-downloaded
198 Current State Test All Onus ${state2test}
199 FOR ${J} IN RANGE 0 ${num_olts}
200 #in case of ATT no flows have to configured. leave loop
201 Exit For Loop If "${workflow}"=="ATT"
202 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
203 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
204 ... ${olt_serial_number}
205 Do Onu Subscriber Add Per OLT ${of_id} ${olt_serial_number} ${print2console}
206 END
207 ${alternativeonustates}= Create List omci-flows-deleted
208 Current State Test All Onus stopping-openomci alternativeonustate=${alternativeonustates}
209 ${list_onu_device_id} Create List
210 Build ONU Device Id List ${list_onu_device_id}
211 ${event}= Set Variable ONU_CONFIG_FAILURE_MISSING_US_PRIORITY_QUEUE
212 Wait Until Keyword Succeeds ${timeout} 5s Validate Events All ONUs ${list_onu_device_id} ${event}
213 [Teardown] Run Keywords Printout ONU Serial Number and Device Id print2console=${print2console}
214 ... AND Run Keyword If ${logging} Collect Logs
215 ... AND Run Keyword If ${pausebeforecleanup} Import Library Dialogs
216 ... AND Run Keyword If ${pausebeforecleanup} Pause Execution Press OK to continue with clean up!
217 ... AND Set Suite Variable ${TechProfile} ${EMPTY}
218 ... AND Remove Tech Profile ${INFRA_NAMESPACE}
219 ... AND Delete All Devices and Verify
220 ... AND Get ONU Go Adapter ETCD Data ${INFRA_NAMESPACE} ${kvstoreprefix}
221 ... AND Delete MIB Template Data ${INFRA_NAMESPACE}
222 ... AND Stop Logging ONUCapabilitiesVsConfigReqPrioQueues
223
224Verify ONU capabilities against configuration requirements TConts
225 [Documentation] Validates the ONU capabilities against configuration requirements regarding number of tconts.
226 ... Require more tconts than ONU capabilities. Check for correct reason/state of ONU and kafka message
227 ... VOL-4826
228 ... Hint: Run this test case using bbsim-kind-multi-uni-tt.yaml!
229 [Tags] functionalMultiUni ONUCapabilitiesVsConfigReqTConts
230 [Setup] Run Keywords Start Logging ONUCapabilitiesVsConfigReqTConts
231 ... AND Set Tech Profile 1T1GEM-multi-instances ${INFRA_NAMESPACE} 64
232 ... AND Set Tech Profile 1T1GEM-multi-instances ${INFRA_NAMESPACE} 65
233 ... AND Set Tech Profile 1T1GEM-multi-instances ${INFRA_NAMESPACE} 66
234 ... AND Setup Test
235 kafka.Records Clear
236 FOR ${I} IN RANGE 0 ${num_olts}
237 #get olt serial number
238 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
239 #validate olt states
240 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
241 Enable Device ${olt_device_id}
242 END
243 ${state2test}= Set Variable If
244 ... "${workflow}"=="DT" initial-mib-downloaded
245 ... "${workflow}"=="TT" initial-mib-downloaded
246 ... "${workflow}"=="ATT" omci-flows-pushed
247 ... initial-mib-downloaded
248 Current State Test All Onus ${state2test}
249 FOR ${J} IN RANGE 0 ${num_olts}
250 ${olt_serial_number}= Set Variable ${list_olts}[${J}][sn]
251 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
252 ... ${olt_serial_number}
253 Do Onu Subscriber Add Per OLT ${of_id} ${olt_serial_number} ${print2console}
254 END
255 ${alternativeonustates}= Create List omci-flows-deleted
256 Current State Test All Onus stopping-openomci alternativeonustate=${alternativeonustates}
257 ${list_onu_device_id} Create List
258 Build ONU Device Id List ${list_onu_device_id}
259 ${list_onu_device_id} Create List
260 Build ONU Device Id List ${list_onu_device_id}
261 ${event}= Set Variable ONU_CONFIG_FAILURE_MISSING_TCONT
262 Wait Until Keyword Succeeds ${timeout} 5s Validate Events All ONUs ${list_onu_device_id} ${event}
263 [Teardown] Run Keywords Printout ONU Serial Number and Device Id print2console=${print2console}
264 ... AND Run Keyword If ${logging} Collect Logs
265 ... AND Run Keyword If ${pausebeforecleanup} Import Library Dialogs
266 ... AND Run Keyword If ${pausebeforecleanup} Pause Execution Press OK to continue with clean up!
267 ... AND Set Suite Variable ${TechProfile} ${EMPTY}
268 ... AND Remove Tech Profile ${INFRA_NAMESPACE}
269 ... AND Remove Tech Profile ${INFRA_NAMESPACE} 65
270 ... AND Remove Tech Profile ${INFRA_NAMESPACE} 66
271 ... AND Delete All Devices and Verify
272 ... AND Get ONU Go Adapter ETCD Data ${INFRA_NAMESPACE} ${kvstoreprefix}
273 ... AND Delete MIB Template Data ${INFRA_NAMESPACE}
274 ... AND Stop Logging ONUCapabilitiesVsConfigReqTConts
275
276*** Keywords ***
277Setup Suite
278 [Documentation] Set up the test suite
279 Start Logging Setup or Teardown Setup-${SUITE NAME}
280 Common Test Suite Setup
281 ${switch_type}= Get Variable Value ${web_power_switch.type}
282 Run Keyword If "${switch_type}"!="" Set Global Variable ${powerswitch_type} ${switch_type}
283 # set ${kafka} depending on environment in case of port-forward is needed
284 ${rc} ${kafka}= Run Keyword If ${PORT_FORWARDING} Run and Return Rc and Output
285 ... kubectl get svc -n ${INFRA_NAMESPACE} | grep kafka-0-external | awk '{print $1}'
286 Run Keyword If ${PORT_FORWARDING} Should Not Be Empty ${kafka} Service kafka-0-external not found
287 # start port forwarding if needed (when voltha runs in k8s)
288 ${portFwdHandle} = Run Keyword If ${PORT_FORWARDING} Start Process
289 ... kubectl port-forward --address 0.0.0.0 --namespace default svc/${kafka} ${KAFKA_PORT}:${KAFKA_SVC_PORT} &
290 ... shell=true
291 Set Suite Variable ${portFwdHandle}
292 Sleep 5s
293 # open connection to read kafka bus
294 Wait Until Keyword Succeeds 3x 5s
295 ... kafka.Connection Open ${KAFKA_IP} ${KAFKA_PORT} voltha.events timestamp_from=0
296 # delete etcd MIB Template Data
297 Delete MIB Template Data ${INFRA_NAMESPACE}
298 # delete etcd onu data
299 Delete ONU Go Adapter ETCD Data namespace=${INFRA_NAMESPACE} validate=True
300 Stop Logging Setup or Teardown Setup-${SUITE NAME}
301
302Teardown Suite
303 [Documentation] tear down the test suite
304 Run Keyword If ${pausebeforecleanup} Import Library Dialogs
305 Run Keyword If ${pausebeforecleanup} Pause Execution Press OK to continue with clean up!
306 Run Keyword If ${pausebeforecleanup} Log Teardown will be continued... console=yes
307 # close connection to kafka
308 kafka.Connection Close
309 # stop port forwarding if started
310 Run Keyword If ${PORT_FORWARDING} Terminate Process ${portFwdHandle} kill=true
311 # call common suite teardown
312 utils.Teardown Suite
313 # delete etcd MIB Template Data
314 Delete MIB Template Data ${INFRA_NAMESPACE}
315 # delete etcd onu data
316 Delete ONU Go Adapter ETCD Data namespace=${INFRA_NAMESPACE} validate=True
317 Close All ONOS SSH Connections
318
319Setup Test
320 [Documentation] Pre-test Setup
321 #test for empty device list
322 Test Empty Device List
323 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
324 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
325 Run Keyword If ${has_dataplane} Sleep 60s
326 Run Keyword If "${firsttest}"=="False" Sleep 35s
327 ${firsttest} Set Variable False
328 Set Suite Variable ${firsttest}
329 # Create a list of olt ids (logical and device_id)
330 ${olt_ids} Create List
331 FOR ${I} IN RANGE 0 ${num_olts}
332 #create/preprovision device
333 ${olt_device_id}= Run Keyword If "${list_olts}[${I}][type]" == "${None}"
334 ... Create Device ${list_olts}[${I}][ip] ${list_olts}[${I}][oltport]
335 ... ELSE Create Device ${list_olts}[${I}][ip] ${list_olts}[${I}][oltport] ${list_olts}[${I}][type]
336 ${olt_serial_number}= Set Variable ${list_olts}[${I}][sn]
337 #validate olt states
338 Wait Until Keyword Succeeds ${timeout} 5s
339 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id} by_dev_id=True
340 Sleep 5s
341 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
342 ${olt} Create Dictionary device_id ${olt_device_id} logical_id ${logical_id}
343 ... of_id ${of_id} sn ${olt_serial_number}
344 Append To List ${olt_ids} ${olt}
345 END
346 Set Global Variable ${olt_ids}