blob: f4af894dd8bd8a6f159cdb4539eadd097b811892 [file] [log] [blame]
Joey Armstrong9fadcbe2024-01-17 19:00:37 -05001# Copyright 2021-2024 Open Networking Foundation (ONF) and the ONF Contributors
Hardik Windlassdd1a9a12021-02-23 15:34:50 +00002#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14# FIXME Can we use the same test against BBSim and Hardware?
15
16*** Settings ***
17Documentation Test Voltha Components Software Upgrade
18Suite Setup Setup Suite
19Test Setup Setup
20Test Teardown Teardown
21Suite Teardown Teardown Suite
22Library Collections
23Library String
24Library OperatingSystem
25Library XML
26Library RequestsLibrary
27Library ../../libraries/DependencyLibrary.py
28Resource ../../libraries/onos.robot
29Resource ../../libraries/voltctl.robot
30Resource ../../libraries/voltha.robot
31Resource ../../libraries/utils.robot
32Resource ../../libraries/k8s.robot
33Resource ../../variables/variables.robot
34Resource ../../libraries/power_switch.robot
35
36*** Variables ***
37${POD_NAME} flex-ocp-cord
38${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
39${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
40#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
41${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
42${HELM_CHARTS_DIR} ~/helm-charts
43${VOLTHA_POD_NUM} 8
44${NAMESPACE} voltha
Hardik Windlass4288c6a2021-09-28 07:22:06 +000045${INFRA_NAMESPACE} default
Hardik Windlassdd1a9a12021-02-23 15:34:50 +000046# For below variable value, using deployment name as using grep for
47# parsing radius pod name, we can also use full radius pod name
48${RESTART_POD_NAME} radius
49${timeout} 60s
50${of_id} 0
51${logical_id} 0
52${has_dataplane} True
53${teardown_device} False
54${scripts} ../../scripts
55
56# Per-test logging on failure is turned off by default; set this variable to enable
57${container_log_dir} ${None}
58
Matteo Scandoloda1c64b2021-07-20 10:14:49 -070059# logging flag to enable Collect Logs, can be passed via the command line too
60# example: -v logging:False
61${logging} True
62
Hardik Windlassdd1a9a12021-02-23 15:34:50 +000063${suppressaddsubscriber} True
64
65# Voltha Components to Test for Software Upgrade need to be passed in the following variable in format:
Hardik Windlassdc2610f2021-03-09 07:33:51 +000066# <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
67# Example: adapter-open-olt,adapter-open-olt,voltha/voltha-openolt-adapter:3.1.3*
Hardik Windlassdd1a9a12021-02-23 15:34:50 +000068${voltha_comps_under_test} ${EMPTY}
69
70*** Test Cases ***
71Test Voltha Components Minor Version Upgrade
72 [Documentation] Validates the Voltha Components Minor Version Upgrade doesn't affect the system functionality
73 ... Performs the sanity and verifies all the ONUs are authenticated/DHCP/pingable
74 ... Requirement: Components to test needs to be passed in robot command variable 'voltha_comps_under_test' in the format:
Hardik Windlassdc2610f2021-03-09 07:33:51 +000075 ... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
Hardik Windlassdd1a9a12021-02-23 15:34:50 +000076 ... Check [VOL-3843] for more details
77 [Tags] functional VolthaCompMinorVerUpgrade
78 [Setup] Start Logging VolthaCompMinorVerUpgrade
79 [Teardown] Run Keywords Collect Logs
80 ... AND Stop Logging VolthaCompMinorVerUpgrade
81 ... AND Delete All Devices and Verify
82 # Add OLT device
83 Setup
84 # Performing Sanity Test to make sure subscribers are all DHCP and pingable
85 Run Keyword If ${has_dataplane} Clean Up Linux
86 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
87 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
88 Log ${podStatusOutput}
89 ${countBeforeUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
90 ${num_comps_under_test}= Get Length ${list_voltha_comps_under_test}
91 FOR ${I} IN RANGE 0 ${num_comps_under_test}
92 ${label}= Set Variable ${list_voltha_comps_under_test}[${I}][label]
93 ${container}= Set Variable ${list_voltha_comps_under_test}[${I}][container]
94 ${image}= Set Variable ${list_voltha_comps_under_test}[${I}][image]
Hardik Windlassdc2610f2021-03-09 07:33:51 +000095 ${pod_image} ${app_ver} ${helm_chart} Get Pod Image And App Version And Helm Chart By Label
96 ... ${NAMESPACE} app ${label}
97 Log ${label}: image, app ver & helm chart before upgrade: ${pod_image}, ${app_ver} & ${helm_chart}
Hardik Windlassdd1a9a12021-02-23 15:34:50 +000098 ${deployment}= Wait Until Keyword Succeeds ${timeout} 15s
99 ... Get K8s Deployment by Pod Label ${NAMESPACE} app ${label}
100 Wait Until Keyword Succeeds ${timeout} 15s Deploy Pod New Image ${NAMESPACE} ${deployment}
101 ... ${container} ${image}
102 Wait Until Keyword Succeeds ${timeout} 3s Validate Pods Status By Label ${NAMESPACE}
103 ... app ${label} Running
104 Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${label}
105 Wait Until Keyword Succeeds ${timeout} 3s Verify Pod Image ${NAMESPACE} app ${label} ${image}
Hardik Windlassdc2610f2021-03-09 07:33:51 +0000106 ${pod_image_1} ${app_ver_1} ${helm_chart_1} Get Pod Image And App Version And Helm Chart By Label
107 ... ${NAMESPACE} app ${label}
108 Log ${label}: image, app ver & helm chart after upgrade: ${pod_image_1}, ${app_ver_1} & ${helm_chart_1}
Matteo Scandolo2769d2b2021-04-14 10:29:24 -0700109 Restart VOLTHA Port Forward voltha-api
Hardik Windlass8a16b1b2022-06-27 13:19:37 +0530110 # Static sleep to let voltctl tcp connection establish
111 Sleep 5s
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000112 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test ${suppressaddsubscriber}
113 END
114 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
115 Log ${podStatusOutput}
116 ${countAfterUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
117 Should Be Equal As Strings ${countAfterUpgrade} ${countBeforeUpgrade}
118 # Additional Verification
119 Wait Until Keyword Succeeds ${timeout} 2s Delete All Devices and Verify
120 Setup
121 Run Keyword If ${has_dataplane} Clean Up Linux
122 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
123
Hardik Windlass07b52452022-01-17 07:35:53 +0000124
125Test Voltha Components Minor Version Rolling Upgrade
126 [Documentation] Validates that the system can handle operations during software upgrade for minor versions.
127 ... Requirement: Components to test needs to be passed in robot command variable 'voltha_comps_under_test' in the format:
128 ... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
129 ... Check [VOL-4534] for more details
Hardik Windlass76f68212022-01-19 08:16:14 +0000130 [Tags] functional VolthaCompMinorVerRollingUpgrade
Hardik Windlass07b52452022-01-17 07:35:53 +0000131 [Setup] Start Logging VolthaCompMinorVerRollingUpgrade
132 [Teardown] Run Keywords Collect Logs
133 ... AND Stop Logging VolthaCompMinorVerRollingUpgrade
134 ... AND Delete All Devices and Verify
135 # Add OLT device
136 Setup
137 Run Keyword If ${has_dataplane} Clean Up Linux
138 FOR ${I} IN RANGE 0 ${olt_count}
139 ${olt_serial_number}= Get From Dictionary ${olt_ids}[${I}] sn
140 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
141 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS
142 ... ${olt_serial_number}
143 Wait Until Keyword Succeeds ${timeout} 2s Device Is Available In ONOS
144 ... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT} ${of_id}
145 END
146 FOR ${I} IN RANGE 0 ${num_all_onus}
147 ${src}= Set Variable ${hosts.src[${I}]}
148 ${dst}= Set Variable ${hosts.dst[${I}]}
149 ${of_id}= Get ofID From OLT List ${src['olt']}
150 ${onu_device_id}= Get Device ID From SN ${src['onu']}
151 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
152 ... ${of_id} ${src['uni_id']}
153 # Check ONU port is Enabled in ONOS
154 Wait Until Keyword Succeeds 120s 2s
155 ... Verify UNI Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']} ${src['uni_id']}
156 Wait Until Keyword Succeeds ${timeout} 5s
157 ... Verify Default Meter Present in ONOS ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
158 Wait Until Keyword Succeeds ${timeout} 2s Verify Eapol Flows Added For ONU ${ONOS_SSH_IP}
159 ... ${ONOS_SSH_PORT} ${of_id} ${onu_port}
160 # Bring up the device and verify it authenticates
161 Wait Until Keyword Succeeds ${timeout} 5s Validate Device ENABLED ACTIVE REACHABLE
162 ... ${onu_device_id} onu=True onu_reason=omci-flows-pushed by_dev_id=True
163 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
164 ... /tmp/wpa ${src['dp_iface_name']} log
165 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
166 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
167 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
168 Wait Until Keyword Succeeds ${timeout} 2s Verify ONU in AAA-Users ${ONOS_SSH_IP}
169 ... ${ONOS_SSH_PORT} ${onu_port}
170 END
171 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
172 Log ${podStatusOutput}
173 ${countBeforeUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
174 ${num_comps_under_test}= Get Length ${list_voltha_comps_under_test}
175 FOR ${I} IN RANGE 0 ${num_comps_under_test}
176 ${label}= Set Variable ${list_voltha_comps_under_test}[${I}][label]
177 ${container}= Set Variable ${list_voltha_comps_under_test}[${I}][container]
178 ${image}= Set Variable ${list_voltha_comps_under_test}[${I}][image]
179 ${pod_image} ${app_ver} ${helm_chart} Get Pod Image And App Version And Helm Chart By Label
180 ... ${NAMESPACE} app ${label}
181 Log ${label}: image, app ver & helm chart before upgrade: ${pod_image}, ${app_ver} & ${helm_chart}
182 ${deployment}= Wait Until Keyword Succeeds ${timeout} 15s
183 ... Get K8s Deployment by Pod Label ${NAMESPACE} app ${label}
184 Wait Until Keyword Succeeds ${timeout} 15s Deploy Pod New Image ${NAMESPACE} ${deployment}
185 ... ${container} ${image}
186 # Static sleep to let image-update progress before initiating subscriber-add system-operation
Hardik Windlass8a16b1b2022-06-27 13:19:37 +0530187 Sleep 10s
Hardik Windlass07b52452022-01-17 07:35:53 +0000188 Provision Subscribers
189 Wait Until Keyword Succeeds ${timeout} 3s Validate Pods Status By Label ${NAMESPACE}
190 ... app ${label} Running
191 Wait Until Keyword Succeeds ${timeout} 3s Pods Are Ready By Label ${NAMESPACE} app ${label}
192 Wait Until Keyword Succeeds ${timeout} 3s Verify Pod Image ${NAMESPACE} app ${label} ${image}
193 ${pod_image_1} ${app_ver_1} ${helm_chart_1} Get Pod Image And App Version And Helm Chart By Label
194 ... ${NAMESPACE} app ${label}
195 Log ${label}: image, app ver & helm chart after upgrade: ${pod_image_1}, ${app_ver_1} & ${helm_chart_1}
196 Restart VOLTHA Port Forward voltha-api
Hardik Windlass515e9ee2022-02-24 11:52:31 +0000197 # Static sleep to let voltctl tcp connection establish
198 Sleep 5s
Hardik Windlass07b52452022-01-17 07:35:53 +0000199 Verify Provisioned Subscribers
200 Unprovision Subscribers
201 END
202 ${podStatusOutput}= Run kubectl get pods -n ${NAMESPACE}
203 Log ${podStatusOutput}
204 ${countAfterUpgrade}= Run kubectl get pods -n ${NAMESPACE} | grep Running | wc -l
205 Should Be Equal As Strings ${countAfterUpgrade} ${countBeforeUpgrade}
206 # Additional Verification
207 Wait Until Keyword Succeeds ${timeout} 2s Delete All Devices and Verify
208 Setup
209 Run Keyword If ${has_dataplane} Clean Up Linux
210 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
211
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000212*** Keywords ***
213Setup Suite
214 [Documentation] Set up the test suite
215 Common Test Suite Setup
216 Create Voltha Comp Under Test List
217
218Teardown Suite
219 [Documentation] Tear down steps for the suite
220 Run Keyword If ${has_dataplane} Clean Up Linux
TorstenThieme4e2168e2021-06-22 14:01:47 +0000221 Close All ONOS SSH Connections
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000222
223Create Voltha Comp Under Test List
224 [Documentation] Creates a list of Voltha Components to Test from the input variable string
225 ... The input string is expected to be in format:
Hardik Windlassdc2610f2021-03-09 07:33:51 +0000226 ... <comp-label>,<comp-container>,<comp-image>*<comp-label>,<comp-container>,<comp-image>*
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000227 ${list_voltha_comps_under_test} Create List
228 @{comps_under_test_arr}= Split String ${voltha_comps_under_test} *
229 ${num_comps_under_test}= Get Length ${comps_under_test_arr}
Hardik Windlassdc2610f2021-03-09 07:33:51 +0000230 FOR ${I} IN RANGE 0 ${num_comps_under_test}-1
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000231 @{comp_under_test_arr}= Split String ${comps_under_test_arr[${I}]} ,
232 ${label}= Set Variable ${comp_under_test_arr[0]}
233 ${container}= Set Variable ${comp_under_test_arr[1]}
234 ${image}= Set Variable ${comp_under_test_arr[2]}
235 ${comp_under_test} Create Dictionary label ${label} container ${container} image ${image}
236 Append To List ${list_voltha_comps_under_test} ${comp_under_test}
237 END
Hardik Windlassdc2610f2021-03-09 07:33:51 +0000238 Log ${list_voltha_comps_under_test}
Hardik Windlassdd1a9a12021-02-23 15:34:50 +0000239 Set Suite Variable ${list_voltha_comps_under_test}
Hardik Windlass07b52452022-01-17 07:35:53 +0000240
241Provision Subscribers
242 [Documentation] This keyword provisions/adds all the subscribers on all the devices
243 FOR ${I} IN RANGE 0 ${num_all_onus}
244 ${src}= Set Variable ${hosts.src[${I}]}
245 ${dst}= Set Variable ${hosts.dst[${I}]}
246 ${of_id}= Get ofID From OLT List ${src['olt']}
247 ${onu_device_id}= Get Device ID From SN ${src['onu']}
248 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
249 ... ${of_id} ${src['uni_id']}
250 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
251 ... ${ONOS_SSH_PORT} volt-add-subscriber-access ${of_id} ${onu_port}
252 END
253
254Verify Provisioned Subscribers
255 [Documentation] This keyword verifies all the subscribers on all the devices
256 FOR ${I} IN RANGE 0 ${num_all_onus}
257 ${src}= Set Variable ${hosts.src[${I}]}
258 ${dst}= Set Variable ${hosts.dst[${I}]}
259 ${of_id}= Get ofID From OLT List ${src['olt']}
260 ${nni_port}= Wait Until Keyword Succeeds ${timeout} 2s Get NNI Port in ONOS ${of_id}
261 ${onu_device_id}= Get Device ID From SN ${src['onu']}
262 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
263 ... ${of_id} ${src['uni_id']}
264 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
265 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
266 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
267 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
268 # Verify subscriber access flows are added for the ONU port
269 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
270 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
271 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
272 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
273 ... Wait Until Keyword Succeeds ${timeout} 2s
274 ... Check Ping True ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
275 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
276 END
277
278Unprovision Subscribers
279 [Documentation] This keyword unprovisions/deletes all the subscribers on all the devices
280 FOR ${I} IN RANGE 0 ${num_all_onus}
281 ${src}= Set Variable ${hosts.src[${I}]}
282 ${dst}= Set Variable ${hosts.dst[${I}]}
283 ${of_id}= Get ofID From OLT List ${src['olt']}
284 ${nni_port}= Wait Until Keyword Succeeds ${timeout} 2s Get NNI Port in ONOS ${of_id}
285 ${onu_device_id}= Get Device ID From SN ${src['onu']}
286 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
287 ... ${of_id} ${src['uni_id']}
288 Wait Until Keyword Succeeds ${timeout} 2s Execute ONOS CLI Command use single connection ${ONOS_SSH_IP}
289 ... ${ONOS_SSH_PORT} volt-remove-subscriber-access ${of_id} ${onu_port}
290 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
291 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
292 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
293 ... Wait Until Keyword Succeeds ${timeout} 2s
294 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
295 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
296 ... ELSE Sleep 15s
297 END