blob: 9f6660b6923100d50be2ea2254d524a3b4e3e82c [file] [log] [blame]
Suchitra Vemurife309412020-12-01 19:19:52 -08001# Copyright 2020 - present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
14
15*** Settings ***
16Documentation Test various failure scenarios
17Suite Setup Setup Suite
18Test Setup Setup
19Test Teardown Teardown
20#Suite Teardown Teardown Suite
21Library Collections
22Library String
23Library OperatingSystem
24Library XML
25Library RequestsLibrary
26Library ../../libraries/DependencyLibrary.py
27Resource ../../libraries/onos.robot
28Resource ../../libraries/voltctl.robot
29Resource ../../libraries/voltha.robot
30Resource ../../libraries/utils.robot
31Resource ../../libraries/k8s.robot
32Resource ../../variables/variables.robot
33Resource ../../libraries/power_switch.robot
34
35*** Variables ***
36${POD_NAME} flex-ocp-cord
37${KUBERNETES_CONF} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
38${KUBERNETES_CONFIGS_DIR} ~/pod-configs/kubernetes-configs
39#${KUBERNETES_CONFIGS_DIR} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.conf
40${KUBERNETES_YAML} ${KUBERNETES_CONFIGS_DIR}/${POD_NAME}.yml
41${HELM_CHARTS_DIR} ~/helm-charts
42${VOLTHA_POD_NUM} 8
43${NAMESPACE} voltha
44${DEFAULTSPACE} default
45# For below variable value, using deployment name as using grep for
46# parsing radius pod name, we can also use full radius pod name
47${RESTART_POD_NAME} radius
48${timeout} 60s
49${of_id} 0
50${logical_id} 0
51${has_dataplane} True
52${teardown_device} False
53${scripts} ../../scripts
54
55# Per-test logging on failure is turned off by default; set this variable to enable
56${container_log_dir} ${None}
57
58*** Test Cases ***
59Verify OLT after rebooting physically - MultipleOLT
60 [Documentation] Test the physical reboot of the OLT
61 ... Prerequisite : Subscriber are authenticated/DHCP/pingable state
62 ... Test performs a physical reboot on one of the OLTs, performs "reboot" from the OLT CLI
63 ... Validates that ONU on other OLTs are still functional
Hardik Windlass5ceb84e2021-02-11 11:17:08 +000064 [Tags] functional MultiOLTPhysicalReboot
Suchitra Vemurife309412020-12-01 19:19:52 -080065 [Setup] Start Logging MultipleOlt-PhysicalOLTReboot
66 [Teardown] Run Keywords Collect Logs
67 ... AND Stop Logging MultipleOlt-PhysicalOLTReboot
Suchitra Vemuri899e91f2020-12-15 13:31:55 -080068 Run Keyword and Ignore Error Delete All Devices and Verify
Suchitra Vemurife309412020-12-01 19:19:52 -080069 # Add OLT device
70 setup
71 # Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
72 Run Keyword If ${has_dataplane} Clean Up Linux
73 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
74 # Execute the test when the number of OLTs are greater than one
75 Pass Execution If ${olt_count} == 1 Skipping test: just one OLT
76 # Reboot the first OLT from the list of olts - rebooting from the OLT CLI
77 ${olt_user}= Get From Dictionary ${list_olts}[0] user
78 ${olt_pass}= Get From Dictionary ${list_olts}[0] pass
79 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[0] sship
80 ${olt_serial_number}= Get From Dictionary ${list_olts}[0] sn
81 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
82 Run Keyword If ${has_dataplane} Login And Run Command On Remote System
Andrea Campanella5ef88142021-02-02 14:34:50 +010083 ... reboot ${olt_ssh_ip} ${olt_user} ${olt_pass} prompt=#
Suchitra Vemurife309412020-12-01 19:19:52 -080084 # validate that the ONUs on the other OLTs are still functional
85 Verify ping is successful for ONUs not on this OLT ${num_all_onus} ${olt_device_id}
86 Run Keyword And Ignore Error Collect Logs
87 # Validate that the ONUs on the rebooted OLT are not pingable
88 FOR ${I} IN RANGE 0 ${num_all_onus}
89 ${src}= Set Variable ${hosts.src[${I}]}
90 ${dst}= Set Variable ${hosts.dst[${I}]}
91 ${olt_serial_number}= Get From Dictionary ${list_olts}[0] sn
92 Continue For Loop If "${olt_serial_number}"!="${src['olt']}"
93 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
94 ... Wait Until Keyword Succeeds 60s 2s
95 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
96 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
97 END
98 # Wait for the rebooted OLT to come back up
99 ${olt_serial_number}= Get From Dictionary ${olt_ids}[0] sn
100 ${olt_user}= Get From Dictionary ${list_olts}[0] user
101 ${olt_pass}= Get From Dictionary ${list_olts}[0] pass
102 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[0] sship
103 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
104 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
105 ... Check Remote System Reachability True ${olt_ssh_ip}
106 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
107 ... Validate OLT Device ENABLED ACTIVE
108 ... REACHABLE ${olt_serial_number}
109 # Waiting extra time for the ONUs to come up
110 Sleep 60s
111 Run Keyword And Ignore Error Collect Logs
112 Run Keyword If ${has_dataplane} Clean Up Linux
113 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
114 # Deleting OLT after test completes
115 #Run Keyword If ${has_dataplane} Delete All Devices and Verify
Suchitra Vemurife309412020-12-01 19:19:52 -0800116
117Verify OLT Soft Reboot - MultipleOLT
118 [Documentation] Test soft reboot of the OLT using voltctl command
119 ... Verifies that when one OLT is rebooted other ONUs on other OLTs are
120 ... still functional
Hardik Windlass5ceb84e2021-02-11 11:17:08 +0000121 [Tags] MultiOLTSoftReboot functional
Suchitra Vemurife309412020-12-01 19:19:52 -0800122 [Setup] Start Logging MultiOlt-OLTSoftReboot
123 [Teardown] Run Keywords Collect Logs
124 ... AND Stop Logging MultiOlt-OLTSoftReboot
Suchitra Vemuri6971c392020-12-08 10:41:33 -0800125 ... AND Delete All Devices and Verify
Suchitra Vemurife309412020-12-01 19:19:52 -0800126 #Delete All Devices and Verify
127 #Setup
128 ## Performing Sanity Test to make sure subscribers are all AUTH+DHCP and pingable
129 #Run Keyword If ${has_dataplane} Clean Up Linux
130 #Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
131 # Execute the test when the number of OLTs are greater than one
132 Pass Execution If ${olt_count} == 1 Skipping test: just one OLT
133 # Reboot the first OLT
134 ${olt_user}= Get From Dictionary ${list_olts}[0] user
135 ${olt_pass}= Get From Dictionary ${list_olts}[0] pass
136 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[0] sship
137 ${olt_serial_number}= Get From Dictionary ${list_olts}[0] sn
138 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
139 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
140 ... Validate OLT Device ENABLED ACTIVE
141 ... REACHABLE ${olt_serial_number}
142 # Reboot the OLT using "voltctl device reboot" command
143 Reboot Device ${olt_device_id}
144 Run Keyword And Ignore Error Collect Logs
145 # validate that the ONUs on the other OLTs are still functional
146 Verify ping is successful for ONUs not on this OLT ${num_all_onus} ${olt_device_id}
147 #Verify that ping fails for the ONUs where the OLT has been rebooted
148 FOR ${I} IN RANGE 0 ${num_all_onus}
149 ${src}= Set Variable ${hosts.src[${I}]}
150 ${dst}= Set Variable ${hosts.dst[${I}]}
151 ${olt_serial_number}= Get From Dictionary ${list_olts}[0] sn
152 Continue For Loop If "${olt_serial_number}"!="${src['olt']}"
153 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
154 ... Wait Until Keyword Succeeds 60s 2s
155 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
156 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
157 END
158 # Check OLT state of the rebooted OLT
159 ${olt_serial_number}= Get From Dictionary ${list_olts}[0] sn
160 ${olt_ssh_ip}= Get From Dictionary ${list_olts}[0] sship
161 ${olt_device_id}= Get OLTDeviceID From OLT List ${olt_serial_number}
162 # Wait for the OLT to come back up
163 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s
164 ... Check Remote System Reachability True ${olt_ssh_ip}
165 # Check OLT states
166 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 360s 5s
167 ... Validate OLT Device ENABLED ACTIVE
168 ... REACHABLE ${olt_serial_number}
169 # Waiting extra time for the ONUs to come up
170 Sleep 60s
171 Run Keyword And Ignore Error Collect Logs
172 #Check after reboot that ONUs are active, authenticated/DHCP/pingable
173 Run Keyword If ${has_dataplane} Clean Up Linux
174 Wait Until Keyword Succeeds ${timeout} 2s Perform Sanity Test
175
176
177*** Keywords ***
178Setup Suite
179 [Documentation] Set up the test suite
180 Common Test Suite Setup
181 #power_switch.robot needs it to support different vendor's power switch
182 ${switch_type}= Get Variable Value ${web_power_switch.type}
183 Run Keyword If "${switch_type}"!="" Set Global Variable ${powerswitch_type} ${switch_type}
184
185Clear All Devices Then Create New Device
186 [Documentation] Remove any devices from VOLTHA and ONOS
187 # Remove all devices from voltha and nos
188 Delete All Devices and Verify
189 # Execute normal test Setup Keyword
190 Setup