blob: 677616dbcb65e72c96fe0e610d83b2f816d95669 [file] [log] [blame]
Kailash92764922019-07-25 08:21:39 -07001# Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Kailash92764922019-07-25 08:21:39 -070014# robot test functions
15
16*** Settings ***
17Documentation Library for various utilities
18Library SSHLibrary
Kailash92764922019-07-25 08:21:39 -070019Library String
20Library DateTime
21Library Process
22Library Collections
23Library RequestsLibrary
24Library OperatingSystem
Andy Bavier63460da2020-02-20 14:35:12 -070025Library CORDRobot
26Library ImportResource resources=CORDRobot
Kailash92764922019-07-25 08:21:39 -070027
28*** Keywords ***
Kailash528f7c02019-07-31 10:55:49 -070029Check CLI Tools Configured
30 [Documentation] Tests that use 'voltctl' and 'kubectl' should execute this keyword in suite setup
31 # check voltctl and kubectl configured
Zack Williamsa8fe75a2020-01-10 14:25:27 -070032 ${voltctl_rc}= Run And Return RC voltctl device list
33 ${kubectl_rc}= Run And Return RC kubectl get pods
Gilles Depatie675a2062019-10-22 12:44:42 -040034 Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR
35 ... VOLTCTL and KUBECTL not configured. Please configure before executing tests.
Gilles Depatieb5682f82019-10-31 10:39:45 -040036
37Send File To Onos
Zack Williamsa8fe75a2020-01-10 14:25:27 -070038 [Documentation] Send the content of the file to Onos to selected section of configuration
39 ... using Post Request
40 [Arguments] ${CONFIG_FILE} ${section}
41 ${Headers}= Create Dictionary Content-Type application/json
42 ${File_Data}= OperatingSystem.Get File ${CONFIG_FILE}
43 Log ${Headers}
44 Log ${File_Data}
45 ${resp}= Post Request ONOS
46 ... /onos/v1/network/configuration/${section} headers=${Headers} data=${File_Data}
47 Should Be Equal As Strings ${resp.status_code} 200
Andy Bavier88cd9f62019-11-26 16:22:33 -070048
49Common Test Suite Setup
50 [Documentation] Setup the test suite
Andy Bavier88cd9f62019-11-26 16:22:33 -070051 Set Global Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
52 Set Global Variable ${VOLTCTL_CONFIG} export VOLTCONFIG=%{VOLTCONFIG}
53 ${k8s_node_ip}= Evaluate ${nodes}[0].get("ip")
ubuntu6b6e7d42020-03-02 12:35:42 -080054 ${ONOS_REST_IP}= Get Environment Variable ONOS_REST_IP ${k8s_node_ip}
55 ${ONOS_SSH_IP}= Get Environment Variable ONOS_SSH_IP ${k8s_node_ip}
56 Set Global Variable ${ONOS_REST_IP}
57 Set Global Variable ${ONOS_SSH_IP}
Andy Bavier88cd9f62019-11-26 16:22:33 -070058 ${k8s_node_user}= Evaluate ${nodes}[0].get("user")
59 ${k8s_node_pass}= Evaluate ${nodes}[0].get("pass")
60 Check CLI Tools Configured
61 ${onos_auth}= Create List karaf karaf
62 ${HEADERS} Create Dictionary Content-Type=application/json
ubuntu6b6e7d42020-03-02 12:35:42 -080063 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
Andy Bavier88cd9f62019-11-26 16:22:33 -070064 ${olt_ip}= Evaluate ${olts}[0].get("ip")
65 ${olt_user}= Evaluate ${olts}[0].get("user")
66 ${olt_pass}= Evaluate ${olts}[0].get("pass")
67 ${olt_serial_number}= Evaluate ${olts}[0].get("serial")
68 ${num_onus}= Get Length ${hosts.src}
69 ${num_onus}= Convert to String ${num_onus}
70 #send sadis file to onos
Andy Baviera2691752019-11-27 12:18:38 -070071 ${sadis_file}= Get Variable Value ${sadis.file}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070072 Log To Console \nSadis File:${sadis_file}
Andy Bavier88cd9f62019-11-26 16:22:33 -070073 Run Keyword Unless '${sadis_file}' is '${None}' Send File To Onos ${sadis_file} apps/
74 Set Suite Variable ${num_onus}
75 Set Suite Variable ${olt_serial_number}
76 Set Suite Variable ${olt_ip}
77 Set Suite Variable ${olt_user}
78 Set Suite Variable ${olt_pass}
Andy Bavier88cd9f62019-11-26 16:22:33 -070079 @{container_list}= Create List adapter-open-olt adapter-open-onu voltha-api-server
80 ... voltha-ro-core voltha-rw-core-11 voltha-rw-core-12 voltha-ofagent
81 Set Suite Variable ${container_list}
82 ${datetime}= Get Current Date
suraj gour7f6d5fe2019-11-29 10:56:35 +000083 Set Suite Variable ${datetime}
84
85WPA Reassociate
suraj gour7f6d5fe2019-11-29 10:56:35 +000086 [Documentation] Executes a particular wpa_cli reassociate, which performs force reassociation
Zack Williamsa8fe75a2020-01-10 14:25:27 -070087 [Arguments] ${iface} ${ip} ${user} ${pass}=${None}
88 ... ${container_type}=${None} ${container_name}=${None}
suraj gour7f6d5fe2019-11-29 10:56:35 +000089 #Below for loops are used instead of sleep time, to execute reassociate command and check status
Zack Williamsa8fe75a2020-01-10 14:25:27 -070090 FOR ${i} IN RANGE 70
91 ${output}= Login And Run Command On Remote System
92 ... wpa_cli -i ${iface} reassociate ${ip} ${user}
93 ... ${pass} ${container_type} ${container_name}
94 ${passed}= Run Keyword And Return Status Should Contain ${output} OK
Andy Bavierb63f6d22020-03-12 15:34:37 -070095 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070096 END
Andy Bavierb63f6d22020-03-12 15:34:37 -070097 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -070098 FOR ${i} IN RANGE 70
99 ${output}= Login And Run Command On Remote System
Andy Bavierae4a17e2020-02-25 14:54:36 -0700100 ... wpa_cli -i ${iface} status | grep SUCCESS ${ip} ${user}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700101 ... ${pass} ${container_type} ${container_name}
102 ${passed}= Run Keyword And Return Status Should Contain ${output} SUCCESS
Andy Bavierb63f6d22020-03-12 15:34:37 -0700103 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700104 END
Andy Bavierb63f6d22020-03-12 15:34:37 -0700105 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700106
suraj gour7f6d5fe2019-11-29 10:56:35 +0000107Validate Authentication After Reassociate
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700108 [Arguments] ${auth_pass} ${iface} ${ip} ${user} ${pass}=${None}
109 ... ${container_type}=${None} ${container_name}=${None}
110 [Documentation]
111 ... Executes a particular reassociate request on the RG using wpa_cli.
112 ... auth_pass determines if authentication should pass
Andy Bavier84834d42020-02-25 13:49:50 -0700113 ${wpa_log}= Catenate SEPARATOR=. /tmp/wpa ${iface} log
114 ${output}= Login And Run Command On Remote System truncate -s 0 ${wpa_log}; cat ${wpa_log}
115 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
116 Log ${output}
117 Should Not Contain ${output} authentication completed successfully
suraj gour7f6d5fe2019-11-29 10:56:35 +0000118 WPA Reassociate ${iface} ${ip} ${user} ${pass} ${container_type} ${container_name}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700119 Run Keyword If '${auth_pass}' == 'True' Wait Until Keyword Succeeds ${timeout} 2s
Andy Bavier84834d42020-02-25 13:49:50 -0700120 ... Check Remote File Contents True ${wpa_log} ${iface}.*authentication completed successfully
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700121 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
suraj gour7f6d5fe2019-11-29 10:56:35 +0000122 Run Keyword If '${auth_pass}' == 'False' Sleep 20s
Suchitra Vemuri3cb29f82020-02-25 13:59:58 -0800123 Run Keyword If '${auth_pass}' == 'False' Check Remote File Contents False /tmp/wpa.log
Andy Bavier84834d42020-02-25 13:49:50 -0700124 ... ${iface}.*authentication completed successfully ${ip} ${user} ${pass}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700125 ... ${container_type} ${container_name}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000126
127Send Dhclient Request To Release Assigned IP
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700128 [Arguments] ${iface} ${ip} ${user} ${path_dhcpleasefile} ${pass}=${None}
129 ... ${container_type}=${None} ${container_name}=${None}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000130 [Documentation] Executes a dhclient with option to release ip against a particular interface on the RG (src)
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700131 ${result}= Login And Run Command On Remote System
132 ... dhclient -nw -r ${iface} && rm ${path_dhcpleasefile}/dhclient.* ${ip} ${user}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000133 ... ${pass} ${container_type} ${container_name}
134 Log ${result}
135 #Should Contain ${result} DHCPRELEASE
136 [Return] ${result}
suraj gour1ecfae92019-12-20 15:11:40 +0000137
138Check Remote File Contents For WPA Logs
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700139 [Arguments] ${file_should_exist} ${file} ${pattern} ${ip} ${user} ${pass}=${None}
140 ... ${container_type}=${None} ${container_name}=${None} ${prompt}=~$
suraj gour1ecfae92019-12-20 15:11:40 +0000141 [Documentation] Checks for particular pattern count in a file
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700142 ${result}= Login And Run Command On Remote System
143 ... cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
suraj gour1ecfae92019-12-20 15:11:40 +0000144 ... ${container_type} ${container_name} ${prompt}
145 [Return] ${result}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000146
147Perform Sanity Test
148 [Documentation] This keyword performs Sanity Test Procedure
149 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
150 ... This keyword can be used to call in any other tests where sanity check is required
151 ... and avoids duplication of code.
152 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
153 Set Global Variable ${of_id}
Hardik Windlass21807632020-04-14 16:24:55 +0530154 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
155 ... Get NNI Port in ONOS ${of_id}
156 Set Global Variable ${nni_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000157 FOR ${I} IN RANGE 0 ${num_onus}
158 ${src}= Set Variable ${hosts.src[${I}]}
159 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800160 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000161 ${onu_device_id}= Get Device ID From SN ${src['onu']}
162 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
163 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800164 # Check ONU port is Enabled in ONOS
165 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800166 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800167 # Verify EAPOL flows are added for the ONU port
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000168 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800169 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800170 # Verify ONU state in voltha
171 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
172 ... ENABLED ACTIVE REACHABLE
173 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
174 # Perform Authentication
Andy Bavier84834d42020-02-25 13:49:50 -0700175 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
176 ... /tmp/wpa ${src['dp_iface_name']} log
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000177 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
178 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700179 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000180 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800181 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000182 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800183 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000184 ... volt-add-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri9da44302020-03-04 14:24:49 -0800185 # Verify that no pending flows exist for the ONU port
186 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
187 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530188 # Verify subscriber access flows are added for the ONU port
189 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
190 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
191 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000192 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
193 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
194 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
195 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
196 ... ${dst['container_name']}
197 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800198 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000199 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
200 Run Keyword and Ignore Error Collect Logs
201 END
202
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000203Perform Sanity Test DT
204 [Documentation] This keyword performs Sanity Test Procedure for DT Workflow
205 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
206 ... This keyword can be used to call in any other tests where sanity check is required
207 ... and avoids duplication of code.
208 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
209 Set Global Variable ${of_id}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530210 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
211 ... Get NNI Port in ONOS ${of_id}
212 Set Global Variable ${nni_port}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000213 FOR ${I} IN RANGE 0 ${num_onus}
214 ${src}= Set Variable ${hosts.src[${I}]}
215 ${dst}= Set Variable ${hosts.dst[${I}]}
216 ${onu_device_id}= Get Device ID From SN ${src['onu']}
217 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
218 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
219 # Check ONU port is Enabled in ONOS
220 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800221 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000222 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800223 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000224 ... volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530225 # Verify subscriber access flows are added for the ONU port
226 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530227 ... Verify Subscriber Access Flows Added For ONU DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
228 ... ${onu_port} ${nni_port} ${src['s_tag']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000229 # Verify ONU state in voltha
230 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
231 ... ENABLED ACTIVE REACHABLE
232 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530233 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000234 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
235 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
236 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
237 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
238 ... ${dst['container_name']}
239 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
240 Run Keyword and Ignore Error Collect Logs
241 END
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530242 # Verify ONOS Flows
243 # Number of Access Flows on ONOS equals 4 * the Number of Active ONUs (2 for each downstream and upstream)
244 ${onos_flows_count}= Evaluate 4 * ${num_onus}
245 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530246 ... Verify Subscriber Access Flows Added Count DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
247 ... ${onos_flows_count}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530248 # Verify VOLTHA Flows
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000249 # Number of per OLT Flows equals Twice the Number of Active ONUs (each for downstream and upstream) + 1 for LLDP
250 ${olt_flows}= Evaluate 2 * ${num_onus} + 1
251 # Number of per ONU Flows equals 2 (one each for downstream and upstream)
252 ${onu_flows}= Set Variable 2
253 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Flows ${olt_flows}
254 ${List_ONU_Serial} Create List
255 Set Suite Variable ${List_ONU_Serial}
256 Build ONU SN List ${List_ONU_Serial}
257 Log ${List_ONU_Serial}
258 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate ONU Flows
259 ... ${List_ONU_Serial} ${onu_flows}
260
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000261Setup
262 [Documentation] Pre-test Setup
263 #test for empty device list
264 Test Empty Device List
265 Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
266 ... ${olt_ip} ${olt_user} ${olt_pass}
Matteo Scandolof6cea382020-04-20 14:32:00 -0700267 Run Keyword If ${has_dataplane} Sleep 60s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000268 #create/preprovision device
269 ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
270 Set Suite Variable ${olt_device_id}
271 #validate olt states
272 Wait Until Keyword Succeeds ${timeout} 5s
ubuntu6b6e7d42020-03-02 12:35:42 -0800273 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id}
Suchitra Vemuria7e8ad22020-02-11 18:02:40 -0800274 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000275 Enable Device ${olt_device_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800276 Wait Until Keyword Succeeds 180s 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000277 ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
278 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
279 Set Suite Variable ${logical_id}
280
Hardik Windlassaaea3402020-03-10 19:45:45 +0530281Validate ONUs After OLT Disable
282 [Documentation] Validates the ONUs state in Voltha, ONUs port state in ONOS
283 ... and that pings do not succeed After corresponding OLT is Disabled
284 FOR ${I} IN RANGE 0 ${num_onus}
285 ${src}= Set Variable ${hosts.src[${I}]}
286 ${dst}= Set Variable ${hosts.dst[${I}]}
287 ${onu_device_id}= Get Device ID From SN ${src['onu']}
288 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
289 ... ${of_id}
290 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
291 ... Validate Device ENABLED DISCOVERED
292 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
293 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
294 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
295 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
296 ... Wait Until Keyword Succeeds 60s 2s
297 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
298 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
299 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
300 Run Keyword and Ignore Error Collect Logs
301 END
302
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000303Delete All Devices and Verify
304 [Documentation] Remove any devices from VOLTHA and ONOS
305 # Clear devices from VOLTHA
306 Disable Devices In Voltha Root=true
Suchitra Vemuri970747f2020-03-30 17:31:34 -0700307 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000308 Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
309 Delete Devices In Voltha Root=true
Matteo Scandolof6cea382020-04-20 14:32:00 -0700310 Run Keyword If ${has_dataplane} Sleep 30s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000311 Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
312 # Clear devices from ONOS
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800313 #Remove All Devices From ONOS
ubuntu6b6e7d42020-03-02 12:35:42 -0800314 #... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000315
316Teardown
317 [Documentation] kills processes and cleans up interfaces on src+dst servers
318 Run Keyword If ${has_dataplane} Clean Up Linux
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000319
320Teardown Suite
321 [Documentation] Clean up device if desired
322 Run Keyword If ${teardown_device} Delete All Devices and Verify
323
324Delete Device and Verify
325 [Documentation] Disable -> Delete devices via voltctl and verify its removed
Suchitra Vemuri2fa9bba2020-01-22 17:38:48 -0800326 ${olt_device_id}= Get Device ID From SN ${olt_serial_number}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000327 ${rc} ${output}= Run and Return Rc and Output
328 ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
329 Should Be Equal As Integers ${rc} 0
330 Sleep 5s
331 Wait Until Keyword Succeeds ${timeout} 5s
332 ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
333 ${rc} ${output}= Run and Return Rc and Output
334 ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
Matteo Scandolof6cea382020-04-20 14:32:00 -0700335 Run Keyword If ${has_dataplane} Sleep 50s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000336 Should Be Equal As Integers ${rc} 0
337 Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
338
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000339Repeat Sanity Test
340 [Documentation] This keyword performs Sanity Test Procedure
341 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
342 ... This keyword can be used to call in any other tests where sanity check is required
343 ... with wpa reassociation
344 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
345 Set Global Variable ${of_id}
346 FOR ${I} IN RANGE 0 ${num_onus}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800347 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000348 ${src}= Set Variable ${hosts.src[${I}]}
349 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800350 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
351 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
352 # Check ONU port is Enabled in ONOD
353 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800354 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800355 # Verify EAPOL flows are added for the ONU port
356 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800357 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800358 # Verify ONU state in voltha
Suchitra Vemurid1e27fb2020-02-01 16:34:37 -0800359 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
Suchitra Vemuri81e4f022020-02-01 13:16:02 -0800360 ... ENABLED ACTIVE REACHABLE
361 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800362 # Perform Authentication
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000363 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
364 ... Validate Authentication After Reassociate
365 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
366 ... ${src['container_type']} ${src['container_name']}
367 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800368 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000369 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800370 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000371 ... volt-add-subscriber-access ${of_id} ${onu_port}
372 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
373 ... Validate DHCP and Ping True True
374 ... ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
375 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
376 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
377 ... ${dst['container_name']}
378 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800379 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000380 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
381 Run Keyword and Ignore Error Collect Logs
382 END
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000383
384Collect Logs
385 [Documentation] Collect Logs from voltha and onos cli for various commands
386 Run Keyword and Ignore Error Get Device List from Voltha
387 Run Keyword and Ignore Error Get Device Output from Voltha ${olt_device_id}
388 Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
ubuntu6b6e7d42020-03-02 12:35:42 -0800389 Get ONOS Status ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000390
391Verify ping is succesful except for given device
392 [Arguments] ${num_onus} ${exceptional_onu_id}
393 [Documentation] Checks that ping for all the devices are successful except the given ONU.
394 ${pingStatus} Set Variable True
395 FOR ${I} IN RANGE 0 ${num_onus}
396 ${src}= Set Variable ${hosts.src[${I}]}
397 ${dst}= Set Variable ${hosts.dst[${I}]}
398 ${onu_device_id}= Get Device ID From SN ${src['onu']}
399 ${pingStatus} Run Keyword If '${onu_device_id}' == '${exceptional_onu_id}' Set Variable False
400 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s
401 ... Check Ping ${pingStatus} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
402 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
403 END
Andy Bavier4a8450e2020-02-04 08:58:37 -0700404
Andy Bavierabeba262020-02-07 16:22:16 -0700405Start Logging
406 [Arguments] ${label}
407 [Documentation] Start logging for test ${label}
408 ${kail_process}= Run Keyword If "${container_log_dir}" != "${None}" Start Process kail -n default
409 ... -n voltha cwd=${container_log_dir} stdout=${label}-combined.log
410 Set Test Variable ${kail_process}
411
412Stop Logging
413 [Arguments] ${label}
414 [Documentation] End logging for test; remove logfile if test passed
415 Run sync
416 Run Keyword If ${kail_process} Terminate Process ${kail_process}
417 ${test_logfile}= Run Keyword If "${container_log_dir}" != "${None}"
418 ... Join Path ${container_log_dir} ${label}-combined.log
419 Run Keyword If Test Passed Run Keyword If "${test_logfile}" != "${None}" Remove File ${test_logfile}
420
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800421Clean Up Linux
422 [Documentation] Kill processes and clean up interfaces on src+dst servers
423 FOR ${I} IN RANGE 0 ${num_onus}
424 ${src}= Set Variable ${hosts.src[${I}]}
425 ${dst}= Set Variable ${hosts.dst[${I}]}
426 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
427 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
428 Run Keyword And Ignore Error Kill Linux Process [d]hclient ${src['ip']}
429 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
430 Run Keyword If '${dst['ip']}' != '${None}' Run Keyword And Ignore Error
431 ... Kill Linux Process [d]hcpd ${dst['ip']} ${dst['user']}
432 ... ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
433 Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
434 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
435 Run Keyword If '${dst['ip']}' != '${None}' Delete Interface on Remote Host
436 ... ${dst['dp_iface_name']}.${src['s_tag']} ${dst['ip']} ${dst['user']} ${dst['pass']}
437 ... ${dst['container_type']} ${dst['container_name']}
438 END
Scott Baker60e570d2020-02-02 22:10:13 -0800439
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800440Clean WPA Process
441 [Documentation] Kills wpa_supplicant processes only for all RGs
442 FOR ${I} IN RANGE 0 ${num_onus}
443 ${src}= Set Variable ${hosts.src[${I}]}
444 ${dst}= Set Variable ${hosts.dst[${I}]}
445 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
446 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
447 END
448
Scott Baker60e570d2020-02-02 22:10:13 -0800449Should Be Larger Than
450 [Documentation] Verify that value_1 is > value_2
451 [Arguments] ${value_1} ${value_2}
452 Run Keyword If ${value_1} <= ${value_2}
453 ... Fail The value ${value_1} is not larger than ${value_2}
454
Scott Bakeree675552020-02-11 10:43:34 -0800455Should Be Larger Than Or Equal To
456 [Documentation] Verify that value_1 is >= value_2
457 [Arguments] ${value_1} ${value_2}
458 Run Keyword If ${value_1} < ${value_2}
459 ... Fail The value ${value_1} is not larger than or equal to ${value_2}
460
Scott Baker60e570d2020-02-02 22:10:13 -0800461Should Be Float
462 [Documentation] Verify that value is a floating point number type
463 [Arguments] ${value}
464 ${type} Evaluate type(${value}).__name__
465 Should Be Equal ${type} float
466
Scott Bakeree675552020-02-11 10:43:34 -0800467Should Be Newer Than Or Equal To
468 [Documentation] Compare two RFC3339 dates
469 [Arguments] ${value_1} ${value_2}
470 ${unix_v1} Parse RFC3339 ${value_1}
471 ${unix_v2} Parse RFC3339 ${value_2}
472 Run Keyword If ${unix_v1} < ${unix_v2}
473 ... Fail The value ${value_1} is not newer than or equal to ${value_2}
474
Scott Baker60e570d2020-02-02 22:10:13 -0800475Get Current Time
476 [Documentation] Return the current time in RFC3339 format
477 ${output}= Run date -u +"%FT%T%:z"
478 [return] ${output}
Scott Bakeree675552020-02-11 10:43:34 -0800479
480Parse RFC3339
481 [Documentation] Parse an RFC3339 timestamp
482 [Arguments] ${dateStr}
483 ${rc} ${output}= Run and Return Rc and Output date --date="${dateStr}" "+%s"
484 Should Be Equal As Numbers ${rc} 0
485 [return] ${output}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700486
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000487Get Bandwidth Profile Name For Given Subscriber
488 [Arguments] ${subscriber_id} ${stream_type}=upstreamBandwidthProfile
489 [Documentation] Keyword to get the bandwidth details of the given subscriber
490 ${bandwidth_profile_output}= Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
491 ... volt-programmed-subscribers | grep ${subscriber_id}
492 @{bandwidth_profile_array}= Split String ${bandwidth_profile_output} ,
493 Log ${bandwidth_profile_array}
494 FOR ${value} IN @{bandwidth_profile_array}
495 @{row_value}= Split String ${value} =
496 ${bandwidth_profile_name}= Set Variable If '${row_value[0]}' == ' ${stream_type}'
497 ... ${row_value[1]}
498 ${bandwidth_profile_name}= Convert To String ${bandwidth_profile_name}
499 Run Keyword If "${bandwidth_profile_name}" != "None" Exit For Loop
500 END
501 Log ${bandwidth_profile_name}
502 [Return] ${bandwidth_profile_name}
503
504Execute Remote Command
505 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
506 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
507 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
508 ... ${container_type}=${None} ${container_name}=${None}
509 ${conn_id}= SSHLibrary.Open Connection ${ip}
510 Run Keyword If '${pass}' != '${None}'
511 ... SSHLibrary.Login ${user} ${pass}
512 ... ELSE
513 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
514 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
515 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
516 ${output}= Run Keyword If '${container_type}' == 'LXC'
517 ... SSHLibrary.Execute Command lxc exec ${container_name} -- ${cmd}
518 ... ELSE IF '${container_type}' == 'K8S'
519 ... SSHLibrary.Execute Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
520 ... ELSE
521 ... SSHLibrary.Execute Command ${cmd}
522 Log ${output}
523 SSHLibrary.Close Connection
524 [Return] ${output}
525
526Run Iperf3 Test Client
527 [Arguments] ${src} ${server} ${args}
528 [Documentation] Login to ${src} and run the iperf3 client against ${server} using ${args}.
529 ... Return a Dictionary containing the results of the test.
530 ${output}= Execute Remote Command iperf3 -J -c ${server} ${args} | jq -M -c '.'
531 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
532 ${object}= Evaluate json.loads(r'''${output}''') json
533 [Return] ${object}