blob: 43872e836a82bd910455252d80c1c96212610b9f [file] [log] [blame]
Kailash92764922019-07-25 08:21:39 -07001# Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Kailash92764922019-07-25 08:21:39 -070014# robot test functions
15
16*** Settings ***
17Documentation Library for various utilities
18Library SSHLibrary
Kailash92764922019-07-25 08:21:39 -070019Library String
20Library DateTime
21Library Process
22Library Collections
23Library RequestsLibrary
24Library OperatingSystem
Andy Bavier63460da2020-02-20 14:35:12 -070025Library CORDRobot
26Library ImportResource resources=CORDRobot
Kailash92764922019-07-25 08:21:39 -070027
28*** Keywords ***
Kailash528f7c02019-07-31 10:55:49 -070029Check CLI Tools Configured
30 [Documentation] Tests that use 'voltctl' and 'kubectl' should execute this keyword in suite setup
31 # check voltctl and kubectl configured
Zack Williamsa8fe75a2020-01-10 14:25:27 -070032 ${voltctl_rc}= Run And Return RC voltctl device list
33 ${kubectl_rc}= Run And Return RC kubectl get pods
Gilles Depatie675a2062019-10-22 12:44:42 -040034 Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR
35 ... VOLTCTL and KUBECTL not configured. Please configure before executing tests.
Gilles Depatieb5682f82019-10-31 10:39:45 -040036
37Send File To Onos
Zack Williamsa8fe75a2020-01-10 14:25:27 -070038 [Documentation] Send the content of the file to Onos to selected section of configuration
39 ... using Post Request
40 [Arguments] ${CONFIG_FILE} ${section}
41 ${Headers}= Create Dictionary Content-Type application/json
42 ${File_Data}= OperatingSystem.Get File ${CONFIG_FILE}
43 Log ${Headers}
44 Log ${File_Data}
45 ${resp}= Post Request ONOS
46 ... /onos/v1/network/configuration/${section} headers=${Headers} data=${File_Data}
47 Should Be Equal As Strings ${resp.status_code} 200
Andy Bavier88cd9f62019-11-26 16:22:33 -070048
49Common Test Suite Setup
50 [Documentation] Setup the test suite
Andy Bavier88cd9f62019-11-26 16:22:33 -070051 Set Global Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
52 Set Global Variable ${VOLTCTL_CONFIG} export VOLTCONFIG=%{VOLTCONFIG}
53 ${k8s_node_ip}= Evaluate ${nodes}[0].get("ip")
ubuntu6b6e7d42020-03-02 12:35:42 -080054 ${ONOS_REST_IP}= Get Environment Variable ONOS_REST_IP ${k8s_node_ip}
55 ${ONOS_SSH_IP}= Get Environment Variable ONOS_SSH_IP ${k8s_node_ip}
56 Set Global Variable ${ONOS_REST_IP}
57 Set Global Variable ${ONOS_SSH_IP}
Andy Bavier88cd9f62019-11-26 16:22:33 -070058 ${k8s_node_user}= Evaluate ${nodes}[0].get("user")
59 ${k8s_node_pass}= Evaluate ${nodes}[0].get("pass")
60 Check CLI Tools Configured
61 ${onos_auth}= Create List karaf karaf
62 ${HEADERS} Create Dictionary Content-Type=application/json
ubuntu6b6e7d42020-03-02 12:35:42 -080063 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
Andy Bavier88cd9f62019-11-26 16:22:33 -070064 ${olt_ip}= Evaluate ${olts}[0].get("ip")
65 ${olt_user}= Evaluate ${olts}[0].get("user")
66 ${olt_pass}= Evaluate ${olts}[0].get("pass")
67 ${olt_serial_number}= Evaluate ${olts}[0].get("serial")
68 ${num_onus}= Get Length ${hosts.src}
69 ${num_onus}= Convert to String ${num_onus}
70 #send sadis file to onos
Andy Baviera2691752019-11-27 12:18:38 -070071 ${sadis_file}= Get Variable Value ${sadis.file}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070072 Log To Console \nSadis File:${sadis_file}
Andy Bavier88cd9f62019-11-26 16:22:33 -070073 Run Keyword Unless '${sadis_file}' is '${None}' Send File To Onos ${sadis_file} apps/
74 Set Suite Variable ${num_onus}
75 Set Suite Variable ${olt_serial_number}
76 Set Suite Variable ${olt_ip}
77 Set Suite Variable ${olt_user}
78 Set Suite Variable ${olt_pass}
Andy Bavier88cd9f62019-11-26 16:22:33 -070079 @{container_list}= Create List adapter-open-olt adapter-open-onu voltha-api-server
80 ... voltha-ro-core voltha-rw-core-11 voltha-rw-core-12 voltha-ofagent
81 Set Suite Variable ${container_list}
82 ${datetime}= Get Current Date
suraj gour7f6d5fe2019-11-29 10:56:35 +000083 Set Suite Variable ${datetime}
84
85WPA Reassociate
suraj gour7f6d5fe2019-11-29 10:56:35 +000086 [Documentation] Executes a particular wpa_cli reassociate, which performs force reassociation
Zack Williamsa8fe75a2020-01-10 14:25:27 -070087 [Arguments] ${iface} ${ip} ${user} ${pass}=${None}
88 ... ${container_type}=${None} ${container_name}=${None}
suraj gour7f6d5fe2019-11-29 10:56:35 +000089 #Below for loops are used instead of sleep time, to execute reassociate command and check status
Zack Williamsa8fe75a2020-01-10 14:25:27 -070090 FOR ${i} IN RANGE 70
91 ${output}= Login And Run Command On Remote System
92 ... wpa_cli -i ${iface} reassociate ${ip} ${user}
93 ... ${pass} ${container_type} ${container_name}
94 ${passed}= Run Keyword And Return Status Should Contain ${output} OK
Andy Bavierb63f6d22020-03-12 15:34:37 -070095 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070096 END
Andy Bavierb63f6d22020-03-12 15:34:37 -070097 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -070098 FOR ${i} IN RANGE 70
99 ${output}= Login And Run Command On Remote System
Andy Bavierae4a17e2020-02-25 14:54:36 -0700100 ... wpa_cli -i ${iface} status | grep SUCCESS ${ip} ${user}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700101 ... ${pass} ${container_type} ${container_name}
102 ${passed}= Run Keyword And Return Status Should Contain ${output} SUCCESS
Andy Bavierb63f6d22020-03-12 15:34:37 -0700103 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700104 END
Andy Bavierb63f6d22020-03-12 15:34:37 -0700105 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700106
suraj gour7f6d5fe2019-11-29 10:56:35 +0000107Validate Authentication After Reassociate
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700108 [Arguments] ${auth_pass} ${iface} ${ip} ${user} ${pass}=${None}
109 ... ${container_type}=${None} ${container_name}=${None}
110 [Documentation]
111 ... Executes a particular reassociate request on the RG using wpa_cli.
112 ... auth_pass determines if authentication should pass
Andy Bavier84834d42020-02-25 13:49:50 -0700113 ${wpa_log}= Catenate SEPARATOR=. /tmp/wpa ${iface} log
114 ${output}= Login And Run Command On Remote System truncate -s 0 ${wpa_log}; cat ${wpa_log}
115 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
116 Log ${output}
117 Should Not Contain ${output} authentication completed successfully
suraj gour7f6d5fe2019-11-29 10:56:35 +0000118 WPA Reassociate ${iface} ${ip} ${user} ${pass} ${container_type} ${container_name}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700119 Run Keyword If '${auth_pass}' == 'True' Wait Until Keyword Succeeds ${timeout} 2s
Andy Bavier84834d42020-02-25 13:49:50 -0700120 ... Check Remote File Contents True ${wpa_log} ${iface}.*authentication completed successfully
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700121 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
suraj gour7f6d5fe2019-11-29 10:56:35 +0000122 Run Keyword If '${auth_pass}' == 'False' Sleep 20s
Suchitra Vemuri3cb29f82020-02-25 13:59:58 -0800123 Run Keyword If '${auth_pass}' == 'False' Check Remote File Contents False /tmp/wpa.log
Andy Bavier84834d42020-02-25 13:49:50 -0700124 ... ${iface}.*authentication completed successfully ${ip} ${user} ${pass}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700125 ... ${container_type} ${container_name}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000126
127Send Dhclient Request To Release Assigned IP
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700128 [Arguments] ${iface} ${ip} ${user} ${path_dhcpleasefile} ${pass}=${None}
129 ... ${container_type}=${None} ${container_name}=${None}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000130 [Documentation] Executes a dhclient with option to release ip against a particular interface on the RG (src)
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700131 ${result}= Login And Run Command On Remote System
132 ... dhclient -nw -r ${iface} && rm ${path_dhcpleasefile}/dhclient.* ${ip} ${user}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000133 ... ${pass} ${container_type} ${container_name}
134 Log ${result}
135 #Should Contain ${result} DHCPRELEASE
136 [Return] ${result}
suraj gour1ecfae92019-12-20 15:11:40 +0000137
138Check Remote File Contents For WPA Logs
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700139 [Arguments] ${file_should_exist} ${file} ${pattern} ${ip} ${user} ${pass}=${None}
140 ... ${container_type}=${None} ${container_name}=${None} ${prompt}=~$
suraj gour1ecfae92019-12-20 15:11:40 +0000141 [Documentation] Checks for particular pattern count in a file
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700142 ${result}= Login And Run Command On Remote System
143 ... cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
suraj gour1ecfae92019-12-20 15:11:40 +0000144 ... ${container_type} ${container_name} ${prompt}
145 [Return] ${result}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000146
147Perform Sanity Test
148 [Documentation] This keyword performs Sanity Test Procedure
149 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
150 ... This keyword can be used to call in any other tests where sanity check is required
151 ... and avoids duplication of code.
152 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
153 Set Global Variable ${of_id}
Hardik Windlass21807632020-04-14 16:24:55 +0530154 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
155 ... Get NNI Port in ONOS ${of_id}
156 Set Global Variable ${nni_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000157 FOR ${I} IN RANGE 0 ${num_onus}
158 ${src}= Set Variable ${hosts.src[${I}]}
159 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800160 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000161 ${onu_device_id}= Get Device ID From SN ${src['onu']}
162 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
163 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800164 # Check ONU port is Enabled in ONOS
165 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700166 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800167 # Verify EAPOL flows are added for the ONU port
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000168 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800169 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800170 # Verify ONU state in voltha
171 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
172 ... ENABLED ACTIVE REACHABLE
173 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
174 # Perform Authentication
Andy Bavier84834d42020-02-25 13:49:50 -0700175 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
176 ... /tmp/wpa ${src['dp_iface_name']} log
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000177 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
178 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700179 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000180 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800181 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000182 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800183 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000184 ... volt-add-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri9da44302020-03-04 14:24:49 -0800185 # Verify that no pending flows exist for the ONU port
186 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
187 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530188 # Verify subscriber access flows are added for the ONU port
189 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
190 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
191 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000192 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
193 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
194 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
195 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
196 ... ${dst['container_name']}
197 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800198 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000199 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
200 Run Keyword and Ignore Error Collect Logs
201 END
202
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000203Perform Sanity Test DT
204 [Documentation] This keyword performs Sanity Test Procedure for DT Workflow
205 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
206 ... This keyword can be used to call in any other tests where sanity check is required
207 ... and avoids duplication of code.
208 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
209 Set Global Variable ${of_id}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530210 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
211 ... Get NNI Port in ONOS ${of_id}
212 Set Global Variable ${nni_port}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000213 FOR ${I} IN RANGE 0 ${num_onus}
214 ${src}= Set Variable ${hosts.src[${I}]}
215 ${dst}= Set Variable ${hosts.dst[${I}]}
216 ${onu_device_id}= Get Device ID From SN ${src['onu']}
217 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
218 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
219 # Check ONU port is Enabled in ONOS
220 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700221 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000222 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800223 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000224 ... volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530225 # Verify subscriber access flows are added for the ONU port
226 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530227 ... Verify Subscriber Access Flows Added For ONU DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
228 ... ${onu_port} ${nni_port} ${src['s_tag']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000229 # Verify ONU state in voltha
230 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
231 ... ENABLED ACTIVE REACHABLE
232 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530233 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000234 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
235 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
236 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
237 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
238 ... ${dst['container_name']}
239 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
240 Run Keyword and Ignore Error Collect Logs
241 END
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530242 # Verify ONOS Flows
243 # Number of Access Flows on ONOS equals 4 * the Number of Active ONUs (2 for each downstream and upstream)
244 ${onos_flows_count}= Evaluate 4 * ${num_onus}
245 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530246 ... Verify Subscriber Access Flows Added Count DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
247 ... ${onos_flows_count}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530248 # Verify VOLTHA Flows
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000249 # Number of per OLT Flows equals Twice the Number of Active ONUs (each for downstream and upstream) + 1 for LLDP
250 ${olt_flows}= Evaluate 2 * ${num_onus} + 1
251 # Number of per ONU Flows equals 2 (one each for downstream and upstream)
252 ${onu_flows}= Set Variable 2
253 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Flows ${olt_flows}
254 ${List_ONU_Serial} Create List
255 Set Suite Variable ${List_ONU_Serial}
256 Build ONU SN List ${List_ONU_Serial}
257 Log ${List_ONU_Serial}
258 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate ONU Flows
259 ... ${List_ONU_Serial} ${onu_flows}
260
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700261Perform Sanity Test TT
262 [Documentation] This keyword performs Sanity Test Procedure for TT Workflow
263 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
264 ... This keyword can be used to call in any other tests where sanity check is required
265 ... and avoids duplication of code.
266 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
267 Set Global Variable ${of_id}
268 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
269 ... Get NNI Port in ONOS ${of_id}
270 Set Global Variable ${nni_port}
271 FOR ${I} IN RANGE 0 ${num_onus}
272 ${src}= Set Variable ${hosts.src[${I}]}
273 ${dst}= Set Variable ${hosts.dst[${I}]}
274 ${onu_device_id}= Get Device ID From SN ${src['onu']}
275 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
276 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
277 # Check ONU port is Enabled in ONOS
278 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
279 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
280 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
281 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
282 ... volt-add-subscriber-access ${of_id} ${onu_port}
283 Sleep 30s
284 # Verify ONU state in voltha
285 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
286 ... ENABLED ACTIVE REACHABLE
287 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
288 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
289 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
290 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
291 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
292 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
293 ... ${dst['container_name']}
294 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
295 Run Keyword and Ignore Error Collect Logs
296 END
297
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000298Setup
299 [Documentation] Pre-test Setup
300 #test for empty device list
301 Test Empty Device List
Suchitra Vemuri3158f3c2020-05-28 17:58:26 -0700302 #Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
303 #... ${olt_ip} ${olt_user} ${olt_pass}
Suchitra Vemuri1d8e8172020-06-17 17:43:56 -0700304 Run Keyword If ${has_dataplane} Sleep 230s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000305 #create/preprovision device
306 ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
307 Set Suite Variable ${olt_device_id}
308 #validate olt states
309 Wait Until Keyword Succeeds ${timeout} 5s
ubuntu6b6e7d42020-03-02 12:35:42 -0800310 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id}
Suchitra Vemuria7e8ad22020-02-11 18:02:40 -0800311 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000312 Enable Device ${olt_device_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800313 Wait Until Keyword Succeeds 180s 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000314 ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
315 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
316 Set Suite Variable ${logical_id}
317
Hardik Windlassaaea3402020-03-10 19:45:45 +0530318Validate ONUs After OLT Disable
319 [Documentation] Validates the ONUs state in Voltha, ONUs port state in ONOS
320 ... and that pings do not succeed After corresponding OLT is Disabled
321 FOR ${I} IN RANGE 0 ${num_onus}
322 ${src}= Set Variable ${hosts.src[${I}]}
323 ${dst}= Set Variable ${hosts.dst[${I}]}
324 ${onu_device_id}= Get Device ID From SN ${src['onu']}
325 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
326 ... ${of_id}
327 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
328 ... Validate Device ENABLED DISCOVERED
329 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
330 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700331 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassaaea3402020-03-10 19:45:45 +0530332 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
333 ... Wait Until Keyword Succeeds 60s 2s
334 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
335 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
336 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
337 Run Keyword and Ignore Error Collect Logs
338 END
339
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000340Delete All Devices and Verify
341 [Documentation] Remove any devices from VOLTHA and ONOS
342 # Clear devices from VOLTHA
343 Disable Devices In Voltha Root=true
Suchitra Vemuri970747f2020-03-30 17:31:34 -0700344 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000345 Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
346 Delete Devices In Voltha Root=true
Matteo Scandolof6cea382020-04-20 14:32:00 -0700347 Run Keyword If ${has_dataplane} Sleep 30s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000348 Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
349 # Clear devices from ONOS
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800350 #Remove All Devices From ONOS
ubuntu6b6e7d42020-03-02 12:35:42 -0800351 #... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000352
353Teardown
354 [Documentation] kills processes and cleans up interfaces on src+dst servers
355 Run Keyword If ${has_dataplane} Clean Up Linux
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000356
357Teardown Suite
358 [Documentation] Clean up device if desired
359 Run Keyword If ${teardown_device} Delete All Devices and Verify
360
361Delete Device and Verify
362 [Documentation] Disable -> Delete devices via voltctl and verify its removed
Suchitra Vemuri2fa9bba2020-01-22 17:38:48 -0800363 ${olt_device_id}= Get Device ID From SN ${olt_serial_number}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000364 ${rc} ${output}= Run and Return Rc and Output
365 ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
366 Should Be Equal As Integers ${rc} 0
367 Sleep 5s
368 Wait Until Keyword Succeeds ${timeout} 5s
369 ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
370 ${rc} ${output}= Run and Return Rc and Output
371 ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
Matteo Scandolof6cea382020-04-20 14:32:00 -0700372 Run Keyword If ${has_dataplane} Sleep 50s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000373 Should Be Equal As Integers ${rc} 0
374 Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
375
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000376Repeat Sanity Test
377 [Documentation] This keyword performs Sanity Test Procedure
378 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
379 ... This keyword can be used to call in any other tests where sanity check is required
380 ... with wpa reassociation
381 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
382 Set Global Variable ${of_id}
383 FOR ${I} IN RANGE 0 ${num_onus}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800384 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000385 ${src}= Set Variable ${hosts.src[${I}]}
386 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800387 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
388 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
389 # Check ONU port is Enabled in ONOD
390 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700391 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800392 # Verify EAPOL flows are added for the ONU port
393 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800394 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800395 # Verify ONU state in voltha
Suchitra Vemurid1e27fb2020-02-01 16:34:37 -0800396 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
Suchitra Vemuri81e4f022020-02-01 13:16:02 -0800397 ... ENABLED ACTIVE REACHABLE
398 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800399 # Perform Authentication
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000400 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
401 ... Validate Authentication After Reassociate
402 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
403 ... ${src['container_type']} ${src['container_name']}
404 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800405 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000406 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800407 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000408 ... volt-add-subscriber-access ${of_id} ${onu_port}
409 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
410 ... Validate DHCP and Ping True True
411 ... ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
412 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
413 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
414 ... ${dst['container_name']}
415 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800416 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000417 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
418 Run Keyword and Ignore Error Collect Logs
419 END
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000420
421Collect Logs
422 [Documentation] Collect Logs from voltha and onos cli for various commands
423 Run Keyword and Ignore Error Get Device List from Voltha
424 Run Keyword and Ignore Error Get Device Output from Voltha ${olt_device_id}
425 Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
ubuntu6b6e7d42020-03-02 12:35:42 -0800426 Get ONOS Status ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000427
428Verify ping is succesful except for given device
429 [Arguments] ${num_onus} ${exceptional_onu_id}
430 [Documentation] Checks that ping for all the devices are successful except the given ONU.
431 ${pingStatus} Set Variable True
432 FOR ${I} IN RANGE 0 ${num_onus}
433 ${src}= Set Variable ${hosts.src[${I}]}
434 ${dst}= Set Variable ${hosts.dst[${I}]}
435 ${onu_device_id}= Get Device ID From SN ${src['onu']}
436 ${pingStatus} Run Keyword If '${onu_device_id}' == '${exceptional_onu_id}' Set Variable False
437 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s
438 ... Check Ping ${pingStatus} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
439 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
440 END
Andy Bavier4a8450e2020-02-04 08:58:37 -0700441
Andy Bavier1923ecc2020-06-18 14:06:40 -0700442Echo Message to OLT Logs
443 [Arguments] ${message}
444 [Documentation] Echoes ${message} into the OLT logs
Andy Bavier690ce432020-06-20 16:31:10 -0700445 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700446 ... printf '%s\n' '' '' '${message}' '' >> /var/log/openolt.log
Andy Bavier1923ecc2020-06-18 14:06:40 -0700447 ... ${olt_ip} ${olt_user} ${olt_pass}
Andy Bavier690ce432020-06-20 16:31:10 -0700448 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700449 ... printf '%s\n' '' '' '${message}' '' >> /var/log/dev_mgmt_daemon.log
Andy Bavier1923ecc2020-06-18 14:06:40 -0700450 ... ${olt_ip} ${olt_user} ${olt_pass}
451
Andy Bavierabeba262020-02-07 16:22:16 -0700452Start Logging
453 [Arguments] ${label}
454 [Documentation] Start logging for test ${label}
455 ${kail_process}= Run Keyword If "${container_log_dir}" != "${None}" Start Process kail -n default
456 ... -n voltha cwd=${container_log_dir} stdout=${label}-combined.log
457 Set Test Variable ${kail_process}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700458 Run Keyword If ${has_dataplane} Echo Message to OLT Logs START ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700459
460Stop Logging
461 [Arguments] ${label}
462 [Documentation] End logging for test; remove logfile if test passed
463 Run sync
464 Run Keyword If ${kail_process} Terminate Process ${kail_process}
465 ${test_logfile}= Run Keyword If "${container_log_dir}" != "${None}"
466 ... Join Path ${container_log_dir} ${label}-combined.log
467 Run Keyword If Test Passed Run Keyword If "${test_logfile}" != "${None}" Remove File ${test_logfile}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700468 Run Keyword If ${has_dataplane} Echo Message to OLT Logs END ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700469
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800470Clean Up Linux
471 [Documentation] Kill processes and clean up interfaces on src+dst servers
472 FOR ${I} IN RANGE 0 ${num_onus}
473 ${src}= Set Variable ${hosts.src[${I}]}
474 ${dst}= Set Variable ${hosts.dst[${I}]}
Andy Bavier84b83282020-06-30 05:47:49 -0700475 Execute Remote Command sudo pkill wpa_supplicant ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800476 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700477 Execute Remote Command sudo pkill dhclient ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800478 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Baviereff938d2020-06-29 18:27:49 -0700479 Execute Remote Command sudo pkill mausezahn ${src['ip']}
480 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
481 Run Keyword If '${dst['ip']}' != '${None}' Execute Remote Command pkill dhcpd
482 ... ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800483 Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
484 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
485 Run Keyword If '${dst['ip']}' != '${None}' Delete Interface on Remote Host
486 ... ${dst['dp_iface_name']}.${src['s_tag']} ${dst['ip']} ${dst['user']} ${dst['pass']}
487 ... ${dst['container_type']} ${dst['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700488 ${bng_ip}= Get Variable Value ${dst['noroot_ip']}
489 ${bng_user}= Get Variable Value ${dst['noroot_user']}
490 ${bng_pass}= Get Variable Value ${dst['noroot_pass']}
Andy Bavierf4b7d1d2020-06-30 11:14:04 -0700491 Run Keyword If "${bng_ip}" != "${NONE}" and "${bng_user}" != "${NONE}" and "${bng_pass}" != "${NONE}"
492 ... Execute Remote Command sudo pkill mausezahn ${bng_ip} ${bng_user} ${bng_pass}
Andy Baviereff938d2020-06-29 18:27:49 -0700493 ... ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800494 END
Scott Baker60e570d2020-02-02 22:10:13 -0800495
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800496Clean WPA Process
497 [Documentation] Kills wpa_supplicant processes only for all RGs
498 FOR ${I} IN RANGE 0 ${num_onus}
499 ${src}= Set Variable ${hosts.src[${I}]}
500 ${dst}= Set Variable ${hosts.dst[${I}]}
501 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
502 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
503 END
504
Scott Baker60e570d2020-02-02 22:10:13 -0800505Should Be Larger Than
506 [Documentation] Verify that value_1 is > value_2
507 [Arguments] ${value_1} ${value_2}
508 Run Keyword If ${value_1} <= ${value_2}
509 ... Fail The value ${value_1} is not larger than ${value_2}
510
Scott Bakeree675552020-02-11 10:43:34 -0800511Should Be Larger Than Or Equal To
512 [Documentation] Verify that value_1 is >= value_2
513 [Arguments] ${value_1} ${value_2}
514 Run Keyword If ${value_1} < ${value_2}
515 ... Fail The value ${value_1} is not larger than or equal to ${value_2}
516
Scott Baker60e570d2020-02-02 22:10:13 -0800517Should Be Float
518 [Documentation] Verify that value is a floating point number type
519 [Arguments] ${value}
520 ${type} Evaluate type(${value}).__name__
521 Should Be Equal ${type} float
522
Scott Bakeree675552020-02-11 10:43:34 -0800523Should Be Newer Than Or Equal To
524 [Documentation] Compare two RFC3339 dates
525 [Arguments] ${value_1} ${value_2}
526 ${unix_v1} Parse RFC3339 ${value_1}
527 ${unix_v2} Parse RFC3339 ${value_2}
528 Run Keyword If ${unix_v1} < ${unix_v2}
529 ... Fail The value ${value_1} is not newer than or equal to ${value_2}
530
Scott Baker60e570d2020-02-02 22:10:13 -0800531Get Current Time
532 [Documentation] Return the current time in RFC3339 format
533 ${output}= Run date -u +"%FT%T%:z"
534 [return] ${output}
Scott Bakeree675552020-02-11 10:43:34 -0800535
536Parse RFC3339
537 [Documentation] Parse an RFC3339 timestamp
538 [Arguments] ${dateStr}
539 ${rc} ${output}= Run and Return Rc and Output date --date="${dateStr}" "+%s"
540 Should Be Equal As Numbers ${rc} 0
541 [return] ${output}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700542
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000543Get Bandwidth Profile Name For Given Subscriber
544 [Arguments] ${subscriber_id} ${stream_type}=upstreamBandwidthProfile
545 [Documentation] Keyword to get the bandwidth details of the given subscriber
546 ${bandwidth_profile_output}= Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
547 ... volt-programmed-subscribers | grep ${subscriber_id}
548 @{bandwidth_profile_array}= Split String ${bandwidth_profile_output} ,
549 Log ${bandwidth_profile_array}
550 FOR ${value} IN @{bandwidth_profile_array}
551 @{row_value}= Split String ${value} =
552 ${bandwidth_profile_name}= Set Variable If '${row_value[0]}' == ' ${stream_type}'
553 ... ${row_value[1]}
554 ${bandwidth_profile_name}= Convert To String ${bandwidth_profile_name}
555 Run Keyword If "${bandwidth_profile_name}" != "None" Exit For Loop
556 END
557 Log ${bandwidth_profile_name}
558 [Return] ${bandwidth_profile_name}
559
560Execute Remote Command
561 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
562 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
563 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
Andy Baviereff938d2020-06-29 18:27:49 -0700564 ... ${container_type}=${None} ${container_name}=${None} ${timeout}=${None}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000565 ${conn_id}= SSHLibrary.Open Connection ${ip}
566 Run Keyword If '${pass}' != '${None}'
567 ... SSHLibrary.Login ${user} ${pass}
568 ... ELSE
569 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
570 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
571 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700572 ${stdout} ${stderr} ${rc}= Run Keyword If '${container_type}' == 'LXC'
573 ... SSHLibrary.Execute Command lxc exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700574 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000575 ... ELSE IF '${container_type}' == 'K8S'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700576 ... SSHLibrary.Execute Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700577 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000578 ... ELSE
Andy Baviereff938d2020-06-29 18:27:49 -0700579 ... SSHLibrary.Execute Command ${cmd} return_stderr=True return_rc=True timeout=${timeout}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700580
581 Log ${stdout}
582 Log ${stderr}
583 Log ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000584 SSHLibrary.Close Connection
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700585 [Return] ${stdout} ${stderr} ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000586
Suraj Gour33651272020-05-04 10:10:16 +0530587Start Remote Command
588 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
589 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
590 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
591 ... ${container_type}=${None} ${container_name}=${None}
592 ${conn_id}= SSHLibrary.Open Connection ${ip}
593 Run Keyword If '${pass}' != '${None}'
594 ... SSHLibrary.Login ${user} ${pass}
595 ... ELSE
596 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
597 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
598 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
599 Run Keyword If '${container_type}' == 'LXC'
600 ... SSHLibrary.Start Command lxc exec ${container_name} -- ${cmd}
601 ... ELSE IF '${container_type}' == 'K8S'
602 ... SSHLibrary.Start Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
603 ... ELSE
604 ... SSHLibrary.Start Command ${cmd}
Andy Bavier9fb23232020-06-30 15:12:20 -0700605 # It seems that closing the connection immediately will sometimes kill the command
606 Sleep 1s
Suraj Gour33651272020-05-04 10:10:16 +0530607 SSHLibrary.Close Connection
608
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000609Run Iperf3 Test Client
610 [Arguments] ${src} ${server} ${args}
611 [Documentation] Login to ${src} and run the iperf3 client against ${server} using ${args}.
612 ... Return a Dictionary containing the results of the test.
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700613 ${output} ${stderr} ${rc}= Execute Remote Command iperf3 -J -c ${server} ${args} | jq -M -c '.'
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000614 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700615 Should Be Equal As Integers ${rc} 0
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000616 ${object}= Evaluate json.loads(r'''${output}''') json
617 [Return] ${object}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000618
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000619RestoreONUs
620 [Documentation] Restore all connected ONUs
621 [Arguments] ${num_onus}
622 FOR ${I} IN RANGE 0 ${num_onus}
623 ${src}= Set Variable ${hosts.src[${I}]}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700624 ${container_type}= Get Variable Value ${src['container_type']} "null"
625 ${container_name}= Get Variable Value ${src['container_name']} "null"
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000626 ${onu_type}= Get Variable Value ${src['onu_type']} "null"
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700627 #Get ens6f0 from ens6f0.22
628 ${if_name}= Replace String Using Regexp ${src['dp_iface_name']} \\..* \
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000629 Run Keyword IF '${onu_type}' == 'alpha' AlphaONURestoreDefault ${src['ip']} ${src['user']}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700630 ... ${src['pass']} ${if_name} admin admin ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000631 END
632
633AlphaONURestoreDefault
634 [Documentation] Restore the Alpha ONU to factory setting
635 [Arguments] ${rg_ip} ${rg_user} ${rg_pass} ${onu_ifname}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700636 ... ${onu_user} ${onu_pass} ${container_type}=${None} ${container_name}=${None}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000637 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 192.168.1.3/24
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700638 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000639 ${cmd} Catenate
640 ... (echo open "192.168.1.1"; sleep 1;
641 ... echo "${onu_user}"; sleep 1;
642 ... echo "${onu_pass}"; sleep 1;
643 ... echo "restoredefault"; sleep 1) | telnet
644 ${output}= Login And Run Command On Remote System ${cmd}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700645 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000646 Log To Console ${output}
647 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 0
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700648 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Suraj Gour33651272020-05-04 10:10:16 +0530649
650Create traffic with each pbit and capture at other end
651 [Documentation] Generates upstream traffic using Mausezahn tool
652 ... with each pbit and validates reception at other end using tcpdump
653 [Arguments] ${target_ip} ${target_iface} ${src_iface}
654 ... ${packet_count} ${packet_type} ${target_port} ${vlan} ${tcpdump_filter}
655 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
656 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
657 FOR ${pbit} IN RANGE 8
Andy Baviereff938d2020-06-29 18:27:49 -0700658 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +0530659 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -0700660 ${var1}= Set Variable sudo mausezahn ${src_iface} -B ${target_ip} -c ${packet_count} -d 100m
Suraj Gour33651272020-05-04 10:10:16 +0530661 ${var2}= Set Variable -t ${packet_type} "dp=${target_port}" -p 1472 -Q ${pbit}:${vlan}
662 ${cmd}= Set Variable ${var1} ${var2}
663 Start Remote Command ${cmd} ${src_ip} ${src_user} ${src_pass}
664 ... ${src_container_type} ${src_container_name}
665 ${output} ${stderr} ${rc}= Execute Remote Command
Andy Baviereff938d2020-06-29 18:27:49 -0700666 ... sudo tcpdump -l -U -c 30 -i ${target_iface} -e ${tcpdump_filter}
Suraj Gour33651272020-05-04 10:10:16 +0530667 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -0700668 ... timeout=30 seconds
669 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +0530670 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
671 # VOL-3262: I'm seeing untagged downstream traffic at RG for pbit 0. According to Girish this is
672 # incorrect behavior. Simplify the following check when VOL-3262 is resolved.
673 Run Keyword If ${pbit}==0 and "${tcpdump_filter}"=="udp"
674 ... Should Match Regexp ${output} \\.${target_port}: UDP,
675 ... ELSE Should Match Regexp ${output} , p ${pbit},.*\\.${target_port}: UDP,
676 END