blob: 5a6ecdf2bf1a3418b6946a49910bff4b4c8c4449 [file] [log] [blame]
Kailash92764922019-07-25 08:21:39 -07001# Copyright 2017-present Open Networking Foundation
2#
3# Licensed under the Apache License, Version 2.0 (the "License");
4# you may not use this file except in compliance with the License.
5# You may obtain a copy of the License at
6#
7# http://www.apache.org/licenses/LICENSE-2.0
8#
9# Unless required by applicable law or agreed to in writing, software
10# distributed under the License is distributed on an "AS IS" BASIS,
11# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12# See the License for the specific language governing permissions and
13# limitations under the License.
Kailash92764922019-07-25 08:21:39 -070014# robot test functions
15
16*** Settings ***
17Documentation Library for various utilities
18Library SSHLibrary
Kailash92764922019-07-25 08:21:39 -070019Library String
20Library DateTime
21Library Process
22Library Collections
23Library RequestsLibrary
24Library OperatingSystem
Andy Bavier63460da2020-02-20 14:35:12 -070025Library CORDRobot
26Library ImportResource resources=CORDRobot
Kailash92764922019-07-25 08:21:39 -070027
28*** Keywords ***
Kailash528f7c02019-07-31 10:55:49 -070029Check CLI Tools Configured
30 [Documentation] Tests that use 'voltctl' and 'kubectl' should execute this keyword in suite setup
31 # check voltctl and kubectl configured
Zack Williamsa8fe75a2020-01-10 14:25:27 -070032 ${voltctl_rc}= Run And Return RC voltctl device list
33 ${kubectl_rc}= Run And Return RC kubectl get pods
Gilles Depatie675a2062019-10-22 12:44:42 -040034 Run Keyword If ${voltctl_rc} != 0 or ${kubectl_rc} != 0 FATAL ERROR
35 ... VOLTCTL and KUBECTL not configured. Please configure before executing tests.
Gilles Depatieb5682f82019-10-31 10:39:45 -040036
37Send File To Onos
Zack Williamsa8fe75a2020-01-10 14:25:27 -070038 [Documentation] Send the content of the file to Onos to selected section of configuration
39 ... using Post Request
40 [Arguments] ${CONFIG_FILE} ${section}
41 ${Headers}= Create Dictionary Content-Type application/json
42 ${File_Data}= OperatingSystem.Get File ${CONFIG_FILE}
43 Log ${Headers}
44 Log ${File_Data}
45 ${resp}= Post Request ONOS
46 ... /onos/v1/network/configuration/${section} headers=${Headers} data=${File_Data}
47 Should Be Equal As Strings ${resp.status_code} 200
Andy Bavier88cd9f62019-11-26 16:22:33 -070048
49Common Test Suite Setup
50 [Documentation] Setup the test suite
Andy Bavier88cd9f62019-11-26 16:22:33 -070051 Set Global Variable ${KUBECTL_CONFIG} export KUBECONFIG=%{KUBECONFIG}
52 Set Global Variable ${VOLTCTL_CONFIG} export VOLTCONFIG=%{VOLTCONFIG}
53 ${k8s_node_ip}= Evaluate ${nodes}[0].get("ip")
ubuntu6b6e7d42020-03-02 12:35:42 -080054 ${ONOS_REST_IP}= Get Environment Variable ONOS_REST_IP ${k8s_node_ip}
55 ${ONOS_SSH_IP}= Get Environment Variable ONOS_SSH_IP ${k8s_node_ip}
56 Set Global Variable ${ONOS_REST_IP}
57 Set Global Variable ${ONOS_SSH_IP}
Andy Bavier88cd9f62019-11-26 16:22:33 -070058 ${k8s_node_user}= Evaluate ${nodes}[0].get("user")
59 ${k8s_node_pass}= Evaluate ${nodes}[0].get("pass")
60 Check CLI Tools Configured
61 ${onos_auth}= Create List karaf karaf
62 ${HEADERS} Create Dictionary Content-Type=application/json
ubuntu6b6e7d42020-03-02 12:35:42 -080063 Create Session ONOS http://${ONOS_REST_IP}:${ONOS_REST_PORT} auth=${ONOS_AUTH}
Andy Bavier88cd9f62019-11-26 16:22:33 -070064 ${olt_ip}= Evaluate ${olts}[0].get("ip")
Suchitra Vemurib7253a52020-07-14 22:35:17 -070065 ${olt_ssh_ip}= Evaluate ${olts}[0].get("sship")
Andy Bavier88cd9f62019-11-26 16:22:33 -070066 ${olt_user}= Evaluate ${olts}[0].get("user")
67 ${olt_pass}= Evaluate ${olts}[0].get("pass")
68 ${olt_serial_number}= Evaluate ${olts}[0].get("serial")
69 ${num_onus}= Get Length ${hosts.src}
70 ${num_onus}= Convert to String ${num_onus}
71 #send sadis file to onos
Andy Baviera2691752019-11-27 12:18:38 -070072 ${sadis_file}= Get Variable Value ${sadis.file}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070073 Log To Console \nSadis File:${sadis_file}
Andy Bavier88cd9f62019-11-26 16:22:33 -070074 Run Keyword Unless '${sadis_file}' is '${None}' Send File To Onos ${sadis_file} apps/
75 Set Suite Variable ${num_onus}
76 Set Suite Variable ${olt_serial_number}
77 Set Suite Variable ${olt_ip}
Suchitra Vemurib7253a52020-07-14 22:35:17 -070078 Set Suite Variable ${olt_ssh_ip}
Andy Bavier88cd9f62019-11-26 16:22:33 -070079 Set Suite Variable ${olt_user}
80 Set Suite Variable ${olt_pass}
Andy Bavier88cd9f62019-11-26 16:22:33 -070081 @{container_list}= Create List adapter-open-olt adapter-open-onu voltha-api-server
82 ... voltha-ro-core voltha-rw-core-11 voltha-rw-core-12 voltha-ofagent
83 Set Suite Variable ${container_list}
84 ${datetime}= Get Current Date
suraj gour7f6d5fe2019-11-29 10:56:35 +000085 Set Suite Variable ${datetime}
86
87WPA Reassociate
suraj gour7f6d5fe2019-11-29 10:56:35 +000088 [Documentation] Executes a particular wpa_cli reassociate, which performs force reassociation
Zack Williamsa8fe75a2020-01-10 14:25:27 -070089 [Arguments] ${iface} ${ip} ${user} ${pass}=${None}
90 ... ${container_type}=${None} ${container_name}=${None}
suraj gour7f6d5fe2019-11-29 10:56:35 +000091 #Below for loops are used instead of sleep time, to execute reassociate command and check status
Zack Williamsa8fe75a2020-01-10 14:25:27 -070092 FOR ${i} IN RANGE 70
93 ${output}= Login And Run Command On Remote System
94 ... wpa_cli -i ${iface} reassociate ${ip} ${user}
95 ... ${pass} ${container_type} ${container_name}
96 ${passed}= Run Keyword And Return Status Should Contain ${output} OK
Andy Bavierb63f6d22020-03-12 15:34:37 -070097 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -070098 END
Andy Bavierb63f6d22020-03-12 15:34:37 -070099 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700100 FOR ${i} IN RANGE 70
101 ${output}= Login And Run Command On Remote System
Andy Bavierae4a17e2020-02-25 14:54:36 -0700102 ... wpa_cli -i ${iface} status | grep SUCCESS ${ip} ${user}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700103 ... ${pass} ${container_type} ${container_name}
104 ${passed}= Run Keyword And Return Status Should Contain ${output} SUCCESS
Andy Bavierb63f6d22020-03-12 15:34:37 -0700105 Exit For Loop If ${passed}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700106 END
Andy Bavierb63f6d22020-03-12 15:34:37 -0700107 Should Be True ${passed} Status does not contain 'SUCCESS'
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700108
suraj gour7f6d5fe2019-11-29 10:56:35 +0000109Validate Authentication After Reassociate
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700110 [Arguments] ${auth_pass} ${iface} ${ip} ${user} ${pass}=${None}
111 ... ${container_type}=${None} ${container_name}=${None}
112 [Documentation]
113 ... Executes a particular reassociate request on the RG using wpa_cli.
114 ... auth_pass determines if authentication should pass
Andy Bavier84834d42020-02-25 13:49:50 -0700115 ${wpa_log}= Catenate SEPARATOR=. /tmp/wpa ${iface} log
116 ${output}= Login And Run Command On Remote System truncate -s 0 ${wpa_log}; cat ${wpa_log}
117 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
118 Log ${output}
119 Should Not Contain ${output} authentication completed successfully
suraj gour7f6d5fe2019-11-29 10:56:35 +0000120 WPA Reassociate ${iface} ${ip} ${user} ${pass} ${container_type} ${container_name}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700121 Run Keyword If '${auth_pass}' == 'True' Wait Until Keyword Succeeds ${timeout} 2s
Andy Bavier84834d42020-02-25 13:49:50 -0700122 ... Check Remote File Contents True ${wpa_log} ${iface}.*authentication completed successfully
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700123 ... ${ip} ${user} ${pass} ${container_type} ${container_name}
suraj gour7f6d5fe2019-11-29 10:56:35 +0000124 Run Keyword If '${auth_pass}' == 'False' Sleep 20s
Suchitra Vemuri3cb29f82020-02-25 13:59:58 -0800125 Run Keyword If '${auth_pass}' == 'False' Check Remote File Contents False /tmp/wpa.log
Andy Bavier84834d42020-02-25 13:49:50 -0700126 ... ${iface}.*authentication completed successfully ${ip} ${user} ${pass}
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700127 ... ${container_type} ${container_name}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000128
129Send Dhclient Request To Release Assigned IP
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700130 [Arguments] ${iface} ${ip} ${user} ${path_dhcpleasefile} ${pass}=${None}
131 ... ${container_type}=${None} ${container_name}=${None}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000132 [Documentation] Executes a dhclient with option to release ip against a particular interface on the RG (src)
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700133 ${result}= Login And Run Command On Remote System
134 ... dhclient -nw -r ${iface} && rm ${path_dhcpleasefile}/dhclient.* ${ip} ${user}
suraj gourd5cfdbb2019-12-13 12:44:55 +0000135 ... ${pass} ${container_type} ${container_name}
136 Log ${result}
137 #Should Contain ${result} DHCPRELEASE
138 [Return] ${result}
suraj gour1ecfae92019-12-20 15:11:40 +0000139
140Check Remote File Contents For WPA Logs
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700141 [Arguments] ${file_should_exist} ${file} ${pattern} ${ip} ${user} ${pass}=${None}
142 ... ${container_type}=${None} ${container_name}=${None} ${prompt}=~$
suraj gour1ecfae92019-12-20 15:11:40 +0000143 [Documentation] Checks for particular pattern count in a file
Zack Williamsa8fe75a2020-01-10 14:25:27 -0700144 ${result}= Login And Run Command On Remote System
145 ... cat ${file} | grep '${pattern}' | wc -l ${ip} ${user} ${pass}
suraj gour1ecfae92019-12-20 15:11:40 +0000146 ... ${container_type} ${container_name} ${prompt}
147 [Return] ${result}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000148
149Perform Sanity Test
150 [Documentation] This keyword performs Sanity Test Procedure
151 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
152 ... This keyword can be used to call in any other tests where sanity check is required
153 ... and avoids duplication of code.
154 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
155 Set Global Variable ${of_id}
Hardik Windlass21807632020-04-14 16:24:55 +0530156 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
157 ... Get NNI Port in ONOS ${of_id}
158 Set Global Variable ${nni_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000159 FOR ${I} IN RANGE 0 ${num_onus}
160 ${src}= Set Variable ${hosts.src[${I}]}
161 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800162 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000163 ${onu_device_id}= Get Device ID From SN ${src['onu']}
164 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
165 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800166 # Check ONU port is Enabled in ONOS
167 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700168 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800169 # Verify EAPOL flows are added for the ONU port
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000170 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800171 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800172 # Verify ONU state in voltha
173 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
174 ... ENABLED ACTIVE REACHABLE
175 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
176 # Perform Authentication
Andy Bavier84834d42020-02-25 13:49:50 -0700177 ${wpa_log}= Run Keyword If ${has_dataplane} Catenate SEPARATOR=.
178 ... /tmp/wpa ${src['dp_iface_name']} log
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000179 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate Authentication True
180 ... ${src['dp_iface_name']} wpa_supplicant.conf ${src['ip']} ${src['user']} ${src['pass']}
Andy Bavier84834d42020-02-25 13:49:50 -0700181 ... ${src['container_type']} ${src['container_name']} ${wpa_log}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000182 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800183 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000184 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800185 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000186 ... volt-add-subscriber-access ${of_id} ${onu_port}
Suchitra Vemuri9da44302020-03-04 14:24:49 -0800187 # Verify that no pending flows exist for the ONU port
188 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
189 ... Verify No Pending Flows For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Hardik Windlass21807632020-04-14 16:24:55 +0530190 # Verify subscriber access flows are added for the ONU port
191 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
192 ... Verify Subscriber Access Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
193 ... ${onu_port} ${nni_port} ${src['c_tag']} ${src['s_tag']}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000194 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
195 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
196 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
197 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
198 ... ${dst['container_name']}
199 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
ubuntu6b6e7d42020-03-02 12:35:42 -0800200 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000201 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
202 Run Keyword and Ignore Error Collect Logs
203 END
204
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000205Perform Sanity Test DT
206 [Documentation] This keyword performs Sanity Test Procedure for DT Workflow
207 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
208 ... This keyword can be used to call in any other tests where sanity check is required
209 ... and avoids duplication of code.
210 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
211 Set Global Variable ${of_id}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530212 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
213 ... Get NNI Port in ONOS ${of_id}
214 Set Global Variable ${nni_port}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000215 FOR ${I} IN RANGE 0 ${num_onus}
216 ${src}= Set Variable ${hosts.src[${I}]}
217 ${dst}= Set Variable ${hosts.dst[${I}]}
218 ${onu_device_id}= Get Device ID From SN ${src['onu']}
219 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
220 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
221 # Check ONU port is Enabled in ONOS
222 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700223 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000224 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
ubuntu6b6e7d42020-03-02 12:35:42 -0800225 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000226 ... volt-add-subscriber-access ${of_id} ${onu_port}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530227 # Verify subscriber access flows are added for the ONU port
228 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530229 ... Verify Subscriber Access Flows Added For ONU DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
230 ... ${onu_port} ${nni_port} ${src['s_tag']}
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000231 # Verify ONU state in voltha
232 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
233 ... ENABLED ACTIVE REACHABLE
234 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530235 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000236 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
237 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
238 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
239 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
240 ... ${dst['container_name']}
241 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
242 Run Keyword and Ignore Error Collect Logs
243 END
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530244 # Verify ONOS Flows
245 # Number of Access Flows on ONOS equals 4 * the Number of Active ONUs (2 for each downstream and upstream)
246 ${onos_flows_count}= Evaluate 4 * ${num_onus}
247 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
Hardik Windlass25e11702020-03-30 20:05:19 +0530248 ... Verify Subscriber Access Flows Added Count DT ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${of_id}
249 ... ${onos_flows_count}
Hardik Windlassfb5eace2020-03-26 14:49:01 +0530250 # Verify VOLTHA Flows
Hardik Windlassa3d04b92020-02-17 15:06:05 +0000251 # Number of per OLT Flows equals Twice the Number of Active ONUs (each for downstream and upstream) + 1 for LLDP
252 ${olt_flows}= Evaluate 2 * ${num_onus} + 1
253 # Number of per ONU Flows equals 2 (one each for downstream and upstream)
254 ${onu_flows}= Set Variable 2
255 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate OLT Flows ${olt_flows}
256 ${List_ONU_Serial} Create List
257 Set Suite Variable ${List_ONU_Serial}
258 Build ONU SN List ${List_ONU_Serial}
259 Log ${List_ONU_Serial}
260 Run Keyword Wait Until Keyword Succeeds ${timeout} 5s Validate ONU Flows
261 ... ${List_ONU_Serial} ${onu_flows}
262
Suchitra Vemuri2f1e07d2020-06-26 19:18:01 -0700263Perform Sanity Test TT
264 [Documentation] This keyword performs Sanity Test Procedure for TT Workflow
265 ... Sanity test performs dhcp and pings (without EAPOL and DHCP flows) for all the ONUs given for the POD
266 ... This keyword can be used to call in any other tests where sanity check is required
267 ... and avoids duplication of code.
268 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
269 Set Global Variable ${of_id}
270 ${nni_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
271 ... Get NNI Port in ONOS ${of_id}
272 Set Global Variable ${nni_port}
273 FOR ${I} IN RANGE 0 ${num_onus}
274 ${src}= Set Variable ${hosts.src[${I}]}
275 ${dst}= Set Variable ${hosts.dst[${I}]}
276 ${onu_device_id}= Get Device ID From SN ${src['onu']}
277 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
278 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
279 # Check ONU port is Enabled in ONOS
280 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
281 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
282 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2
283 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
284 ... volt-add-subscriber-access ${of_id} ${onu_port}
285 Sleep 30s
286 # Verify ONU state in voltha
287 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
288 ... ENABLED ACTIVE REACHABLE
289 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
290 # TODO: Yet to Verify on the GPON based Physical POD (VOL-2652)
291 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure Validate DHCP and Ping True
292 ... True ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
293 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
294 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
295 ... ${dst['container_name']}
296 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
297 Run Keyword and Ignore Error Collect Logs
298 END
299
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000300Setup
301 [Documentation] Pre-test Setup
302 #test for empty device list
303 Test Empty Device List
Suchitra Vemuri3158f3c2020-05-28 17:58:26 -0700304 #Run Keyword If ${has_dataplane} Wait Until Keyword Succeeds 120s 10s Openolt is Up
305 #... ${olt_ip} ${olt_user} ${olt_pass}
Suchitra Vemuri1d8e8172020-06-17 17:43:56 -0700306 Run Keyword If ${has_dataplane} Sleep 230s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000307 #create/preprovision device
308 ${olt_device_id}= Create Device ${olt_ip} ${OLT_PORT}
309 Set Suite Variable ${olt_device_id}
310 #validate olt states
311 Wait Until Keyword Succeeds ${timeout} 5s
ubuntu6b6e7d42020-03-02 12:35:42 -0800312 ... Validate OLT Device PREPROVISIONED UNKNOWN UNKNOWN ${olt_device_id}
Suchitra Vemuria7e8ad22020-02-11 18:02:40 -0800313 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000314 Enable Device ${olt_device_id}
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800315 Wait Until Keyword Succeeds 180s 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000316 ... Validate OLT Device ENABLED ACTIVE REACHABLE ${olt_serial_number}
317 ${logical_id}= Get Logical Device ID From SN ${olt_serial_number}
318 Set Suite Variable ${logical_id}
319
Hardik Windlassaaea3402020-03-10 19:45:45 +0530320Validate ONUs After OLT Disable
321 [Documentation] Validates the ONUs state in Voltha, ONUs port state in ONOS
322 ... and that pings do not succeed After corresponding OLT is Disabled
323 FOR ${I} IN RANGE 0 ${num_onus}
324 ${src}= Set Variable ${hosts.src[${I}]}
325 ${dst}= Set Variable ${hosts.dst[${I}]}
326 ${onu_device_id}= Get Device ID From SN ${src['onu']}
327 ${onu_port}= Wait Until Keyword Succeeds ${timeout} 2s Get ONU Port in ONOS ${src['onu']}
328 ... ${of_id}
329 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s
330 ... Validate Device ENABLED DISCOVERED
331 ... UNREACHABLE ${src['onu']} onu=True onu_reason=stopping-openomci
332 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700333 ... Verify ONU Port Is Disabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Hardik Windlassaaea3402020-03-10 19:45:45 +0530334 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
335 ... Wait Until Keyword Succeeds 60s 2s
336 ... Check Ping False ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
337 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
338 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
339 Run Keyword and Ignore Error Collect Logs
340 END
341
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000342Delete All Devices and Verify
343 [Documentation] Remove any devices from VOLTHA and ONOS
344 # Clear devices from VOLTHA
345 Disable Devices In Voltha Root=true
Suchitra Vemuri970747f2020-03-30 17:31:34 -0700346 Sleep 5s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000347 Wait Until Keyword Succeeds ${timeout} 2s Test Devices Disabled In Voltha Root=true
348 Delete Devices In Voltha Root=true
Matteo Scandolof6cea382020-04-20 14:32:00 -0700349 Run Keyword If ${has_dataplane} Sleep 30s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000350 Wait Until Keyword Succeeds ${timeout} 2s Test Empty Device List
351 # Clear devices from ONOS
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800352 #Remove All Devices From ONOS
ubuntu6b6e7d42020-03-02 12:35:42 -0800353 #... http://karaf:karaf@${ONOS_REST_IP}:${ONOS_REST_PORT}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000354
355Teardown
356 [Documentation] kills processes and cleans up interfaces on src+dst servers
357 Run Keyword If ${has_dataplane} Clean Up Linux
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000358
359Teardown Suite
360 [Documentation] Clean up device if desired
361 Run Keyword If ${teardown_device} Delete All Devices and Verify
362
363Delete Device and Verify
364 [Documentation] Disable -> Delete devices via voltctl and verify its removed
Suchitra Vemuri2fa9bba2020-01-22 17:38:48 -0800365 ${olt_device_id}= Get Device ID From SN ${olt_serial_number}
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000366 ${rc} ${output}= Run and Return Rc and Output
367 ... ${VOLTCTL_CONFIG}; voltctl device disable ${olt_device_id}
368 Should Be Equal As Integers ${rc} 0
369 Sleep 5s
370 Wait Until Keyword Succeeds ${timeout} 5s
371 ... Validate OLT Device DISABLED UNKNOWN REACHABLE ${olt_serial_number}
372 ${rc} ${output}= Run and Return Rc and Output
373 ... ${VOLTCTL_CONFIG}; voltctl device delete ${olt_device_id}
Matteo Scandolof6cea382020-04-20 14:32:00 -0700374 Run Keyword If ${has_dataplane} Sleep 50s
Gayathri.Selvanee4a91b2020-01-17 06:49:53 +0000375 Should Be Equal As Integers ${rc} 0
376 Wait Until Keyword Succeeds ${timeout} 5s Validate Device Removed ${olt_device_id}
377
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000378Repeat Sanity Test
379 [Documentation] This keyword performs Sanity Test Procedure
380 ... Sanity test performs authentication, dhcp and pings for all the ONUs given for the POD
381 ... This keyword can be used to call in any other tests where sanity check is required
382 ... with wpa reassociation
383 ${of_id}= Wait Until Keyword Succeeds ${timeout} 15s Validate OLT Device in ONOS ${olt_serial_number}
384 Set Global Variable ${of_id}
385 FOR ${I} IN RANGE 0 ${num_onus}
Suchitra Vemuri03005912020-02-20 18:59:46 -0800386 Run Keyword and Ignore Error Collect Logs
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000387 ${src}= Set Variable ${hosts.src[${I}]}
388 ${dst}= Set Variable ${hosts.dst[${I}]}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800389 ${onu_port}= Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
390 ... Get ONU Port in ONOS ${src['onu']} ${of_id}
391 # Check ONU port is Enabled in ONOD
392 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 120s 2s
Suchitra Vemuri760bdd32020-06-17 12:34:48 -0700393 ... Verify ONU Port Is Enabled ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${src['onu']}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800394 # Verify EAPOL flows are added for the ONU port
395 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800396 ... Verify Eapol Flows Added For ONU ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800397 # Verify ONU state in voltha
Suchitra Vemurid1e27fb2020-02-01 16:34:37 -0800398 Run Keyword And Continue On Failure Wait Until Keyword Succeeds ${timeout} 5s Validate Device
Suchitra Vemuri81e4f022020-02-01 13:16:02 -0800399 ... ENABLED ACTIVE REACHABLE
400 ... ${src['onu']} onu=True onu_reason=omci-flows-pushed
Suchitra Vemuri8078b972020-02-06 19:07:41 -0800401 # Perform Authentication
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000402 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
403 ... Validate Authentication After Reassociate
404 ... True ${src['dp_iface_name']} ${src['ip']} ${src['user']} ${src['pass']}
405 ... ${src['container_type']} ${src['container_name']}
406 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800407 ... Verify ONU in AAA-Users ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000408 Wait Until Keyword Succeeds ${timeout} 2s
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800409 ... Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000410 ... volt-add-subscriber-access ${of_id} ${onu_port}
411 Run Keyword If ${has_dataplane} Run Keyword And Continue On Failure
412 ... Validate DHCP and Ping True True
413 ... ${src['dp_iface_name']} ${src['s_tag']} ${src['c_tag']} ${dst['dp_iface_ip_qinq']}
414 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
415 ... ${dst['dp_iface_name']} ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']}
416 ... ${dst['container_name']}
417 Wait Until Keyword Succeeds ${timeout} 2s Run Keyword And Continue On Failure
David K. Bainbridgec0617c92020-03-05 21:17:19 -0800418 ... Validate Subscriber DHCP Allocation ${ONOS_SSH_IP} ${ONOS_SSH_PORT} ${onu_port}
Gayathri.Selvan283a63c2020-01-23 04:09:18 +0000419 Run Keyword and Ignore Error Get Device Output from Voltha ${onu_device_id}
420 Run Keyword and Ignore Error Collect Logs
421 END
Gayathri.Selvanb6a2b542020-01-24 07:24:23 +0000422
423Collect Logs
424 [Documentation] Collect Logs from voltha and onos cli for various commands
425 Run Keyword and Ignore Error Get Device List from Voltha
426 Run Keyword and Ignore Error Get Device Output from Voltha ${olt_device_id}
427 Run Keyword and Ignore Error Get Logical Device Output from Voltha ${logical_id}
ubuntu6b6e7d42020-03-02 12:35:42 -0800428 Get ONOS Status ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
Gayathri.Selvanf68ea4b2020-02-03 07:36:39 +0000429
430Verify ping is succesful except for given device
431 [Arguments] ${num_onus} ${exceptional_onu_id}
432 [Documentation] Checks that ping for all the devices are successful except the given ONU.
433 ${pingStatus} Set Variable True
434 FOR ${I} IN RANGE 0 ${num_onus}
435 ${src}= Set Variable ${hosts.src[${I}]}
436 ${dst}= Set Variable ${hosts.dst[${I}]}
437 ${onu_device_id}= Get Device ID From SN ${src['onu']}
438 ${pingStatus} Run Keyword If '${onu_device_id}' == '${exceptional_onu_id}' Set Variable False
439 Run Keyword And Continue On Failure Wait Until Keyword Succeeds 60s 2s
440 ... Check Ping ${pingStatus} ${dst['dp_iface_ip_qinq']} ${src['dp_iface_name']}
441 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
442 END
Andy Bavier4a8450e2020-02-04 08:58:37 -0700443
Andy Bavier1923ecc2020-06-18 14:06:40 -0700444Echo Message to OLT Logs
445 [Arguments] ${message}
446 [Documentation] Echoes ${message} into the OLT logs
Andy Bavier690ce432020-06-20 16:31:10 -0700447 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700448 ... printf '%s\n' '' '' '${message}' '' >> /var/log/openolt.log
Suchitra Vemurib7253a52020-07-14 22:35:17 -0700449 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
Andy Bavier690ce432020-06-20 16:31:10 -0700450 Wait Until Keyword Succeeds 180s 10s Execute Remote Command
Andy Bavier6f613612020-06-19 15:08:30 -0700451 ... printf '%s\n' '' '' '${message}' '' >> /var/log/dev_mgmt_daemon.log
Suchitra Vemurib7253a52020-07-14 22:35:17 -0700452 ... ${olt_ssh_ip} ${olt_user} ${olt_pass}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700453
Andy Bavierabeba262020-02-07 16:22:16 -0700454Start Logging
455 [Arguments] ${label}
456 [Documentation] Start logging for test ${label}
457 ${kail_process}= Run Keyword If "${container_log_dir}" != "${None}" Start Process kail -n default
458 ... -n voltha cwd=${container_log_dir} stdout=${label}-combined.log
459 Set Test Variable ${kail_process}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700460 Run Keyword If ${has_dataplane} Echo Message to OLT Logs START ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700461
462Stop Logging
463 [Arguments] ${label}
464 [Documentation] End logging for test; remove logfile if test passed
465 Run sync
466 Run Keyword If ${kail_process} Terminate Process ${kail_process}
467 ${test_logfile}= Run Keyword If "${container_log_dir}" != "${None}"
468 ... Join Path ${container_log_dir} ${label}-combined.log
469 Run Keyword If Test Passed Run Keyword If "${test_logfile}" != "${None}" Remove File ${test_logfile}
Andy Bavier1923ecc2020-06-18 14:06:40 -0700470 Run Keyword If ${has_dataplane} Echo Message to OLT Logs END ${label}
Andy Bavierabeba262020-02-07 16:22:16 -0700471
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800472Clean Up Linux
473 [Documentation] Kill processes and clean up interfaces on src+dst servers
474 FOR ${I} IN RANGE 0 ${num_onus}
475 ${src}= Set Variable ${hosts.src[${I}]}
476 ${dst}= Set Variable ${hosts.dst[${I}]}
Andy Bavier84b83282020-06-30 05:47:49 -0700477 Execute Remote Command sudo pkill wpa_supplicant ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800478 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700479 Execute Remote Command sudo pkill dhclient ${src['ip']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800480 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Baviereff938d2020-06-29 18:27:49 -0700481 Execute Remote Command sudo pkill mausezahn ${src['ip']}
482 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
483 Run Keyword If '${dst['ip']}' != '${None}' Execute Remote Command pkill dhcpd
484 ... ${dst['ip']} ${dst['user']} ${dst['pass']} ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800485 Delete IP Addresses from Interface on Remote Host ${src['dp_iface_name']} ${src['ip']}
486 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
487 Run Keyword If '${dst['ip']}' != '${None}' Delete Interface on Remote Host
488 ... ${dst['dp_iface_name']}.${src['s_tag']} ${dst['ip']} ${dst['user']} ${dst['pass']}
489 ... ${dst['container_type']} ${dst['container_name']}
Andy Bavier84b83282020-06-30 05:47:49 -0700490 ${bng_ip}= Get Variable Value ${dst['noroot_ip']}
491 ${bng_user}= Get Variable Value ${dst['noroot_user']}
492 ${bng_pass}= Get Variable Value ${dst['noroot_pass']}
Andy Bavierf4b7d1d2020-06-30 11:14:04 -0700493 Run Keyword If "${bng_ip}" != "${NONE}" and "${bng_user}" != "${NONE}" and "${bng_pass}" != "${NONE}"
494 ... Execute Remote Command sudo pkill mausezahn ${bng_ip} ${bng_user} ${bng_pass}
Andy Baviereff938d2020-06-29 18:27:49 -0700495 ... ${dst['container_type']} ${dst['container_name']}
Suchitra Vemuria5fa3dd2020-02-04 14:38:49 -0800496 END
Scott Baker60e570d2020-02-02 22:10:13 -0800497
Suchitra Vemurif7a033c2020-02-26 17:22:41 -0800498Clean WPA Process
499 [Documentation] Kills wpa_supplicant processes only for all RGs
500 FOR ${I} IN RANGE 0 ${num_onus}
501 ${src}= Set Variable ${hosts.src[${I}]}
502 ${dst}= Set Variable ${hosts.dst[${I}]}
503 Run Keyword And Ignore Error Kill Linux Process [w]pa_supplicant ${src['ip']}
504 ... ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
505 END
506
Scott Baker60e570d2020-02-02 22:10:13 -0800507Should Be Larger Than
508 [Documentation] Verify that value_1 is > value_2
509 [Arguments] ${value_1} ${value_2}
510 Run Keyword If ${value_1} <= ${value_2}
511 ... Fail The value ${value_1} is not larger than ${value_2}
512
Scott Bakeree675552020-02-11 10:43:34 -0800513Should Be Larger Than Or Equal To
514 [Documentation] Verify that value_1 is >= value_2
515 [Arguments] ${value_1} ${value_2}
516 Run Keyword If ${value_1} < ${value_2}
517 ... Fail The value ${value_1} is not larger than or equal to ${value_2}
518
Scott Baker60e570d2020-02-02 22:10:13 -0800519Should Be Float
520 [Documentation] Verify that value is a floating point number type
521 [Arguments] ${value}
522 ${type} Evaluate type(${value}).__name__
523 Should Be Equal ${type} float
524
Scott Bakeree675552020-02-11 10:43:34 -0800525Should Be Newer Than Or Equal To
526 [Documentation] Compare two RFC3339 dates
527 [Arguments] ${value_1} ${value_2}
528 ${unix_v1} Parse RFC3339 ${value_1}
529 ${unix_v2} Parse RFC3339 ${value_2}
530 Run Keyword If ${unix_v1} < ${unix_v2}
531 ... Fail The value ${value_1} is not newer than or equal to ${value_2}
532
Scott Baker60e570d2020-02-02 22:10:13 -0800533Get Current Time
534 [Documentation] Return the current time in RFC3339 format
535 ${output}= Run date -u +"%FT%T%:z"
536 [return] ${output}
Scott Bakeree675552020-02-11 10:43:34 -0800537
538Parse RFC3339
539 [Documentation] Parse an RFC3339 timestamp
540 [Arguments] ${dateStr}
541 ${rc} ${output}= Run and Return Rc and Output date --date="${dateStr}" "+%s"
542 Should Be Equal As Numbers ${rc} 0
543 [return] ${output}
Matteo Scandolo142e6272020-04-29 17:36:59 -0700544
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000545Get Bandwidth Profile Name For Given Subscriber
546 [Arguments] ${subscriber_id} ${stream_type}=upstreamBandwidthProfile
547 [Documentation] Keyword to get the bandwidth details of the given subscriber
548 ${bandwidth_profile_output}= Execute ONOS CLI Command ${ONOS_SSH_IP} ${ONOS_SSH_PORT}
549 ... volt-programmed-subscribers | grep ${subscriber_id}
550 @{bandwidth_profile_array}= Split String ${bandwidth_profile_output} ,
551 Log ${bandwidth_profile_array}
552 FOR ${value} IN @{bandwidth_profile_array}
553 @{row_value}= Split String ${value} =
554 ${bandwidth_profile_name}= Set Variable If '${row_value[0]}' == ' ${stream_type}'
555 ... ${row_value[1]}
556 ${bandwidth_profile_name}= Convert To String ${bandwidth_profile_name}
557 Run Keyword If "${bandwidth_profile_name}" != "None" Exit For Loop
558 END
559 Log ${bandwidth_profile_name}
560 [Return] ${bandwidth_profile_name}
561
562Execute Remote Command
563 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
564 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
565 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
Andy Baviereff938d2020-06-29 18:27:49 -0700566 ... ${container_type}=${None} ${container_name}=${None} ${timeout}=${None}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000567 ${conn_id}= SSHLibrary.Open Connection ${ip}
568 Run Keyword If '${pass}' != '${None}'
569 ... SSHLibrary.Login ${user} ${pass}
570 ... ELSE
571 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
572 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
573 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700574 ${stdout} ${stderr} ${rc}= Run Keyword If '${container_type}' == 'LXC'
575 ... SSHLibrary.Execute Command lxc exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700576 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000577 ... ELSE IF '${container_type}' == 'K8S'
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700578 ... SSHLibrary.Execute Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
Andy Baviereff938d2020-06-29 18:27:49 -0700579 ... return_stderr=True return_rc=True timeout=${timeout}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000580 ... ELSE
Andy Baviereff938d2020-06-29 18:27:49 -0700581 ... SSHLibrary.Execute Command ${cmd} return_stderr=True return_rc=True timeout=${timeout}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700582
583 Log ${stdout}
584 Log ${stderr}
585 Log ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000586 SSHLibrary.Close Connection
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700587 [Return] ${stdout} ${stderr} ${rc}
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000588
Suraj Gour33651272020-05-04 10:10:16 +0530589Start Remote Command
590 [Documentation] SSH into a remote host and execute a command on the bare host or in a container.
591 ... This replaces and simplifies the Login And Run Command On Remote System keyword in CORDRobot.
592 [Arguments] ${cmd} ${ip} ${user} ${pass}=${None}
593 ... ${container_type}=${None} ${container_name}=${None}
594 ${conn_id}= SSHLibrary.Open Connection ${ip}
595 Run Keyword If '${pass}' != '${None}'
596 ... SSHLibrary.Login ${user} ${pass}
597 ... ELSE
598 ... SSHLibrary.Login With Public Key ${user} %{HOME}/.ssh/id_rsa
599 ${namespace}= Run Keyword If '${container_type}' == 'K8S' SSHLibrary.Execute Command
600 ... kubectl get pods --all-namespaces | grep ${container_name} | awk '{print $1}'
601 Run Keyword If '${container_type}' == 'LXC'
602 ... SSHLibrary.Start Command lxc exec ${container_name} -- ${cmd}
603 ... ELSE IF '${container_type}' == 'K8S'
604 ... SSHLibrary.Start Command kubectl -n ${namespace} exec ${container_name} -- ${cmd}
605 ... ELSE
606 ... SSHLibrary.Start Command ${cmd}
Andy Bavier9fb23232020-06-30 15:12:20 -0700607 # It seems that closing the connection immediately will sometimes kill the command
608 Sleep 1s
Suraj Gour33651272020-05-04 10:10:16 +0530609 SSHLibrary.Close Connection
610
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000611Run Iperf3 Test Client
612 [Arguments] ${src} ${server} ${args}
613 [Documentation] Login to ${src} and run the iperf3 client against ${server} using ${args}.
614 ... Return a Dictionary containing the results of the test.
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700615 ${output} ${stderr} ${rc}= Execute Remote Command iperf3 -J -c ${server} ${args} | jq -M -c '.'
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000616 ... ${src['ip']} ${src['user']} ${src['pass']} ${src['container_type']} ${src['container_name']}
Andy Bavierf6ab19c2020-05-14 10:34:47 -0700617 Should Be Equal As Integers ${rc} 0
Gayathri.Selvan92d16862020-03-19 14:47:58 +0000618 ${object}= Evaluate json.loads(r'''${output}''') json
619 [Return] ${object}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000620
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000621RestoreONUs
622 [Documentation] Restore all connected ONUs
623 [Arguments] ${num_onus}
624 FOR ${I} IN RANGE 0 ${num_onus}
625 ${src}= Set Variable ${hosts.src[${I}]}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700626 ${container_type}= Get Variable Value ${src['container_type']} "null"
627 ${container_name}= Get Variable Value ${src['container_name']} "null"
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000628 ${onu_type}= Get Variable Value ${src['onu_type']} "null"
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700629 #Get ens6f0 from ens6f0.22
630 ${if_name}= Replace String Using Regexp ${src['dp_iface_name']} \\..* \
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000631 Run Keyword IF '${onu_type}' == 'alpha' AlphaONURestoreDefault ${src['ip']} ${src['user']}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700632 ... ${src['pass']} ${if_name} admin admin ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000633 END
634
635AlphaONURestoreDefault
636 [Documentation] Restore the Alpha ONU to factory setting
637 [Arguments] ${rg_ip} ${rg_user} ${rg_pass} ${onu_ifname}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700638 ... ${onu_user} ${onu_pass} ${container_type}=${None} ${container_name}=${None}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000639 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 192.168.1.3/24
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700640 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000641 ${cmd} Catenate
642 ... (echo open "192.168.1.1"; sleep 1;
643 ... echo "${onu_user}"; sleep 1;
644 ... echo "${onu_pass}"; sleep 1;
645 ... echo "restoredefault"; sleep 1) | telnet
646 ${output}= Login And Run Command On Remote System ${cmd}
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700647 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Hung-Wei Chiue0bec7c2020-05-29 22:16:56 +0000648 Log To Console ${output}
649 ${output}= Login And Run Command On Remote System sudo ifconfig ${onu_ifname} 0
Hung-Wei Chiucc6e6552020-06-04 12:57:38 -0700650 ... ${rg_ip} ${rg_user} ${rg_pass} ${container_type} ${container_name}
Suraj Gour33651272020-05-04 10:10:16 +0530651
652Create traffic with each pbit and capture at other end
653 [Documentation] Generates upstream traffic using Mausezahn tool
654 ... with each pbit and validates reception at other end using tcpdump
655 [Arguments] ${target_ip} ${target_iface} ${src_iface}
656 ... ${packet_count} ${packet_type} ${target_port} ${vlan} ${tcpdump_filter}
657 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
658 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
659 FOR ${pbit} IN RANGE 8
Andy Baviereff938d2020-06-29 18:27:49 -0700660 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +0530661 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -0700662 ${var1}= Set Variable sudo mausezahn ${src_iface} -B ${target_ip} -c ${packet_count} -d 100m
Suraj Gour33651272020-05-04 10:10:16 +0530663 ${var2}= Set Variable -t ${packet_type} "dp=${target_port}" -p 1472 -Q ${pbit}:${vlan}
664 ${cmd}= Set Variable ${var1} ${var2}
665 Start Remote Command ${cmd} ${src_ip} ${src_user} ${src_pass}
666 ... ${src_container_type} ${src_container_name}
667 ${output} ${stderr} ${rc}= Execute Remote Command
Andy Baviereff938d2020-06-29 18:27:49 -0700668 ... sudo tcpdump -l -U -c 30 -i ${target_iface} -e ${tcpdump_filter}
Suraj Gour33651272020-05-04 10:10:16 +0530669 ... ${dst_ip} ${dst_user} ${dst_pass} ${dst_container_type} ${dst_container_name}
Andy Baviereff938d2020-06-29 18:27:49 -0700670 ... timeout=30 seconds
671 Execute Remote Command sudo pkill mausezahn
Suraj Gour33651272020-05-04 10:10:16 +0530672 ... ${src_ip} ${src_user} ${src_pass} ${src_container_type} ${src_container_name}
673 # VOL-3262: I'm seeing untagged downstream traffic at RG for pbit 0. According to Girish this is
674 # incorrect behavior. Simplify the following check when VOL-3262 is resolved.
675 Run Keyword If ${pbit}==0 and "${tcpdump_filter}"=="udp"
676 ... Should Match Regexp ${output} \\.${target_port}: UDP,
677 ... ELSE Should Match Regexp ${output} , p ${pbit},.*\\.${target_port}: UDP,
678 END